3,748+ c204 Smaller-but-Needed - Page 23

EdH · 2021-11-11, 13:48

Eleven "stuck" clients greeted me this morning! Was this another IP refresh? I might drop a note to the CADO Team.

On a bright note, our Team CPU has turned 21 and is now fully considered an adult in the U.S.

kruoli · 2021-11-11, 14:07

Yes, I had to restart the server this morning once again.

The problem is not strictly with renewing the IP. My IP gets renewed every day and I had no problems for at least a few days. It is much more likely that the problem occurs when the server and/or client is doing something specific while the IP gets renewed. The HTTP server on the server side gets stuck. Thus, I have to Strg+C twice to kill the server. I am not sure at which point the client gets stuck. I would advise to add a timeout 1h ./cado-nfs-client.py … --single to your command line (that already uses --single, as you said, but not timeout). I thought about this only just now and will add that to my client script.

I was irritated to see that a few WUs came through after the server got stuck, our status page showed the last incoming WU at least three hours after the IP renewal.

EdH · 2021-11-11, 14:50

Quote:

Originally Posted by kruoli

Yes, I had to restart the server this morning once again.

The problem is not strictly with renewing the IP. My IP gets renewed every day and I had no problems for at least a few days. It is much more likely that the problem occurs when the server and/or client is doing something specific while the IP gets renewed. The HTTP server on the server side gets stuck. Thus, I have to Strg+C twice to kill the server. I am not sure at which point the client gets stuck. I would advise to add a timeout 1h ./cado-nfs-client.py … --single to you command line (that already uses --single, as you said, but not timeout). I thought about this only just now and will add that to my client script.

I was irritated to see that a few WUs came through after the server got stuck, our status page showed the last incoming WU at least three hours after the IP renewal.

I suspect that a client is in the middle of a WU transfer when the renewal occurs and since they're locked, the IP change confuses them.

Thanks for the mention of "timeout." I will look into adding that to my scripts as well. (Probably, not today, though.)

Our status page should be looking at WUs being sent out for the check, rather than WUs received. I think it's doing that, isn't it? That should flag if there is no new WU sent out shown in the log after our set time.

kruoli · 2021-11-11, 14:51

Quote:

Originally Posted by EdH

Our status page should be looking at WUs being sent out for the check, rather than WUs received. I think it's doing that, isn't it? That should flag if there is no new WU sent out shown in the log after our set time.

I thought it was doing both? But this should not change the weirdness of this behaviour.

EdH · 2021-11-11, 15:01

Quote:

Originally Posted by kruoli

I thought it was doing both? But this should not change the weirdness of this behaviour.

It's curious if the clients are taking that long to process a WU, although that is possible. But it may just be that those clients took that long retrying and finally uploaded their results. It's good you have a long timeout for the WUs.

pinhodecarlos · 2021-11-12, 20:56

Oliver, can you please provide me with the full command to run the CADO client to point to your server? TIA

EdH · 2021-11-13, 19:23

This is very rough and it's quite possible that I am off in my calculations, but I tried modifying some clients on a machine with enough RAM, based on info from VBCurtis some time ago.

Since I have some machines with a little more RAM, I cut the threads on some of the clients and made new clients with those cut threads. Specifically, I tried it with a Z620 12c/24t dual Xeon that has 48GB RAM. It was running three clients, each using 8 threads and 5.5GB RAM. I am now running 6 clients, each using 4 threads and 5.5GB RAM. It still has some RAM room.

If my timing data calculations are correct, I have gained about 9% in throughput. I believe this corresponds with the info from VBCurtis about such.

Based on my perceived success, I will be trying to change some other machines to see if this holds true for others capable of meeting the RAM needs.

Thanks Curtis! It took me a while to implement this, but it may very well get me to modify the scripts for all future CADO-NFS work, both local and team.

kruoli · 2021-11-13, 22:01

We just hit 1,000,000,000 relations.

charybdis · 2021-11-15, 23:38

Quote:

Maximum relations returned from single WU: 12408 (483824000-483826000)

What's going on here? Was this something like A=32? I hope it wasn't higher lpbs because then the extra relations would be useless.

kruoli · 2021-11-16, 06:18

Quote:

Originally Posted by charybdis

Was this something like A=32?

Exactly, I hit it after three WUs.

kruoli · 2021-11-16, 09:57

Finally, Seth's stats view is also available. It is linked to from within the other status page. He anonymised client details after I wished for it until we find a better solution (some did not want to share them, and it was quicker to do it this way rather than removing all the client details (I guess)).

We have one more client than real participants because there is a machine I could not associate safely with its owner. Thus, it is listed like it was a separate user.

Also, Seth added a graph which shows the WU yield against Q. You can quickly spot where we changed the sieving strategy (at least I thought the first big jump is that, but IIRC we switched the strategy later, didn't we?), where I experimented with A=31 and maybe even the few outliers because of A=32.

The page gets refreshed every half an hour.

2021-11-11, 13:48	#243
EdH "Ed Hall" Dec 2009 Adirondack Mtns 11FC₁₆ Posts	Not quite a dozen Eleven "stuck" clients greeted me this morning! Was this another IP refresh? I might drop a note to the CADO Team. On a bright note, our Team CPU has turned 21 and is now fully considered an adult in the U.S.

2021-11-11, 14:07	#244
kruoli "Oliver" Sep 2017 Porta Westfalica, DE 2⁴·5·13 Posts	Yes, I had to restart the server this morning once again. The problem is not strictly with renewing the IP. My IP gets renewed every day and I had no problems for at least a few days. It is much more likely that the problem occurs when the server and/or client is doing something specific while the IP gets renewed. The HTTP server on the server side gets stuck. Thus, I have to Strg+C twice to kill the server. I am not sure at which point the client gets stuck. I would advise to add a timeout 1h ./cado-nfs-client.py … --single to your command line (that already uses --single, as you said, but not timeout). I thought about this only just now and will add that to my client script. I was irritated to see that a few WUs came through after the server got stuck, our status page showed the last incoming WU at least three hours after the IP renewal. Last fiddled with by kruoli on 2021-11-11 at 14:50 Reason: Grammar.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Using 16e on smaller numbers	fivemack	Factoring	3	2017-09-19 08:52
NFS on smaller numbers?	skan	YAFU	6	2013-02-26 13:57
Bernoulli(200) c204	akruppa	Factoring	114	2012-08-20 14:01
checking smaller number	fortega	Data	2	2005-06-16 22:48
Factoring Smaller Numbers	marc	Factoring	6	2004-10-09 14:17

2021-11-12, 20:56	#248
pinhodecarlos "Carlos Pinho" Oct 2011 Milton Keynes, UK 3·41² Posts	Oliver, can you please provide me with the full command to run the CADO client to point to your server? TIA

2021-11-13, 19:23	#249
EdH "Ed Hall" Dec 2009 Adirondack Mtns 2²×1,151 Posts	This is very rough and it's quite possible that I am off in my calculations, but I tried modifying some clients on a machine with enough RAM, based on info from VBCurtis some time ago. Since I have some machines with a little more RAM, I cut the threads on some of the clients and made new clients with those cut threads. Specifically, I tried it with a Z620 12c/24t dual Xeon that has 48GB RAM. It was running three clients, each using 8 threads and 5.5GB RAM. I am now running 6 clients, each using 4 threads and 5.5GB RAM. It still has some RAM room. If my timing data calculations are correct, I have gained about 9% in throughput. I believe this corresponds with the info from VBCurtis about such. Based on my perceived success, I will be trying to change some other machines to see if this holds true for others capable of meeting the RAM needs. Thanks Curtis! It took me a while to implement this, but it may very well get me to modify the scripts for all future CADO-NFS work, both local and team.

2021-11-13, 22:01	#250
kruoli "Oliver" Sep 2017 Porta Westfalica, DE 2⁴×5×13 Posts	We just hit 1,000,000,000 relations.

2021-11-16, 09:57	#253
kruoli "Oliver" Sep 2017 Porta Westfalica, DE 2⁴×5×13 Posts	Finally, Seth's stats view is also available. It is linked to from within the other status page. He anonymised client details after I wished for it until we find a better solution (some did not want to share them, and it was quicker to do it this way rather than removing all the client details (I guess)). We have one more client than real participants because there is a machine I could not associate safely with its owner. Thus, it is listed like it was a separate user. Also, Seth added a graph which shows the WU yield against Q. You can quickly spot where we changed the sieving strategy (at least I thought the first big jump is that, but IIRC we switched the strategy later, didn't we?), where I experimented with A=31 and maybe even the few outliers because of A=32. The page gets refreshed every half an hour.