The Next Big Development for GIMPS

[ewmayer]

Originally Posted by axn

In which case, you'd need to save the current iteration, the last verified GEC check (for rolling back), and the current GEC cumulative product. Still 3 needed.

I'm wondering now how you're managing with just two?

Each redundant p[exponent] and q[exponent] savefile saves current PRP residue and GEC cumulative product. Every 1M iters we do an actual GEC check; if it passes we write both the usual savefiles and a last-good-GEC copy named p[exponent].G. If the next GEC fails, we roll back to the .G file.

I've not yet had that procedure fail, even after a dozen runs on my ultra-flaky (and hence great for testing this stuff) Haswell system, but in the as-yet-unencountered case that the run data and the .G file get corrupted, the user can manually copy the last p[exponent].[k*10]M persistent savefile into both p[exponent] and p[exponent].G and restart from that, maximal loss 10Miters.

[Prime95]

Latest improvements/updates from the technical discussions.

For an exponent around 100,000,000. We can default to power==7, requiring 1.5GB of disk space during the test, and a 100MB proof file to upload. The server can relatively quickly convert this to a 12.5MB file to store on the server until verification is complete. This 12.5MB file can be sent to a verifier that runs 0.8% of a full PRP test, returning a 256-bit code to the server. Assuming a match, we're all done!

The math looks good that results cannot be faked. The original PRP tester can even be sent the verification work and cannot fake the verification!

The PRP tester could go one power higher which doubles required temp disk space and proof size increases by 12.5MB, verification time is cut in half. Or conversely PRP tester could go one power lower, halving required temp disk space, 12.5MB smaller proof file, and doubling verification time.

[axn]

Originally Posted by Prime95

The PRP tester could go one power higher which doubles required temp disk space and proof size increases by 12.5MB, verification time is cut in half. Or conversely PRP tester could go one power lower, halving required temp disk space, 12.5MB smaller proof file, and doubling verification time.

If this is going to be a dedicated worktype outsourced to users, you might as well go one (or even two!!) step lower to reduce the logistics aspect of it.

[R. Gerbicz]

Originally Posted by axn

If this is going to be a dedicated worktype outsourced to users, you might as well go one (or even two!!) step lower to reduce the logistics aspect of it.

With power=5 would you need more than 3% verification time of the original prp test.

[axn]

Originally Posted by R. Gerbicz

With power=5 would you need more than 3% verification time of the original prp test.

Which is still 97% better than the 100% verification time today (aka double check). We should optimize for the amount of bandwidth which the authors (i.e. first time testers), verifiers, and the server consume.

Again, someone like Curtis Cooper will feel the pinch when a whole batch of clients suddenly decide to upload gigabytes of data in the middle of the day clogging the university network.

EDIT:- If this ever goes live, we might have to ask Curtis C to change over a significant fraction of their resource to verification work.

[Prime95]

Originally Posted by axn

Which is still 97% better than the 100% verification time today (aka double check). We should optimize for the amount of bandwidth which the authors (i.e. first time testers), verifiers, and the server consume.

Again, someone like Curtis Cooper will feel the pinch when a whole batch of clients suddenly decide to upload gigabytes of data in the middle of the day clogging the university network.

I suspect a University is least likely to be affected by bandwidth. Temp disk space could be another matter. Random secretary/student/professor will not be happy if prime95 eats up the last few GB of disk space.

I foresee options to tell prime95 when to do uploads, how much disk space each worker can use, and the ability to choose the proof power to some degree. Prime95 default setup should be more multithreaded. 4 workers chewing up 2GB disk each is inferior to one quad-threaded worker using only 2GB total.

[pinhodecarlos]

It’s a good idea setting up Prime95 when to upload since more people will start working from home due to the current circumstances and i already noticed that conference calls by teams and work VPN’s mess up the internet connection even on fibre broadband.

[ATH]

Primenet should still store the normal 64 bit residue just in case there is a problem, and someone wants to run a real double check, or years later if someone wants to double check some results when the calculation is a lot faster.

[kriesel]

Originally Posted by Prime95

I suspect a University is least likely to be affected by bandwidth. Temp disk space could be another matter. Random secretary/student/professor will not be happy if prime95 eats up the last few GB of disk space.

I foresee options to tell prime95 when to do uploads, how much disk space each worker can use, and the ability to choose the proof power to some degree. Prime95 default setup should be more multithreaded. 4 workers chewing up 2GB disk each is inferior to one quad-threaded worker using only 2GB total.

Please consider implementing settable download and upload rate limits also. Many universities have fat pipes. And some parts of a university may be modest sized remote sites with thin pipes. I worked at one such for decades. As far as I know, it has never had access faster than 10Mbps. There was a several year stretch of T1 (1.544Mbps). 2GB at T1 is 2.9 hours. At 10Mbs line rate it is 0.44 hours each. Saturating a link can lead to issues for other things, like page loads from a web server. We experienced multiple cases over the years where the incoming or outgoing got saturated and made everyone's internet use difficult or fruitless for the duration. (Third party IP spoofed packet to cause a Swiss HP printer to produce a flood to us, or someone opening a malicious email attachment that launched a spam server, that sort of thing.) Periodically creating sustained link saturation may get the activity banned from a network.

User settable:

• schedule window for downloads of verification work
• schedule window for uploads of proof files
• transmission rate limit(s); separate up and down for asymmetric connections?
• disk quota
• participation in verification work
• proof power preference

[preda]

Originally Posted by ewmayer

Here gpuowl - this one is also for a PRP tests but is clearly just of single-residue size, so gpuowl must stash the GEC residue somewhere else ... but the 104960929-old.owl file is the only other one of that size I see. Ah, I get it - gpuowl is storing last-GEC-good residue in the 'old' file, and latest PRP-test residue in the non-old file, and using an on-the-fly method to generate a GEC residue at each restart, rather than a whole-run-cumulative one as I use:

I have to correct that interpretation of gpuowl's checkpoint size: gpuowl does store a single residue, and that is just the GEC (the "from the start" one). The actual residue can be computed from the GEC residue (but not the other way around), and this is done on load.

[preda]

Originally Posted by axn

In which case, you'd need to save the current iteration, the last verified GEC check (for rolling back), and the current GEC cumulative product. Still 3 needed.

I'm wondering now how you're managing with just two?

Just one! gpuowl only ever saves verified residues, and only writes the base-3 GEC residue, and the res64 of the normal residue. On load the residue is recomputed from the GEC and res64 compared as a check.