Experimental n=1.7M sieving

[The Carnivore]

Originally Posted by Happy5214

I got the program to compile with GCC 11 on Linux. I had to edit the code to change the starting n, as there was no parameter. At this time, it is not suitable for production use. I directed it to sieve n=[1700001, 1700064], k_max = 6e9 (1/100 of what Yves suggested in https://www.primegrid.com/forum_thre...ap=true#165303), to p_max = 1e10. It said it would only sieve to 2^20 when the program started, and after about 14 minutes, it only ended up sieving to p_max = 8100. It claimed to have "23950114 candidates" remaining, and I decided to kill the program rather than fill up my SSD with a huge temporary file. We can shelve this idea until Yves finishes a production-ready GPU version.

Speaking of benchmarks, is it just me, or does TwinGenX have bad scaling after a certain number of threads? There's a big speedup by going from 2 threads to 4 threads, a small speedup by going from 4 threads to 8 threads, and no speedup when going from 8 threads to 16 threads. This is on a 16-core Ryzen 9.

[storm5510]

Originally Posted by VBCurtis

Huh? ECM area for what? Are you talking about some project totally unrelated to this one, like GIMPS?

Yes, GIMPS. I failed to pick-up on this being related to FFT sizes. Sorry!

[gd_barnes]

Originally Posted by The Carnivore

So are we switching to n=1950000 and n=2580000 then? They're both big enough for the top 5000 list.

My opinion: Don't make choices for twin searches based on the top-5000 level. n=1.95M will be gone from top-5000 before you know it. n=2.58M will probably drop off in 2 years or so. Either search for twins or search for top-5000 primes but don't try to do both. You either have next to zero chance of twin or your top-5000 primes drop off too quickly.

n=1.7M seemed like a good choice based on where it feel on the FFT and no one had any notions that they would get top-5000 primes from it. I don't know why a change is being considered. If any change is done, I would suggest running the remainder of PrimeGrid's n=1.29M file when they shut their effort down. That would give a reasonable chance at a #1 twin. I'll go further and say that we should replace our port 13000 work for n=3.322M with the n=1.29M effort.

[The Carnivore]

Originally Posted by gd_barnes

My opinion: Don't make choices for twin searches based on the top-5000 level. n=1.95M will be gone from top-5000 before you know it. n=2.58M will probably drop off in 2 years or so. Either search for twins or search for top-5000 primes but don't try to do both. You either have next to zero chance of twin or your top-5000 primes drop off too quickly.

n=1.7M seemed like a good choice based on where it feel on the FFT and no one had any notions that they would get top-5000 primes from it. I don't know why a change is being considered. If any change is done, I would suggest running the remainder of PrimeGrid's n=1.29M file when they shut their effort down. That would give a reasonable chance at a #1 twin. I'll go further and say that we should replace our port 13000 work for n=3.322M with the n=1.29M effort.

Or how about this - replace the n=1.7M file with n=1.95M or n=2.58M in port 12000, but don't delete the 1.7M file. When the n=1.95M/2.58M primes fall off the top 5000 list, resume the n=1.7M search where we left off. Of course, a sieve file would first need to be generated.

[MooMoo2]

Originally Posted by The Carnivore

So are we switching to n=1950000 and n=2580000 then? They're both big enough for the top 5000 list.

n=2580000 is not being considered at this time since finding a twin or SG for that n will require over 10x as much work as n=1290000. Since only one twin and one SG was found in the ~10 years that n=1290000 was running, the odds of finding a n=2.58M twin or SG within the next 20 or so years are pretty slim. Besides, if you're going for moonshots, you might as well try n=3.322M, which is not that much higher.

For n=1950000, the graph and timings I posted do show an efficiency gain over n=1700000, but the effect is quite small. On a core i7 7700K, n=1.95M tests will take ~1115 seconds to complete, vs. ~947 seconds to complete for n=1.7M.

Originally Posted by gd_barnes

My opinion: Don't make choices for twin searches based on the top-5000 level. n=1.95M will be gone from top-5000 before you know it. n=2.58M will probably drop off in 2 years or so.

FWIW, the most recent graphs at: https://www.primegrid.com/forum_thread.php?id=10031 show that n=1950000 will be out of the top 5000 list in a little under a year. n=2.58M will probably drop off in around 6 years.

Or how about this - replace the n=1.7M file with n=1.95M or n=2.58M in port 12000, but don't delete the 1.7M file. When the n=1.95M/2.58M primes fall off the top 5000 list, resume the n=1.7M search where we left off. Of course, a sieve file would first need to be generated.

A replacement with n=2.58M won't happen. A temporary replacement with n=1.95M may be a reasonable option, but that'll depend on how quickly I can generate a file that isn't horribly undersieved.

[gd_barnes]

Originally Posted by The Carnivore

Or how about this - replace the n=1.7M file with n=1.95M or n=2.58M in port 12000, but don't delete the 1.7M file. When the n=1.95M/2.58M primes fall off the top 5000 list, resume the n=1.7M search where we left off. Of course, a sieve file would first need to be generated.

Why would we do that? How about this: Either do a twin search or do a top-5000 search but not both.

Why deviate from the n=1.7M search for top-5000 primes that will only be there for a few months? It takes n=1.833M to make the top-5000 now. What would be the point of n=1.95M? The bottom of the list would likely be near ~1.9M before we started. You may as well have stayed at n=1.7M at that point. And if you go to n=2.58M, you have next to zero chance of twin.

If anything, we could consider replacing the n=3.322M tests with 1.95M/2.58M. The idea being that this is a twin search, not a top-5000 search. Otherwise we are continuously chasing the top-5000 list.

[Happy5214]

PrimeGrid has already mentioned that a reboot of their Sophie project at a larger n is a possibility. If so, there is no point in doing work at these larger n's, as PrimeGrid would likely test (or surpass) them anyway. Chasing the top 5000 is pointless, as it would require constantly sieving new n's and ditching old ones as soon as they fall below the threshold (likely before the files are done).

This is the Twin Prime Search. We aim to find (large) twin primes, with Sophie Germain primes as a side effect. Any project decisions should be made with the goal of finding twin and Sophie Germain primes as the primary objective.

[Bottom Quark]

Originally Posted by MooMoo2

For n=1950000, the graph and timings I posted do show an efficiency gain over n=1700000, but the effect is quite small. On a core i7 7700K, n=1.95M tests will take ~1115 seconds to complete, vs. ~947 seconds to complete for n=1.7M.

Yeah, if you look at the next post in the thread that Carnivore linked to, the tiny efficiency gain isn't worth it: https://www.mersenneforum.org/showpo...46&postcount=9

"With a decently well-coded set of non-power-of-2 FFT-pass routines this effect should be on the 5%-penalty-or-less ballpark, when normalized by FFT length. If you see a timing effect much larger than that it's usually due to cache-mapping issues or quirks of the multithreading, both of which can be difficult to track down."

Finding a twin or Sophie at n=1.95M requires 55% more work than finding one at n=1.7M even after those efficiency gains are included: (1115/947)*(1.95/1.7)*(1.95/1.7) = 1.549

I'm in favor of sticking with n=1.7M. There's already n=3.322M at port 13000 for those who want a prime in the top 5000.

On a related note, will there be any additional sieving done on either n=1.7M or n=3.322M beyond the current p=1047T / p=1130T limits?

[Cybertronic]

Don't want to out myself now...
Has anyone ever had the idea to pick out an area after intiution where a twin could be, without systematically searching everything. I have the ability to do that. That can save years !

[Bottom Quark]

Originally Posted by Cybertronic

Has anyone ever had the idea to pick out an area after intiution where a twin could be, without systematically searching everything. I have the ability to do that. That can save years !

[MooMoo2]

Originally Posted by Bottom Quark

On a related note, will there be any additional sieving done on...

There is some additional sieving in the works