Bad sieve files found for 2020 and 2021 - Page 3

gd_barnes · 2022-11-22, 22:27

Quote:

Originally Posted by rebirther

R88 250-500k testing range complete, no primes found, 300-350k double check range matching all residues with the first run

If you have another important base to double check where a prime is expected but doesn't find anything let me know.

Great. Sounds good. Assumed to be no problems with R88. Thank you for the effort involved!

Future clarification for me and anyone else: The 250-500k testing range only included tests that were sieved out between P=100e12 and 500e12.

A big thanks to Luminescence for doing the sieve to 100e12 and providing factors!

We will be double-checking a good portion of the 2k bases from this year. I need to think about the best way to do that.

MisterBitcoin · 2022-12-23, 20:08

I lately used srsieve2 a lot, even versions prior v1.5. Now updated to the latest version.
Does anyone know which version yoyo is using for yoyo@home; I asked him just in case.

Edit: Quick reply: He still uses sr1sieve and sr2sieve. So we should be fine with the deeper sieved files; now it would depend on which software the sieve file was started I guess?

gd_barnes · 2022-12-23, 22:36

Quote:

Originally Posted by MisterBitcoin

I lately used srsieve2 a lot, even versions prior v1.5. Now updated to the latest version.
Does anyone know which version yoyo is using for yoyo@home; I asked him just in case.

Edit: Quick reply: He still uses sr1sieve and sr2sieve. So we should be fine with the deeper sieved files; now it would depend on which software the sieve file was started I guess?

Thanks for the info! That's good to know. I am having them re-sieve a few of the files that looked a little strange on the tests remaining for the sieve depth shown. I estimate they will be done in the next couple of weeks. At that point, we will begin a double-check for the lower half of the n-range done on most of the 2k bases. It is mostly the 2k bases where we should have had primes. Not having primes on the 1k bases was not so unusual. If we end up finding missing primes on the 2kers, we will likely have to expand the double-check.

MisterBitcoin · 2022-12-26, 16:12

Quote:

Originally Posted by rebirther

I hope this does not affect this old bug from srsieve2:

Code:

2.2.2 - October 22, 2021

   srsieve2, srsieve2cl:  version 1.5.3
      Modified to not remove terms that are prime as that defeats the purpose of
      Sierpinski/Riesel searches.

Hmmm. I just realised that I used srsieve2 1.3.1 for my R71 reservation. Up to n=5K I found a bit over 2.2K primes; sounds about fine by me.
Just for fun I will now run a sieve with 1.6.8 and compare both sieve files. If there was a bug, we should notice it in those lower n-ranges I think.

MisterBitcoin · 2022-12-26, 20:37

Quote:

Originally Posted by MisterBitcoin

Hmmm. I just realised that I used srsieve2 1.3.1 for my R71 reservation. Up to n=5K I found a bit over 2.2K primes; sounds about fine by me.
Just for fun I will now run a sieve with 1.6.8 and compare both sieve files. If there was a bug, we should notice it in those lower n-ranges I think.

Code:

2022-12-20 01:06:48: Sieve started: 3 < p < 5e7 with 22403873 terms (2500 < n < 5000, k*71^n+c) (expecting 21015458 factors)
2022-12-20 01:51:57: Sieve interrupted at p=32456407.  Primes tested 2000152.  Found 21045607 factors.  1358266 terms remaining.  Time 2708.77 seconds

2022-12-26 21:13:02: Sieve started: 3 < p < 32456407 with 22403873 terms (2500 < n < 5000, k*71^n-1) (expecting 20980769 factors)
2022-12-26 21:30:14: Sieve completed at p=32456441.  Primes tested 1999996.  Found 21045607 factors.  1358266 terms remaining.  Time 1030.92 seconds

Everything looks about fine for me

Luminescence · 2022-12-30, 01:36

I looked around a bit in the SRBase forum and found this thread:

https://srbase.my-firewall.org/sr5/f...ad.php?id=1368

and this part of a reply:

Quote:

Using zero-padded FMA3 FFT length 560K, Pass1=448, Pass2=1280, 5 threads, a = 3

As mentioned by George, two functions introduced with Gwnum 30.4 have problems with zero-padded FFTs.

I thought this kind of FFT is used if |c| != 1 but the post on the SRBase forum shows that this is not necessarily the case.

Any opinions on this?

rebirther · 2022-12-30, 08:09

Quote:

Originally Posted by Luminescence

I looked around a bit in the SRBase forum and found this thread:

https://srbase.my-firewall.org/sr5/f...ad.php?id=1368

and this part of a reply:

As mentioned by George, two functions introduced with Gwnum 30.4 have problems with zero-padded FFTs.

I thought this kind of FFT is used if |c| != 1 but the post on the SRBase forum shows that this is not necessarily the case.

Any opinions on this?

This was and old thread where we have used llr, in the meantime all llr2 apps were updated to the latest version with gwnum 3.10.

gd_barnes · 2023-02-04, 19:23

To all,

We have identified a problem with some sieve files that were prepared for the project in 2020 and 2021. Specifically they were only files that were sieved by Yoyo to P=5e14 between about Aug. 2020 and Sept. 2021, although it is possible that some could have happened a little before or after those dates.

There are no problems with Yoyo's process and there are no problems with any sieving software that we observed.

The problem is related to a scripting error on our end when removing Yoyo's factors from our sieve files. The error caused more tests to be removed than it should have.

All indications are that the testing that we have done are only on good tests. It's that many other good tests were missed. Therefore this alone does not necessitate a double-check. There will be additional testing that will need to be done.

We have identified 201 different sieve files that were received from Jan. 2020 and March 2022. Approximately 100 or a little more may have problems with them. We kept the date-range wide because we did not know exactly when the problem started and stopped. The date range includes research on 44 sieve files. Out of those 44 files, 24 were found to have problems (55%). Hence the ~100 problem file estimate.

It is the lack of primes on 2k bases from our goals list in 2022 done by SRBase that lead us to research our sieve files. There were 25 bases that they did. Out of those 25 bases, 13 were files sieved to 5e14. 6 of them had a problem (46%). Out of the other 12 bases that were sieved to something other than 5e14, none of them had a problem. We were also able to observe that the tests were only missing from one out of the 2 k's in the bases. It is this oddity of tests only being missing from 1 out of 2 k's that led us to the source of the problem.

TL;DR:
The scripting error caused all tests with duplicate n-values to be removed when there was more than one k with the same n-value (!!) This is regardless of the number of k's.
!TL;DR

Therefore 1k bases are not affected -and- the problem is the biggest in files with the most number of k's. Fortunately Yoyo doesn't do any sieving on bases with a large number of k's. The biggest one that we found was for R25 where 85 k's were sieved for n=150M-1M. It is this base that has the biggest problem of all since it has been tested for n=150K-300K. Other than that, we have not found any 5e14 sieves where Yoyo has worked on with more than 25 k's remaining.

Now for the action plan:
1. For all files sieved to 5e14, run a preliminary re-sieve to 1e9.
2. Using the output in step 1, run sr2sieve or srsieve2 to have it give an estimate of the number of factors for a sieve to 5e14.
3. If the number of estimated tests remaining at 5e14 is more than 1 st. dev. above that in the old file, continue sieving to 1e12.
4. Using the output in step 3, repeat step 2. If the number of estimated tests remaining at 5e14 is more than 1.5 st. devs. above that in the old file, send the file to Yoyo for re-sieving to 5e14.
5. Compare the old and new files sieved to 5e14 and pull out the tests that were not done.
6. Test the differences in step 5.

Note: 1 and 1.5 st. devs. is a somewhat low threshold for error. I think it's better to error on the side of caution. If it turns out there are no missing tests then nothing is lost. In my testing so far, most are blatantly obvious...some with 100 st. devs. or more. Further verification can be found by visually observing the old sieve file (or have Excel or another program do some checking on the file). If there are no duplicate n-values for different k's in the original file, there is a good chance that the file has a problem even if the sieve is only showing in this lower threshold of error.

Prioritizing the sieves: Since we expect likely over 100 different files to have problems, we have to prioritize what we do.

1. Current reservations. They have been identified and files have been sent to Yoyo for re-sieving. They are R111 (MisterBitcoin), S529 (carpetpool), and S536 (Luminescence). There are 2 other reservations with 5e14 sieve files where the file does not have a problem.

2. Recommended list and 2023 goals. They have been identified and files have been sent to Yoyo for re-sieving. The only bad ones have 2 k's remaining. They are R534 and R667.

3. Goals from last year. That is done for all of the bases with 2 k's remaining. We've done 5 of the 6 steps above on them including having them re-sieved by YoYo. Only step 6 for additional testing has not been done. This helped confirm that there was a problem and led us to expanding the search to all of our 1000+ bases. Goals with other than 2 k's remaining have not been looked into or identified yet.

4. Files with no testing yet done on them. Let's get those pulled out quickly.

5a. Files with some testing already done on them. That is a large majority of them. We want to get them corrected before we continue on them.

5b. Files fully tested where there is a sieve file available for a higher n-range. As in 5a, we want to make sure no primes were missed before anyone begins on a higher n-range.

(5a & 5b would have about the same priority.)

6. Files fully tested where no file is available for a higher n-range. This is the lowest priority because it isn't hurting anything currently but we need to eventually run the missing tests to make sure our historical record is accurate.

Research and preliminary sieving is ongoing on all of our files. I will be in contact with people who have affected reservations. If you are among those with reservations, you can continue testing your files. We will eventually have additional tests that we can send you that will need to be done -or- I/we can run those tests if the effort required is not large.

If anyone has any additional input, please feel free.

Gary

VBCurtis · 2023-02-04, 23:54

Quote:

Originally Posted by gd_barnes

No, sr2sieve is not multi-threaded.

I'm super late to notice this, but sr2sieve is definitely multi-threaded. The linux version runs up to -t 8. There is not a separate thread to manage the subthreads, and running multi-threaded loses a bit of efficiency: -t 8 might be 5-6x as fast as single-threaded.

The windows version is not multi-threaded that I know of.

gd_barnes · 2023-02-05, 01:28

Quote:

Originally Posted by VBCurtis

I'm super late to notice this, but sr2sieve is definitely multi-threaded. The linux version runs up to -t 8. There is not a separate thread to manage the subthreads, and running multi-threaded loses a bit of efficiency: -t 8 might be 5-6x as fast as single-threaded.

The windows version is not multi-threaded that I know of.

Hmmm. Interesting to know. It's been a long time since I sieved on one of my old Linux machines.

pepi37 · 2023-02-05, 15:02

Yes , under Linux srsieve2 is multi-threaded, Windows version never support MT ( it was left on TODO list)
But in any case one instance per core give bet output as VBCurtsi say

2022-12-23, 20:08	#24
MisterBitcoin "Nuri, the dragon :P" Jul 2016 Good old Germany 2³×3×37 Posts	I lately used srsieve2 a lot, even versions prior v1.5. Now updated to the latest version. Does anyone know which version yoyo is using for yoyo@home; I asked him just in case. Edit: Quick reply: He still uses sr1sieve and sr2sieve. So we should be fine with the deeper sieved files; now it would depend on which software the sieve file was started I guess? Last fiddled with by MisterBitcoin on 2022-12-23 at 20:12

2023-02-04, 19:23	#30
gd_barnes "Gary" May 2007 Overland Park, KS 2×19×317 Posts	To all, We have identified a problem with some sieve files that were prepared for the project in 2020 and 2021. Specifically they were only files that were sieved by Yoyo to P=5e14 between about Aug. 2020 and Sept. 2021, although it is possible that some could have happened a little before or after those dates. There are no problems with Yoyo's process and there are no problems with any sieving software that we observed. The problem is related to a scripting error on our end when removing Yoyo's factors from our sieve files. The error caused more tests to be removed than it should have. All indications are that the testing that we have done are only on good tests. It's that many other good tests were missed. Therefore this alone does not necessitate a double-check. There will be additional testing that will need to be done. We have identified 201 different sieve files that were received from Jan. 2020 and March 2022. Approximately 100 or a little more may have problems with them. We kept the date-range wide because we did not know exactly when the problem started and stopped. The date range includes research on 44 sieve files. Out of those 44 files, 24 were found to have problems (55%). Hence the ~100 problem file estimate. It is the lack of primes on 2k bases from our goals list in 2022 done by SRBase that lead us to research our sieve files. There were 25 bases that they did. Out of those 25 bases, 13 were files sieved to 5e14. 6 of them had a problem (46%). Out of the other 12 bases that were sieved to something other than 5e14, none of them had a problem. We were also able to observe that the tests were only missing from one out of the 2 k's in the bases. It is this oddity of tests only being missing from 1 out of 2 k's that led us to the source of the problem. TL;DR: The scripting error caused all tests with duplicate n-values to be removed when there was more than one k with the same n-value (!!) This is regardless of the number of k's. !TL;DR Therefore 1k bases are not affected -and- the problem is the biggest in files with the most number of k's. Fortunately Yoyo doesn't do any sieving on bases with a large number of k's. The biggest one that we found was for R25 where 85 k's were sieved for n=150M-1M. It is this base that has the biggest problem of all since it has been tested for n=150K-300K. Other than that, we have not found any 5e14 sieves where Yoyo has worked on with more than 25 k's remaining. Now for the action plan: 1. For all files sieved to 5e14, run a preliminary re-sieve to 1e9. 2. Using the output in step 1, run sr2sieve or srsieve2 to have it give an estimate of the number of factors for a sieve to 5e14. 3. If the number of estimated tests remaining at 5e14 is more than 1 st. dev. above that in the old file, continue sieving to 1e12. 4. Using the output in step 3, repeat step 2. If the number of estimated tests remaining at 5e14 is more than 1.5 st. devs. above that in the old file, send the file to Yoyo for re-sieving to 5e14. 5. Compare the old and new files sieved to 5e14 and pull out the tests that were not done. 6. Test the differences in step 5. Note: 1 and 1.5 st. devs. is a somewhat low threshold for error. I think it's better to error on the side of caution. If it turns out there are no missing tests then nothing is lost. In my testing so far, most are blatantly obvious...some with 100 st. devs. or more. Further verification can be found by visually observing the old sieve file (or have Excel or another program do some checking on the file). If there are no duplicate n-values for different k's in the original file, there is a good chance that the file has a problem even if the sieve is only showing in this lower threshold of error. Prioritizing the sieves: Since we expect likely over 100 different files to have problems, we have to prioritize what we do. 1. Current reservations. They have been identified and files have been sent to Yoyo for re-sieving. They are R111 (MisterBitcoin), S529 (carpetpool), and S536 (Luminescence). There are 2 other reservations with 5e14 sieve files where the file does not have a problem. 2. Recommended list and 2023 goals. They have been identified and files have been sent to Yoyo for re-sieving. The only bad ones have 2 k's remaining. They are R534 and R667. 3. Goals from last year. That is done for all of the bases with 2 k's remaining. We've done 5 of the 6 steps above on them including having them re-sieved by YoYo. Only step 6 for additional testing has not been done. This helped confirm that there was a problem and led us to expanding the search to all of our 1000+ bases. Goals with other than 2 k's remaining have not been looked into or identified yet. 4. Files with no testing yet done on them. Let's get those pulled out quickly. 5a. Files with some testing already done on them. That is a large majority of them. We want to get them corrected before we continue on them. 5b. Files fully tested where there is a sieve file available for a higher n-range. As in 5a, we want to make sure no primes were missed before anyone begins on a higher n-range. (5a & 5b would have about the same priority.) 6. Files fully tested where no file is available for a higher n-range. This is the lowest priority because it isn't hurting anything currently but we need to eventually run the missing tests to make sure our historical record is accurate. Research and preliminary sieving is ongoing on all of our files. I will be in contact with people who have affected reservations. If you are among those with reservations, you can continue testing your files. We will eventually have additional tests that we can send you that will need to be done -or- I/we can run those tests if the effort required is not large. If anyone has any additional input, please feel free. Gary Last fiddled with by gd_barnes on 2023-02-06 at 06:09

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
RSP Sieve Files for k*2^n-1 from PrimeGrid	pinhodecarlos	Riesel Prime Search	103	2022-11-26 15:02
Largest k*2^n-1 Primes Found In 2020	andyhedges	Riesel Prime Search	5	2021-01-02 00:09
I found a sieve to search all pairs of twin primes	Pietro Maiorana	Twin Prime Search	8	2019-09-26 23:07
Algebraic factors in sieve files	pepi37	Conjectures 'R Us	95	2017-07-04 13:37
program to verify factors found by sr(x)sieve?	mdettweiler	Software	16	2009-03-08 02:06

2023-02-05, 15:02	#33
pepi37 Dec 2011 After milion nines:) 11·149 Posts	Yes , under Linux srsieve2 is multi-threaded, Windows version never support MT ( it was left on TODO list) But in any case one instance per core give bet output as VBCurtsi say