mersenneforum.org CRUS sieving questions
 User Name Remember Me? Password
 Register FAQ Search Today's Posts Mark Forums Read

2022-12-09, 01:41   #23
chalsall
If I May

"Chris Halsall"
Sep 2002

22·5·7·79 Posts

Quote:
 Originally Posted by storm5510 So, I begin, again. Live and learn...
It happens. From time to time. It is how we learn... 8^)

 2022-12-09, 02:15 #24 storm5510 Random Account     Aug 2009 Not U. + S.A. 23×32×5×7 Posts So, here is what I have set up: Base 60 Machine 1 runs -p 3 to -P 4e11 Machine 2 runs -p 4e11 to -P 650e9 Machine 3 runs -p 650e9 to -P 135e10 They all finish within an hour of each other. Once I got the first two to finish relatively close. I had to adjust the -P on the GPU machine up so it would match the first two. They finish in about 3 hours. Caveat: As the k's grow, the time will increase. Each will produce a .abcd file with slightly different names. I can then merge them together into a single file. Once I conform this setup works, then I can start "stacking" the k's in a single batch file. This will require more batch file editing, but they can run far longer without interaction from me. I hope this is what everyone has in mind. Edit: The below is what I referred to as "stacking:" Code: srsieve2cl -n 100e3 -N 250e3 -p 650e9 -P 135e10 -g 16 -M 9000 -s "36*60^n-1" srsieve2cl -n 100e3 -N 250e3 -p 650e9 -P 135e10 -g 16 -M 9000 -s "1700*60^n-1" srsieve2cl -n 100e3 -N 250e3 -p 650e9 -P 135e10 -g 16 -M 9000 -s "4708*60^n-1" srsieve2cl -n 100e3 -N 250e3 -p 650e9 -P 135e10 -g 16 -M 9000 -s "5317*60^n-1" srsieve2cl -n 100e3 -N 250e3 -p 650e9 -P 135e10 -g 16 -M 9000 -s "5611*60^n-1" Last fiddled with by storm5510 on 2022-12-09 at 02:36 Reason: Additional
2022-12-09, 02:54   #25
rogue

"Mark"
Apr 2003
Between here and the

2·72·71 Posts

Quote:
 Originally Posted by storm5510 So, here is what I have set up: Base 60 Machine 1 runs -p 3 to -P 4e11 Machine 2 runs -p 4e11 to -P 650e9 Machine 3 runs -p 650e9 to -P 135e10 They all finish within an hour of each other. Once I got the first two to finish relatively close. I had to adjust the -P on the GPU machine up so it would match the first two. They finish in about 3 hours. Caveat: As the k's grow, the time will increase. Each will produce a .abcd file with slightly different names. I can then merge them together into a single file. Once I conform this setup works, then I can start "stacking" the k's in a single batch file. This will require more batch file editing, but they can run far longer without interaction from me. I hope this is what everyone has in mind.
Yes, something like this:

srsieve2cl -n 100e3 -N 250e3 -P 135e10 -g 16 -M 9000 -s "36*60^n-1" -s "1700*60^n-1" -s"4708*60^n-1" -s "5317*60^n-1" -s "5611*60^n-1"

or

srsieve2cl -n 100e3 -N 250e3 -P 135e10 -g 16 -M 9000 -s b60.in

where b60.in is a file where each line is a separate sequence for the same base.

If you are going to sieve the same base across multiple computers, pre-sieve to 1e9 to eliminate 90% of the factors. Use that output file as input to each machine so you get

srsieve2 -p1e9 -P4e11 -i b60_n.abcd -O f1.txt
srsieve2 -p4e11 -P8e11 -i b60_n.abcd -O f2.txt
srsieve2 -p8e11 -P12e11 -i b60_n.abcd -O f3.txt

When all of those are done:

srsieve2 -A -i b60_n.abcd -I f1.txt
srsieve2 -A -i b60_n.abcd -I f2.txt
srsieve2 -A -i b60_n.abcd -I f3.txt

srsieve2 and srsieve2cl are interchangeable, but if the machines are different, speed wise, then the range of p will differ.

My recommendation is to run different bases on different machines so that you don't need to piece together factor files. I think you will be surprised at how quickly you can sieve to 1e12 or 1e13 when you have dozens or hundreds of sequences on a GPU. I could sieve over 2000 sequences to 1e12 in less than 3 days on a GPU.

2022-12-09, 04:58   #26
gd_barnes

"Gary"
May 2007
Overland Park, KS

13×907 Posts

Quote:
 Originally Posted by storm5510 So, here is what I have set up: Base 60 Machine 1 runs -p 3 to -P 4e11 Machine 2 runs -p 4e11 to -P 650e9 Machine 3 runs -p 650e9 to -P 135e10 They all finish within an hour of each other. Once I got the first two to finish relatively close. I had to adjust the -P on the GPU machine up so it would match the first two. They finish in about 3 hours. Caveat: As the k's grow, the time will increase. Each will produce a .abcd file with slightly different names. I can then merge them together into a single file. Once I conform this setup works, then I can start "stacking" the k's in a single batch file. This will require more batch file editing, but they can run far longer without interaction from me. I hope this is what everyone has in mind. Edit: The below is what I referred to as "stacking:" Code: srsieve2cl -n 100e3 -N 250e3 -p 650e9 -P 135e10 -g 16 -M 9000 -s "36*60^n-1" srsieve2cl -n 100e3 -N 250e3 -p 650e9 -P 135e10 -g 16 -M 9000 -s "1700*60^n-1" srsieve2cl -n 100e3 -N 250e3 -p 650e9 -P 135e10 -g 16 -M 9000 -s "4708*60^n-1" srsieve2cl -n 100e3 -N 250e3 -p 650e9 -P 135e10 -g 16 -M 9000 -s "5317*60^n-1" srsieve2cl -n 100e3 -N 250e3 -p 650e9 -P 135e10 -g 16 -M 9000 -s "5611*60^n-1"
No. It is not what we have in mind. Without using the -O command, the first part of it will not even work properly. You are not creating factor files but instead sieve files. How will you be able to "merge" together sieve files for the same base that have different tests remaining? Merging them won't logically work. You have to create factor files and merge those together and then use those to remove terms from the sieve file.

Why do you insist on running one k at a time as you show in the last part of your process? We don't want to continue helping you if you insist on running one k at a time no matter how many batch jobs you set up. Please trust us that the below is the way to do this.

I will create an input file for you and spell it out as best as I can.

1. Using the attached file of k's called R60-remain.txt, run the following command on machine 1:
srsieve2cl -n 100e3 -N 250e3 -P 1e9 -g 16 -M 9000 -s R60-remain.txt

This will run quickly and is a precursor to running the big jobs next. It will create an output file. I suggest renaming it to something relevant. Let's call it R60-sieve.txt

The following assumes that you have 3 equal machines. You can tweak the sieving ranges to make them complete at about the same time.

2. Using the examples that Mark gave, run the following command on machine 1:
srsieve2cl -p1e9 -P4e11 -i R60-sieve.txt -O factors1.txt
(tweak as needed for the relevant number of cores/threads)

3. Run the following command on machine 2:
srsieve2cl -p4e11 -P8e11 -i R60-sieve.txt -O factors2.txt

4. Run the following command on machine 3:
srsieve2cl -p8e11 -P12e11 -i R60-sieve.txt -O factors3.txt

5. You now have 3 factor files factors1.txt, factors2.txt, and factors3.txt. Merge them all together. I just copy and paste each one to the end of the first one. To check yourself, add up how many total lines are in all of the files. Then after the copy and paste, make sure your final file has that many lines. You can also use a DOS command (or there's probably a myriad of other ways) to merge all of the files together. Rename that merged file as factors-all.txt

6. Run the following command:
srsieve2cl -A -i R60-sieve-txt -I factors-all.txt

This will create a new output file with all of the factors removed.

7. Rename the output file from #6 to something relevant again like R60-sieveb.txt

8. Correct the sieve depth in R60-sieveb.txt to show 120000000000 since you sieved to 12e11.

9. Run an LLR test at 60% of the n-range using approximately the median k-value in the file (60% of n=100000 to 250000 = 190000 and the median k is 12061, so use 12061*60^190000-1 in this case) to see how long a test takes.

10. Begin a short trial sieve using R60-sieveb.txt to see what the removal rate is. If you are removing factors at twice the rate of an LLR test, then you need to sieve to slightly less than double the current sieve depth. Note that this is not exact math-wise but it gets you close. If you are way off, say you are removing factors 10 times faster than the LLR test, don't automatically sieve 10 times as far. Do something less than that, say 5-7 times as far, and then see what the removal rate is again. Nothing is ever quite a straight-line when it comes to a removal rate.

11. Delete all extraneous factor or sieve files except for R60-sieveb.txt

12. Repeat steps 2 thru 11 as many times as necessary until you've reached approximately the correct optimum sieve depth. On the 2nd time through, use the new file R60-sieveb.txt. Each time you go through it, continue using the final file from step 8 as input to the new step 2.

Have you looked at some of the documentation that comes with the sieving programs, such as README? If not, that can be very helpful in understanding the difference between factor files and sieve files and how to merge and remove one from the other.
Attached Files
 R60-remain.txt (446 Bytes, 10 views)

Last fiddled with by gd_barnes on 2022-12-09 at 05:13

 2022-12-09, 05:15 #27 storm5510 Random Account     Aug 2009 Not U. + S.A. 252010 Posts @gd_barnes OK, all of this is sailing over my head so it would probably be best to just drop the assignment. Please!
2022-12-09, 05:32   #28
gd_barnes

"Gary"
May 2007
Overland Park, KS

13×907 Posts

Quote:
 Originally Posted by storm5510 @gd_barnes OK, all of this is sailing over my head so it would probably be best to just drop the assignment. Please!
OK well...sorry to hear that.

What part of it is confusing you? Have you done factor removal from sieve files before? Once you've done it a couple of times, it becomes a lot easier.

On the optimum sieve depth part, if you want to skip that and just provide us with a preliminary sieve file, say, sieved to 1e12 or 5e12 or 10e12, then that would be OK too. Others can then pick up the sieve file, determine optimum sieve depth, and finish it off.

2022-12-09, 05:51   #29
storm5510
Random Account

Aug 2009
Not U. + S.A.

23·32·5·7 Posts

Quote:
 Originally Posted by gd_barnes OK well...sorry to hear that. What part of it is confusing you? Have you done factor removal from sieve files before? Once you've done it a couple of times, it becomes a lot easier. On the optimum sieve depth part, if you want to skip that and just provide us with a preliminary sieve file, say, sieved to 1e12 or 5e12 or 10e12, then that would be OK too. Others can then pick up the sieve file, determine optimum sieve depth, and finish it off.
Scratch my request! It was a decision made in haste because of being really tired.

What I need to do is study your last instructional post in detail, follow it, and see what I can come up with.

I had an issue with srsieve2cl which I have submitted for rogue to look into. I found a condition where it refused to start and displayed an error message.

Now, sleep!

 2022-12-09, 07:48 #30 gd_barnes     "Gary" May 2007 Overland Park, KS 1179110 Posts The instructions that I gave are essentially a turn-key operation of a complete sieving process using multiple machines. Doing all of that, the file would be completely ready for LLR testing in the most efficient way possible, both from a personal time and CPU time perspective. Mark had suggested that you might find it easier to run one base on only one machine. That is true. But I had the impression that you wanted to use multiple machines for a single base. If you want to skip the optimum sieve calculations, that is OK. I can guarantee that the optimum sieve depth is > 10e12, likely > 20e12. srsieve2cl is amazingly fast for multiple k's on multiple cores so it could be higher than that but I cannot say for sure without setting it up myself. To make it worth your while, you should at least sieve to 1e12. If you can sieve to 10e12, that would be great. That's the maximum I'd suggest that you do without computing an optimum sieve depth. There is an alternative that you might find easier that is only a little less efficient CPU-wise. Split that k-file up that I posted (R60-remain.txt) into 3 files, 1 for each machine, of 11 k's each or whatever ratio works since you have machines of varying capabilities. You still get the efficiency of multiple k's in each sieve without having to copy factor and sieve files across multiple machines, albeit somewhat less efficient. Doing this, you'd have 3 separate sieves. You'd have separate factor files to remove from the separate sieve files each time they finish. That might make it a little easier to wrap your head around. Then when all 3 sieves are done sieving to the same depth, you can use srfile to combine them into one big sieve file to send to us.
 2022-12-09, 16:55 #31 storm5510 Random Account     Aug 2009 Not U. + S.A. 1001110110002 Posts I was not able to sleep last night so I got back up to work on this. I was up until 5 AM EST. I took a lot from the post rogue made. I created a list as he mentioned and called it "input.txt" Code: 36*60^n-1 1700*60^n-1 4708*60^n-1 5317*60^n-1 5611*60^n-1 6101*60^n-1 6162*60^n-1 6274*60^n-1 7060*60^n-1 7870*60^n-1 I developed my batch process as I read along, one step at a time. and testing each line individually. Below is the batch from Machine 3: 1 and 2 as different as they are not running a GPU version. The basics are the same. Code: @echo off cls srsieve2cl -n 100e3 -N 250e3 -p 1e12 -P 2e12 -g 16 -M 9000 -s input.txt srsieve2cl -p 1e12 -P 2e12 -g 16 -M 9000 -i b60_n.abcd -O f3.txt srsieve2cl -A -g 16 -M 9000 -i b60_n.abcd -I f3.txt I wrote these in such a way to make them reusable by simply changing the input file. Below are the "p" settings for each. Code: Machine 1: -p 3 -P 45e10. Machine 2: -p 45e10 to -P 1e12. Machine 3: -p 1e12 to 2e12. All three should finish in a 3-hour window tomorrow.
 2022-12-09, 17:40 #32 rogue     "Mark" Apr 2003 Between here and the 2×72×71 Posts Once you do the initial sieving to 1e9, then the first instance will start with 1e9. All three instances will the output from the the initial sieving as their input.
2022-12-09, 18:10   #33
storm5510
Random Account

Aug 2009
Not U. + S.A.

47308 Posts

Quote:
 Originally Posted by rogue Once you do the initial sieving to 1e9, then the first instance will start with 1e9. All three instances will the output from the the initial sieving as their input.
You lose me here. I am not doing any sieving which stops at 1e9. My first batch is from 3 to 45e10.

 Similar Threads Thread Thread Starter Forum Replies Last Post rebirther Conjectures 'R Us 826 2023-01-25 15:49 The Carnivore Twin Prime Search 13 2019-07-06 03:11 gd_barnes Conjectures 'R Us 143 2014-10-21 23:55 nstaab1 Lounge 15 2013-03-06 13:48 CRGreathouse Puzzles 24 2011-10-28 18:30

All times are UTC. The time now is 10:48.

Fri Jan 27 10:48:29 UTC 2023 up 162 days, 8:17, 0 users, load averages: 1.15, 0.96, 0.98

Copyright ©2000 - 2023, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.

≠ ± ∓ ÷ × · − √ ‰ ⊗ ⊕ ⊖ ⊘ ⊙ ≤ ≥ ≦ ≧ ≨ ≩ ≺ ≻ ≼ ≽ ⊏ ⊐ ⊑ ⊒ ² ³ °
∠ ∟ ° ≅ ~ ‖ ⟂ ⫛
≡ ≜ ≈ ∝ ∞ ≪ ≫ ⌊⌋ ⌈⌉ ∘ ∏ ∐ ∑ ∧ ∨ ∩ ∪ ⨀ ⊕ ⊗ 𝖕 𝖖 𝖗 ⊲ ⊳
∅ ∖ ∁ ↦ ↣ ∩ ∪ ⊆ ⊂ ⊄ ⊊ ⊇ ⊃ ⊅ ⊋ ⊖ ∈ ∉ ∋ ∌ ℕ ℤ ℚ ℝ ℂ ℵ ℶ ℷ ℸ 𝓟
¬ ∨ ∧ ⊕ → ← ⇒ ⇐ ⇔ ∀ ∃ ∄ ∴ ∵ ⊤ ⊥ ⊢ ⊨ ⫤ ⊣ … ⋯ ⋮ ⋰ ⋱
∫ ∬ ∭ ∮ ∯ ∰ ∇ ∆ δ ∂ ℱ ℒ ℓ
𝛢𝛼 𝛣𝛽 𝛤𝛾 𝛥𝛿 𝛦𝜀𝜖 𝛧𝜁 𝛨𝜂 𝛩𝜃𝜗 𝛪𝜄 𝛫𝜅 𝛬𝜆 𝛭𝜇 𝛮𝜈 𝛯𝜉 𝛰𝜊 𝛱𝜋 𝛲𝜌 𝛴𝜎𝜍 𝛵𝜏 𝛶𝜐 𝛷𝜙𝜑 𝛸𝜒 𝛹𝜓 𝛺𝜔