mersenneforum.org Parameter explorations for CADO 165-170 digits
 Register FAQ Search Today's Posts Mark Forums Read

 2021-05-20, 15:23 #34 VBCurtis     "Curtis" Feb 2005 Riverside, CA 37·137 Posts As you go up in relations, you will go up in TD too. I think if you do this for 190 vs 195 vs 200M rels you'll find out the interesting parts about how extra sieving -> higher TD -> smaller-er matrix -> possible saved time overall. If TD 110 fails to build a matrix, try 105? If you work from the top down, you won't need to try all the TDs for all the rels sizes- the idea is to locate the largest TD that works for each relation set, as that "should" save the most time. I don't think you can run multiple copies of filtering at the same time unless you're really careful about specifying all the output file names to be different for each copy; else they'll trample each other.
2021-05-20, 16:19   #35
VBCurtis

"Curtis"
Feb 2005
Riverside, CA

37×137 Posts

Quote:
 Originally Posted by bur BTW, I am just factoring a C147 with your experimental params.c145, should I keep the relations to do similar tests?
No, the files at C150 and below are pretty well tested and refined. It's C155+ that have few jobs run.

 2021-05-22, 06:50 #36 bur     Aug 2020 79*6581e-4;3*2539e-3 41910 Posts Unfortunately you were right about the simultaneous runs, I hardlinked the rels.dat file to different directories, as I thought it was only read from. But apparently the processes want write access to it, because they all quit at the first singleton removal with an error about writing access. I should build an HDD into that computer... But if working down from TD=110 on 190,195,200 is sufficient, then it's not too many tests anyway.
2021-05-22, 12:25   #37
charybdis

Apr 2020

54610 Posts

Quote:
 Originally Posted by bur Unfortunately you were right about the simultaneous runs, I hardlinked the rels.dat file to different directories, as I thought it was only read from. But apparently the processes want write access to it, because they all quit at the first singleton removal with an error about writing access. I should build an HDD into that computer...

 2021-05-23, 06:56 #38 bur     Aug 2020 79*6581e-4;3*2539e-3 1A316 Posts Here are the results, no major gains, but if using more than 10 cores it will be worth to use 195/105 instead of 190/95. On my 10 core i9 the sieving would take 3:30 h longer with 2:40 h saved at LA, so already at 20 cores it would help a bit. 200M rels / TD=110 Code: commencing 2-way merge reduce to 15092846 relation sets and 14808858 unique ideals commencing full merge memory use: 682.9 MB found 21197 cycles, need 3146938 too few cycles, matrix probably cannot build 200M rels / TD=105 (took 11:54 h for TD=90) Code: matrix is 7010462 x 7010687 (2900.8 MB) with weight 774175808 (110.43/col) sparse part has weight 690320583 (98.47/col) [...] linear algebra completed 70075 of 7010687 dimensions (1.0%, ETA 11h36m) 195M rels / TD=105 (took 13:25 h for TD=90) Code: matrix is 7353588 x 7353812 (3046.7 MB) with weight 812669648 (110.51/col) sparse part has weight 725127644 (98.61/col) [...] linear algebra completed 73595 of 7353812 dimensions (1.0%, ETA 12h49m] 190M rels / TD=100 Code: commencing 2-way merge reduce to 17035108 relation sets and 16791673 unique ideals commencing full merge memory use: 776.3 MB found 80952 cycles, need 3526279 too few cycles, matrix probably cannot build 190M rels / TD=95 (took 15:43 h for TD=90) Code: matrix is 8132721 x 8132946 (3102.7 MB) with weight 822289451 (101.11/col) sparse part has weight 732019314 (90.01/col) [...] linear algebra completed 81232 of 8132946 dimensions (1.0%, ETA 15h30m)
 2021-05-27, 07:00 #39 bur     Aug 2020 79*6581e-4;3*2539e-3 6438 Posts The c159 is in progress, found 65 % of the relations. One thing I notice compared to the c163 is that the number of relations per q decreases strongly. Already at q ~ 29e6 it's only 35,000 rels per 10,000 range, whereas with the c163 it still was 35,000 rels at q ~ 66e6. The c159 will end up at q around 43e6. So apparently that was a lucky polynomial with the c163. I am not sure if that changes anything regarding the parameters.
 2021-05-29, 06:48 #40 bur     Aug 2020 79*6581e-4;3*2539e-3 1A316 Posts The matrix was build at first try with 160,000,000 relations. Final q sieved: 45300000 matrix is 5597069 x 5597293 (2017.8 MB) with weight 529305051 (94.56/col) sparse part has weight 472974971 (84.50/col) Is this as expected or does it make sense to see if fewer relations would work? TD was again the msieve default of 90, while your params.c160 aims for 145. To be honest, I forgot to adjust the msieve parameters... Last fiddled with by bur on 2021-05-29 at 06:50
 2021-05-29, 16:30 #41 VBCurtis     "Curtis" Feb 2005 Riverside, CA 37·137 Posts 5.5M matrix is still a bit on the small side. I bet 155M relations would work fine; if you don't mind spending the time to check, I'd appreciate it. Note that CADO and msieve measure density in different ways; I subtract 50 from CADO density to get a rough msieve density, so your msieve default 90 isn't far from the density I guessed is best. Thanks for the stats report! Charybdis sent me a params trial and stats report on PM this week for a C165. These two reports provide ideas for a bit of tinkering with the settings. If you're going to do another 158-167 sized job, let me know and I'll send you a new file to try out. If we're going to try to refine these settings, we will need one more data point: The poly score as reported by *msieve* (or by the optimal skew calculator at http://myfactors.mooo.com/). That's because poly scores can vary by 5% for a given size of input, and we should take that into account when comparing runs.
2021-05-29, 18:37   #42
charybdis

Apr 2020

54610 Posts

Quote:
 Originally Posted by VBCurtis If we're going to try to refine these settings, we will need one more data point: The poly score as reported by *msieve* (or by the optimal skew calculator at http://myfactors.mooo.com/). That's because poly scores can vary by 5% for a given size of input, and we should take that into account when comparing runs.
Another interesting data point is the number of unique relations, as reported by msieve filtering, which is a more consistent indicator of when a matrix will build than the number of raw relations. Some polynomials seem to produce more duplicate relations than others, for reasons that I'm not really sure about.

 2021-06-10, 08:06 #43 bur     Aug 2020 79*6581e-4;3*2539e-3 419 Posts I missed your reply, I'll run tests on the minimum number of relations after that strange C165 SNFS factorization finishes. I entered the poly parameters into myfactor, but it came up with a different skew (3.3e6 instead of 2.6e6) than the one I used for the factorization and the scores were on the order of 10^-12 instead of 10^7. I found this surprising, is the score calculated that differently or were the myfactor skews so bad? How can I calculate scores by using msieve? Regarding unique relations, I'll record those for the c159 tests. I don't have that data for the c164, I could re-run the matrix tests on it, should I just do it for the lowest number of relations that worked?
2021-06-10, 12:17   #44
swellman

Jun 2012

2×1,621 Posts

Quote:
 Originally Posted by bur I entered the poly parameters into myfactor, but it came up with a different skew (3.3e6 instead of 2.6e6) than the one I used for the factorization and the scores were on the order of 10^-12 instead of 10^7. I found this surprising, is the score calculated that differently or were the myfactor skews so bad? How can I calculate scores by using msieve?
See this post. The method works quite well.

 Similar Threads Thread Thread Starter Forum Replies Last Post EdH CADO-NFS 127 2020-10-07 01:47 storm5510 Information & Answers 4 2019-11-30 21:32 ksteczk PrimeNet 6 2018-03-26 15:11 R.D. Silverman Cunningham Tables 14 2010-09-29 19:56 R.D. Silverman Cunningham Tables 11 2006-03-06 18:46

All times are UTC. The time now is 10:11.

Wed Dec 1 10:11:16 UTC 2021 up 131 days, 4:40, 1 user, load averages: 0.84, 1.17, 1.22