mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > CADO-NFS

Reply
 
Thread Tools
Old 2021-05-20, 15:23   #34
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

37·137 Posts
Default

As you go up in relations, you will go up in TD too.

I think if you do this for 190 vs 195 vs 200M rels you'll find out the interesting parts about how extra sieving -> higher TD -> smaller-er matrix -> possible saved time overall. If TD 110 fails to build a matrix, try 105? If you work from the top down, you won't need to try all the TDs for all the rels sizes- the idea is to locate the largest TD that works for each relation set, as that "should" save the most time.

I don't think you can run multiple copies of filtering at the same time unless you're really careful about specifying all the output file names to be different for each copy; else they'll trample each other.
VBCurtis is offline   Reply With Quote
Old 2021-05-20, 16:19   #35
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

37×137 Posts
Default

Quote:
Originally Posted by bur View Post
BTW, I am just factoring a C147 with your experimental params.c145, should I keep the relations to do similar tests?
No, the files at C150 and below are pretty well tested and refined. It's C155+ that have few jobs run.
VBCurtis is offline   Reply With Quote
Old 2021-05-22, 06:50   #36
bur
 
bur's Avatar
 
Aug 2020
79*6581e-4;3*2539e-3

41910 Posts
Default

Unfortunately you were right about the simultaneous runs, I hardlinked the rels.dat file to different directories, as I thought it was only read from. But apparently the processes want write access to it, because they all quit at the first singleton removal with an error about writing access. I should build an HDD into that computer...

But if working down from TD=110 on 190,195,200 is sufficient, then it's not too many tests anyway.
bur is offline   Reply With Quote
Old 2021-05-22, 12:25   #37
charybdis
 
charybdis's Avatar
 
Apr 2020

54610 Posts
Default

Quote:
Originally Posted by bur View Post
Unfortunately you were right about the simultaneous runs, I hardlinked the rels.dat file to different directories, as I thought it was only read from. But apparently the processes want write access to it, because they all quit at the first singleton removal with an error about writing access. I should build an HDD into that computer...
Yes - msieve needs write access to add free relations.
charybdis is offline   Reply With Quote
Old 2021-05-23, 06:56   #38
bur
 
bur's Avatar
 
Aug 2020
79*6581e-4;3*2539e-3

1A316 Posts
Default

Here are the results, no major gains, but if using more than 10 cores it will be worth to use 195/105 instead of 190/95. On my 10 core i9 the sieving would take 3:30 h longer with 2:40 h saved at LA, so already at 20 cores it would help a bit.


200M rels / TD=110
Code:
commencing 2-way merge
reduce to 15092846 relation sets and 14808858 unique ideals
commencing full merge
memory use: 682.9 MB
found 21197 cycles, need 3146938
 too few cycles, matrix probably cannot build
200M rels / TD=105 (took 11:54 h for TD=90)
Code:
matrix is 7010462 x 7010687 (2900.8 MB) with weight 774175808 (110.43/col)
sparse part has weight 690320583 (98.47/col)
[...]
 linear algebra completed 70075 of 7010687 dimensions (1.0%, ETA 11h36m)
195M rels / TD=105 (took 13:25 h for TD=90)
Code:
matrix is 7353588 x 7353812 (3046.7 MB) with weight 812669648 (110.51/col)
sparse part has weight 725127644 (98.61/col)
[...]
 linear algebra completed 73595 of 7353812 dimensions (1.0%, ETA 12h49m]
190M rels / TD=100
Code:
commencing 2-way merge
reduce to 17035108 relation sets and 16791673 unique ideals
commencing full merge
memory use: 776.3 MB
found 80952 cycles, need 3526279
too few cycles, matrix probably cannot build
190M rels / TD=95 (took 15:43 h for TD=90)
Code:
matrix is 8132721 x 8132946 (3102.7 MB) with weight 822289451 (101.11/col)
sparse part has weight 732019314 (90.01/col)
[...]
linear algebra completed 81232 of 8132946 dimensions (1.0%, ETA 15h30m)
bur is offline   Reply With Quote
Old 2021-05-27, 07:00   #39
bur
 
bur's Avatar
 
Aug 2020
79*6581e-4;3*2539e-3

6438 Posts
Default

The c159 is in progress, found 65 % of the relations. One thing I notice compared to the c163 is that the number of relations per q decreases strongly. Already at q ~ 29e6 it's only 35,000 rels per 10,000 range, whereas with the c163 it still was 35,000 rels at q ~ 66e6. The c159 will end up at q around 43e6.

So apparently that was a lucky polynomial with the c163. I am not sure if that changes anything regarding the parameters.
bur is offline   Reply With Quote
Old 2021-05-29, 06:48   #40
bur
 
bur's Avatar
 
Aug 2020
79*6581e-4;3*2539e-3

1A316 Posts
Default

The matrix was build at first try with 160,000,000 relations.

Final q sieved: 45300000

matrix is 5597069 x 5597293 (2017.8 MB) with weight 529305051 (94.56/col)
sparse part has weight 472974971 (84.50/col)

Is this as expected or does it make sense to see if fewer relations would work? TD was again the msieve default of 90, while your params.c160 aims for 145. To be honest, I forgot to adjust the msieve parameters...

Last fiddled with by bur on 2021-05-29 at 06:50
bur is offline   Reply With Quote
Old 2021-05-29, 16:30   #41
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

37·137 Posts
Default

5.5M matrix is still a bit on the small side. I bet 155M relations would work fine; if you don't mind spending the time to check, I'd appreciate it.

Note that CADO and msieve measure density in different ways; I subtract 50 from CADO density to get a rough msieve density, so your msieve default 90 isn't far from the density I guessed is best.

Thanks for the stats report!

Charybdis sent me a params trial and stats report on PM this week for a C165. These two reports provide ideas for a bit of tinkering with the settings. If you're going to do another 158-167 sized job, let me know and I'll send you a new file to try out.

If we're going to try to refine these settings, we will need one more data point: The poly score as reported by *msieve* (or by the optimal skew calculator at http://myfactors.mooo.com/). That's because poly scores can vary by 5% for a given size of input, and we should take that into account when comparing runs.
VBCurtis is offline   Reply With Quote
Old 2021-05-29, 18:37   #42
charybdis
 
charybdis's Avatar
 
Apr 2020

54610 Posts
Default

Quote:
Originally Posted by VBCurtis View Post
If we're going to try to refine these settings, we will need one more data point: The poly score as reported by *msieve* (or by the optimal skew calculator at http://myfactors.mooo.com/). That's because poly scores can vary by 5% for a given size of input, and we should take that into account when comparing runs.
Another interesting data point is the number of unique relations, as reported by msieve filtering, which is a more consistent indicator of when a matrix will build than the number of raw relations. Some polynomials seem to produce more duplicate relations than others, for reasons that I'm not really sure about.
charybdis is offline   Reply With Quote
Old 2021-06-10, 08:06   #43
bur
 
bur's Avatar
 
Aug 2020
79*6581e-4;3*2539e-3

419 Posts
Default

I missed your reply, I'll run tests on the minimum number of relations after that strange C165 SNFS factorization finishes.

I entered the poly parameters into myfactor, but it came up with a different skew (3.3e6 instead of 2.6e6) than the one I used for the factorization and the scores were on the order of 10^-12 instead of 10^7. I found this surprising, is the score calculated that differently or were the myfactor skews so bad? How can I calculate scores by using msieve?


Regarding unique relations, I'll record those for the c159 tests. I don't have that data for the c164, I could re-run the matrix tests on it, should I just do it for the lowest number of relations that worked?
bur is offline   Reply With Quote
Old 2021-06-10, 12:17   #44
swellman
 
swellman's Avatar
 
Jun 2012

2×1,621 Posts
Default

Quote:
Originally Posted by bur View Post
I entered the poly parameters into myfactor, but it came up with a different skew (3.3e6 instead of 2.6e6) than the one I used for the factorization and the scores were on the order of 10^-12 instead of 10^7. I found this surprising, is the score calculated that differently or were the myfactor skews so bad? How can I calculate scores by using msieve?
See this post. The method works quite well.
swellman is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Some CADO-NFS Work At Around 175-180 Decimal Digits EdH CADO-NFS 127 2020-10-07 01:47
Sigma parameter in ecm storm5510 Information & Answers 4 2019-11-30 21:32
PrimeNet error 7: Invalid parameter ksteczk PrimeNet 6 2018-03-26 15:11
Parameter Underestimation R.D. Silverman Cunningham Tables 14 2010-09-29 19:56
ECM Work and Parameter Choices R.D. Silverman Cunningham Tables 11 2006-03-06 18:46

All times are UTC. The time now is 10:11.


Wed Dec 1 10:11:16 UTC 2021 up 131 days, 4:40, 1 user, load averages: 0.84, 1.17, 1.22

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.