Register FAQ Search Today's Posts Mark Forums Read

 2022-04-01, 19:47 #1 EdH     "Ed Hall" Dec 2009 Adirondack Mtns 468910 Posts CADO-NFS Data Harvesting This is mostly a question for VBCurtis: If I'm only running CADO-NFS through sieving, are there any bits of timing data vs. composite sizes you may be interested in? I won't be able to flag whether I'm using a params file modified by you or me, or an original, but I can probably harvest certain details contained in the log file, or even from the snapshot. This is the CADO-NFS portion of a normal run for my scripts: - CADO-NFS is called by a script and given a few standard items. The rest are supplied by the params files. - CADO-NFS performs the Polyselect and Lattice Sieving via server/clients. - A side script is started (depending on the composite being >125 digits) that examines the relations and performs Msieve filtering until a matrix can be built. - - Once successful, CADO-NFS is told to shut down. - - - The shutdown may occur anywhere from Lattice Sieving to LA. - If the composite is <125 digits, CADO-NFS completes the factorization. In light of the fact the process may or may not get to/through filtering, is there info that would be of value to gather?
 2022-04-01, 23:30 #2 VBCurtis     "Curtis" Feb 2005 Riverside, CA 22×32×149 Posts Yes, if you can connect the timing data to the params used for the job and to the composite size (including first digit). Also, small jobs are easy to collect data, and I think I'm done with params below 135-140 digits. So, maybe only for 150+ digit jobs? That data can be used to build a bit of a size - vs - sievetime curve, and any outliers mean "find better params for that size, please." If you'd like to do that, I'll be happy to review the data to see where I likely have suboptimal params.
 2022-04-02, 01:40 #3 EdH     "Ed Hall" Dec 2009 Adirondack Mtns 468910 Posts It's only for a c140, but how would this look for data: Code: N = 336... <140 digits> tasks.I = 14 tasks.lim0 = 8800000 tasks.lim1 = 14400000 tasks.lpb0 = 30 tasks.lpb1 = 31 tasks.qmin = 900000 tasks.filter.target_density = 130.0 tasks.filter.purge.keep = 180 tasks.polyselect.P = 182500 tasks.polyselect.admax = 146e3 tasks.polyselect.admin = 1800 tasks.polyselect.degree = 5 tasks.polyselect.incr = 120 tasks.polyselect.nq = 15625 tasks.polyselect.nrkeep = 66 tasks.polyselect.ropteffort = 16 tasks.sieve.lambda0 = 1.82 tasks.sieve.lambda1 = 1.81 tasks.sieve.mfb0 = 56 tasks.sieve.mfb1 = 58 tasks.sieve.ncurves0 = 18 tasks.sieve.ncurves1 = 24 tasks.sieve.qrange = 10000 Polynomial Selection (root optimized): Total time: 6113.74 Polynomial Selection (root optimized): Rootsieve time: 6112.81 Generate Factor Base: Total cpu/real time for makefb: 22.71/1.43919 Lattice Sieving: Total number of relations: 100246494 Lattice Sieving: Total time: 322777s Filtering - Duplicate Removal, splitting pass: CPU time for dup1: 245.5s Anything missing or not really of interest?
2022-04-02, 02:07   #4
charybdis

Apr 2020

11000110012 Posts

Quote:
 Originally Posted by EdH Code: Lattice Sieving: Total time: 322777s
Unhelpfully, while the total time for a completed job is given in wall-clock time and in thread-time, this value for the sieving step is neither of these: it's client-time, so should be about thread-time/2 unless you've changed threads-per-client from the default. Just something that needs to be kept in mind when making comparisons.

2022-04-02, 02:50   #5
EdH

"Ed Hall"
Dec 2009

32×521 Posts

Quote:
 Originally Posted by charybdis Unhelpfully, while the total time for a completed job is given in wall-clock time and in thread-time, this value for the sieving step is neither of these: it's client-time, so should be about thread-time/2 unless you've changed threads-per-client from the default. Just something that needs to be kept in mind when making comparisons.
I'm a tiny bit confused, but all my clients are based on four threads, except an occasional two thread machine that is very rarely engaged. I could make that rarely into never without issue, since it is also my GPU machine and I'm normally doing other stuff with it. Are you saying I should divide this value by 4? Or, should I just note on that line, 4 threads per client?

Also, would it be helpful if I translate that value into 89:39:37?

 2022-04-02, 06:00 #6 VBCurtis     "Curtis" Feb 2005 Riverside, CA 22·32·149 Posts A note of 4 threads per client is enough- I can double the time if I compare to my own machines, or just leave it as-is when comparing to other runs of yours. I do think you should have everything run 4-threaded so that the mix of 2-threaded clients and 4-threaded clients doesn't dirty the data. Polyselect params aren't really of interest, but poly score is of interest. Alas, Cado's score report is only comparable to other polys that use the exact-same siever & lim's, which is annoying. Poly select time is useful, as I am still not convinced I'm doing the right amount of poly select relative to sieve time! You may be using some older params files before I learned that the ratio of last Q sieved to starting-Q should be no more than 8. If you notice any final-Q that's much higher than 8x the starting-Q for that params file, boost starting-Q accordingly. I'm seeing best performance with this ratio around 6 for C140+ jobs, a bit higher ratio works fine for smaller jobs.
 2022-04-02, 15:12 #8 VBCurtis     "Curtis" Feb 2005 Riverside, CA 536410 Posts If there's a cownoise score, the actual poly has no use to the data summary. I agree that 1-2 two-threaded instances won't color the data from a 50+ client farm!
 2022-04-02, 15:34 #9 EdH     "Ed Hall" Dec 2009 Adirondack Mtns 10010010100012 Posts Here's another run against the c140: Code: N = 336... <140 digits> tasks.I = 14 tasks.lim0 = 8800000 tasks.lim1 = 14400000 tasks.lpb0 = 30 tasks.lpb1 = 31 tasks.qmin = 900000 tasks.filter.target_density = 130.0 tasks.filter.purge.keep = 180 tasks.sieve.lambda0 = 1.82 tasks.sieve.lambda1 = 1.81 tasks.sieve.mfb0 = 56 tasks.sieve.mfb1 = 58 tasks.sieve.ncurves0 = 18 tasks.sieve.ncurves1 = 24 tasks.sieve.qrange = 10000 Polynomial Selection (root optimized): Total time: 6113.74 Lattice Sieving: Total number of relations: 100246494 Lattice Sieving: Total time: 322777s (all clients used 4 threads) Found 55764577 unique, 23084551 duplicate, and 0 bad relations. cownoise Best MurphyE for polynomial is 2.51691527e-11 The discrepancy in relations counts is due to the filtering tests for Msieve running while CADO-NFS is still sieving. Would a ratio of duplication be a less confusing value? Or, would you rather be able to look at the actual numbers as shown?
2022-04-02, 15:54   #10
charybdis

Apr 2020

31916 Posts

Quote:
 Originally Posted by EdH Is there interest in the separate Rootsieve time?
Surely there's not much point in reporting it unless you report the time for the rest of polyselect too?

2022-04-02, 16:20   #11
EdH

"Ed Hall"
Dec 2009

125116 Posts

Quote:
 Originally Posted by charybdis Surely there's not much point in reporting it unless you report the time for the rest of polyselect too?
Found it! Is this better?:
Code:
N = 336... <140 digits>
Polynomial Selection (size optimized): Total time: 29411.9
Polynomial Selection (root optimized): Total time: 6113.74
Lattice Sieving: Total number of relations: 100246494
Lattice Sieving: Total time: 322777s (all clients used 4 threads)
Found 55764577 unique, 23084551 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 2.51691527e-11

 Similar Threads Thread Thread Starter Forum Replies Last Post Shaopu Lin CADO-NFS 522 2021-05-04 18:28 EdH EdH 8 2019-05-20 15:07 henryzz CADO-NFS 4 2017-11-20 15:14 skan Information & Answers 1 2013-10-22 07:00 R.D. Silverman Factoring 4 2008-11-06 12:35

All times are UTC. The time now is 02:30.

Mon Aug 8 02:30:57 UTC 2022 up 31 days, 21:18, 1 user, load averages: 1.36, 1.32, 1.26