CADO NFS

[EdH]

Preliminary results do not look very favorable. Using scripts to run msieve and ggnfs, it took my conglomeration of machines 14:43:33 to complete the factorization of the C150 described above, if my time math is correct. ( 10:53:46 to 01:45:00 )

I started the CADO-NFS run today at 08:39:34. Currently, the LA (krylov) stage is estimating 20:19:55 tomorrow and with each print line the ETA is drifting later, sometimes adding more than 10 minutes.

I'm not sure how well the parameters can be tweaked, but it would need to be a pretty huge change to be anywhere toward the msieve/ggnfs package.

The only machine running now is the server doing LA. A quick check shows it's only using 5 of the 16 available GB, so memory shouldn't be an issue ATM. But, I don't remember the time comparisons for the rest of the steps after krylov, or how the memory use may change.

If the time holds, it's going to be way more than double that of the other setup.

[VBCurtis]

My C130 test with improved parameters lists 62k thread-seconds for LA, while C140 lists 126K thread-seconds (2.3ghz dual-quad-xeon, core2 era). If C150 with good parameters doubles time again, 260K thread-seconds rounded generously for newer hardware would be 12ish hours on a quad-core for just the LA. That definitely does not compare favorably with msieve/ggnfs!

I was really hoping overall package performance would converge in the 150s or 160s...

[Dubslow]

But isn't CADO's polyselect+siever better though? Why can't we combine the best of each?

[EdH]

Originally Posted by Dubslow

But isn't CADO's polyselect+siever better though? Why can't we combine the best of each?

I was actually thinking this morning of trying to see if I can copy the relations over and use msieve for LA on a different machine. In README.msieve, this is one of the sections. However, I don't want to "distract" the server from what it is doing.

Now that I'm able to distribute the msieve poly selection among my machines, I think msieve/ggnfs is the way to go for a LAN based system. CADO_NFS would still have appeal if I was to add outside machines.

If further details are of interest:

170 cores of various flavors were used for the msieve/ggnfs run. A weak dual core laptop failed to run CADO-NFS. This should only be a very minor loss.

The msieve/ggnfs run started poly selection at 10:53:46.
Sieving started at 11:01:27.
Matrix solving began at 19:01:45
LA began at 19:01:46.
Square root started at 00:57:58
The three factors were logged at 01:45:00

The advantage I had originally considered for CADO-NFS was (and still is to some degree), the ease of setting it up and the distributed poly select. But, now I have the scripts and the ability to distribute poly select.

[EdH]

Hey Curtis,

I'm not sure you can adjust the params enough to make up the difference. Now the server is running LA (mksol) and giving an ETA of 07:46:58.

It's looking well over three times as long for CADO-NFS.

Ed

[VBCurtis]

Ed-
Even if you don't run this factorization again with my params, your default run's timing results will help me choose parameters for my own future use. I'll post them, of course, in case someone wishes to try them.

C150 is about where I started to tinker with factmsieve parameters too, so I think CADO will be even worse off against those.

[EdH]

Originally Posted by VBCurtis

Ed-
Even if you don't run this factorization again with my params, your default run's timing results will help me choose parameters for my own future use. I'll post them, of course, in case someone wishes to try them.

C150 is about where I started to tinker with factmsieve parameters too, so I think CADO will be even worse off against those.

Hi Curtis,

I still expect to give your params a test run when you get them set. I just can't see running CADO-NFS as a norm at this point, but I have a script alternative for msieve/ggnfs.

And, the results are in:

Info:Polynomial Selection (size optimized): Aggregate statistics:
Info:Polynomial Selection (size optimized): potential collisions: 56795
Info:Polynomial Selection (size optimized): raw lognorm (nr/min/av/max/std): 57461/46.070/54.041/58.840/0.891
Info:Polynomial Selection (size optimized): optimized lognorm (nr/min/av/max/std): 57461/44.100/48.552/54.860/1.446
Info:Polynomial Selection (size optimized): Total time: 91122.7
Info:Polynomial Selection (root optimized): Aggregate statistics:
Info:Polynomial Selection (root optimized): Total time: 8478.5
Info:Polynomial Selection (root optimized): Rootsieve time: 8477.14
Info:Generate Factor Base: Total cpu/real time for makefb: 36.18/8.17465
Info:Generate Free Relations: Total cpu/real time for freerel: 624.23/82.0787
Info:Lattice Sieving: Aggregate statistics:
Info:Lattice Sieving: Total number of relations: 53833827
Info:Lattice Sieving: Average J: 3798.46 for 2342851 special-q, max bucket fill: 0.697496
Info:Lattice Sieving: Total CPU time: 6.52511e+06s
Info:Filtering - Duplicate Removal, splitting pass: Total cpu/real time for dup1: 165.89/578.793
Info:Filtering - Duplicate Removal, splitting pass: Aggregate statistics:
Info:Filtering - Duplicate Removal, splitting pass: CPU time for dup1: 577.6000000000001s
Info:Filtering - Duplicate Removal, removal pass: Total cpu/real time for dup2: 1273.34/1043.39
Info:Filtering - Singleton removal: Total cpu/real time for purge: 1078.85/1214.3
Info:Filtering - Merging: Total cpu/real time for merge: 3665.72/3278.51
Info:Filtering - Merging: Total cpu/real time for replay: 230.56/198.062
Info:Linear Algebra: Total cpu/real time for bwc: 945170/0.000172853
Info:Linear Algebra: Aggregate statistics:
Info:Linear Algebra: Krylov: WCT time 78981.07
Info:Linear Algebra: Lingen CPU time 2012.76, WCT time 306.77
Info:Linear Algebra: Mksol: WCT time 42773.62
Info:Quadratic Characters: Total cpu/real time for characters: 205.42/64.8022
Info:Square Root: Total cpu/real time for sqrt: 11575.4/1607.64
Info:HTTP server: Shutting down HTTP server
Info:Complete Factorization: Total cpu/elapsed time for entire factorization: 7.58874e+06/171384
Info:root: Cleaning up computation data in /tmp/cado.vzafj00h
35912223503197268109418424875344813700437442706880566137398291217213 22947545427314151445011966017377 584458412373341050558641690854880477452541557046361

It was finished in just less than two days - 47h 36m 24s.

Let me know when you get some changes made for the params...

Ed

[VictordeHolland]

http://mersenneforum.org/showpost.ph...53&postcount=6

GGNFS+Mieve vs. CADO-NFS C150
Poly select: 7:39 vs. 9:46 (99,601 cpusec /170 cores=~586 sec realtime)
Sieving: 8:00:28 vs. 10:39:43 (6.52511e+06s cpusec /170 cores =~38,383 sec realtime)
LA: 5:56:13 vs. 33:54:21 (78,981.07 + 306.77 + 42,773.62 = 122,061 sec)
SQR: 47:02 vs. 26:47 (1607.64 sec)

So the big difference is really in the LA phase??

[henryzz]

Originally Posted by VictordeHolland

http://mersenneforum.org/showpost.ph...53&postcount=6
So the big difference is really in the LA phase??

How many cores was the LA run on?

[EdH]

Originally Posted by henryzz

How many cores was the LA run on?

The same i7-2600 @ 3.4 GHz with 16GB ran both LAs on 8 threads (4c x 2t).

edit: Also note that I hard stopped the polyselect for the msieve/ggnfs run. I don't know what msieve would have preferred.

edit2: msieve wanted 193.43 CPU-hours.I think this would have far out done the time CADO-NFS used.

[VBCurtis]

Originally Posted by EdH

Hi Curtis,

Info:Polynomial Selection (size optimized): Total time: 91122.7
Info:Polynomial Selection (root optimized): Rootsieve time: 8477.14
Info:Lattice Sieving: Total number of relations: 53833827
Info:Lattice Sieving: Total CPU time: 6.52511e+06s
Info:Linear Algebra: Total cpu/real time for bwc: 945170/0.000172853
Info:Complete Factorization: Total cpu/elapsed time for entire factorization: 7.58874e+06/171384

It was finished in just less than two days - 47h 36m 24s.
Let me know when you get some changes made for the params...
Ed

Ed-
Here are my best-guess params for C150:

tasks.polyselect.degree = 5

tasks.polyselect.P = 600000
tasks.polyselect.admax = 25e4
tasks.polyselect.adrange = 5e2
tasks.polyselect.incr = 60
tasks.polyselect.nq = 15625
tasks.polyselect.nrkeep = 100
tasks.polyselect.ropteffort = 12

###########################################################################
# Sieve
###########################################################################

lim0 = 13000000
lim1 = 32000000
lpb0 = 29
lpb1 = 30
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 60
tasks.sieve.ncurves0 = 16
tasks.sieve.ncurves1 = 21
tasks.I = 14

tasks.sieve.qrange = 2000
tasks.sieve.qmin = 3000000

###########################################################################
# Filtering
###########################################################################

tasks.filter.purge.keep = 175
tasks.filter.maxlevel = 30
tasks.filter.target_density = 155.0

Major changes:
1. Ditched 3 large primes.
2. I = 14 rather than 13. I'm not sure this is correct, as I think with CADO the transition from 13 to 14 is right around 150. However, this means that if it's a mistake it should be a small one.
3. Reduced target density from 170 to 155.
4. Almost tripled poly-select effort. Your data showed 100k sec on poly select on a job that took ~7.6M seconds, meaning poly select was around 1.5% of total time. I've found best results at 4-5%, so I added a bunch in hopes that overall time will drop a bunch to make my selection ~5%.
5. Set qmin = 3M. I think in the absence of this parameter CADO defaults to begin sieving at lim0, which was 16M. I expect you'll complete sieving before 16M, so almost no Q's will overlap your previous run!

I look forward to your test results.