mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   CADO-NFS (https://www.mersenneforum.org/forumdisplay.php?f=170)
-   -   CADO-NFS Data Harvesting (https://www.mersenneforum.org/showthread.php?t=27689)

EdH 2022-04-01 19:47

CADO-NFS Data Harvesting
 
This is mostly a question for VBCurtis:

If I'm only running CADO-NFS through sieving, are there any bits of timing data vs. composite sizes you may be interested in?

I won't be able to flag whether I'm using a params file modified by you or me, or an original, but I can probably harvest certain details contained in the log file, or even from the snapshot.

This is the CADO-NFS portion of a normal run for my scripts:
- CADO-NFS is called by a script and given a few standard items. The rest are supplied by the params files.
- CADO-NFS performs the Polyselect and Lattice Sieving via server/clients.
- A side script is started (depending on the composite being >125 digits) that examines the relations and performs Msieve filtering until a matrix can be built.
- - Once successful, CADO-NFS is told to shut down.
- - - The shutdown may occur anywhere from Lattice Sieving to LA.
- If the composite is <125 digits, CADO-NFS completes the factorization.

In light of the fact the process may or may not get to/through filtering, is there info that would be of value to gather?

VBCurtis 2022-04-01 23:30

Yes, if you can connect the timing data to the params used for the job and to the composite size (including first digit).

Also, small jobs are easy to collect data, and I think I'm done with params below 135-140 digits. So, maybe only for 150+ digit jobs? That data can be used to build a bit of a size - vs - sievetime curve, and any outliers mean "find better params for that size, please."

If you'd like to do that, I'll be happy to review the data to see where I likely have suboptimal params.

EdH 2022-04-02 01:40

It's only for a c140, but how would this look for data:[code]N = 336... <140 digits>
tasks.I = 14
tasks.lim0 = 8800000
tasks.lim1 = 14400000
tasks.lpb0 = 30
tasks.lpb1 = 31
tasks.qmin = 900000
tasks.filter.target_density = 130.0
tasks.filter.purge.keep = 180
tasks.polyselect.P = 182500
tasks.polyselect.admax = 146e3
tasks.polyselect.admin = 1800
tasks.polyselect.degree = 5
tasks.polyselect.incr = 120
tasks.polyselect.nq = 15625
tasks.polyselect.nrkeep = 66
tasks.polyselect.ropteffort = 16
tasks.sieve.lambda0 = 1.82
tasks.sieve.lambda1 = 1.81
tasks.sieve.mfb0 = 56
tasks.sieve.mfb1 = 58
tasks.sieve.ncurves0 = 18
tasks.sieve.ncurves1 = 24
tasks.sieve.qrange = 10000
Polynomial Selection (root optimized): Total time: 6113.74
Polynomial Selection (root optimized): Rootsieve time: 6112.81
Generate Factor Base: Total cpu/real time for makefb: 22.71/1.43919
Lattice Sieving: Total number of relations: 100246494
Lattice Sieving: Total time: 322777s
Filtering - Duplicate Removal, splitting pass: CPU time for dup1: 245.5s[/code]Anything missing or not really of interest?

charybdis 2022-04-02 02:07

[QUOTE=EdH;603076][code]Lattice Sieving: Total time: 322777s[/code][/QUOTE]

Unhelpfully, while the total time for a completed job is given in wall-clock time and in thread-time, this value for the sieving step is neither of these: it's client-time, so should be about thread-time/2 unless you've changed threads-per-client from the default. Just something that needs to be kept in mind when making comparisons.

EdH 2022-04-02 02:50

[QUOTE=charybdis;603079]Unhelpfully, while the total time for a completed job is given in wall-clock time and in thread-time, this value for the sieving step is neither of these: it's client-time, so should be about thread-time/2 unless you've changed threads-per-client from the default. Just something that needs to be kept in mind when making comparisons.[/QUOTE]I'm a tiny bit confused, but all my clients are based on four threads, except an occasional two thread machine that is very rarely engaged. I could make that rarely into never without issue, since it is also my GPU machine and I'm normally doing other stuff with it. Are you saying I should divide this value by 4? Or, should I just note on that line, 4 threads per client?

Also, would it be helpful if I translate that value into 89:39:37?

VBCurtis 2022-04-02 06:00

A note of 4 threads per client is enough- I can double the time if I compare to my own machines, or just leave it as-is when comparing to other runs of yours.
I do think you should have everything run 4-threaded so that the mix of 2-threaded clients and 4-threaded clients doesn't dirty the data.
Polyselect params aren't really of interest, but poly score is of interest. Alas, Cado's score report is only comparable to other polys that use the exact-same siever & lim's, which is annoying. Poly select time is useful, as I am still not convinced I'm doing the right amount of poly select relative to sieve time!

You may be using some older params files before I learned that the ratio of last Q sieved to starting-Q should be no more than 8. If you notice any final-Q that's much higher than 8x the starting-Q for that params file, boost starting-Q accordingly. I'm seeing best performance with this ratio around 6 for C140+ jobs, a bit higher ratio works fine for smaller jobs.

EdH 2022-04-02 12:52

I was also thinking a note about clients would be better, since then you know. If I simply adjusted the value, we'd never be sure. And, I can leave out the 2-thread easy enough, although I wonder the actual effect of one, 2-thread client alongside 57-79, four-thread clients. Then, again, what's its contribution among the set? I doubt it would be missed.

Should I add in the full polynomial? There are at least two different polynomials (of three) in my current sample. I would expect the final one to be the one used. I should be able to harvest that.

(I'm pretty sure) I could add in a cownoise score.

I'm currently running a c160, that is supposed to finish tonight. I can start using it as my sample, instead of the current c140.

If I understand, I can drop all tasks.polyselect values.

Is there interest in the separate Rootsieve time?

What about the Factor Base value?

ETA: If I achieve my goal of having CADO-NFS continue sieving until I tell it to stop, there will be no filtering time. I will probably remove that item from my data list.

VBCurtis 2022-04-02 15:12

If there's a cownoise score, the actual poly has no use to the data summary.
I agree that 1-2 two-threaded instances won't color the data from a 50+ client farm!

EdH 2022-04-02 15:34

Here's another run against the c140:[code]N = 336... <140 digits>
tasks.I = 14
tasks.lim0 = 8800000
tasks.lim1 = 14400000
tasks.lpb0 = 30
tasks.lpb1 = 31
tasks.qmin = 900000
tasks.filter.target_density = 130.0
tasks.filter.purge.keep = 180
tasks.sieve.lambda0 = 1.82
tasks.sieve.lambda1 = 1.81
tasks.sieve.mfb0 = 56
tasks.sieve.mfb1 = 58
tasks.sieve.ncurves0 = 18
tasks.sieve.ncurves1 = 24
tasks.sieve.qrange = 10000
Polynomial Selection (root optimized): Total time: 6113.74
Lattice Sieving: Total number of relations: 100246494
Lattice Sieving: Total time: 322777s (all clients used 4 threads)
Found 55764577 unique, 23084551 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 2.51691527e-11[/code]The discrepancy in relations counts is due to the filtering tests for Msieve running while CADO-NFS is still sieving. Would a ratio of duplication be a less confusing value? Or, would you rather be able to look at the actual numbers as shown?

charybdis 2022-04-02 15:54

[QUOTE=EdH;603094]Is there interest in the separate Rootsieve time?[/QUOTE]

Surely there's not much point in reporting it unless you report the time for the rest of polyselect too?

EdH 2022-04-02 16:20

[QUOTE=charybdis;603115]Surely there's not much point in reporting it unless you report the time for the rest of polyselect too?[/QUOTE]Found it! Is this better?:[code]N = 336... <140 digits>
tasks.I = 14
tasks.lim0 = 8800000
tasks.lim1 = 14400000
tasks.lpb0 = 30
tasks.lpb1 = 31
tasks.qmin = 900000
tasks.filter.target_density = 130.0
tasks.filter.purge.keep = 180
tasks.sieve.lambda0 = 1.82
tasks.sieve.lambda1 = 1.81
tasks.sieve.mfb0 = 56
tasks.sieve.mfb1 = 58
tasks.sieve.ncurves0 = 18
tasks.sieve.ncurves1 = 24
tasks.sieve.qrange = 10000
Polynomial Selection (size optimized): Total time: 29411.9
Polynomial Selection (root optimized): Total time: 6113.74
Lattice Sieving: Total number of relations: 100246494
Lattice Sieving: Total time: 322777s (all clients used 4 threads)
Found 55764577 unique, 23084551 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 2.51691527e-11[/code]

EdH 2022-04-02 21:38

Today is not turning ot to be a "better day!" I'm causing duplication of wok in another thread, and now I'm finding out that if CADO-NFS is told to stop prior to its filtering, it doesn't give a tme for las. I will have to sort out something else. For now, the c160 will not be useful for much. I might just take a break. . .

EdH 2022-04-02 22:45

Maybe I've found a solution. Here's the data for the c160:[code]N = 516... <160 digits>
tasks.I = 14
tasks.lim0 = 45000000
tasks.lim1 = 70000000
tasks.lpb0 = 31
tasks.lpb1 = 32
tasks.qmin = 17000000
tasks.filter.target_density = 150.0
tasks.filter.purge.keep = 190
tasks.sieve.lambda0 = 1.84
tasks.sieve.mfb0 = 59
tasks.sieve.mfb1 = 62
tasks.sieve.ncurves0 = 20
tasks.sieve.ncurves1 = 25
tasks.sieve.qrange = 10000
Polynomial Selection (size optimized): Total time: 505021
Polynomial Selection (root optimized): Total time: 26267.6
Lattice Sieving: Total number of relations: 212441669
Lattice Sieving: Total time: 3.16477e+06s (all clients used 4 threads)
Found 149733097 unique, 40170110 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 1.43789954e-12[/code]

EdH 2022-04-05 14:03

Here's a c162:[code]N = 235... <162 digits>
tasks.I = 14
tasks.lim0 = 45000000
tasks.lim1 = 70000000
tasks.lpb0 = 31
tasks.lpb1 = 32
tasks.qmin = 17000000
tasks.filter.target_density = 150.0
tasks.filter.purge.keep = 190
tasks.sieve.lambda0 = 1.84
tasks.sieve.mfb0 = 59
tasks.sieve.mfb1 = 62
tasks.sieve.ncurves0 = 20
tasks.sieve.ncurves1 = 25
tasks.sieve.qrange = 10000
Polynomial Selection (size optimized): Total time: 508246
Polynomial Selection (root optimized): Total time: 25518.1
Lattice Sieving: Total time: 3.77171e+06s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 218448391
Found 149733097 unique, 40170110 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 1.16869325e-12[/code]

EdH 2022-04-09 00:13

Here's a c168:[code]N = 385... <168 digits>
tasks.I = 14
tasks.lim0 = 65000000
tasks.lim1 = 100000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 10000000
tasks.filter.target_density = 170.0
tasks.filter.purge.keep = 160
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 60
tasks.sieve.ncurves0 = 19
tasks.sieve.ncurves1 = 25
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 999726
Polynomial Selection (root optimized): Total time: 6873.68
Lattice Sieving: Total time: 6.3694e+06s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 179907757
Found 149733097 unique, 40170110 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 5.83275752e-13[/code]

EdH 2022-04-18 17:50

Here's a c161:[code]N = 235... <161 digits>
tasks.I = 14
tasks.lim0 = 45000000
tasks.lim1 = 70000000
tasks.lpb0 = 31
tasks.lpb1 = 32
tasks.qmin = 17000000
tasks.filter.target_density = 150.0
tasks.filter.purge.keep = 190
tasks.sieve.lambda0 = 1.84
tasks.sieve.mfb0 = 59
tasks.sieve.mfb1 = 62
tasks.sieve.ncurves0 = 20
tasks.sieve.ncurves1 = 25
tasks.sieve.qrange = 10000
Polynomial Selection (size optimized): Total time: 493855
Polynomial Selection (root optimized): Total time: 27925.7
Lattice Sieving: Total time: 2.84944e+06s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 202173233
Found 149733097 unique, 40170110 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 1.47600121e-12[/code]

EdH 2022-04-20 21:51

I'm currently running a c164 with [C]A = 28[/C] and [C]adjust_strategy = 2[/C]. Will the data from this one compare to the data from others. What additional things might I need to mention, if any?

VBCurtis 2022-04-20 23:14

I don't think any. We're looking for deviances from the trendline of "twice as hard every 5.5 digits". When a job is above that trend, it's a sign the params for that job size might benefit from some more attention.

At least, that's what I've found working my way up from 95 to 150 digits.

Charybdis occasionally runs two jobs very similar in length with one setting changed between them, as an A/B comparison to determine which setting is better. This is time-consuming at 160+, but it's the sort of work that lets us refine the params set. For instance, we *still* don't have a clear idea of when 3LP pulls ahead of 2LP for CADO. In principle, there should be a single cutoff above which we always use 3LP.

If you find yourself running a second job in the 160s the same size as one you've already documented, give 3LP a shot (I can be more specific on settings if you like).

EdH 2022-04-21 00:12

I've got a pretty large pool of numbers I'm playing with. If you can get me specific params you'd like me to use, I'll try them on another 164 digit or close. The current one is 345. . . I've got about a dozen to check for something close, that I'll hope (ironically) doesn't ECM. I have no idea how to use 3LP, so please be quite specific in what I should do.

EdH 2022-04-21 14:24

[QUOTE=EdH;604411]I've got a pretty large pool of numbers I'm playing with. . .[/QUOTE]Of course that meant that all the c164s are falling to ECM, now.* If I don't find a suitable c164, would you prefer I move up or down a digit? The leading digits are 345... on the current one, which should be finished tomorrow.

[SIZE=2]* The best way to get them to succeed at ECM is to look for GNFS candidates, unless you actually try that. . .[/SIZE]
[SIZE=2]
[/SIZE][SIZE=2]Edit: By posting the above I hope that the final c164 will fail ECM.[SIZE=1] But then because I posted such, it will succeed. But. . .:smile:[/SIZE]
[/SIZE]

VBCurtis 2022-04-22 01:57

Here's my 3LP settings for params.c165, tested exactly once:
[code]tasks.I = 14
tasks.qmin = 10000000
tasks.lim0 = 40000000
tasks.lim1 = 60000000
tasks.sieve.lambda0 = 1.83
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 88
tasks.sieve.ncurves0 = 18
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
tasks.sieve.rels_wanted = 175000000[/code]

If you can compare sieve time to your current settings, that would help us decide if C165 is big enough to run 3LP. If you have multiple jobs, please also try mfb1=89, as 88 might be too small.
3LP makes the sieve faster, at the expense of a jump in matrix size. It's not to our benefit to log a 10% improvement in sieve time if we lose 50% to matrix time! Hopefully that's an exaggeration, but that's why we take data.

charybdis 2022-04-22 02:05

Are you sure that swapping the lims won't improve yield? I thought larger lim on the 2LP side was pretty well established by now. Too lazy to dig up an old polynomial and test-sieve it myself.

EdH 2022-04-22 02:30

[QUOTE=VBCurtis;604513]Here's my 3LP settings for params.c165, tested exactly once:
. . ..[/QUOTE]Sorry for my 3LP ignorance, but will these params tell it to use 3LP or do I need to add something else?

Also, do you need the full 175M relations? If Msieve successfully filters earlier than 175M, do you still want the rest?

BTW, I found a c164 candidate. I think it has a 6 leading digit. The current c164 should be done tomorrow, so I can start the 3LP job after that.

VBCurtis 2022-04-22 03:07

[QUOTE=charybdis;604514]Are you sure that swapping the lims won't improve yield? I thought larger lim on the 2LP side was pretty well established by now. Too lazy to dig up an old polynomial and test-sieve it myself.[/QUOTE]

Definitely not sure. With GGNFS that's clear, but CADO sieves below the factor base so I didn't make any assumptions.

Ed-
Nothing else needs to be changed. mfb at 88 is the key setting that causes 3LP (any setting larger than 3 * log_2(lim) will do it).

I don't mind if you don't get to 175M; whatever your scripts do is just fine with me- all the better to compare to a previous run with your script. May wish to swap lim's per Charybdis' suggestion, though.

charybdis 2022-04-22 10:51

Haven't done any big GNFS jobs myself for a while, but larger lims on the 2LP side definitely sieve better for the big SNFS jobs I've been doing instead. I think we used larger lims on the 2LP side for 3,748+ too.

EdH 2022-04-22 12:42

Here's the first c164 (note: A=28 and adjust_strategy=2):[code]N = 345... <164 digits>
tasks.lim0 = 50000000
tasks.lim1 = 70000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 10000000
tasks.filter.target_density = 170.0
tasks.filter.purge.keep = 160
tasks.sieve.lambda0 = 2.07
tasks.sieve.lambda1 = 2.17
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 61
tasks.sieve.ncurves0 = 18
tasks.sieve.ncurves1 = 25
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 529277
Polynomial Selection (root optimized): Total time: 31468
Lattice Sieving: Total time: 4.6221e+06s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 171561952
Found 149733097 unique, 40170110 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 8.37946014e-13[/code]Anything else I should grab before I do the next one?

EdH 2022-04-22 12:48

One last question:

Should I have a [C]tasks.sieve.lambda1[/C] value?

I currently have 2.17 (as can be seen above). Should I just keep that?

VBCurtis 2022-04-22 14:01

No lambda- if you did use one, it would have to be close to 3 for 3LP. Best leave it default.

EdH 2022-04-22 14:22

[QUOTE=VBCurtis;604538]No lambda- if you did use one, it would have to be close to 3 for 3LP. Best leave it default.[/QUOTE]
Thanks! It is in work. Here are the significant parts of the snapshot:[code]N = 685. . .<164 digits>
tasks.I = 14
tasks.lim0 = 60000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 10000000
tasks.sieve.lambda0 = 1.83
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 88
tasks.sieve.ncurves0 = 18
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
tasks.sieve.rels_wanted = 175000000[/code]

EdH 2022-04-23 12:24

Here is the latest c164:[code]N = 685... <164 digits>
tasks.I = 14
tasks.lim0 = 60000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 10000000
tasks.filter.target_density = 170.0
tasks.filter.purge.keep = 160
tasks.sieve.lambda0 = 1.83
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 88
tasks.sieve.ncurves0 = 18
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 526394
Polynomial Selection (root optimized): Total time: 31614.9
Lattice Sieving: Total time: 4.67967e+06s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 175012772
Found 149733097 unique, 40170110 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 8.31589954e-13[/code]I don't see too much difference. Murphy_E is lower and sieving took a little bit longer. However, the previous c164 used strategy 2, while this one did not. What effect would that have had at this size?

If you like, provide some changes and I'll put them in the params file for the next ~c165 composite. It may not be real soon, but maybe this upcoming week.

VBCurtis 2022-04-23 14:20

Try with strategy2, please? I don't use that setting because it seems to trigger errors with CADO postprocessing, so I forgot to include it for you.
My guess is 4% faster from strat-2? Your next data point will tell us. :)

Was the resulting matrix notably bigger than your previous C164?

EdH 2022-04-23 14:54

[QUOTE=VBCurtis;604613]Try with strategy2, please? I don't use that setting because it seems to trigger errors with CADO postprocessing, so I forgot to include it for you.
My guess is 4% faster from strat-2? Your next data point will tell us. :)

Was the resulting matrix notably bigger than your previous C164?[/QUOTE]I'll run the next one with A=28 and strategy 2, then.* I didn't bring up strategy 2 because you used I=14. Here are the matrix sections from the two logs - first c164:[code]Thu Apr 21 08:30:37 2022 matrix is 9822977 x 9823172 (3015.8 MB) with weight 932199937 (94.90/col)
Thu Apr 21 08:30:37 2022 sparse part has weight 672685141 (68.48/col)
Thu Apr 21 08:32:25 2022 filtering completed in 2 passes
Thu Apr 21 08:32:27 2022 matrix is 9792967 x 9793156 (3013.3 MB) with weight 931103496 (95.08/col)
Thu Apr 21 08:32:27 2022 sparse part has weight 672400393 (68.66/col)
Thu Apr 21 08:33:10 2022 matrix starts at (0, 0)
Thu Apr 21 08:33:11 2022 matrix is 9792967 x 9793156 (3013.3 MB) with weight 931103496 (95.08/col)
Thu Apr 21 08:33:11 2022 sparse part has weight 672400393 (68.66/col)
Thu Apr 21 08:33:11 2022 saving the first 48 matrix rows for later
Thu Apr 21 08:33:12 2022 matrix includes 64 packed rows
Thu Apr 21 08:33:13 2022 matrix is 9792919 x 9793156 (2895.1 MB) with weight 745879127 (76.16/col)[/code]and, the second c164:[code]Sat Apr 23 07:29:12 2022 matrix is 10949079 x 10949259 (3349.2 MB) with weight 1042919866 (95.25/col)
Sat Apr 23 07:29:12 2022 sparse part has weight 746571916 (68.18/col)
Sat Apr 23 07:32:13 2022 filtering completed in 2 passes
Sat Apr 23 07:32:17 2022 matrix is 10934410 x 10934588 (3348.1 MB) with weight 1042422445 (95.33/col)
Sat Apr 23 07:32:17 2022 sparse part has weight 746467122 (68.27/col)
Sat Apr 23 07:33:16 2022 matrix starts at (0, 0)
Sat Apr 23 07:33:19 2022 matrix is 10934410 x 10934588 (3348.1 MB) with weight 1042422445 (95.33/col)
Sat Apr 23 07:33:19 2022 sparse part has weight 746467122 (68.27/col)
Sat Apr 23 07:33:19 2022 saving the first 48 matrix rows for later
Sat Apr 23 07:33:21 2022 matrix includes 64 packed rows
Sat Apr 23 07:33:23 2022 matrix is 10934362 x 10934588 (3228.6 MB) with weight 832280012 (76.11/col)[/code]Yes, the matrix is a little bit larger, but is that just due to when Msieve happened to succeed in its filtering tests?

* I'm guessing that's the only change you would like for the next ~c164 run (I don't have another c164 handy just yet), or do you want something else modified, too?

VBCurtis 2022-04-23 23:49

Please try A=28 separately from strat 2. I'd like to know the speed gained from start 2 on I=14.
I expect A=28 would be slower than I=14 here, anyway; perhaps we can test-sieve that rather than run a full job.

EdH 2022-04-24 00:20

[QUOTE=VBCurtis;604640]Please try A=28 separately from strat 2. I'd like to know the speed gained from start 2 on I=14.
I expect A=28 would be slower than I=14 here, anyway; perhaps we can test-sieve that rather than run a full job.[/QUOTE]OK. I have a c164 (685...) candidate and I should be able to start it running tomorrow. I'll run I=14 [B]with[/B] adjust_strategy=2.

(For some reason, I had it in the back of my head that strategy=2 wouldn't work with I=14.)

charybdis 2022-04-24 01:21

[QUOTE=VBCurtis;604640]I expect A=28 would be slower than I=14 here, anyway; perhaps we can test-sieve that rather than run a full job.[/QUOTE]

Unfortunately I don't think that's something you can figure out by test-sieving, because A=28 should have a lower duplication rate. The crossover point for "time to find rels_wanted raw relations" is probably a few digits higher than the true crossover.

@EdH: I think you might need to take a look at your script, as all your summaries seem to include
[code]Found 149733097 unique, 40170110 duplicate, and 0 bad relations.[/code]

EdH 2022-04-24 02:38

[QUOTE=charybdis;604646]. . .
@EdH: I think you might need to take a look at your script, as all your summaries seem to include
[code]Found 149733097 unique, 40170110 duplicate, and 0 bad relations.[/code][/QUOTE]Indeed! I can't find where the report gets written in any of my scripts, but I do have a file with those values from sometime, that I harvest for each run. Thanks for catching that. I will definitely have to work on it. I might have to skip remdups4 and let Msieve report duplication and harvest the values from there.

EdH 2022-04-24 11:36

OK, I'm losing it! The new candidate is the one I just factored. It got mixed into the list because it wasn't finished yet. I need to do some more work before I get to the next candidate.

EdH 2022-04-24 14:21

I have a c164 underway:[code]N = 712...<164 digits>
tasks.I = 14
tasks.lim0 = 60000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 10000000
tasks.sieve.adjust_strategy = 2
tasks.sieve.lambda0 = 1.83
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 88
tasks.sieve.ncurves0 = 18
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
tasks.sieve.rels_wanted = 175000000[/code]

EdH 2022-04-25 13:19

Here's the next c164 (I=14 and adjust_strategy=2):[code]N = 712... <164 digits>
tasks.I = 14
tasks.lim0 = 60000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 10000000
tasks.filter.target_density = 170.0
tasks.filter.purge.keep = 160
tasks.sieve.adjust_strategy = 2
tasks.sieve.lambda0 = 1.83
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 88
tasks.sieve.ncurves0 = 18
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 524425
Polynomial Selection (root optimized): Total time: 30333.8
Lattice Sieving: Total time: 4.46548e+06s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 175001545
Found 122488916 unique, 45564734 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 8.11818879e-13[/code]

VBCurtis 2022-04-25 14:18

Poly score 2.5% worse, but sieve time roughly 5% better. Nice!

The next settings to test are A=28 and mfb1 = 89. A=28 is more important a test (mfb should not change sieve time very much).

EdH 2022-04-25 16:12

[QUOTE=VBCurtis;604744]. . .
The next settings to test are A=28 and mfb1 = 89. A=28 is more important a test (mfb should not change sieve time very much).[/QUOTE]Are you saying I should only change to A=28 first, or go ahead and change both, and with or without strategy=2?

VBCurtis 2022-04-25 16:18

I think / hope each change should be independent- that is, you have determined strat 2 is faster (really, Charybdis determined this over a year ago), now it's the default. Next, try A = 28; once we know the best setting there, try mfb's.

One change at a time with A/B comparisons give us "clear" evidence for what to use; once the big settings like A and lp are set, the little settings (mfb, starting Q, lambda, target rels) can be dialed in hopes of finding a few more % of speed.

EdH 2022-04-25 16:55

[QUOTE=VBCurtis;604756]I think / hope each change should be independent- that is, you have determined strat 2 is faster (really, Charybdis determined this over a year ago), now it's the default. Next, try A = 28; once we know the best setting there, try mfb's.

One change at a time with A/B comparisons give us "clear" evidence for what to use; once the big settings like A and lp are set, the little settings (mfb, starting Q, lambda, target rels) can be dialed in hopes of finding a few more % of speed.[/QUOTE]I just wanted to make sure I'm on the same page. The only thing I'll change for next time is A=28.

Again, I'm out of c164 candidates, I may have some lower c165s.

charybdis 2022-04-25 17:34

[QUOTE=EdH;604764]I just wanted to make sure I'm on the same page. The only thing I'll change for next time is A=28.[/QUOTE]

A=28 should probably have a slightly lower qmin - maybe 7M? - as you'll be sieving a smaller range of Q. We'll see what Curtis says.

VBCurtis 2022-04-25 19:40

Excellent point- we want the ratio of qmin to qmax to be something between 6 and 8. I suspect your C164 with I=14 ran Q roughly 10-60M; since A=28 should yield 40% better, a Q-range of 7-43M might be expected.

So, I agree with Charybdis' suggestion to change qmin to 7M. Good idea!

EdH 2022-04-25 21:37

[QUOTE=VBCurtis;604771]Excellent point- we want the ratio of qmin to qmax to be something between 6 and 8. I suspect your C164 with I=14 ran Q roughly 10-60M; since A=28 should yield 40% better, a Q-range of 7-43M might be expected.

So, I agree with Charybdis' suggestion to change qmin to 7M. Good idea![/QUOTE]It has been written - it will be done.:smile:

One more c164 (231...) has shown up in my work listings and I should be able to start it tomorrow afternoon.

EdH 2022-04-26 13:39

The latest is underway:[code]N = 231...<164 digits>
tasks.A = 28
tasks.lim0 = 60000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 7000000
tasks.sieve.adjust_strategy = 2
tasks.sieve.lambda0 = 1.83
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 88
tasks.sieve.ncurves0 = 18
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
tasks.sieve.rels_wanted = 175000000[/code]In my reports, is there any interest in the two "filter" lines? I'm thinking of removing them.

VBCurtis 2022-04-26 15:14

The filter lines have no effect, since you're using msieve for postprocessing.

EdH 2022-04-26 16:45

[QUOTE=VBCurtis;604815]The filter lines have no effect, since you're using msieve for postprocessing.[/QUOTE]Those were my thoughts, but wondered if they may have any use, anyway. Thanks!

EdH 2022-04-27 14:29

The latest c164:[code]N = 231... <164 digits>
tasks.A = 28
tasks.lim0 = 60000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 7000000
tasks.sieve.adjust_strategy = 2
tasks.sieve.lambda0 = 1.83
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 88
tasks.sieve.ncurves0 = 18
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 496571
Polynomial Selection (root optimized): Total time: 31172.8
Lattice Sieving: Total time: 4.28663e+06s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 169728713
Found 119819021 unique, 42731179 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 8.60775398e-13[/code]

VBCurtis 2022-04-27 18:33

Looks like a 6% better scoring poly than the last test, but only 4% lower sieve time.
Nearly a wash, but I=14 is also lower memory while being not-slower.

EdH 2022-04-27 19:30

[QUOTE=VBCurtis;604891]Looks like a 6% better scoring poly than the last test, but only 4% lower sieve time.
Nearly a wash, but I=14 is also lower memory while being not-slower.[/QUOTE]Sounds like a suggestion to move back to I=14. I'm thinking I'm going to have to move up or down a digit for my next few tests. Is there a preference if I move to mfb1=89?

VBCurtis 2022-04-28 01:54

As long as the input number is close, comparing E-score is pretty accurate (as I did in my previous post). It shouldn't matter whether you go with 163 or 165 or 166 next time for the mfb 89 trial.
Note that a larger mfb "should" improve yield, but more raw relations are likely necessary. The tradeoff is murky- if sec/rel is not better when test-sieving, I go with the smaller mfb. When sec/rel is better, a full factorization is likely to educate us. So, please try a full factorization as you planned.

EdH 2022-04-28 12:25

All my runs are full runs, unless they break. At these sizes, full runs are less than 48 hours, so my patience is still holding.

EdH 2022-04-28 18:05

I've started a c165. I went back to I=14, but forgot to take qmin back up to 10M. Will that throw things off?:[code]N = 309...<165>
tasks.I = 14
tasks.lim0 = 60000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 7000000
tasks.sieve.adjust_strategy = 2
tasks.sieve.lambda0 = 1.83
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 89
tasks.sieve.ncurves0 = 18
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
tasks.sieve.rels_wanted = 175000000[/code]

VBCurtis 2022-04-28 21:17

Not really; you'll have a slightly higher duplicate rate, meaning you may need an extra 1-3M raw relations to get the matrix size you expect. Sieve time will barely change.

EdH 2022-04-29 17:37

Here's the c165:[code]N = 309... <165 digits>
tasks.I = 14
tasks.lim0 = 60000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 7000000
tasks.sieve.adjust_strategy = 2
tasks.sieve.lambda0 = 1.83
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 89
tasks.sieve.ncurves0 = 18
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 523377
Polynomial Selection (root optimized): Total time: 35527
Lattice Sieving: Total time: 4.96142e+06s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 175009615
Found 122644730 unique, 53361072 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 7.58493957e-13[/code]Although I have another c165 available, I'm going to move back to working on the easier Aliquot sequences list for a little bit, since the number of sequences is diminishing and I want to try to make sure it doesn't get too low.

VBCurtis 2022-04-29 20:50

If I've got the scaling of poly score vs sieve time, looks like 88 is quicker.

EdH 2022-04-29 21:08

[QUOTE=VBCurtis;604981]If I've got the scaling of poly score vs sieve time, looks like 88 is quicker.[/QUOTE]Not at all empirical, but I had the feeling this was more difficult. I encountered a CADO-NFS filtering crash (possibly the strategy 2 trouble you mentioned?) and Msieve LA came back with over 32.5 hours ETA with 40 threads. I should probably save a copy of the Msieve log just for later perusal. (I wonder if I should see if a Colab instance would give me a good GPU and what it might show for ETA. . .)

ETA: I'm planning to try the Colab session, but the compression and upload times, as well as Colab inialization, are going to take quite a while, so I'm not sure there's an advantage.

ETA2: After nearly an hour and nowhere near completion, I gave up on my msieve.dat.mat.tbz upload to Google Drive for a Colab test. It had already taken over a half-hour to copy and compress it.

EdH 2022-05-27 20:04

Here's a c162 I just finished:[code]N = 314... <162 digits>
tasks.I = 14
tasks.lim0 = 45000000
tasks.lim1 = 70000000
tasks.lpb0 = 31
tasks.lpb1 = 32
tasks.qmin = 17000000
tasks.sieve.lambda0 = 1.84
tasks.sieve.mfb0 = 59
tasks.sieve.mfb1 = 62
tasks.sieve.ncurves0 = 20
tasks.sieve.ncurves1 = 25
tasks.sieve.qrange = 10000
Polynomial Selection (size optimized): Total time: 494305
Polynomial Selection (root optimized): Total time: 28582
Lattice Sieving: Total time: 3.96443e+06s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 209104683
Found 146771345 unique, 45989579 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 1.11487621e-12
[/code]I'm guessing I should adjust some of these values to be closer to the params for the previous c165?

VBCurtis 2022-05-27 20:23

Looks like 7-8% slower than the scaling one would expect for a C162 given the timing data from the C165 (that is, using the "double effort every 5.5 digits" heuristic).

I'd expect 3.72 megaseconds based on the 4.96 megasecond time for C165, calculated by inverting the ratio of the poly scores; the run was 3.96 megaseconds, 8% slower than target.

This may indicate 3 large primes is valuable on the C160 file, or perhaps some other parameter is not ideal. I'd try the lims and mfbs from the C165 run, with 31/31LP and 5% lower target relations (because the job is smaller/easier). I'd also use the q-min from the C165.

Ignoring the changes made for 3LP, I've been scaling the lim's to double every 10 digits. I'm not sure what is best for 3LP cases, but C165 used 60M/40M; perhaps 50M for lim0 makes sense on C160 file.

EdH 2022-05-27 21:46

Thanks! I've possibly got a c160 (615...) coming up. I think I've got the changes made to the params.* I'll post what turns up.

* I've got a high rels_wanted, but Msieve is going to break in when a matrix is possible.

EdH 2022-05-28 16:28

Here's the latest:[code]N = 615... <160 digits>
tasks.I = 14
tasks.lim0 = 50000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 7000000
tasks.sieve.lambda0 = 1.83
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 89
tasks.sieve.ncurves0 = 20
tasks.sieve.ncurves1 = 25
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 493434
Polynomial Selection (root optimized): Total time: 27103.8
Lattice Sieving: Total time: 3.40375e+06s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 169962886
Found 119992365 unique, 35401629 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 1.64176010e-12[/code]

VBCurtis 2022-05-28 18:19

Hrmmm.... poly score nearly 50% better than the last one, but only 15% less sieve time. That's bad- especially when raw relations needed went from 209M to 169M.
Aha! I forgot: Breaking 3LP cofactors uses a small ncurves value. Leaving ncurves1 at 25 may have caused this slow job. I think the CADO default is to use ncurves of 9 or 10 on the 3LP side (20-25 on the 2LP side is still good).
So, when using mfb1 > 2 * lpb1 which is how we do 3LP on that side, drop ncurves1 from 25 to 10ish.

EdH 2022-05-28 21:02

I'm running a sequence that is currently at 169 digits on its way down. It has a good chance of providing some more 16x composites. I'll drop ncurves1 to 10 for the next one, which could actually be any of the three param files (160, 165, 170). I'm somewhat set for 160 and 165, but what would you suggest for 170 if I start with the 165 params?

EdH 2022-06-14 13:38

Another c163
 
[code]N = 344... <163 digits>
tasks.I = 14
tasks.lim0 = 60000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 7000000
tasks.sieve.lambda0 = 1.83
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 89
tasks.sieve.ncurves0 = 18
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 529204
Polynomial Selection (root optimized): Total time: 30833.4
Lattice Sieving: Total time: 4.95571e+06s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 185882690
Found 125042962 unique, 43355067 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 9.45211518e-13[/code]

VBCurtis 2022-06-14 14:13

Weird; two digits easier than the C165 from a few posts ago, with the same params, yet needed 10M more raw relations, more unique relations, and took the same amount of time as a poly that scored 20%+ worse.

EdH 2022-06-14 15:27

[QUOTE=VBCurtis;607821]Weird; two digits easier than the C165 from a few posts ago, with the same params, yet needed 10M more raw relations, more unique relations, and took the same amount of time as a poly that scored 20%+ worse.[/QUOTE]
Is all of this skewed because of my use of Msieve telling CADO-NFS when to stop sieving?

Edit: Msieve runs filtering to test for a matrix success and then waits a short time and tests again.

VBCurtis 2022-06-14 16:48

No, I think your way is actually more accurate, since it measures when a matrix can be built of a certain density rather than using user-set guesses for number of relations to find.

EdH 2022-07-20 02:15

One more:[code]N = 164... <160 digits>
tasks.I = 14
tasks.lim0 = 50000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 7000000
tasks.sieve.lambda0 = 1.83
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 89
tasks.sieve.ncurves0 = 20
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 480487
Polynomial Selection (root optimized): Total time: 27393
Lattice Sieving: Total time: 3.16854e+06s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 166474932
Found 113244194 unique, 35828316 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 1.62682689e-12[/code]

EdH 2022-07-23 22:59

And, a c172:[code]N = 237... <172 digits>
tasks.I = 14
tasks.lim0 = 65000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 7000000
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 89
tasks.sieve.ncurves0 = 19
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 985587
Polynomial Selection (root optimized): Total time: 10003
Lattice Sieving: Total time: 1.28816e+07s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 240648323
Found 151223351 unique, 81169538 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 3.16608561e-13[/code]

EdH 2022-08-18 13:38

Here's a c173:[code]N = 107... <173 digits>
tasks.A = 28
tasks.lim0 = 65000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 7000000
tasks.sieve.adjust_strategy = 2
tasks.sieve.mfb0 = 59
tasks.sieve.mfb1 = 89
tasks.sieve.ncurves0 = 19
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 994417
Polynomial Selection (root optimized): Total time: 9945.86
Lattice Sieving: Total time: 1.28671e+07s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 233696828
Found 145036125 unique, 73955642 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 2.69267673e-13[/code]What changes, other than increasing qmin would be helpful? A suggestion for qmin=15M has been made. What about strategy = 2? Is that good here? It seems we removed that in the past.

charybdis 2022-08-18 14:03

[QUOTE=EdH;611693]What changes, other than increasing qmin would be helpful? A suggestion for qmin=15M has been made. What about strategy = 2? Is that good here? It seems we removed that in the past.[/QUOTE]

Strategy 2 is pretty much essential when using even values of A. The performance benefit is over 10% if I remember correctly. As already discussed in this thread, it also tends to be a few percent faster at this size even for odd values of A (i.e. when I is used instead of A).

Presumably strategy 2 was removed because of the occasional CADO filtering crashes. But with your script setup that shouldn't be an issue, as long as CADO filtering doesn't run until msieve can build a matrix.

Apart from increasing qmin, the other thing to try for small c17x would be raising lpb1 to 32, and correspondingly mfb1 to 92.

EdH 2022-08-18 15:28

Thanks! I've been trying to adjust rels_wanted to make sure it exceeds Msieve matrix build relations needed, so I'll stick with strategy 2. Part of this is so the CADO filtering isn't fighting for resources while Msieve filtering is testing for a matrix. I'll adjust qmin to 15M, and raise lpb1 and mfb1 for my next c17x run, whenever I do another one.

I do run CADO filtering briefly at the end to gather the data for this thread. At that point, if it crashes, it doesn't bother my scripts, but I might miss some of the values in these posts. I can look at that later if it does occur.

EdH 2022-08-22 13:11

Here's another c160:[code]N = 126... <160 digits>
tasks.I = 14
tasks.lim0 = 50000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 7000000
tasks.sieve.lambda0 = 1.83
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 89
tasks.sieve.ncurves0 = 20
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 488894
Polynomial Selection (root optimized): Total time: 25992.8
Lattice Sieving: Total time: 3.49294e+06s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 172663455
Found 118538364 unique, 41243958 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 1.50177701e-12[/code]I have since, increased qmin for both c160 and c165 params. I should have another c165 tomorrow.

BTW, I have been harvesting all my runs for a while now. If you have some specific sizes you'd be interested in, let me know and I'll post them.

charybdis 2022-08-22 13:28

Comparing with earlier in the thread, it looks to me like 3LP is slower at c160.

EdH 2022-08-23 18:56

Here's a c165:[code]N = 322... <165 digits>
tasks.I = 14
tasks.lim0 = 60000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 12000000
tasks.sieve.lambda0 = 1.83
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 89
tasks.sieve.ncurves0 = 18
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 513630
Polynomial Selection (root optimized): Total time: 34447.3
Lattice Sieving: Total time: 5.58991e+06s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 174610477
Found 128257371 unique, 38152963 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 7.81486672e-13[/code]How does this compare?

I had less duplication with 12M, but I can't say how time turned out. It seems to be a bit longer than a c163, as expected. You think I should increase qmin some more and/or take mfb1 down a bit?

charybdis 2022-08-23 19:33

[QUOTE=EdH;611938]How does this compare?

I had less duplication with 12M, but I can't say how time turned out. It seems to be a bit longer than a c163, as expected. You think I should increase qmin some more and/or take mfb1 down a bit?[/QUOTE]

This is actually a bit slower than your previous c165 from post 57, which had the same params except qmin=7M and strategy 2. My comment (in a PM, I think) about raising qmin was intended for the small c17x numbers, not c160-165. Granted, strategy 2 is probably responsible for some of the difference.

For 2LP at c165, I'd drop mfb1 to 61 and bump lim1 up to something like 80M to compensate. Also ncurves1 back up to 25. Let's wait to see what VBCurtis thinks.

VBCurtis 2022-08-23 21:08

[QUOTE=charybdis;611943]This is actually a bit slower than your previous c165 from post 57, which had the same params except qmin=7M and strategy 2. My comment (in a PM, I think) about raising qmin was intended for the small c17x numbers, not c160-165. Granted, strategy 2 is probably responsible for some of the difference.

For 2LP at c165, I'd drop mfb1 to 61 and bump lim1 up to something like 80M to compensate. Also ncurves1 back up to 25. Let's wait to see what VBCurtis thinks.[/QUOTE]

I think all of this is accurate: 58/61 for mfb's sounds right, with 59/61 also worth a test. Ditto for doubling lim1 for 2LP compared to 3LP.
I'm finding best speed/duplicate compromise when Q-max is 6 to 7 times Q-min. If Q-max exceeds 8 * Q-min, I raise Q-min accordingly on the params file for that size.

For 31-bit LP and no 3LP, I'd use ncurves of 21 and 25, but up or down a couple seems to not matter.

Edit: Try lambda0 = 1.86 for mfb0 = 58, and 1.89 for mfb0 = 59. That should improve yield.

EdH 2022-09-11 15:25

Here's the latest, a c171:[code]N = 902... <171 digits>
tasks.I = 14
tasks.lim0 = 65000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 15000000
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 89
tasks.sieve.ncurves0 = 19
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 979823
Polynomial Selection (root optimized): Total time: 9797.37
Lattice Sieving: Total time: 1.25403e+07s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 225750239
Found 136126593 unique, 61871366 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 2.98902086e-13[/code]

VBCurtis 2022-09-11 16:57

Let me know what size you plan to run next; I'll do my first batch of las test-sieving on that size so we can make progress on these settings. I plan to test a bunch of settings, and then have you run a full job on the ones I think are fastest.
There remain good reasons to test full jobs- for instance, test-sieving doesn't indicate just how many relations will be needed when I change LP bounds. Our experience provides reasonable estimates, but when I'm trying to eke out 5% more speed an estimation error of 5% can swamp whatever speed I think I'm finding.


All times are UTC. The time now is 15:49.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.