mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > CADO-NFS

Reply
 
Thread Tools
Old 2021-09-02, 15:33   #56
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

499910 Posts
Default

Each polyselect process runs two-threaded, with each thread working independently. Using adrange of 8 * incr means each thread gets four leading coefficients to search. It is unavoidable that one set of four will finish before the other, so every polyselect process will run single-threaded for some time at the end of its workunit.

This is part of the reason why using hyperthreads gains speed with CADO.

I haven't changed anything else on params.c165; in fact, I haven't solved a job that big since you did so.
VBCurtis is offline   Reply With Quote
Old 2021-09-02, 17:30   #57
bur
 
bur's Avatar
 
Aug 2020
79*6581e-4;3*2539e-3

1100100012 Posts
Default

Quote:
This is part of the reason why using hyperthreads gains speed with CADO.
I didn't know that. For a 10 core should I use 5 clients à 4 threads or 10 à 2? And does sieving also benefit from hyperthreading? I didn't notice that problem during sieving.
bur is offline   Reply With Quote
Old 2021-09-02, 21:28   #58
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

10011100001112 Posts
Default

I leave all CADO processes the default two-threaded unless I run low on memory. But number of threads per process is separate from number of total threads used- CADO defaults to the total number of hyperthreads on a machine, and that is clearly faster than number of cores. In fact, someone 'round these parts said they run 40-threaded on an 18 core HT machine because the overhead threads for DB recording etc are quite slow. Can't say I've tried that extreme myself...

Anyway, if you run 20 threads on a 10-core the specifics don't seem to matter.

Last fiddled with by VBCurtis on 2021-09-02 at 21:28
VBCurtis is offline   Reply With Quote
Old 2021-09-07, 12:06   #59
bur
 
bur's Avatar
 
Aug 2020
79*6581e-4;3*2539e-3

401 Posts
Default

Ok, good to know. Does it apply to sieving as well? I never observed non-utilized cores during sieving.


I started sieving a c167 from 1992:1644 on Sunday using a polynomial I searched previously with the parameters from your file but instead using admax=5e6 instead of 1e6. Cownoise is 5.7e-13 so nothing too extreme, but will this interfere with adjusting the other parameters?


Specifically in my two runs on ~165 digit numbers I had a low number of duplicates (+70% uniques) and could go with 190M rels. Though I think that the quality of the poly does not influence the number of duplicates? Unfortunately I never checked the polys from those two previous composites.

Last fiddled with by bur on 2021-09-07 at 12:07
bur is offline   Reply With Quote
Old 2021-09-07, 14:22   #60
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

4,999 Posts
Default

More poly select doesn't interfere with anything else- perhaps 2x more poly select saves you 5% of sieve time; that's around half a digit of difficulty. I have poly select calibrated to take about 5% of the total job time; if you double poly select and find a poly that sieves 5% faster, the net gain in job time is roughly zero.

The number of uniques varies from poly to poly, and not in a way correlated to score. A much-better poly might sieve over a substantially smaller Q-region, which ought to lead to fewer duplicates, but otherwise it's pretty random.
VBCurtis is offline   Reply With Quote
Old 2021-09-08, 06:04   #61
bur
 
bur's Avatar
 
Aug 2020
79*6581e-4;3*2539e-3

401 Posts
Default

In an older post I saw why that initial c163 sieved so fast:

Code:
skew 8385289.96, size 8.965e-16, alpha -7.745, combined = 1.077e-12 rroots = 5
The record is 1.096e-12.

Last fiddled with by bur on 2021-09-08 at 06:05
bur is offline   Reply With Quote
Old 2021-09-19, 16:08   #62
bur
 
bur's Avatar
 
Aug 2020
79*6581e-4;3*2539e-3

401 Posts
Default

I'm currently at mksol for the c167 (took 197M relations). I started sieving at q = 17M. Since you mentioned earlier that a value of 17M was likely too large and the unique percentage is high (76%) I wanted to see what happens when I resieve the lower q-ranges and try and build a matrix with those included.



Could you post the appropriate command line for las? I checked the help but there are a lot of parameters that apparently aren't used normally in the command line, at least from what I remember seeing in top.
bur is offline   Reply With Quote
Old 2021-09-20, 01:23   #63
swellman
 
swellman's Avatar
 
Jun 2012

3,203 Posts
Default

Quote:
Originally Posted by bur View Post
I'm currently at mksol for the c167 (took 197M relations). I started sieving at q = 17M. Since you mentioned earlier that a value of 17M was likely too large and the unique percentage is high (76%) I wanted to see what happens when I resieve the lower q-ranges and try and build a matrix with those included.



Could you post the appropriate command line for las? I checked the help but there are a lot of parameters that apparently aren't used normally in the command line, at least from what I remember seeing in top.
See https://www.mersenneforum.org/showthread.php?t=26319 for a worked example using las. Might not have all the answers you seek but it could help.
swellman is online now   Reply With Quote
Old 2021-09-20, 09:04   #64
bur
 
bur's Avatar
 
Aug 2020
79*6581e-4;3*2539e-3

6218 Posts
Default

Thanks, that's very helpful. I also found that the commandline is saved to the sieving file.
bur is offline   Reply With Quote
Old 2021-09-23, 16:00   #65
bur
 
bur's Avatar
 
Aug 2020
79*6581e-4;3*2539e-3

401 Posts
Default

I did some tests on a C167 to see the differences between various q-min values.

I sieved in the region q0=10M to q1=108M with the parameters from curtis' file. The I concatenated the sieve files of various regions and made msieve build a matrix.


These are the results:
Code:
--
q=15M-104M (89M)

202836566 relations
50758954 duplicates and 152199568 unique relations

matrix is 10451195 x 10451420 (3780.8 MB) with weight 983955554 (94.15/col)
sparse part has weight 886591531 (84.83/col)
--

--
q=15M-105M (90M)

204611251 relations
51190337 duplicates and 153542870 unique relations

matrix is 10205132 x 10205356 (3689.9 MB) with weight 960718330 (94.14/col)
sparse part has weight 865223080 (84.78/col)
--

--
q=15M-106M (91M)

206510998 relations
51623746 duplicates and 154887252 unique relations

matrix is 9999089 x 9999314 (3614.4 MB) with weight 941369223 (94.14/col)
sparse part has weight 847506249 (84.76/col)
--

--
q=16M-105M (89M)

201149205 relations
found 49063275 duplicates and 152207886 unique relations

matrix is 10458080 x 10458305 (3784.1 MB) with weight 984860664 (94.17/col)
sparse part has weight 887403885 (84.85/col)
--

--
q=16M-106M (90M)

202926996 relations
found 49492280 duplicates and 153556672 unique relations

matrix is 10212921 x 10213146 (3693.3 MB) with weight 961603325 (94.15/col)
sparse part has weight 866042288 (84.80/col)
--

--
q=16M-107M (91M)

204688978 relations
49920242 duplicates and 154890692 unique relations

matrix is 10010922 x 10011147 (3618.0 MB) with weight 942278762 (94.12/col)
sparse part has weight 848335435 (84.74/col)
--

--
q=17M-106M (89M)

199442289 relations
found 47446144 duplicates and 152118101 unique relations

matrix is 10476553 x 10476778 (3789.4 MB) with weight 986164792 (94.13/col)
sparse part has weight 888588171 (84.82/col)
--

--
q=17M-107M (90M)

201204271 relations
found 47869683 duplicates and 153456544 unique relations

matrix is 10231547 x 10231772 (3699.3 MB) with weight 963128811 (94.13/col)
sparse part has weight 867441644 (84.78/col)
--

--
q=17M-108M (91M)

202962678 relations
48294306 duplicates and 154790328 unique relations

matrix is 10028735 x 10028960 (3624.6 MB) with weight 943951416 (94.12/col)
sparse part has weight 849864687 (84.74/col)
 --
I compared runs with the same length of the q-range, i.e. 89M, 90M and 91M. Strangely a q-min of 16M is the best choice in terms of uniques yield, while the smallest matrix was achieved with q-min of 15M.

I don't know exactly how many unique relations are required to build a matrix, but 150.7M failed, 152M succeeded.

My conclusion would be to start at q0=15M with rels_wanted of 203M-205M. Are there any more ranges or other tests that would be of interest?
bur is offline   Reply With Quote
Old 2021-09-23, 16:40   #66
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

4,999 Posts
Default

I'd like to see 10-100M and 10-105M. The CADO group reports that duplicate ratio rises meaningfully when q-max is beyond 8 * q-min. Your data is all within that ratio, but you started sieving at 10M so you have a chance to measure a range outside that ratio to see how duplicates and matrix building behave.

A faster test would be to try remdups on 15-108, 12-108. 10-108; we can do some subtraction to see how the duplicate ratio is on specifically 12-15 and 10-12 within the data set of "sieved to 108M". Those Q-ranges look much faster than higher ranges, but if there are over 50% duplicates down there (when filtered with the entire dataset) then the faster sieving is an illusion. We don't need full filtering / matrix generation runs there, just the part until "xxx raw relations, yyy unique".
VBCurtis is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Some CADO-NFS Work At Around 175-180 Decimal Digits EdH CADO-NFS 127 2020-10-07 01:47
Sigma parameter in ecm storm5510 Information & Answers 4 2019-11-30 21:32
PrimeNet error 7: Invalid parameter ksteczk PrimeNet 6 2018-03-26 15:11
Parameter Underestimation R.D. Silverman Cunningham Tables 14 2010-09-29 19:56
ECM Work and Parameter Choices R.D. Silverman Cunningham Tables 11 2006-03-06 18:46

All times are UTC. The time now is 19:42.


Fri Oct 22 19:42:59 UTC 2021 up 91 days, 14:11, 0 users, load averages: 3.19, 2.84, 2.42

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.