mersenneforum.org Some CADO-NFS Work At Around 175-180 Decimal Digits
 Register FAQ Search Today's Posts Mark Forums Read

2020-09-02, 23:36   #122
charybdis

Apr 2020

10716 Posts

Quote:
 Originally Posted by VBCurtis Sorry for the delay, been busy with some data-gathering for nfs@home queue planning. A params.C185 file should have the usual 25-30% increase in lim's, and we should test 32/32 against the current setting. If we stay with 31/32, I'd add another 20-30M relations wanted. 32/32 should be 30% higher than that to start with. Poly select should be about double the C180 file- say, 60% increase in admax and 25% increase in P.
This is what I've got for c180:

Code:
###########################################################################
# Polynomial selection
###########################################################################

###########################################################################
# Sieve
###########################################################################

# tasks.sieve.lambda1 = 3.01 ?? would match what we've done with lambda0
tasks.sieve.rels_wanted = 300000000 # for a single machine; I've been aiming for around 320M
tasks.sieve.qrange = 5000
The polyselect parameters won't be optimal, but at least they produce decent polys.
The lims probably aren't optimal either. Optimising them would probably require running the same number lots of times - easy enough at c120, but a bit of an issue at c180...

Quote:
 Edit: I'd also raise qmin to 25M or 30M. The most recent CADO-factorization paper mentions that controlling the qmax/qmin ratio helps to control the duplicate rate; so as our jobs get tougher and sieve up to larger Q's, qmin should rise as well. If I understood what they said properly (a weak assumption), a ratio of 7 is a decent target, and duplicate-rates get poor once the ratio exceeds 10. We saw that back when I suggested qmin of 500k, and their paper agrees with the data you gathered. We expect Q-max of 175-200M, I think?
Thanks for sharing this! A ratio of 7 does indeed line up well with what I found. I'll try (edit: changed a bit to reflect Curtis's draft c185.params)
Code:
tasks.I = 15
tasks.sieve.ncurves1 = 13
for the first c184, and we'll see if you're right about needing an extra 20M-30M relations. The next number can be the trial run for 32/32.

Last fiddled with by charybdis on 2020-09-02 at 23:40

 2020-09-12, 18:53 #123 charybdis     Apr 2020 1000001112 Posts First successful attempt at a matrix for the c184 from 4+3_466: Code: Sat Sep 12 17:35:10 2020 Msieve v. 1.54 (SVN 1030M) Sat Sep 12 17:35:10 2020 random seeds: 24ad7ccb b689eb64 Sat Sep 12 17:35:10 2020 factoring 1288420870966541327457423401697563128187770527956826832585494726405215165841980470637608224215821394194152554733348168114523551339058369701611634537845108590062264466133701035309124657 (184 digits) Sat Sep 12 17:35:11 2020 searching for 15-digit factors Sat Sep 12 17:35:11 2020 commencing number field sieve (184-digit input) Sat Sep 12 17:35:11 2020 R0: -400690796617504803266434417211713435 Sat Sep 12 17:35:11 2020 R1: 17433125019345016821281 Sat Sep 12 17:35:11 2020 A0: 974638980914360897045449737219395727892611072 Sat Sep 12 17:35:11 2020 A1: 4219797827063945184925953662650167666 Sat Sep 12 17:35:11 2020 A2: -494543283474842279204870729055 Sat Sep 12 17:35:11 2020 A3: -10129443552045375616276 Sat Sep 12 17:35:11 2020 A4: 34163940584578 Sat Sep 12 17:35:11 2020 A5: 124740 Sat Sep 12 17:35:11 2020 skew 101124293.72, size 6.600e-18, alpha -7.549, combined = 5.544e-14 rroots = 3 Sat Sep 12 17:35:11 2020 Sat Sep 12 17:35:11 2020 commencing relation filtering Sat Sep 12 17:35:11 2020 setting target matrix density to 100.0 Sat Sep 12 17:35:11 2020 estimated available RAM is 15845.8 MB Sat Sep 12 17:35:11 2020 commencing duplicate removal, pass 1 Sat Sep 12 18:05:22 2020 found 83600412 hash collisions in 300946431 relations Sat Sep 12 18:05:43 2020 commencing duplicate removal, pass 2 Sat Sep 12 18:11:32 2020 found 98891655 duplicates and 202054776 unique relations Sat Sep 12 18:11:32 2020 memory use: 2387.0 MB Sat Sep 12 18:11:33 2020 reading ideals above 183959552 Sat Sep 12 18:11:33 2020 commencing singleton removal, initial pass Sat Sep 12 18:27:10 2020 memory use: 5512.0 MB Sat Sep 12 18:27:11 2020 reading all ideals from disk Sat Sep 12 18:27:32 2020 memory use: 3530.8 MB Sat Sep 12 18:27:37 2020 commencing in-memory singleton removal Sat Sep 12 18:27:42 2020 begin with 202054776 relations and 195951001 unique ideals Sat Sep 12 18:28:34 2020 reduce to 91808076 relations and 69726710 ideals in 18 passes Sat Sep 12 18:28:34 2020 max relations containing the same ideal: 30 Sat Sep 12 18:28:40 2020 reading ideals above 720000 Sat Sep 12 18:28:40 2020 commencing singleton removal, initial pass Sat Sep 12 18:40:14 2020 memory use: 2756.0 MB Sat Sep 12 18:40:14 2020 reading all ideals from disk Sat Sep 12 18:40:38 2020 memory use: 3788.1 MB Sat Sep 12 18:40:45 2020 keeping 89701206 ideals with weight <= 200, target excess is 486271 Sat Sep 12 18:40:53 2020 commencing in-memory singleton removal Sat Sep 12 18:40:59 2020 begin with 91808076 relations and 89701206 unique ideals Sat Sep 12 18:42:20 2020 reduce to 91576225 relations and 89469280 ideals in 14 passes Sat Sep 12 18:42:20 2020 max relations containing the same ideal: 200 Sat Sep 12 18:42:54 2020 removing 4408357 relations and 4008357 ideals in 400000 cliques Sat Sep 12 18:42:56 2020 commencing in-memory singleton removal Sat Sep 12 18:43:02 2020 begin with 87167868 relations and 89469280 unique ideals Sat Sep 12 18:43:52 2020 reduce to 86998225 relations and 85290168 ideals in 9 passes Sat Sep 12 18:43:52 2020 max relations containing the same ideal: 199 Sat Sep 12 18:44:24 2020 removing 3296653 relations and 2896653 ideals in 400000 cliques Sat Sep 12 18:44:25 2020 commencing in-memory singleton removal Sat Sep 12 18:44:31 2020 begin with 83701572 relations and 85290168 unique ideals Sat Sep 12 18:45:14 2020 reduce to 83597784 relations and 82289128 ideals in 8 passes Sat Sep 12 18:45:14 2020 max relations containing the same ideal: 195 Sat Sep 12 18:45:45 2020 removing 2943680 relations and 2543680 ideals in 400000 cliques Sat Sep 12 18:45:46 2020 commencing in-memory singleton removal Sat Sep 12 18:45:51 2020 begin with 80654104 relations and 82289128 unique ideals Sat Sep 12 18:46:33 2020 reduce to 80566167 relations and 79657103 ideals in 8 passes Sat Sep 12 18:46:33 2020 max relations containing the same ideal: 192 Sat Sep 12 18:47:02 2020 removing 2416391 relations and 2071402 ideals in 344989 cliques Sat Sep 12 18:47:04 2020 commencing in-memory singleton removal Sat Sep 12 18:47:09 2020 begin with 78149776 relations and 79657103 unique ideals Sat Sep 12 18:47:49 2020 reduce to 78087981 relations and 77523656 ideals in 8 passes Sat Sep 12 18:47:49 2020 max relations containing the same ideal: 189 Sat Sep 12 18:48:28 2020 relations with 0 large ideals: 1755 Sat Sep 12 18:48:28 2020 relations with 1 large ideals: 1308 Sat Sep 12 18:48:28 2020 relations with 2 large ideals: 26394 Sat Sep 12 18:48:28 2020 relations with 3 large ideals: 279678 Sat Sep 12 18:48:28 2020 relations with 4 large ideals: 1664509 Sat Sep 12 18:48:28 2020 relations with 5 large ideals: 6065505 Sat Sep 12 18:48:28 2020 relations with 6 large ideals: 14102865 Sat Sep 12 18:48:28 2020 relations with 7+ large ideals: 55945967 Sat Sep 12 18:48:28 2020 commencing 2-way merge Sat Sep 12 18:49:10 2020 reduce to 47771699 relation sets and 47207374 unique ideals Sat Sep 12 18:49:10 2020 commencing full merge Sat Sep 12 19:01:29 2020 memory use: 5643.5 MB Sat Sep 12 19:01:33 2020 found 22554044 cycles, need 22541574 Sat Sep 12 19:01:39 2020 weight of 22541574 cycles is about 2254542524 (100.02/cycle) Sat Sep 12 19:01:39 2020 distribution of cycle lengths: Sat Sep 12 19:01:39 2020 1 relations: 2505630 Sat Sep 12 19:01:39 2020 2 relations: 2281308 Sat Sep 12 19:01:39 2020 3 relations: 2240340 Sat Sep 12 19:01:39 2020 4 relations: 2050168 Sat Sep 12 19:01:39 2020 5 relations: 1873938 Sat Sep 12 19:01:39 2020 6 relations: 1689463 Sat Sep 12 19:01:39 2020 7 relations: 1480064 Sat Sep 12 19:01:39 2020 8 relations: 1281871 Sat Sep 12 19:01:39 2020 9 relations: 1129919 Sat Sep 12 19:01:39 2020 10+ relations: 6008873 Sat Sep 12 19:01:39 2020 heaviest cycle: 28 relations Sat Sep 12 19:01:42 2020 commencing cycle optimization Sat Sep 12 19:02:11 2020 start with 157972026 relations Sat Sep 12 19:05:44 2020 pruned 4774693 relations Sat Sep 12 19:05:45 2020 memory use: 4817.7 MB Sat Sep 12 19:05:45 2020 distribution of cycle lengths: Sat Sep 12 19:05:45 2020 1 relations: 2505630 Sat Sep 12 19:05:45 2020 2 relations: 2342283 Sat Sep 12 19:05:45 2020 3 relations: 2330295 Sat Sep 12 19:05:45 2020 4 relations: 2110706 Sat Sep 12 19:05:45 2020 5 relations: 1931570 Sat Sep 12 19:05:45 2020 6 relations: 1721214 Sat Sep 12 19:05:45 2020 7 relations: 1502020 Sat Sep 12 19:05:45 2020 8 relations: 1291081 Sat Sep 12 19:05:45 2020 9 relations: 1129255 Sat Sep 12 19:05:45 2020 10+ relations: 5677520 Sat Sep 12 19:05:45 2020 heaviest cycle: 28 relations Sat Sep 12 19:06:23 2020 RelProcTime: 5472 Sat Sep 12 19:06:30 2020 Sat Sep 12 19:06:30 2020 commencing linear algebra Sat Sep 12 19:06:31 2020 read 22541574 cycles Sat Sep 12 19:07:06 2020 cycles contain 77557788 unique relations Sat Sep 12 19:14:45 2020 read 77557788 relations Sat Sep 12 19:16:28 2020 using 20 quadratic characters above 4294917295 Sat Sep 12 19:21:25 2020 building initial matrix Sat Sep 12 19:33:28 2020 memory use: 10853.9 MB Sat Sep 12 19:34:23 2020 read 22541574 cycles Sat Sep 12 19:34:26 2020 matrix is 22541397 x 22541574 (9232.7 MB) with weight 2833980271 (125.72/col) Sat Sep 12 19:34:26 2020 sparse part has weight 2149800401 (95.37/col) Sat Sep 12 19:37:38 2020 filtering completed in 2 passes Sat Sep 12 19:37:42 2020 matrix is 22540213 x 22540390 (9232.6 MB) with weight 2833930299 (125.73/col) Sat Sep 12 19:37:42 2020 sparse part has weight 2149790384 (95.38/col) Sat Sep 12 19:40:48 2020 matrix starts at (0, 0) Sat Sep 12 19:40:51 2020 matrix is 22540213 x 22540390 (9232.6 MB) with weight 2833930299 (125.73/col) Sat Sep 12 19:40:51 2020 sparse part has weight 2149790384 (95.38/col) Sat Sep 12 19:40:51 2020 saving the first 48 matrix rows for later Sat Sep 12 19:40:53 2020 matrix includes 64 packed rows Sat Sep 12 19:40:56 2020 matrix is 22540165 x 22540390 (9014.3 MB) with weight 2387432529 (105.92/col) Sat Sep 12 19:40:56 2020 sparse part has weight 2137643846 (94.84/col) Sat Sep 12 19:40:56 2020 using block size 8192 and superblock size 884736 for processor cache size 9216 kB Sat Sep 12 19:42:01 2020 commencing Lanczos iteration (6 threads) Sat Sep 12 19:42:01 2020 memory use: 8657.6 MB Sat Sep 12 19:43:06 2020 linear algebra at 0.0%, ETA 255h42m Sat Sep 12 19:43:26 2020 checkpointing every 90000 dimensions Increasing the lims doesn't seem to have changed the number of required relations all that much, but the matrix has got bigger. It also looks like the higher lims give a bit of a speedup; I'll provide some more data once I have a more manageable matrix.
 2020-09-13, 16:21 #124 charybdis     Apr 2020 263 Posts Sieving Q from 30M to 201M took 109.9M CPU-seconds, and produced: Code: Sun Sep 13 13:55:54 2020 commencing relation filtering Sun Sep 13 13:55:54 2020 setting target matrix density to 110.0 Sun Sep 13 13:55:54 2020 estimated available RAM is 15845.8 MB Sun Sep 13 13:55:54 2020 commencing duplicate removal, pass 1 Sun Sep 13 14:28:58 2020 found 92069979 hash collisions in 327933354 relations Sun Sep 13 14:29:20 2020 commencing duplicate removal, pass 2 Sun Sep 13 14:35:46 2020 found 109446093 duplicates and 218487261 unique relations Sun Sep 13 14:35:46 2020 memory use: 2387.0 MB Sun Sep 13 14:35:46 2020 reading ideals above 200998912 Sun Sep 13 14:35:46 2020 commencing singleton removal, initial pass Sun Sep 13 14:52:41 2020 memory use: 5512.0 MB Sun Sep 13 14:52:42 2020 reading all ideals from disk Sun Sep 13 14:53:09 2020 memory use: 3752.4 MB Sun Sep 13 14:53:14 2020 commencing in-memory singleton removal Sun Sep 13 14:53:20 2020 begin with 218487261 relations and 200826425 unique ideals ... Sun Sep 13 16:09:52 2020 matrix is 18797898 x 18798123 (8118.4 MB) with weight 2166084557 (115.23/col) Sun Sep 13 16:09:52 2020 sparse part has weight 1940221126 (103.21/col) Sun Sep 13 16:09:52 2020 using block size 8192 and superblock size 884736 for processor cache size 9216 kB Sun Sep 13 16:10:46 2020 commencing Lanczos iteration (6 threads) Sun Sep 13 16:10:46 2020 memory use: 7721.7 MB Sun Sep 13 16:11:38 2020 linear algebra at 0.0%, ETA 171h51m Higher lims don't seem to require more unique relations to build a matrix, though they do increase the duplication rate because more of the sieving is below lim1. The matrix hasn't got much larger either. Taking into account the poly scores, speedup relative to the lower lims is about 5%. I'm going to do the c183 from 4-3_443 next, as Sean's test-sieving showed that GNFS ought to be slightly faster than SNFS. I'll use lpb 32/32, mfb 60/90.
 2020-09-24, 17:40 #125 charybdis     Apr 2020 263 Posts Decided to try an early filtering run to get an idea of how many relations would be needed to build a matrix, and unexpectedly found I already had enough: Code: Thu Sep 24 13:58:53 2020 Msieve v. 1.54 (SVN 1030M) Thu Sep 24 13:58:53 2020 random seeds: ef4162b7 451c1899 Thu Sep 24 13:58:53 2020 factoring 804578163904697763240436614199214815395940531796301633839037380172577453559153269748944116985171617459275488471139711654222671571416371470668646221173373030303086767051992380630291851 (183 digits) Thu Sep 24 13:58:54 2020 searching for 15-digit factors Thu Sep 24 13:58:54 2020 commencing number field sieve (183-digit input) Thu Sep 24 13:58:54 2020 R0: -271295764750414187331730229412933346 Thu Sep 24 13:58:54 2020 R1: 1815599719426695021034001 Thu Sep 24 13:58:54 2020 A0: -157759777296013619586250253003236919241978 Thu Sep 24 13:58:54 2020 A1: 208393498976386503111847358887987939 Thu Sep 24 13:58:54 2020 A2: 1903348446157907742464167536 Thu Sep 24 13:58:54 2020 A3: -28655357853458133620365 Thu Sep 24 13:58:54 2020 A4: -145343027262384 Thu Sep 24 13:58:54 2020 A5: 59126760 Thu Sep 24 13:58:54 2020 skew 5738604.63, size 5.111e-18, alpha -6.698, combined = 4.768e-14 rroots = 5 Thu Sep 24 13:58:54 2020 Thu Sep 24 13:58:54 2020 commencing relation filtering Thu Sep 24 13:58:54 2020 setting target matrix density to 100.0 Thu Sep 24 13:58:54 2020 estimated available RAM is 15845.8 MB Thu Sep 24 13:58:54 2020 commencing duplicate removal, pass 1 ... Thu Sep 24 14:37:47 2020 found 96535007 hash collisions in 371534711 relations Thu Sep 24 14:38:09 2020 added 121654 free relations Thu Sep 24 14:38:09 2020 commencing duplicate removal, pass 2 Thu Sep 24 14:46:01 2020 found 108284827 duplicates and 263371538 unique relations Thu Sep 24 14:46:01 2020 memory use: 2387.0 MB Thu Sep 24 14:46:01 2020 reading ideals above 186974208 Thu Sep 24 14:46:01 2020 commencing singleton removal, initial pass Thu Sep 24 15:06:32 2020 memory use: 6024.0 MB Thu Sep 24 15:06:32 2020 reading all ideals from disk Thu Sep 24 15:07:11 2020 memory use: 4787.5 MB Thu Sep 24 15:07:19 2020 commencing in-memory singleton removal Thu Sep 24 15:07:26 2020 begin with 263371538 relations and 255561661 unique ideals Thu Sep 24 15:08:37 2020 reduce to 123121291 relations and 95945973 ideals in 17 passes Thu Sep 24 15:08:37 2020 max relations containing the same ideal: 35 Thu Sep 24 15:08:46 2020 reading ideals above 720000 Thu Sep 24 15:08:46 2020 commencing singleton removal, initial pass Thu Sep 24 15:24:11 2020 memory use: 3012.0 MB Thu Sep 24 15:24:12 2020 reading all ideals from disk Thu Sep 24 15:24:56 2020 memory use: 5110.3 MB Thu Sep 24 15:25:06 2020 keeping 116085117 ideals with weight <= 200, target excess is 643178 Thu Sep 24 15:25:17 2020 commencing in-memory singleton removal Thu Sep 24 15:25:25 2020 begin with 123121291 relations and 116085117 unique ideals Thu Sep 24 15:27:04 2020 reduce to 123023063 relations and 115986878 ideals in 13 passes Thu Sep 24 15:27:04 2020 max relations containing the same ideal: 200 Thu Sep 24 15:27:50 2020 removing 8828550 relations and 7828550 ideals in 1000000 cliques Thu Sep 24 15:27:52 2020 commencing in-memory singleton removal Thu Sep 24 15:28:00 2020 begin with 114194513 relations and 115986878 unique ideals Thu Sep 24 15:29:18 2020 reduce to 113693675 relations and 107651902 ideals in 11 passes Thu Sep 24 15:29:18 2020 max relations containing the same ideal: 198 Thu Sep 24 15:30:00 2020 removing 6670193 relations and 5670193 ideals in 1000000 cliques Thu Sep 24 15:30:03 2020 commencing in-memory singleton removal Thu Sep 24 15:30:10 2020 begin with 107023482 relations and 107651902 unique ideals Thu Sep 24 15:31:10 2020 reduce to 106698207 relations and 101653297 ideals in 9 passes Thu Sep 24 15:31:10 2020 max relations containing the same ideal: 193 Thu Sep 24 15:31:49 2020 removing 6014282 relations and 5014282 ideals in 1000000 cliques Thu Sep 24 15:31:52 2020 commencing in-memory singleton removal Thu Sep 24 15:31:58 2020 begin with 100683925 relations and 101653297 unique ideals Thu Sep 24 15:32:55 2020 reduce to 100396164 relations and 96348603 ideals in 9 passes Thu Sep 24 15:32:55 2020 max relations containing the same ideal: 184 Thu Sep 24 15:33:33 2020 removing 5675031 relations and 4675031 ideals in 1000000 cliques Thu Sep 24 15:33:35 2020 commencing in-memory singleton removal Thu Sep 24 15:33:41 2020 begin with 94721133 relations and 96348603 unique ideals Thu Sep 24 15:34:28 2020 reduce to 94444992 relations and 91394891 ideals in 8 passes Thu Sep 24 15:34:28 2020 max relations containing the same ideal: 180 Thu Sep 24 15:35:03 2020 removing 5474215 relations and 4474215 ideals in 1000000 cliques Thu Sep 24 15:35:05 2020 commencing in-memory singleton removal Thu Sep 24 15:35:11 2020 begin with 88970777 relations and 91394891 unique ideals Thu Sep 24 15:36:01 2020 reduce to 88694416 relations and 86641692 ideals in 9 passes Thu Sep 24 15:36:01 2020 max relations containing the same ideal: 171 Thu Sep 24 15:36:35 2020 removing 5350364 relations and 4350365 ideals in 1000000 cliques Thu Sep 24 15:36:37 2020 commencing in-memory singleton removal Thu Sep 24 15:36:42 2020 begin with 83344052 relations and 86641692 unique ideals Thu Sep 24 15:37:28 2020 reduce to 83062640 relations and 82007106 ideals in 9 passes Thu Sep 24 15:37:28 2020 max relations containing the same ideal: 165 Thu Sep 24 15:37:59 2020 removing 2035095 relations and 1725648 ideals in 309447 cliques Thu Sep 24 15:38:01 2020 commencing in-memory singleton removal Thu Sep 24 15:38:06 2020 begin with 81027545 relations and 82007106 unique ideals Thu Sep 24 15:38:36 2020 reduce to 80987675 relations and 80241438 ideals in 6 passes Thu Sep 24 15:38:36 2020 max relations containing the same ideal: 163 Thu Sep 24 15:39:17 2020 relations with 0 large ideals: 2387 Thu Sep 24 15:39:17 2020 relations with 1 large ideals: 2644 Thu Sep 24 15:39:17 2020 relations with 2 large ideals: 48967 Thu Sep 24 15:39:17 2020 relations with 3 large ideals: 476525 Thu Sep 24 15:39:17 2020 relations with 4 large ideals: 2541197 Thu Sep 24 15:39:17 2020 relations with 5 large ideals: 8234652 Thu Sep 24 15:39:17 2020 relations with 6 large ideals: 16947934 Thu Sep 24 15:39:17 2020 relations with 7+ large ideals: 52733369 Thu Sep 24 15:39:17 2020 commencing 2-way merge Thu Sep 24 15:40:00 2020 reduce to 49697082 relation sets and 48950844 unique ideals Thu Sep 24 15:40:00 2020 commencing full merge Thu Sep 24 15:51:57 2020 memory use: 5812.0 MB Thu Sep 24 15:52:01 2020 found 23504201 cycles, need 23427044 Thu Sep 24 15:52:07 2020 weight of 23427044 cycles is about 2342739929 (100.00/cycle) Thu Sep 24 15:52:07 2020 distribution of cycle lengths: Thu Sep 24 15:52:07 2020 1 relations: 2500861 Thu Sep 24 15:52:07 2020 2 relations: 2225344 Thu Sep 24 15:52:07 2020 3 relations: 2213648 Thu Sep 24 15:52:07 2020 4 relations: 2063640 Thu Sep 24 15:52:07 2020 5 relations: 1936571 Thu Sep 24 15:52:07 2020 6 relations: 1794380 Thu Sep 24 15:52:07 2020 7 relations: 1614061 Thu Sep 24 15:52:07 2020 8 relations: 1433750 Thu Sep 24 15:52:07 2020 9 relations: 1293116 Thu Sep 24 15:52:07 2020 10+ relations: 6351673 Thu Sep 24 15:52:07 2020 heaviest cycle: 27 relations Thu Sep 24 15:52:11 2020 commencing cycle optimization Thu Sep 24 15:52:40 2020 start with 162941027 relations Thu Sep 24 15:56:11 2020 pruned 4844744 relations Thu Sep 24 15:56:12 2020 memory use: 4991.4 MB Thu Sep 24 15:56:12 2020 distribution of cycle lengths: Thu Sep 24 15:56:12 2020 1 relations: 2500861 Thu Sep 24 15:56:12 2020 2 relations: 2281779 Thu Sep 24 15:56:12 2020 3 relations: 2297419 Thu Sep 24 15:56:12 2020 4 relations: 2125272 Thu Sep 24 15:56:12 2020 5 relations: 1998705 Thu Sep 24 15:56:12 2020 6 relations: 1832292 Thu Sep 24 15:56:12 2020 7 relations: 1647060 Thu Sep 24 15:56:12 2020 8 relations: 1454221 Thu Sep 24 15:56:12 2020 9 relations: 1305171 Thu Sep 24 15:56:12 2020 10+ relations: 5984264 Thu Sep 24 15:56:12 2020 heaviest cycle: 27 relations Thu Sep 24 15:56:51 2020 RelProcTime: 7077 Thu Sep 24 15:56:59 2020 Thu Sep 24 15:56:59 2020 commencing linear algebra Thu Sep 24 15:57:01 2020 read 23427044 cycles Thu Sep 24 15:57:36 2020 cycles contain 80452255 unique relations Thu Sep 24 16:06:39 2020 read 80452255 relations Thu Sep 24 16:08:27 2020 using 20 quadratic characters above 4294917295 Thu Sep 24 16:13:36 2020 building initial matrix Thu Sep 24 16:26:16 2020 memory use: 11291.7 MB Thu Sep 24 16:27:47 2020 read 23427044 cycles Thu Sep 24 16:27:50 2020 matrix is 23426866 x 23427044 (9592.6 MB) with weight 2929149922 (125.03/col) Thu Sep 24 16:27:50 2020 sparse part has weight 2233524910 (95.34/col) Thu Sep 24 16:31:09 2020 filtering completed in 2 passes Thu Sep 24 16:31:13 2020 matrix is 23423858 x 23424036 (9592.4 MB) with weight 2929033628 (125.04/col) Thu Sep 24 16:31:13 2020 sparse part has weight 2233502594 (95.35/col) Thu Sep 24 16:34:27 2020 matrix starts at (0, 0) Thu Sep 24 16:34:30 2020 matrix is 23423858 x 23424036 (9592.4 MB) with weight 2929033628 (125.04/col) Thu Sep 24 16:34:30 2020 sparse part has weight 2233502594 (95.35/col) Thu Sep 24 16:34:30 2020 saving the first 48 matrix rows for later Thu Sep 24 16:34:33 2020 matrix includes 64 packed rows Thu Sep 24 16:34:36 2020 matrix is 23423810 x 23424036 (9296.0 MB) with weight 2454035299 (104.77/col) Thu Sep 24 16:34:36 2020 sparse part has weight 2202649119 (94.03/col) Thu Sep 24 16:34:36 2020 using block size 8192 and superblock size 884736 for processor cache size 9216 kB Thu Sep 24 16:35:39 2020 commencing Lanczos iteration (6 threads) Thu Sep 24 16:35:40 2020 memory use: 8970.1 MB Thu Sep 24 16:36:45 2020 linear algebra at 0.0%, ETA 265h12m Thu Sep 24 16:37:06 2020 checkpointing every 90000 dimensions I was caught out by the fact that the duplication rate is substantially lower than it was on the 31/32 runs; Curtis, you were right about 32/32 needing 30% more *unique* relations than 31/32. I can't immediately see why raising the lpb on the rational side while sieving on the algebraic side ought to have anything to do with the duplication rate, but I'm not a number theorist so I'd appreciate it if someone could explain this. It's looking like 32/32 is a win at this size. As usual I'll have more details once I've got the matrix size down.
2020-09-24, 21:56   #126
VBCurtis

"Curtis"
Feb 2005
Riverside, CA

13×367 Posts

Quote:
 Originally Posted by charybdis I was caught out by the fact that the duplication rate is substantially lower than it was on the 31/32 runs; Curtis, you were right about 32/32 needing 30% more *unique* relations than 31/32. I can't immediately see why raising the lpb on the rational side while sieving on the algebraic side ought to have anything to do with the duplication rate, but I'm not a number theorist so I'd appreciate it if someone could explain this. It's looking like 32/32 is a win at this size. As usual I'll have more details once I've got the matrix size down.
There's a strong chance it's just a lucky polynomial with better-than-avg duplication rate. I could see a small improvement in unique-to-raw ratio if this run needed a smaller Q range (i.e. much better yield), but otherwise I wager it's just luck.

I'm afraid the tradeoff of faster sieving vs larger matrix gets worse if we go above 32LP; but I also believe 32/33 is fastest at C193 (I helped someone off the forum run ~10 jobs at that size, we did a bunch of work on parameters). On this job, I hope that tradeoff proves worth it.

2020-09-25, 17:22   #127
charybdis

Apr 2020

26310 Posts

110.4M CPU-seconds of sieving, from 30M to 207M, gave:
Code:
Fri Sep 25 13:16:07 2020  commencing relation filtering
Fri Sep 25 13:16:07 2020  setting target matrix density to 110.0
...
Fri Sep 25 13:59:11 2020  found 108139037 hash collisions in 408503325 relations
Fri Sep 25 13:59:32 2020  commencing duplicate removal, pass 2
Fri Sep 25 14:08:03 2020  found 121841205 duplicates and 286662120 unique relations
Fri Sep 25 14:08:03 2020  memory use: 2387.0 MB
Fri Sep 25 14:08:04 2020  reading ideals above 207028224
Fri Sep 25 14:08:04 2020  commencing singleton removal, initial pass
Fri Sep 25 14:30:25 2020  memory use: 6024.0 MB
Fri Sep 25 14:30:26 2020  reading all ideals from disk
Fri Sep 25 14:31:11 2020  memory use: 5114.7 MB
Fri Sep 25 14:31:19 2020  commencing in-memory singleton removal
Fri Sep 25 14:31:27 2020  begin with 286662120 relations and 262594881 unique ideals
...
Fri Sep 25 16:00:53 2020  matrix is 19772369 x 19772594 (8473.1 MB) with weight 2251213167 (113.86/col)
Fri Sep 25 16:00:53 2020  sparse part has weight 2023439071 (102.34/col)
Fri Sep 25 16:00:53 2020  using block size 8192 and superblock size 884736 for processor cache size 9216 kB
Fri Sep 25 16:01:50 2020  commencing Lanczos iteration (6 threads)
Fri Sep 25 16:01:50 2020  memory use: 8099.3 MB
Fri Sep 25 16:02:44 2020  linear algebra at 0.0%, ETA 188h45m
Quote:
 Originally Posted by VBCurtis There's a strong chance it's just a lucky polynomial with better-than-avg duplication rate. I could see a small improvement in unique-to-raw ratio if this run needed a smaller Q range (i.e. much better yield), but otherwise I wager it's just luck.
The polynomial score actually isn't great: 4.768e-14 for a large c183, compared to 5.544e-14 for the c184 I ran previously. It did get an unusually large boost of around 3% from the re-scoring that CADO runs at the end of polyselect, which took it up from 4th to 1st in the list of best polys - though even adding 3% to 4.768e-14 only gets you up to 4.911e-14.

If we assume that this poly is indeed ~13% worse than the c184 poly, then 32/32 is a clear win. But if the low duplication rate wasn't picked up by the poly score, then this might not be a fair comparison, so I think another run with identical parameters is in order.

 2020-10-07, 01:47 #128 charybdis     Apr 2020 4078 Posts c184 32/32 run: sieving 30M-205M took 112M CPU-seconds and gave Code: Tue Oct 6 23:43:20 2020 Msieve v. 1.54 (SVN 1030M) Tue Oct 6 23:43:20 2020 random seeds: 7a19fd23 910545fb Tue Oct 6 23:43:20 2020 factoring 1926769016895629095108654427662805571730828819453815038321063771589859906145101426691203081858286347255498278743308679252628417992530572751984914065156690326565318892677376743855798161 (184 digits) Tue Oct 6 23:43:20 2020 searching for 15-digit factors Tue Oct 6 23:43:20 2020 commencing number field sieve (184-digit input) Tue Oct 6 23:43:20 2020 R0: -232800310647131682476873772810105422 Tue Oct 6 23:43:20 2020 R1: 90317080737385565963549 Tue Oct 6 23:43:20 2020 A0: 2439943766309753461417671509774240145609855 Tue Oct 6 23:43:20 2020 A1: 3109448214836060753130916105182937413 Tue Oct 6 23:43:20 2020 A2: -51506391362834289409336135888 Tue Oct 6 23:43:20 2020 A3: -20569441357157925834233 Tue Oct 6 23:43:20 2020 A4: 80188144206473 Tue Oct 6 23:43:20 2020 A5: 2817780 Tue Oct 6 23:43:20 2020 skew 20877130.61, size 5.540e-18, alpha -6.572, combined = 4.934e-14 rroots = 5 Tue Oct 6 23:43:20 2020 Tue Oct 6 23:43:20 2020 commencing relation filtering Tue Oct 6 23:43:20 2020 setting target matrix density to 110.0 Tue Oct 6 23:43:20 2020 estimated available RAM is 15845.8 MB Tue Oct 6 23:43:20 2020 commencing duplicate removal, pass 1 Wed Oct 7 00:24:51 2020 found 114260066 hash collisions in 416382602 relations Wed Oct 7 00:25:13 2020 commencing duplicate removal, pass 2 Wed Oct 7 00:33:42 2020 found 132138239 duplicates and 284244363 unique relations Wed Oct 7 00:33:42 2020 memory use: 2387.0 MB Wed Oct 7 00:33:42 2020 reading ideals above 205193216 Wed Oct 7 00:33:42 2020 commencing singleton removal, initial pass Wed Oct 7 00:55:32 2020 memory use: 6024.0 MB Wed Oct 7 00:55:33 2020 reading all ideals from disk Wed Oct 7 00:56:13 2020 memory use: 5093.3 MB Wed Oct 7 00:56:21 2020 commencing in-memory singleton removal Wed Oct 7 00:56:29 2020 begin with 284244363 relations and 262016506 unique ideals ... Wed Oct 7 02:25:05 2020 matrix is 20333308 x 20333533 (8742.9 MB) with weight 2327221335 (114.45/col) Wed Oct 7 02:25:05 2020 sparse part has weight 2088575957 (102.72/col) Wed Oct 7 02:25:05 2020 using block size 8192 and superblock size 884736 for processor cache size 9216 kB Wed Oct 7 02:26:04 2020 commencing Lanczos iteration (6 threads) Wed Oct 7 02:26:04 2020 memory use: 8351.8 MB Wed Oct 7 02:27:01 2020 linear algebra at 0.0%, ETA 204h37m Duplication rate is more normal after the anomalous poly on the previous job, but 32/32 still looks like a win: compared to the last 31/32 job, sieving ran just a couple percent slower despite the poly scoring 12% worse. Even if we insist on a bit of oversieving with 32/32 to compensate for the larger matrices, that should only add a few more percent to the sieving time. I suppose there's a chance 32/32 is already better at c180, so I ought to test that at some point. For now, though, the mfb bounds could do with some experimentation.

 Similar Threads Thread Thread Starter Forum Replies Last Post enzocreti enzocreti 1 2020-03-03 18:38 tuckerkao Miscellaneous Math 2 2020-02-16 06:23 Nick Puzzles 9 2013-02-13 17:17 vsuite GPU Computing 11 2011-02-02 04:47 Corbyguy Software 3 2008-06-09 18:09

All times are UTC. The time now is 10:07.

Fri May 7 10:07:36 UTC 2021 up 29 days, 4:48, 0 users, load averages: 2.86, 3.03, 2.81