mersenneforum.org Some CADO-NFS Work At Around 175-180 Decimal Digits
 Register FAQ Search Today's Posts Mark Forums Read

2020-04-16, 21:23   #34
charybdis

Apr 2020

2·3·19 Posts

Quote:
 Originally Posted by VBCurtis Hrmmmmm..... I've been using the screen display "total sieving time" that pops up at the end of the sieve phase, and matches the end-of-factorization summary printed to screen. I can't say I delve into the log file hardly ever. Ed's timing falls into my curve-fitting of time vs input size from the other jobs I listed, so I am quite confused. I appreciate you pointing out the inconsistency in the log report!
Did you also use the "total sieving time" to get the times for some of the other jobs you mentioned? If so that might explain the discrepancy.

Here are a couple of sample lines from an old completed job, as an example:
Code:
Info:Lattice Sieving: Total time: 403109s
...
Info:Complete Factorization / Discrete logarithm: Total cpu/elapsed time for entire factorization: 862194/16096.8
The second one seems to give the correct CPU time, whereas the first is just client-seconds; the sieving for this job took just over 2 hours wall clock time on ~110 cores, which is consistent with ~800k CPU-seconds for sieving plus a little extra for polyselect + postprocessing.

2020-04-16, 21:48   #35
EdH

"Ed Hall"
Dec 2009

1101011111002 Posts

Quote:
 Originally Posted by VBCurtis Hrmmmmm..... I've been using the screen display "total sieving time" that pops up at the end of the sieve phase, and matches the end-of-factorization summary printed to screen. I can't say I delve into the log file hardly ever. Ed's timing falls into my curve-fitting of time vs input size from the other jobs I listed, so I am quite confused. I appreciate you pointing out the inconsistency in the log report!
Here are the significant last lines from the screen run to 270M:
Code:
Info:Lattice Sieving: Reached target of 270000000 relations, now have 270013333
Info:Lattice Sieving: Aggregate statistics:
Info:Lattice Sieving: Total number of relations: 270013333
Info:Lattice Sieving: Average J: 16023.9 for 5345903 special-q, max bucket fill -bkmult 1.0,1s:1.416720
Info:Lattice Sieving: Total time: 2.19772e+07s
Info:Filtering - Duplicate Removal, splitting pass: Stopping at duplicates1

 2020-04-16, 21:51 #36 EdH     "Ed Hall" Dec 2009 Adirondack Mtns 65748 Posts t_d=100 will build a matrix: Code: Thu Apr 16 12:15:32 2020 commencing relation filtering Thu Apr 16 12:15:32 2020 setting target matrix density to 100.0 Thu Apr 16 12:15:32 2020 estimated available RAM is 15926.6 MB Thu Apr 16 12:15:32 2020 commencing duplicate removal, pass 1 Thu Apr 16 12:15:34 2020 error -1 reading relation 189590 . . . Thu Apr 16 12:49:54 2020 error -1 reading relation 267865222 Thu Apr 16 12:50:24 2020 found 88770665 hash collisions in 271761268 relations Thu Apr 16 12:50:56 2020 added 122298 free relations Thu Apr 16 12:50:56 2020 commencing duplicate removal, pass 2 Thu Apr 16 12:57:26 2020 found 125086447 duplicates and 146797119 unique relations Thu Apr 16 12:57:26 2020 memory use: 1449.5 MB Thu Apr 16 12:57:26 2020 reading ideals above 139919360 Thu Apr 16 12:57:26 2020 commencing singleton removal, initial pass Thu Apr 16 13:13:00 2020 memory use: 3012.0 MB Thu Apr 16 13:13:00 2020 reading all ideals from disk Thu Apr 16 13:13:19 2020 memory use: 2357.7 MB Thu Apr 16 13:13:24 2020 commencing in-memory singleton removal Thu Apr 16 13:13:28 2020 begin with 146797119 relations and 142617272 unique ideals Thu Apr 16 13:14:09 2020 reduce to 56506171 relations and 38799310 ideals in 18 passes Thu Apr 16 13:14:09 2020 max relations containing the same ideal: 21 Thu Apr 16 13:14:12 2020 reading ideals above 720000 Thu Apr 16 13:14:12 2020 commencing singleton removal, initial pass Thu Apr 16 13:24:49 2020 memory use: 1506.0 MB Thu Apr 16 13:24:49 2020 reading all ideals from disk Thu Apr 16 13:25:08 2020 memory use: 2241.3 MB Thu Apr 16 13:25:14 2020 keeping 54258109 ideals with weight <= 200, target excess is 313347 Thu Apr 16 13:25:20 2020 commencing in-memory singleton removal Thu Apr 16 13:25:25 2020 begin with 56506171 relations and 54258109 unique ideals Thu Apr 16 13:26:29 2020 reduce to 56030216 relations and 53781539 ideals in 13 passes Thu Apr 16 13:26:29 2020 max relations containing the same ideal: 200 Thu Apr 16 13:26:55 2020 removing 3684525 relations and 3284525 ideals in 400000 cliques Thu Apr 16 13:26:57 2020 commencing in-memory singleton removal Thu Apr 16 13:27:01 2020 begin with 52345691 relations and 53781539 unique ideals Thu Apr 16 13:27:47 2020 reduce to 52174610 relations and 50324306 ideals in 10 passes Thu Apr 16 13:27:47 2020 max relations containing the same ideal: 197 Thu Apr 16 13:28:11 2020 removing 2772195 relations and 2372195 ideals in 400000 cliques Thu Apr 16 13:28:13 2020 commencing in-memory singleton removal Thu Apr 16 13:28:17 2020 begin with 49402415 relations and 50324306 unique ideals Thu Apr 16 13:28:55 2020 reduce to 49291975 relations and 47840809 ideals in 9 passes Thu Apr 16 13:28:55 2020 max relations containing the same ideal: 190 Thu Apr 16 13:29:19 2020 removing 2488158 relations and 2088158 ideals in 400000 cliques Thu Apr 16 13:29:20 2020 commencing in-memory singleton removal Thu Apr 16 13:29:24 2020 begin with 46803817 relations and 47840809 unique ideals Thu Apr 16 13:30:00 2020 reduce to 46708746 relations and 45656840 ideals in 9 passes Thu Apr 16 13:30:00 2020 max relations containing the same ideal: 185 Thu Apr 16 13:30:22 2020 removing 2334687 relations and 1934687 ideals in 400000 cliques Thu Apr 16 13:30:23 2020 commencing in-memory singleton removal Thu Apr 16 13:30:27 2020 begin with 44374059 relations and 45656840 unique ideals Thu Apr 16 13:30:57 2020 reduce to 44283467 relations and 43630836 ideals in 8 passes Thu Apr 16 13:30:57 2020 max relations containing the same ideal: 182 Thu Apr 16 13:31:18 2020 removing 1701806 relations and 1412658 ideals in 289148 cliques Thu Apr 16 13:31:19 2020 commencing in-memory singleton removal Thu Apr 16 13:31:23 2020 begin with 42581661 relations and 43630836 unique ideals Thu Apr 16 13:31:52 2020 reduce to 42532132 relations and 42168363 ideals in 8 passes Thu Apr 16 13:31:52 2020 max relations containing the same ideal: 176 Thu Apr 16 13:32:19 2020 relations with 0 large ideals: 1038 Thu Apr 16 13:32:19 2020 relations with 1 large ideals: 1550 Thu Apr 16 13:32:19 2020 relations with 2 large ideals: 21593 Thu Apr 16 13:32:19 2020 relations with 3 large ideals: 198057 Thu Apr 16 13:32:19 2020 relations with 4 large ideals: 1072004 Thu Apr 16 13:32:19 2020 relations with 5 large ideals: 3623143 Thu Apr 16 13:32:19 2020 relations with 6 large ideals: 7969344 Thu Apr 16 13:32:19 2020 relations with 7+ large ideals: 29645403 Thu Apr 16 13:32:19 2020 commencing 2-way merge Thu Apr 16 13:32:48 2020 reduce to 25695897 relation sets and 25332128 unique ideals Thu Apr 16 13:32:48 2020 commencing full merge Thu Apr 16 13:41:06 2020 memory use: 3081.6 MB Thu Apr 16 13:41:09 2020 found 12276374 cycles, need 12244328 Thu Apr 16 13:41:12 2020 weight of 12244328 cycles is about 1224870286 (100.04/cycle) Thu Apr 16 13:41:12 2020 distribution of cycle lengths: Thu Apr 16 13:41:12 2020 1 relations: 1128304 Thu Apr 16 13:41:12 2020 2 relations: 1130284 Thu Apr 16 13:41:12 2020 3 relations: 1190346 Thu Apr 16 13:41:12 2020 4 relations: 1117203 Thu Apr 16 13:41:12 2020 5 relations: 1062965 Thu Apr 16 13:41:12 2020 6 relations: 969179 Thu Apr 16 13:41:12 2020 7 relations: 868868 Thu Apr 16 13:41:12 2020 8 relations: 766062 Thu Apr 16 13:41:12 2020 9 relations: 679349 Thu Apr 16 13:41:12 2020 10+ relations: 3331768 Thu Apr 16 13:41:12 2020 heaviest cycle: 28 relations Thu Apr 16 13:41:15 2020 commencing cycle optimization Thu Apr 16 13:41:35 2020 start with 86442588 relations Thu Apr 16 13:43:59 2020 pruned 2629604 relations Thu Apr 16 13:44:00 2020 memory use: 2639.1 MB Thu Apr 16 13:44:00 2020 distribution of cycle lengths: Thu Apr 16 13:44:00 2020 1 relations: 1128304 Thu Apr 16 13:44:00 2020 2 relations: 1158969 Thu Apr 16 13:44:00 2020 3 relations: 1235456 Thu Apr 16 13:44:00 2020 4 relations: 1152269 Thu Apr 16 13:44:00 2020 5 relations: 1097163 Thu Apr 16 13:44:00 2020 6 relations: 991876 Thu Apr 16 13:44:00 2020 7 relations: 886776 Thu Apr 16 13:44:00 2020 8 relations: 775313 Thu Apr 16 13:44:00 2020 9 relations: 683433 Thu Apr 16 13:44:00 2020 10+ relations: 3134769 Thu Apr 16 13:44:00 2020 heaviest cycle: 27 relations Thu Apr 16 13:44:19 2020 RelProcTime: 5327 Thu Apr 16 13:44:24 2020 Thu Apr 16 13:44:24 2020 commencing linear algebra Thu Apr 16 13:44:25 2020 read 12244328 cycles Thu Apr 16 13:44:47 2020 cycles contain 42246024 unique relations Thu Apr 16 13:51:48 2020 read 42246024 relations Thu Apr 16 13:52:54 2020 using 20 quadratic characters above 4294917295 Thu Apr 16 13:55:56 2020 building initial matrix Thu Apr 16 14:03:01 2020 memory use: 5922.5 MB Thu Apr 16 14:03:06 2020 read 12244328 cycles Thu Apr 16 14:03:09 2020 matrix is 12244151 x 12244328 (5011.5 MB) with weight 1544672270 (126.15/col) Thu Apr 16 14:03:09 2020 sparse part has weight 1166814329 (95.29/col) Thu Apr 16 14:05:56 2020 filtering completed in 2 passes Thu Apr 16 14:05:59 2020 matrix is 12243059 x 12243236 (5011.5 MB) with weight 1544627458 (126.16/col) Thu Apr 16 14:05:59 2020 sparse part has weight 1166804691 (95.30/col) Thu Apr 16 14:07:01 2020 matrix starts at (0, 0) Thu Apr 16 14:07:04 2020 matrix is 12243059 x 12243236 (5011.5 MB) with weight 1544627458 (126.16/col) Thu Apr 16 14:07:04 2020 sparse part has weight 1166804691 (95.30/col) Thu Apr 16 14:07:04 2020 saving the first 48 matrix rows for later Thu Apr 16 14:07:06 2020 matrix includes 64 packed rows Thu Apr 16 14:07:08 2020 matrix is 12243011 x 12243236 (4887.3 MB) with weight 1292507191 (105.57/col) Thu Apr 16 14:07:08 2020 sparse part has weight 1158749915 (94.64/col) Thu Apr 16 14:07:08 2020 using block size 8192 and superblock size 786432 for processor cache size 8192 kB Thu Apr 16 14:08:03 2020 commencing Lanczos iteration (8 threads) Thu Apr 16 14:08:03 2020 memory use: 4077.8 MB Thu Apr 16 14:08:55 2020 linear algebra at 0.0%, ETA 112h 0m
 2020-04-17, 00:33 #37 VBCurtis     "Curtis" Feb 2005 Riverside, CA 3·5·13·23 Posts 12.2M at TD 100 seems pretty good; so maybe just 10-15M more relations will get you TD 120 and 20% faster matrix time.
 2020-04-17, 16:41 #38 EdH     "Ed Hall" Dec 2009 Adirondack Mtns 22×863 Posts t_d=120 success: Code: Fri Apr 17 10:03:58 2020 Msieve v. 1.54 (SVN 1018) Fri Apr 17 10:03:58 2020 random seeds: 947503a5 a978d24a Fri Apr 17 10:03:58 2020 factoring 76552370139504036674890813564032281493867343366619508594816489005834882856199128873928842970710045044111574726594936894404957063604759585302342441093226844531070349677623657609 (176 digits) Fri Apr 17 10:03:59 2020 searching for 15-digit factors Fri Apr 17 10:03:59 2020 commencing number field sieve (176-digit input) Fri Apr 17 10:03:59 2020 R0: -10749206376460432970317818596117873 Fri Apr 17 10:03:59 2020 R1: 4023609444811856477743 Fri Apr 17 10:03:59 2020 A0: 91389778824609164214454779424151524400880 Fri Apr 17 10:03:59 2020 A1: 16573333756774205759678902993899502 Fri Apr 17 10:03:59 2020 A2: -8753197000583595457254903663 Fri Apr 17 10:03:59 2020 A3: -1186820920867031701728 Fri Apr 17 10:03:59 2020 A4: 77519198521772 Fri Apr 17 10:03:59 2020 A5: 533400 Fri Apr 17 10:03:59 2020 skew 1.00, size 3.822e-17, alpha -6.645, combined = 8.280e-16 rroots = 5 Fri Apr 17 10:03:59 2020 Fri Apr 17 10:03:59 2020 commencing relation filtering Fri Apr 17 10:03:59 2020 setting target matrix density to 120.0 Fri Apr 17 10:03:59 2020 estimated available RAM is 15926.6 MB Fri Apr 17 10:03:59 2020 commencing duplicate removal, pass 1 Fri Apr 17 10:04:01 2020 error -1 reading relation 160728 . . . Fri Apr 17 10:42:16 2020 error -1 reading relation 294036055 Fri Apr 17 10:42:19 2020 found 96785463 hash collisions in 294359053 relations Fri Apr 17 10:42:51 2020 added 122298 free relations Fri Apr 17 10:42:51 2020 commencing duplicate removal, pass 2 Fri Apr 17 10:49:52 2020 found 136725894 duplicates and 157755457 unique relations Fri Apr 17 10:49:52 2020 memory use: 2387.0 MB Fri Apr 17 10:49:52 2020 reading ideals above 139919360 Fri Apr 17 10:49:52 2020 commencing singleton removal, initial pass Fri Apr 17 11:06:57 2020 memory use: 3012.0 MB Fri Apr 17 11:06:57 2020 reading all ideals from disk Fri Apr 17 11:07:13 2020 memory use: 2535.2 MB Fri Apr 17 11:07:18 2020 commencing in-memory singleton removal Fri Apr 17 11:07:22 2020 begin with 157755457 relations and 148007529 unique ideals Fri Apr 17 11:08:04 2020 reduce to 67900523 relations and 45789557 ideals in 15 passes Fri Apr 17 11:08:04 2020 max relations containing the same ideal: 22 Fri Apr 17 11:08:08 2020 reading ideals above 720000 Fri Apr 17 11:08:08 2020 commencing singleton removal, initial pass Fri Apr 17 11:20:26 2020 memory use: 1506.0 MB Fri Apr 17 11:20:26 2020 reading all ideals from disk Fri Apr 17 11:20:48 2020 memory use: 2698.7 MB Fri Apr 17 11:20:55 2020 keeping 61205493 ideals with weight <= 200, target excess is 380214 Fri Apr 17 11:21:02 2020 commencing in-memory singleton removal Fri Apr 17 11:21:09 2020 begin with 67900523 relations and 61205493 unique ideals Fri Apr 17 11:22:25 2020 reduce to 67676809 relations and 60981638 ideals in 13 passes Fri Apr 17 11:22:25 2020 max relations containing the same ideal: 200 Fri Apr 17 11:22:56 2020 removing 6407085 relations and 5407085 ideals in 1000000 cliques Fri Apr 17 11:22:59 2020 commencing in-memory singleton removal Fri Apr 17 11:23:04 2020 begin with 61269724 relations and 60981638 unique ideals Fri Apr 17 11:23:52 2020 reduce to 60870327 relations and 55168052 ideals in 9 passes Fri Apr 17 11:23:52 2020 max relations containing the same ideal: 194 Fri Apr 17 11:24:20 2020 removing 4932237 relations and 3932237 ideals in 1000000 cliques Fri Apr 17 11:24:22 2020 commencing in-memory singleton removal Fri Apr 17 11:24:27 2020 begin with 55938090 relations and 55168052 unique ideals Fri Apr 17 11:25:05 2020 reduce to 55656169 relations and 50949518 ideals in 8 passes Fri Apr 17 11:25:05 2020 max relations containing the same ideal: 182 Fri Apr 17 11:25:31 2020 removing 4493365 relations and 3493365 ideals in 1000000 cliques Fri Apr 17 11:25:33 2020 commencing in-memory singleton removal Fri Apr 17 11:25:37 2020 begin with 51162804 relations and 50949518 unique ideals Fri Apr 17 11:26:12 2020 reduce to 50901236 relations and 47190406 ideals in 8 passes Fri Apr 17 11:26:12 2020 max relations containing the same ideal: 175 Fri Apr 17 11:26:36 2020 removing 4277502 relations and 3277502 ideals in 1000000 cliques Fri Apr 17 11:26:37 2020 commencing in-memory singleton removal Fri Apr 17 11:26:42 2020 begin with 46623734 relations and 47190406 unique ideals Fri Apr 17 11:27:17 2020 reduce to 46363498 relations and 43648404 ideals in 9 passes Fri Apr 17 11:27:17 2020 max relations containing the same ideal: 163 Fri Apr 17 11:27:39 2020 removing 4160361 relations and 3160361 ideals in 1000000 cliques Fri Apr 17 11:27:40 2020 commencing in-memory singleton removal Fri Apr 17 11:27:44 2020 begin with 42203137 relations and 43648404 unique ideals Fri Apr 17 11:28:09 2020 reduce to 41929982 relations and 40209959 ideals in 7 passes Fri Apr 17 11:28:09 2020 max relations containing the same ideal: 152 Fri Apr 17 11:28:28 2020 removing 4106362 relations and 3106362 ideals in 1000000 cliques Fri Apr 17 11:28:30 2020 commencing in-memory singleton removal Fri Apr 17 11:28:33 2020 begin with 37823620 relations and 40209959 unique ideals Fri Apr 17 11:28:55 2020 reduce to 37524651 relations and 36798691 ideals in 7 passes Fri Apr 17 11:28:55 2020 max relations containing the same ideal: 144 Fri Apr 17 11:29:13 2020 removing 1493656 relations and 1208745 ideals in 284911 cliques Fri Apr 17 11:29:14 2020 commencing in-memory singleton removal Fri Apr 17 11:29:17 2020 begin with 36030995 relations and 36798691 unique ideals Fri Apr 17 11:29:38 2020 reduce to 35991753 relations and 35550443 ideals in 7 passes Fri Apr 17 11:29:38 2020 max relations containing the same ideal: 141 Fri Apr 17 11:30:01 2020 relations with 0 large ideals: 1293 Fri Apr 17 11:30:01 2020 relations with 1 large ideals: 2792 Fri Apr 17 11:30:01 2020 relations with 2 large ideals: 37024 Fri Apr 17 11:30:01 2020 relations with 3 large ideals: 297491 Fri Apr 17 11:30:01 2020 relations with 4 large ideals: 1403043 Fri Apr 17 11:30:01 2020 relations with 5 large ideals: 4111486 Fri Apr 17 11:30:01 2020 relations with 6 large ideals: 7823119 Fri Apr 17 11:30:01 2020 relations with 7+ large ideals: 22315505 Fri Apr 17 11:30:01 2020 commencing 2-way merge Fri Apr 17 11:30:24 2020 reduce to 22470918 relation sets and 22029608 unique ideals Fri Apr 17 11:30:24 2020 commencing full merge Fri Apr 17 11:38:40 2020 memory use: 2811.5 MB Fri Apr 17 11:38:42 2020 found 10221905 cycles, need 10199808 Fri Apr 17 11:38:45 2020 weight of 10199808 cycles is about 1224250293 (120.03/cycle) Fri Apr 17 11:38:45 2020 distribution of cycle lengths: Fri Apr 17 11:38:45 2020 1 relations: 584468 Fri Apr 17 11:38:45 2020 2 relations: 687298 Fri Apr 17 11:38:45 2020 3 relations: 768063 Fri Apr 17 11:38:45 2020 4 relations: 784213 Fri Apr 17 11:38:45 2020 5 relations: 792232 Fri Apr 17 11:38:45 2020 6 relations: 768076 Fri Apr 17 11:38:45 2020 7 relations: 737052 Fri Apr 17 11:38:45 2020 8 relations: 696318 Fri Apr 17 11:38:45 2020 9 relations: 646480 Fri Apr 17 11:38:45 2020 10+ relations: 3735608 Fri Apr 17 11:38:45 2020 heaviest cycle: 28 relations Fri Apr 17 11:38:47 2020 commencing cycle optimization Fri Apr 17 11:39:06 2020 start with 85373695 relations Fri Apr 17 11:42:01 2020 pruned 3541155 relations Fri Apr 17 11:42:01 2020 memory use: 2367.5 MB Fri Apr 17 11:42:01 2020 distribution of cycle lengths: Fri Apr 17 11:42:01 2020 1 relations: 584468 Fri Apr 17 11:42:01 2020 2 relations: 705356 Fri Apr 17 11:42:01 2020 3 relations: 800146 Fri Apr 17 11:42:01 2020 4 relations: 815349 Fri Apr 17 11:42:01 2020 5 relations: 828312 Fri Apr 17 11:42:01 2020 6 relations: 800399 Fri Apr 17 11:42:01 2020 7 relations: 769761 Fri Apr 17 11:42:01 2020 8 relations: 722256 Fri Apr 17 11:42:01 2020 9 relations: 668701 Fri Apr 17 11:42:01 2020 10+ relations: 3505060 Fri Apr 17 11:42:01 2020 heaviest cycle: 28 relations Fri Apr 17 11:42:21 2020 RelProcTime: 5902 Fri Apr 17 11:42:25 2020 Fri Apr 17 11:42:25 2020 commencing linear algebra Fri Apr 17 11:42:26 2020 read 10199808 cycles Fri Apr 17 11:42:46 2020 cycles contain 35733787 unique relations Fri Apr 17 11:49:55 2020 read 35733787 relations Fri Apr 17 11:50:57 2020 using 20 quadratic characters above 4294917295 Fri Apr 17 11:53:31 2020 building initial matrix Fri Apr 17 12:00:17 2020 memory use: 4960.8 MB Fri Apr 17 12:00:22 2020 read 10199808 cycles Fri Apr 17 12:00:25 2020 matrix is 10199631 x 10199808 (4836.1 MB) with weight 1480894507 (145.19/col) Fri Apr 17 12:00:25 2020 sparse part has weight 1145354678 (112.29/col) Fri Apr 17 12:02:39 2020 filtering completed in 2 passes Fri Apr 17 12:02:42 2020 matrix is 10199506 x 10199683 (4836.1 MB) with weight 1480888728 (145.19/col) Fri Apr 17 12:02:42 2020 sparse part has weight 1145353286 (112.29/col) Fri Apr 17 12:03:55 2020 matrix starts at (0, 0) Fri Apr 17 12:03:57 2020 matrix is 10199506 x 10199683 (4836.1 MB) with weight 1480888728 (145.19/col) Fri Apr 17 12:03:57 2020 sparse part has weight 1145353286 (112.29/col) Fri Apr 17 12:03:57 2020 saving the first 48 matrix rows for later Fri Apr 17 12:03:59 2020 matrix includes 64 packed rows Fri Apr 17 12:04:01 2020 matrix is 10199458 x 10199683 (4728.6 MB) with weight 1264348240 (123.96/col) Fri Apr 17 12:04:01 2020 sparse part has weight 1137575108 (111.53/col) Fri Apr 17 12:04:01 2020 using block size 8192 and superblock size 786432 for processor cache size 8192 kB Fri Apr 17 12:04:51 2020 commencing Lanczos iteration (8 threads) Fri Apr 17 12:04:51 2020 memory use: 3960.1 MB Fri Apr 17 12:05:37 2020 linear algebra at 0.0%, ETA 82h 9m Fri Apr 17 12:05:52 2020 checkpointing every 130000 dimensions About 30 hours saved in LA, with less than 24 extra in sieving. But, as was pointed out elsewhere, the LA could have been running on a sole machine, with the sieving running for the next project. I'll have to think about this for a bit. It just might still be better, at least at this size, to go for the first viable matrix and move the rest of the farm to the next composite. Then again, I'm not nearly that organized.
 2020-04-17, 18:08 #39 VBCurtis     "Curtis" Feb 2005 Riverside, CA 118516 Posts Two data points that these params lead to ~12M matrices even without tons of extra sieving; at that size, I agree that adding more sieving is a waste of power. Looks like 260M relations is enough to get to a matrix, and Ed's job has one of the highest duplicate rates I've ever seen- almost 45%! Pending further testing, let's chalk that up to an unlucky poly, and figure his 270M isn't usually necessary. Charybdis' other filtering runs show us that 5M relations of extra sieving takes half a million dimensions off the matrix; that's a tradeoff I happily take. I suppose I can put a note in the params file that jobs run on a single machine should use 250M rels_wanted. Now that we have that setting fixed, we can tweak lambdas and ncurves etc to try to slice off another 10-20% of sieve time. Ed's log snippet shows Q=93M at the end of sieving; what we don't have is obvious indication that I=15 is faster or slower than A=28. That's not so bad, as it means they're likely fairly close in speed. Loosening lambda settings should help on A=28, since increasing yield will reduce the Q-range and thus reduce the slow-sieving Q's. I'll wait for data from Charybdis' second run (32/32LP, I think?) before I go mucking with lambdas. Perhaps this weekend I'll try some test-sieving, even.
 2020-04-18, 14:53 #40 EdH     "Ed Hall" Dec 2009 Adirondack Mtns 1101011111002 Posts I told CADO-NFS to go ahead and start LA. It currently shows krylov finishing on the 21st: Code: Info:Linear Algebra: krylov: N=51000 ; ETA (N=288000): Tue Apr 21 13:15:12 2020 [1.134 s/iter] vs. msieve LA completion around 1:00 a.m. on Apr 21: Code: linear algebra completed 2752960 of 10199683 dimensions (27.0%, ETA 61h11m) Of course, the rest of the CADO-NFS steps will take quite a bit longer, while the msieve square root step will take a few extra minutes. I tried to harvest some additional data from the CADO-NFS log, but gedit isn't being cooperative. I did extract a bunch of time data: Code: PID2635 2020-04-07 12:01:27,455 Info:Generate Factor Base: Total cpu/real time for makefb: 158.69/31.8918 PID2635 2020-04-07 12:12:09,925 Info:Generate Free Relations: Total cpu/real time for freerel: 4950.49/642.391 PID29604 2020-04-08 22:52:05,674 Info:Generate Factor Base: Total cpu/real time for makefb: 158.69/31.8918 PID29604 2020-04-08 22:52:05,676 Info:Generate Free Relations: Total cpu/real time for freerel: 4950.49/642.391 PID10019 2020-04-09 22:48:52,134 Info:Generate Factor Base: Total cpu/real time for makefb: 158.69/31.8918 PID10019 2020-04-09 22:48:52,137 Info:Generate Free Relations: Total cpu/real time for freerel: 4950.49/642.391 PID12298 2020-04-12 22:30:38,662 Info:Generate Factor Base: Total cpu/real time for makefb: 158.69/31.8918 PID12298 2020-04-12 22:30:38,664 Info:Generate Free Relations: Total cpu/real time for freerel: 4950.49/642.391 PID22249 2020-04-13 19:59:08,517 Info:Generate Factor Base: Total cpu/real time for makefb: 158.69/31.8918 PID22249 2020-04-13 19:59:08,518 Info:Generate Free Relations: Total cpu/real time for freerel: 4950.49/642.391 PID23248 2020-04-13 22:47:27,140 Info:Generate Factor Base: Total cpu/real time for makefb: 158.69/31.8918 PID23248 2020-04-13 22:47:27,150 Info:Generate Free Relations: Total cpu/real time for freerel: 4950.49/642.391 PID15335 2020-04-15 22:45:00,195 Info:Generate Factor Base: Total cpu/real time for makefb: 158.69/31.8918 PID15335 2020-04-15 22:45:00,196 Info:Generate Free Relations: Total cpu/real time for freerel: 4950.49/642.391 PID21322 2020-04-16 12:08:52,799 Info:Generate Factor Base: Total cpu/real time for makefb: 158.69/31.8918 PID21322 2020-04-16 12:08:52,800 Info:Generate Free Relations: Total cpu/real time for freerel: 4950.49/642.391 PID25635 2020-04-16 22:52:09,492 Info:Generate Factor Base: Total cpu/real time for makefb: 158.69/31.8918 PID25635 2020-04-16 22:52:09,494 Info:Generate Free Relations: Total cpu/real time for freerel: 4950.49/642.391 PID31904 2020-04-17 12:46:17,245 Info:Generate Factor Base: Total cpu/real time for makefb: 158.69/31.8918 PID31904 2020-04-17 12:46:17,252 Info:Generate Free Relations: Total cpu/real time for freerel: 4950.49/642.391 PID31904 2020-04-17 14:42:34,649 Info:Filtering - Duplicate Removal, splitting pass: Total cpu/real time for dup1: 2112.98/6974.47 PID31904 2020-04-17 15:07:33,352 Info:Filtering - Duplicate Removal, removal pass: Total cpu/real time for dup2: 5388.95/1496.74 PID31904 2020-04-17 15:18:41,063 Info:Filtering - Singleton removal: Total cpu/real time for purge: 2189.08/664.827 PID31904 2020-04-17 16:45:35,564 Info:Filtering - Merging: Total cpu/real time for merge: 5613.2/4885.08 PID31904 2020-04-17 16:45:35,564 Info:Filtering - Merging: Total cpu/real time for replay: 365.12/328.686 Are there some other specifics you'd like me to find?
2020-04-18, 14:56   #41
EdH

"Ed Hall"
Dec 2009

1101011111002 Posts

Quote:
 Originally Posted by VBCurtis Two data points that these params lead to ~12M matrices even without tons of extra sieving; at that size, I agree that adding more sieving is a waste of power. Looks like 260M relations is enough to get to a matrix, and Ed's job has one of the highest duplicate rates I've ever seen- almost 45%! Pending further testing, let's chalk that up to an unlucky poly, and figure his 270M isn't usually necessary. Charybdis' other filtering runs show us that 5M relations of extra sieving takes half a million dimensions off the matrix; that's a tradeoff I happily take. I suppose I can put a note in the params file that jobs run on a single machine should use 250M rels_wanted. Now that we have that setting fixed, we can tweak lambdas and ncurves etc to try to slice off another 10-20% of sieve time. Ed's log snippet shows Q=93M at the end of sieving; what we don't have is obvious indication that I=15 is faster or slower than A=28. That's not so bad, as it means they're likely fairly close in speed. Loosening lambda settings should help on A=28, since increasing yield will reduce the Q-range and thus reduce the slow-sieving Q's. I'll wait for data from Charybdis' second run (32/32LP, I think?) before I go mucking with lambdas. Perhaps this weekend I'll try some test-sieving, even.
I suppose I'm nearly ready to tackle another 17x sometime soon. If you'd like to suggest some changes to "my" params, I'd welcome giving them a shot.

 2020-04-18, 15:04 #42 EdH     "Ed Hall" Dec 2009 Adirondack Mtns 22·863 Posts I am considering breaking some of these posts into a separate thread with a new, more accurate title. Suggestions are welcome for a title and for whether to keep the new thread in my blog area or move it into factoring. I'm thinking that loading the posts into the CADO-NFS thread would not be good. Perhaps it's time for a CADO-NFS sub-forum?
 2020-04-18, 17:02 #43 VBCurtis     "Curtis" Feb 2005 Riverside, CA 106058 Posts (edited to remove CADO-requesting, since you already asked xyzzy) Sure, I'll tweak a few settings for your next c17x. I'll post the new params here later today. Last fiddled with by VBCurtis on 2020-04-18 at 17:05
2020-04-18, 19:52   #44
EdH

"Ed Hall"
Dec 2009

22×863 Posts

Quote:
 Originally Posted by VBCurtis . . . Sure, I'll tweak a few settings for your next c17x. I'll post the new params here later today.
Thanks. I've reserved 5+2,415 (12586...71 <177 dd>) for this next run.

 Similar Threads Thread Thread Starter Forum Replies Last Post enzocreti enzocreti 1 2020-03-03 18:38 tuckerkao Miscellaneous Math 2 2020-02-16 06:23 Nick Puzzles 9 2013-02-13 17:17 vsuite GPU Computing 11 2011-02-02 04:47 Corbyguy Software 3 2008-06-09 18:09

All times are UTC. The time now is 17:55.

Sat Nov 28 17:55:59 UTC 2020 up 79 days, 15:06, 3 users, load averages: 1.24, 1.12, 1.25