Can more P-1 help the DC effort

[Prime95]

Originally Posted by congsz

One more question: How much can P-1'ing (with v30.8+ on a 32GB+ or even 64GB+ RAM machine) help in completing DCs? Is this (P-1) worth doing?

IMO, someone with a 32GB machine should go to https://www.mersenne.ca/pm1_worst.php and look at the 67M to 68M range. Pick an exponent and time how long P-1 takes using the same B1 bound as the original run. Also report the new probability of finding a factor. Then report on how long the DC would take.

We can then determine if P-1 is profitable. If it is we should immediately start up a sub project for large memory machines to get better P-1 done on the DC candidates in front of the DC wavefront. I'll contribute 2 cores of a 64GB machine.

[kriesel]

Originally Posted by Prime95

IMO, someone with a 32GB machine should go to https://www.mersenne.ca/pm1_worst.php and look at the 67M to 68M range. Pick an exponent and time how long P-1 takes using the same B1 bound as the original run. Also report the new probability of finding a factor. Then report on how long the DC would take.

We can then determine if P-1 is profitable.

I'm trying a few now reserved, on a 128 GB ECC dual-xeon e5-2670 system with p95 v30.8b15, letting it choose its own bounds, given the PFactor lines that link produced. A gotcha is it does not include bits prior TF or # of tests there, so it assumes only 70 bits and 2 tests saved by a found factor, and gives an excessive probability of factoring. For M67177633:

[Aug 31 12:35:45] Worker starting
[Aug 31 12:35:45] Setting affinity to run worker on CPU core #9
[Aug 31 12:35:45] Optimal P-1 factoring of M67177633 using up to 112640MB of memory.
[Aug 31 12:35:45] Assuming no factors below 2^70 and 2 primality tests saved if a factor is found.
[Aug 31 12:35:46] Optimal bounds are B1=958000, B2=980270000
[Aug 31 12:35:46] Chance of finding a factor is an estimated 10.9%

Stop, tweak worktodo entries to 75 bits and 1 test, resume:

[Aug 31 12:43:37] M67177633 stage 1 is 7.95% complete. Time: 42.763 sec.
[Aug 31 12:44:18] Worker stopped.
[Aug 31 12:45:09] Waiting 5 seconds to stagger worker starts.
[Aug 31 12:45:14] Worker starting
[Aug 31 12:45:14] Setting affinity to run worker on CPU core #9
[Aug 31 12:45:14] Optimal P-1 factoring of M67177633 using up to 112640MB of memory.
[Aug 31 12:45:14] Assuming no factors below 2^75 and 1 primality test saved if a factor is found.
[Aug 31 12:45:14] Optimal bounds are B1=455000, B2=326757000
[Aug 31 12:45:14] Chance of finding a factor is an estimated 5.78%
[Aug 31 12:45:14] 
[Aug 31 12:45:15] Setting affinity to run helper thread 1 on CPU core #10
[Aug 31 12:45:15] Setting affinity to run helper thread 2 on CPU core #11
[Aug 31 12:45:15] Setting affinity to run helper thread 3 on CPU core #12
[Aug 31 12:45:15] Setting affinity to run helper thread 4 on CPU core #13
[Aug 31 12:45:15] Setting affinity to run helper thread 5 on CPU core #14
[Aug 31 12:45:15] Setting affinity to run helper thread 6 on CPU core #15
[Aug 31 12:45:15] Using AVX FFT length 3584K, Pass1=896, Pass2=4K, clm=1, 8 threads
[Aug 31 12:45:15] Setting affinity to run helper thread 7 on CPU core #16
[Aug 31 12:45:15] Ignoring suggested B1 value, using B1=958000 from the save file
[Aug 31 12:45:15] M67177633 stage 1 is 8.63% complete.

The second one will be more informative for run time. The 3 should be done in less than a day. Test Status indicates 1 hour 49 minutes each.

A 16GiB i5-1035G1 system yielded much lower bounds for M66209933:

[Aug 31 13:14:10] Worker starting
[Aug 31 13:14:10] Setting affinity to run worker on CPU core #1
[Aug 31 13:14:10] Optimal P-1 factoring of M66209933 using up to 8192MB of memory.
[Aug 31 13:14:10] Assuming no factors below 2^75 and 1 primality test saved if a factor is found.
[Aug 31 13:14:10] Optimal bounds are B1=302000, B2=25472000
[Aug 31 13:14:10] Chance of finding a factor is an estimated 3.65%
[Aug 31 13:14:10] 
[Aug 31 13:14:11] Using AVX-512 FFT length 3528K, Pass1=1344, Pass2=2688, clm=1, 4 threads
[Aug 31 13:14:11] Setting affinity to run helper thread 1 on CPU core #2
[Aug 31 13:14:11] Setting affinity to run helper thread 2 on CPU core #3
[Aug 31 13:14:11] Setting affinity to run helper thread 3 on CPU core #4

Test Status again indicates 1 hour 49 minutes each.

My soon-to-be 32GB system has been in Dell warranty repair delay for 3 weeks so far (completion date unknown, parts out of stock).

[Prime95]

Originally Posted by kriesel

I'm trying a few now reserved, on a 128 GB ECC dual-xeon e5-2670 system with p95 v30.8b15

Thanks. That machine can emulate a 64GB and 32GB machine by reducing the memory prime95 is allowed to use.

Stop, tweak worktodo entries to 75 bits and 1 test, resume:

[Aug 31 12:45:14] Optimal bounds are B1=455000, B2=326757000
[Aug 31 12:45:14] Chance of finding a factor is an estimated 5.78%

As an aside, James hates P-1 with B1 less than the original B1 because the new P-1 test is not a "superset" of the old P-1. Oh well. The actual bounds selected are not optimal as prime95 should be tweaked to deduct the original P-1's chance of finding a factor. We can tune that later.

So, it pays to run more P-1 on your machine if you can do more than 1/(.0578-.0275)=33 P-1 tests in the time it takes to run one full DC. Eagerly awaiting results.

[Prime95]

Originally Posted by Prime95

The actual bounds selected are not optimal as prime95 should be tweaked to deduct the original P-1's chance of finding a factor. We can tune that later.

We might can trick prime95's optimal bounds calculator by noting that the original 2.75% P-1 chance of finding a factor is roughly equivalent to 2 more bits of TF (2/77 = 2.60%).

[kriesel]

16 GiB i5-1035G1 system, Win 10 Pro x64, prime95 v30.8b15

[Aug 31 13:14:10] Worker starting
[Aug 31 13:14:10] Setting affinity to run worker on CPU core #1
[Aug 31 13:14:10] Optimal P-1 factoring of M66209933 using up to 8192MB of memory.
[Aug 31 13:14:10] Assuming no factors below 2^75 and 1 primality test saved if a factor is found.
[Aug 31 13:14:10] Optimal bounds are B1=302000, B2=25472000
[Aug 31 13:14:10] Chance of finding a factor is an estimated 3.65%
[Aug 31 13:14:10] 
[Aug 31 13:14:11] Using AVX-512 FFT length 3528K, Pass1=1344, Pass2=2688, clm=1, 4 threads
[Aug 31 13:14:11] Setting affinity to run helper thread 1 on CPU core #2
[Aug 31 13:14:11] Setting affinity to run helper thread 2 on CPU core #3
[Aug 31 13:14:11] Setting affinity to run helper thread 3 on CPU core #4
[Aug 31 13:20:37] M66209933 stage 1 is 11.47% complete. Time: 386.178 sec.
[Aug 31 13:27:21] M66209933 stage 1 is 22.95% complete. Time: 403.818 sec.
[Aug 31 13:34:07] M66209933 stage 1 is 34.44% complete. Time: 405.458 sec.
[Aug 31 13:40:36] M66209933 stage 1 is 45.92% complete. Time: 389.182 sec.
[Aug 31 13:47:15] M66209933 stage 1 is 57.40% complete. Time: 398.858 sec.
[Aug 31 13:54:06] M66209933 stage 1 is 68.89% complete. Time: 410.978 sec.
[Aug 31 14:00:56] M66209933 stage 1 is 80.37% complete. Time: 410.513 sec.
[Aug 31 14:07:43] M66209933 stage 1 is 91.85% complete. Time: 406.341 sec.
[Aug 31 14:12:34] M66209933 stage 1 complete. 870952 transforms. Total time: 3503.149 sec.
[Aug 31 14:13:20] Inversion of stage 1 result complete. 5 transforms, 1 modular inverse. Time: 45.981 sec.
[Aug 31 14:13:21] Available memory is 8192MB.
[Aug 31 14:13:21] Setting affinity to run helper thread 1 on CPU core #2
[Aug 31 14:13:21] Setting affinity to run helper thread 3 on CPU core #4
[Aug 31 14:13:21] Switching to AVX-512 FFT length 3840K, Pass1=2K, Pass2=1920, clm=1, 4 threads
[Aug 31 14:13:21] Estimated stage 2 vs. stage 1 runtime ratio: 1.015
[Aug 31 14:13:21] Setting affinity to run helper thread 2 on CPU core #3
[Aug 31 14:13:22] Using 8168MB of memory.  D: 462, 60x216 polynomial multiplication.
[Aug 31 14:13:23] Setting affinity to run polymult helper thread on CPU core #2
[Aug 31 14:13:23] Setting affinity to run polymult helper thread on CPU core #3
[Aug 31 14:13:23] Setting affinity to run polymult helper thread on CPU core #4
[Aug 31 14:13:42] Stage 2 init complete. 1419 transforms. Time: 21.819 sec.
[Aug 31 14:19:51] M66209933 stage 2 at B2=7007154 [9.34%].  Time: 368.383 sec.
[Aug 31 14:25:13] M66209933 stage 2 at B2=8958642 [18.91%].  Time: 322.214 sec.
[Aug 31 14:30:24] M66209933 stage 2 at B2=10910130 [28.47%].  Time: 311.203 sec.
[Aug 31 14:35:45] M66209933 stage 2 at B2=12861618 [38.04%].  Time: 320.851 sec.
[Aug 31 14:41:02] M66209933 stage 2 at B2=14813106 [47.60%].  Time: 317.047 sec.
[Aug 31 14:45:59] M66209933 stage 2 at B2=16764594 [57.17%].  Time: 296.710 sec.
[Aug 31 14:51:12] M66209933 stage 2 at B2=18716082 [66.73%].  Time: 312.972 sec.
[Aug 31 14:57:15] M66209933 stage 2 at B2=20667570 [76.30%].  Time: 363.192 sec.
[Aug 31 15:03:04] M66209933 stage 2 at B2=22619058 [85.86%].  Time: 349.151 sec.
[Aug 31 15:08:30] M66209933 stage 2 at B2=24570546 [95.43%].  Time: 326.073 sec.
[Aug 31 15:11:03] M66209933 stage 2 complete. 353759 transforms. Total time: 3440.437 sec.
[Aug 31 15:11:03] Starting stage 2 GCD - please be patient.
[Aug 31 15:11:32] Stage 2 GCD complete. Time: 29.133 sec.
[Aug 31 15:11:32] M66209933 completed P-1, B1=302000, B2=25501938, Wi4: B08DBC65

[Aug 31 13:14:10] Worker starting
P-1 total elapsed time 1:57:22 -> 7042. seconds.

Test, Status estimate for M66209933 PRP&Proof as DC is 5 days 13 hours 45 minutes -> 481500. seconds.

Ratio 481500/7042 = 68.375 estimated payoff > estimated 1/.0365 =27.397 factorings estimated cost of additional P-1 effort finding one exponent's factor. A lot here depends on how correct the 3.65% P-1 factor probability is.
Subtracting off (pessimistically, since the new B1 is smaller than the original B1) all the 1.551% calculated at https://www.mersenne.ca/prob.php?exp...5000&b2=695000 gives 2.1%; 1/ .021 = 47.619, still a win.
481500 - 47.619 * 7042 = 146167 seconds in 47.619 exponents, expected to be saved. (Projected average savings ~0.85 hours per exponent re-P-1 factored after a stage-1-only previous P-1 attempt, using just 8 GiB on the 16 GiB system for the second P-1 attempt.)

How pessimistic was the subtraction? https://www.mersenne.ca/prob.php?exp...2000&b2=695000 gives 1.504% for the B1 302K, B2 695K intersection of the first and second bounds sets, 0.046% lower than the 1.55% I subtracted. So maybe instead use
.0365 - .01504 = .02146 instead of .021. 1/.02146 = 46.598;
481500 - 46.598 * 7042 =153357 seconds net savings in 46.598 P-1 retries; equivalent to 3291 seconds = 0.914 hours per retry (0.68%)

[Prime95]

Originally Posted by kriesel

the 1.55% calculated at https://www.mersenne.ca/prob.php?exp...5000&b2=695000 gives ...

@James: If you are reading this, Ken's link gives a 1.55% probability, but this page on exponent 66209933 gives 2.721%.

[Prime95]

Originally Posted by kriesel

A lot here depends on how correct the 3.65% P-1 factor probability is.

https://www.mersenne.ca/prob.php?exp...00&b2=25501938 shows 3.67%, so the number seems pretty good.

Subtracting off (pessimistically, since the new B1 is smaller than the original B1) all the 1.55% calculated at https://www.mersenne.ca/prob.php?exp...5000&b2=695000 gives 2.1%; 1/ .021 = 47.619, still a win.
481500 - 47.619 * 7042 = 146167 seconds in 47.619 exponents expected to be saved.

Thanks for the data. P-1 a winner with as little as 8GB! Looks like we should immediately start a P-1 subproject while we debate this projects long-term goals and desired server rule changes.

[kriesel]

Originally Posted by Prime95

@James: If you are reading this, Ken's link gives a 1.55% probability, but this page on exponent 66209933 gives 2.721%.

My guess is most of the difference is the assumption of CPU TF levels at that worst-factored link, while I used actual GPU completed levels; 70 vs 75 bits. https://www.mersenne.ca/exponent/66209933

I selected target exponents by clicking "stage 1 only" and "1 primality test" at the worst-p-1-factored link.

[kriesel]

Originally Posted by Prime95

https://www.mersenne.ca/prob.php?exp...00&b2=25501938 shows 3.67%, so the number seems pretty good.

... P-1 a winner with as little as 8GB! Looks like we should immediately start a P-1 subproject while we debate this projects long-term goals and desired server rule changes.

It gets better at both ends; +.02% as above, +.046% after adjusting for the pessimistic treatment, see last paragraph added by edit to https://mersenneforum.org/showpost.p...2&postcount=34.
Could 5-6 GB also be a winner? (At least for single-stage-only previous P-1.)

I suggest selective redo of P-1 64M and up, as 64M is currently issuing as manual DC assignments and via PrimeNet to slow DC systems. It looks like the lowest remaining single-stage-only are ~65.8M.
There are lower exponents with small stage 2s that may or may not pay to redo P-1, in the 64M range.

[kriesel]

Originally Posted by Prime95

@James: If you are reading this, Ken's link gives a 1.55% probability, but this page on exponent 66209933 gives 2.721%.

It also gives a draft worktodo entry Pfactor=1,2,65933107,-1,70,2 as one to re P-1, in response to selecting 1 test only, stage-1 only, even though that is already shown successfully double-checked LL on mersenne.org, and mersenne.ca lists only a single LL result but shows it already trial factored to 75 bits.
Therefore PrimeNet would not issue an AID for its revised worktodo entry Pfactor=1,2,65933107,-1,75,1 to prime95. The ones that actually need a double-check yet, primenet was happy to issue P-1 assignments for.

[kriesel]

prime95 v30.8b15 on Win7 Pro x64 dual xeon e5-2670 dual worker, 1 worker P-1, other PRP

[Aug 31 15:53:40] Optimal P-1 factoring of M67177697 using up to 112640MB of memory.
[Aug 31 15:53:40] Assuming no factors below 2^75 and 1 primality test saved if a factor is found.
[Aug 31 15:53:41] Optimal bounds are B1=455000, B2=326757000
[Aug 31 15:53:41] Chance of finding a factor is an estimated 5.78%
[Aug 31 15:53:41] 
[Aug 31 15:53:42] Setting affinity to run helper thread 1 on CPU core #10
[Aug 31 15:53:42] Setting affinity to run helper thread 3 on CPU core #12
[Aug 31 15:53:42] Setting affinity to run helper thread 2 on CPU core #11
[Aug 31 15:53:42] Setting affinity to run helper thread 4 on CPU core #13
[Aug 31 15:53:42] Setting affinity to run helper thread 5 on CPU core #14
[Aug 31 15:53:42] Setting affinity to run helper thread 6 on CPU core #15
[Aug 31 15:53:42] Using AVX FFT length 3584K, Pass1=896, Pass2=4K, clm=1, 8 threads
[Aug 31 15:53:42] Setting affinity to run helper thread 7 on CPU core #16
[Aug 31 15:54:25] M67177697 stage 1 is 1.51% complete. Time: 43.201 sec.
[Aug 31 15:55:08] M67177697 stage 1 is 3.04% complete. Time: 42.623 sec.
...
[Aug 31 16:41:15] M67177697 stage 1 is 97.50% complete. Time: 42.582 sec.
[Aug 31 16:42:31] M67177697 stage 1 is 99.03% complete. Time: 76.689 sec.
[Aug 31 16:42:58] M67177697 stage 1 complete. 1312770 transforms. Total time: 2956.933 sec.
[Aug 31 16:43:41] Inversion of stage 1 result complete. 5 transforms, 1 modular inverse. Time: 42.686 sec.
[Aug 31 16:43:42] Available memory is 112522MB.
[Aug 31 16:43:44] Setting affinity to run helper thread 1 on CPU core #10
[Aug 31 16:43:44] Setting affinity to run helper thread 2 on CPU core #11
[Aug 31 16:43:44] Setting affinity to run helper thread 3 on CPU core #12
[Aug 31 16:43:44] Switching to AVX FFT length 4M, Pass1=512, Pass2=8K, clm=2, 8 threads
[Aug 31 16:43:44] Estimated stage 2 vs. stage 1 runtime ratio: 0.916
[Aug 31 16:43:44] Setting affinity to run helper thread 4 on CPU core #13
[Aug 31 16:43:44] Setting affinity to run helper thread 5 on CPU core #14
[Aug 31 16:43:44] Setting affinity to run helper thread 6 on CPU core #15
[Aug 31 16:43:44] Setting affinity to run helper thread 7 on CPU core #16
[Aug 31 16:43:44] Using 112519MB of memory.  D: 6930, 720x2839 polynomial multiplication.
[Aug 31 16:45:50] Setting affinity to run polymult helper thread on CPU core #10
[Aug 31 16:45:50] Setting affinity to run polymult helper thread on CPU core #11
[Aug 31 16:45:50] Setting affinity to run polymult helper thread on CPU core #12
[Aug 31 16:45:50] Setting affinity to run polymult helper thread on CPU core #13
[Aug 31 16:45:50] Setting affinity to run polymult helper thread on CPU core #14
[Aug 31 16:45:50] Setting affinity to run polymult helper thread on CPU core #15
[Aug 31 16:45:50] Setting affinity to run polymult helper thread on CPU core #16
[Aug 31 16:50:54] Stage 2 init complete. 21489 transforms. Time: 432.442 sec.
[Aug 31 16:58:52] M67177697 stage 2 at B2=35543970 [3.12%].  Time: 477.391 sec.
[Aug 31 17:01:43] M67177697 stage 2 at B2=45239040 [6.25%].  Time: 170.739 sec.
[Aug 31 17:04:40] M67177697 stage 2 at B2=54934110 [9.37%].  Time: 177.358 sec.
[Aug 31 17:09:38] M67177697 stage 2 at B2=64629180 [12.49%].  Time: 298.434 sec.
[Aug 31 17:12:32] M67177697 stage 2 at B2=74324250 [15.62%].  Time: 173.291 sec.
[Aug 31 17:15:20] M67177697 stage 2 at B2=84019320 [18.75%].  Time: 168.005 sec.
[Aug 31 17:18:07] M67177697 stage 2 at B2=93714390 [21.87%].  Time: 167.309 sec.
[Aug 31 17:20:05] M67177697 stage 2 at B2=103409460 [25.00%].  Time: 118.460 sec.
[Aug 31 17:22:45] M67177697 stage 2 at B2=113104530 [28.12%].  Time: 159.294 sec.
[Aug 31 17:25:35] M67177697 stage 2 at B2=122799600 [31.25%].  Time: 170.198 sec.
[Aug 31 17:28:23] M67177697 stage 2 at B2=132494670 [34.37%].  Time: 168.484 sec.
[Aug 31 17:32:04] M67177697 stage 2 at B2=142189740 [37.49%].  Time: 220.694 sec.
[Aug 31 17:34:47] M67177697 stage 2 at B2=151884810 [40.62%].  Time: 162.808 sec.
[Aug 31 17:37:36] M67177697 stage 2 at B2=161579880 [43.74%].  Time: 169.230 sec.
[Aug 31 17:40:49] M67177697 stage 2 at B2=171274950 [46.87%].  Time: 193.120 sec.
[Aug 31 17:43:27] M67177697 stage 2 at B2=180970020 [49.99%].  Time: 158.214 sec.
[Aug 31 17:45:28] M67177697 stage 2 at B2=190665090 [53.12%].  Time: 120.519 sec.
[Aug 31 17:48:10] M67177697 stage 2 at B2=200360160 [56.24%].  Time: 161.878 sec.
[Aug 31 17:50:42] M67177697 stage 2 at B2=210055230 [59.37%].  Time: 152.601 sec.
[Aug 31 17:52:58] M67177697 stage 2 at B2=219750300 [62.49%].  Time: 135.536 sec.
[Aug 31 17:55:53] M67177697 stage 2 at B2=229445370 [65.62%].  Time: 174.692 sec.
[Aug 31 17:59:32] M67177697 stage 2 at B2=239140440 [68.75%].  Time: 219.190 sec.
[Aug 31 18:02:01] M67177697 stage 2 at B2=248835510 [71.87%].  Time: 149.083 sec.
[Aug 31 18:04:21] M67177697 stage 2 at B2=258530580 [75.00%].  Time: 139.591 sec.
[Aug 31 18:06:53] M67177697 stage 2 at B2=268225650 [78.12%].  Time: 152.236 sec.
[Aug 31 18:11:06] M67177697 stage 2 at B2=277920720 [81.25%].  Time: 252.877 sec.
[Aug 31 18:13:36] M67177697 stage 2 at B2=287615790 [84.37%].  Time: 150.325 sec.
[Aug 31 18:15:49] M67177697 stage 2 at B2=297310860 [87.50%].  Time: 133.055 sec.
[Aug 31 18:18:16] M67177697 stage 2 at B2=307005930 [90.62%].  Time: 147.090 sec.
[Aug 31 18:20:19] M67177697 stage 2 at B2=316701000 [93.75%].  Time: 122.658 sec.
[Aug 31 18:24:05] M67177697 stage 2 at B2=326396070 [96.87%].  Time: 226.646 sec.
[Aug 31 18:26:41] M67177697 stage 2 complete. 363907 transforms. Total time: 5746.962 sec.
[Aug 31 18:26:41] Starting stage 2 GCD - please be patient.
[Aug 31 18:27:09] Stage 2 GCD complete. Time: 27.955 sec.
[Aug 31 18:27:09] M67177697 completed P-1, B1=455000, B2=336091140, Wi4: DE0F6F75

Aug 31 15:53:40 begin; 2:33:29 elapsed = 9209 seconds ~ 2.56 hours.
For B1=455000, B2=326757000 Chance of finding a factor is an estimated 5.78%.
https://www.mersenne.ca/prob.php?exp...0&b2=326757000 gives 5.838%.
Prior P-1 was B1=B2=710000. which gives 1.569% at https://www.mersenne.ca/prob.php?exp...0000&b2=710000
Estimated overlap is 1.557% https://www.mersenne.ca/prob.php?exp...5000&b2=710000
.05838 total chance - .01557 overlap = .04281 chance of a factor found

DC on same xeon/worker est 9/10 9:45 - 9/14 05:55 = 3d 20 h 10 min est = 331800. sec est. (~92.17 hours)
DC time estimate is again from prime95 test, status output and is unconfirmed.

.04281 * 331800 = 14204; - 9209 = 4995. seconds =1.39 hours expected net savings (1.505% of a DC duration, 54.2% of a P-1 re-do duration). 1/.04281 ~ 23.36 exponents for one factored. (~59.76 hours P-1 for a factor found.)