P-1 on OBD candidates

[kriesel]

(edits in quote are in bold)

Originally Posted by kriesel

Approximate example: if system A is twice as fast as system B, the two will begin work at the same time, and B2start = B1 (an oversimplification) = 17,000,000, B2range=1G-17M = 983M. B2range/3=~328M.
System A runs stage 2 for bounds 17M to 328*2+17 =673M; system B runs stage 2 for bounds 673M to 1G.
...
I expect multiple OBD candidates to be TF completed to 92 bits within 6 months.
On an RTX2080, OBD TF 89-90 bits is ~19 days, and to 91 ~+35 days initially indicated, so 92 ~+70 days.
(All using mfaktc GPU kernel "barrett92_mul32_gs")
Starting from 88 done (level 22) that would be ~10 +19 +35 +70 = 134 days ~ 4.5 months.

There are one candidate each now at 91 bits TF completed, 90, and ~halfway between 89 and 90.
RTX2080 run indicates ~2143. GHD/day; GTX1650 at such levels is ~753. GHD/day; GTX1650 Super 954.
A GTX1650 is estimated to complete 88-92 in 12.8 months; a GTX 1650 Super in 10.1 months.

Mlucas s1 files for OBD are ~405MB each, so for p & q pairs, ~810MB per exponent, just two exponents' sets fit in the dropbox 2GB free space. Google drive at 15 GB is roomier, allowing 18 pairs.

It would be good to avoid a surplus of stage-1-completed and paucity of stage-2 attempts.
Operationally, it would be simplest if stage 1 and stage 2 occur on the same system, or at least same LAN (avoiding the cloud-storage shuffle and multiuser coordination on a single exponent).
There is some throughput advantage to doing stage 1 on 16 GiB systems and running exclusively stage 2 on big-ram systems.

WSL vs. native Linux performance should be investigated before committing to lengthy runs. (And I'm seeing substantial indications of perhaps WSL-related memory leak on a test system running Mlucas v20.1 P-1 on 665M. Of order 100MB/hour in first day or two after a system restart. Haven't attempted an mprime on WSL on the same system comparison yet.)

[clowns789]

It also begs the question if P-1 should be done following reaching 92 bits or if we should wait until one system can do both P-1 stages and PRP consecutively. Since gpuowl 7 requires this and perhaps future CPU-based software will as well, I want to avoid any redundancy or at least any failure to optimize should it be deemed that P-1 and PRP should run on the same system the same way Ken mentioned that P-1 Stage 1 and Stage 2 should be on the same system.

[kriesel]

Originally Posted by clowns789

It also begs the question if P-1 should be done following reaching 92 bits or if we should wait until one system can do both P-1 stages and PRP consecutively. ...

A reasonable question. One might also ask whether P-1 should occur between 91 and 92 bits TF. Since GPU TF 91-92 is ~0.2 years for ~1.1% chance of a factor, while P-1 is ~1 year+ on multicore CPU for ~4% chance of a factor, I think it's better to go to 92 on TF first, now, while performing P-1 system qualification in parallel, and P-1 software debugging has been occurring.

CUDAPm1 is currently moot for OBD P-1. It has demonstrated in light testing an inability to perform both stages above ~430M exponent or achieve resume from a save file above some higher exponent. It is limited by its source code to 2³¹-1. It is slower than Gpuowl on same hardware. There are numerous bugs. It is no longer being maintained. Etc.

Gpuowl is currently moot for OBD P-1. No existing version of Gpuowl supports OBD P-1. (So consecutive P-1/PRP as in gpuowl 6.x, or combined P-1 stage 1/PRP powering of 3, as in gpuowl V7.x, are both moot. Gpuowl development has ceased, at least for now, with the last GitHub commit occurring in late March 2021. Note that on the same hardware and exponent, on Windows, gpuowl v7.x combined P-1/prp has been observed to be slower than gpuowl v6.11 serial P-1 standalone followed by PRP for wavefront or 100Mdigit or ~gigabit IIRC. Some versions of gpuowl nominally support OBD PRP but they lack PRP proof, and the prp time is ~5 years/exponent on a Radeon VII.)

I think mprime/prime95 will remain moot for OBD P-1 and PRP also. On AVX512 only, prime95 supports 64M max fft, 1/3 the size needed for OBD P-1 or PRP. George may implement parallel P-1/PRP in prime95/mprime in the future, but is likely to only support up to ~64M fft length ~1.17G exponent. He's not interested in coding for cases that would take decades to run to completion. There are lots of things competing for George's attention.

Mlucas recently added standalone P-1 capability. It might someday get the combined P-1/PRP capability, but not until after it gets PRP proof generation capability. Mlucas now supports Mersenne exponents up to 2³³, but OBD PRP would take decades on affordable/available hardware in either case.

CUDALucas is moot for OBD PRP. It's LL only, has no Jacobi symbol based error check, per the source code maxes out at 2³¹-1, and in practice crashes above ~1.4G exponent. It is not being maintained. If fixed it would likely be slower than gpuowl on the same NVIDIA hardware and exponent. https://www.mersenneforum.org/showpo...6&postcount=14

Demonstrating feasibility of OBD standalone P-1 appeals to me. Usage of software informs future development, finds bugs, and independently confirms resolution of bugs.

Suggesting running both P-1 stages on the same system was motivated by an operational efficiency consideration, not a feasibility or primarily performance consideration. Consider some alternate scenarios following, all of which presume Mlucas v20.x or later:

1) Same system, standalone P-1: system with plenty of ram runs stage 1 using little of the ram, then stage 2 using most of the ram. No user intervention or file transfer between systems or between participants. Simplest.

2) Same LAN, mixed systems, parallelism at stage level:
~16 GiB systems run stage 1.
~64+ GiB systems run stage 2. Manually copy files across LAN via private server as needed, or, have different systems' program instances run in same private LAN server folders sequentially (which would in my case involve lots of fiddling with jobs.sh and mlucas.cfg at the stage 1/stage 2 switch because of disparate processor instruction sets, core counts etc).
Less simple; somewhat better utilization of high-ram systems; greater total throughput by letting 16GiB systems help.

3) Same LAN, mixed systems, parallelism at stage level and more:
~16 GiB systems run stage 1. Perhaps two run the same first exponent and bounds and then the resulting p<exponent> files are compared, to try to detect stage 1 error / crudely assess reliability. If no match, try a third afterward.
~64+ GiB systems run stage 2. Manually copy stage 1 files across LAN via private server to the stage 2 systems as needed.
Split stage 2 to two or more large-ram systems with B2 subranges chosen to roughly equalize run times.
Even less simple; somewhat better utilization of high-ram systems; greater throughput by letting 16 GiB systems help; reduced latency for a given task.

4) Collaboration among participants via internet on a single exponent:
One participant (A) runs stage 1 solo, on a ~16 GiB system. Stage 1 is not amenable to splitting or parallelism at the systems level.
Participant A posts stage 1 result files p<exponent>, and optionally q<exponent>, on a cloud drive accessible by others, such as Dropbox or google drive with sharing enabled.
Multiple participants previously prequalified hardware by run-time scaling tests and posting those results.
The other participants must basically trust that A did stage 1 accurately, and did not just generate random bits.
(Or a helpful or cautious participant could run stage 1 again to the same B1 and compare the resulting files.)
By mutual agreement, participant(s) A, B (, etc?) subdivide the stage 2 bounds of the single exponent, for roughly equal stage 2 run time on probably disparate hardware.
The other participants download A's stage 1 results files for their respective parallel stage 2 runs on different B2 subranges. They then work in parallel to complete the OBD P-1 stage 2 and report the results of each distinct B2 subrange.
This is probably the fastest way to get one OBD P-1 completed, and the most complicated. It might be the only feasible way for some participants to constructively contribute in under a year or two run time with the hardware they have.


The collaboration scenario is the same model Ernst has chosen for the F33 P-1 effort, which is ~order of magnitude larger than one OBD P-1. Participation in that requires >120 GiB ram.
It may also be how one of the earliest OBD P-1 completed gets run.

[Viliam Furik]

Originally Posted by kriesel

Mlucas s1 files for OBD are ~405MB each, so for p & q pairs, ~810MB per exponent, just two exponents' sets fit in the dropbox 2GB free space. Google drive at 15 GB is roomier, allowing 18 pairs.

I didn't pay enough attention to this thread to know why you need that storage, but I can offer at least 1 TB on OneDrive cloud storage in case you are interested.

[kriesel]

Run time scaling completed twice, from 10M to 500M or higher plus short stage 1 OBD interim timing, yields multiple extrapolations to OBD GPU72 bounds P-1 both stages completion within one year of continuous running, if not loaded with too much other work simultaneously. About 10-11 months solo looks feasible on this dual xeon e5-2697v2 system with 128 GiB ram on Ubuntu, WSL1, Win10. The initial two attachments at the link below describe some possibilities for yet faster run time. (Run time scaling results for additional systems may be posted at the same link later.)

See https://www.mersenneforum.org/showpo...5&postcount=17 for more background

[kriesel]

There is a November 28 patch for Mlucas v20.1.1.
Mersenne.ca is now accepting P-1 results from Mlucas and Gpuowl for p>10⁹, via bulk factor file upload.
As far as I know, there is no reservation system for P-1 work for p>10⁹ at this time other than posting in this thread.

Current TF status is shown at https://www.mersenne.ca/obd. Current indicated Level is 22.06.
3321928171 30% complete from 91 to 92, ETA to 92 2022-01-31 by johnny_jack
3321928307 83% complete from 90 to 91, ETA to 91 2021-12-04 & will continue immediately to 92 by kriesel
3321928319 31% complete from 90 to 91, ETA to 91 2021-12-26 & will continue immediately to 92 by kriesel
3321938373 01% complete from 89 to 90, ETA to 90 2022-01-21 by kriesel
There are 13 others in progress at bit levels 86-87 to 88-89 by kriesel as part of an effort to go expeditiously to OBD Level 24 or higher.
Faster GPUs are running the higher of the bit levels indicated above; slower GPUs and colab sessions are running the lower of the bit levels mentioned above.
All exponents 3321928171 to 3321929987 (currently 33 exponents) are being taken higher, with the expectation that some will have a factor discovered before reaching level 26 (TF to 92 bits completed on 26 surviving unfactored exponents) or level 28 (28 to 92 bits TF and through stage 2 P-1 to good bounds without factors found). These P-1 survivors would be preparation for someone someday after additional considerable hardware advance, using hardware that does not yet exist and won't for years, using software that does not yet exist either, going to level 29 (29 OBD PRP/GEC/proof-gen) & level 30 (30 OBD PRP/GEC/proof-gen & verified).

So at the moment:

OBD TF completed to 91, reserved to 92 bits: 1

OBD TF completed to 92 bits: 0
Systems with qualification(s) completed for OBD P-1: 1
Exponents reserved for P-1: 0
Exponents completed thru stage1 P-1: 0
Exponents completed thru stage2 P-1: 0
There's no expectation of being able to qualify less than supercomputer hardware for PRP, and PRP/GEC/proof software for OBD or higher does not exist, so as expected, systems with qualifications completed for OBD PRP etc = 0.

[kriesel]

Run time scaling performed, from 10M to 500M or higher plus short stage 1 OBD interim timing, and experimentation with core counts and 1 vs 2 instances, etc., yields multiple extrapolations to OBD GPU72 bounds P-1 both stages completion within one year of continuous running, if stage 2 is done in tandem with another system at least as fast. About 1.25-1.7 years solo looks feasible on this dual xeon e5-2690v2 system with 64 GiB ECC ram on Ubuntu, WSL1, Win10. The third attachment at the link below describe some possibilities for yet faster run time. (Run time scaling results for additional systems may be posted at the same link later.)

See https://www.mersenneforum.org/showpo...5&postcount=17 for more background.
Note the previously qualified system also contains ECC ram.

[kriesel]

Run time scaling performed, from 10M to 332M or higher plus short stage 1 OBD interim timing, yields multiple extrapolations to OBD GPU72 bounds P-1 both stages completion within 1.4 years of continuous running, if stage 2 is done in tandem with another system at least as fast. About 2 years solo looks feasible on this i5-7600T system with 64 GiB nonECC ram on CentOS 7.9. Because it lacks ECC on the ram, its use for OBD P-1 is likely to be brief. The fourth attachment at the link below describes tests and results. (Run time scaling results for additional systems may be posted at the same link later.)

See https://www.mersenneforum.org/showpo...5&postcount=17 for more background.

[kriesel]

The following are known to be too slow for OBD P-1 solo or tandem to proper bounds within a year, lack adequate 64 GiB ram maximum capacity for running tandem stage 2 (at least in the systems I had available to test, if not intrinsically by CPU design), and lack support for ECC ram.
i7-7500U (my system 16 GiB max; Intel specs now say 32 GiB max)
i7-4790 or lesser model number in that series (4770 etc), or alternate CPUs in the same motherboard models, e.g. i3-4170, Celeron G1840 (my systems 16 GiB max; Intel specs now say 32 GiB max)
i5-1035G1 (my systems 32 GiB max; Intel specs now say 64 GiB max)
i7-8750H (my system 32 GiB max; Intel specs now say 64 GiB max)

The following support adequate amounts of ECC ram, but would be too slow, judging by relative prime95 performance:
dual-Xeon X5650
dual-Xeon E5645
dual-Xeon E5520

The following do not support sufficient ram for even OBD stage 1 P-1, lack support for ECC ram, and would be too slow, judging by relative prime95 performance:
Core 2 Duo E8200
i3-M370
Pentium M750

The following are too inefficient and slow under WSL, and limited to access to 16 cores/64 hyperthreads there:
Xeon Phi 7210
Xeon Phi 7250
These will require a native Linux boot to probably qualify.

[kriesel]

OBD P-1 is now feasible. For now, Mlucas v20.x is the only known software capable of OBD Mersenne P-1 factoring, with sufficiently large fft lengths and several months of QA testing and revision (patching) accomplished. The latest version is recommended, which is V20.1.1 2022-12-02. See http://www.mersenneforum.org/mayer/README.html and https://mersenneforum.org/showthread.php?t=27295

I propose the following:

1. OBD candidates are taken to 92 bits TF done, before P-1 is attempted. (Due to run time, RTX206x or faster GPUs and mfaktx are recommended. TF reservations that would take longer than a year on available hardware ought be avoided.)
2. P-1 is only attempted on OBD candidates with no known factors, except for software testing in which finding again a known factor supports confidence in the P-1 software on the tested FFT length.
3. Whoever does the TF from 91 bits to 92 (which is half the total work of trial factoring an exponent up to 92 bits) gets first choice at reserving the P-1 for the same exponent. The individual may be operating under different usernames in the Mersenne forum and on mersenne.ca. Announcement in the OBD subforum of intent to complete a specific exponent's TF to 92 bits, followed by initiation and completion of that exponent to 92 bits under a different mersenne.ca user name shall constitute sufficient proof that the two user names correspond to the same individual. That individual may retain priority on P-1 on the exponent by making a clear reservation in this thread, or cede the option to another specific individual or group, by posting clearly his intent in this thread including the recipient(s), or surrender the option entirely. (Something analogous to "I cede the option to reserve exponent 332192xxxx P-1 to user(s) insert-forum-name(s)-here".) Prior agreement by PM that the recipient(s) will commit to timely reserving and completing the P-1 is recommended in the case of ceding the option, as is prior qualification of hardware by the recipient(s). The option period does not reset upon ceding. Ceding recursively is discouraged. If the TF-to-92 user decides not to attempt P-1 on the exponent, announcing that in this thread promptly would be helpful.
4. In the absence of a reservation by the TF-to-92-bits user (or if the option is ceded to others, by those others) and after a month past TF completion to 92 bits, or upon the TF-to-92 user posting its release before a month passes, the exponent becomes fair game for anyone with sufficient (qualified) hardware resources to reserve for P-1.
5. P-1 is estimated to be ~21,663. GHzDays per attempt of both stages without factor found early, 5,220. GHzDays per attempt of stage 1 only, and require ~16 GiB of ECC ram in stage 1, ~64 GiB of ECC ram per system in stage 2. (128 GiB of ECC ram per system would be better in stage 2.) Those who haven't the hardware resources to complete that within a year ought not make solo P-1 reservations or attempts. (~59.4 GHzD/day minimum at ~192M fft length needed for completion of both stages solo within one year running continuously nonstop. Roughly a Xeon Phi 7210 or dual-12-core Xeon e5-2697v2)
6. To qualify for reserving OBD exponents for P-1, post in this thread, results of run-time scaling derived from multiple widely-spaced exponents full P-1 runs, including above 100Mdigit (and ideally also nearer 1Gbit) that demonstrates scaling consistent with completion within a year for OBD P-1, on the hardware planned to be used, along with a description of the software and hardware, sufficient for OBD P-1. Some run time scaling examples are posted at the Mlucas v20.1.x P-1 run time scaling reference post. Post title beginning with "Qualification" is suggested. To increase total possible throughput and participation, qualification either unconditionally, capable of completion of both stages within one year on one system, or conditionally, capable of completion within one year for a single stage or by splitting stage 2 with another equal or faster system, are allowed. Conditional qualifications should include " (conditional)" in the post title. Allow plenty of time for performing qualification, which may take weeks or months. Users intending to qualify multiple systems are encouraged to include a system name in each qualification post. Stating OS, Mlucas version, CPU model, available ram and type, and estimated OBD P-1 completion time are required.
7. Optionally, systems tested and found to fail qualification may be posted, possibly saving others the time and effort of futile attempts to qualify similar hardware. At least the CPU model should be stated, and additional details required for qualification are encouraged. Post title should begin with "Disqualification".
8. Reservation is by posting a statement in this thread, of intent to P-1 factor a specific exponent. Post title "Reservation " followed by the exponent and whether the reservation is joint, solo, for both stages or only stage 1, is suggested. For example, "Reservation 3321928307 solo both P-1 stages", "Reservation 3321928171 joint both P-1 stages", "Reservation 3321928319 solo P-1 stage 1", "Reservation 3321928319 joint P-1 stage 2". Stage 2 is possible to do as a split effort with multiple systems, or joint effort with systems by multiple users; stage 1 is not. "Joint P-1 stage 1" is an oxymoron and constitutes an invalid reservation that will be ignored. Explicitly stating P-1 is proposed, to preclude possible uncertainty if P+1 or other multistage factoring becomes available for OBD exponents in the future.
9. Joint reservations should be posted by the user who completed the TF to 92 bits (or by whomever he ceded the right to), if posting in the one-month option period. Otherwise reservation posting may be done by whoever is selected by mutual agreement in PM among the joint effort participants.
10. P-1 reservations omitting joint, solo, or stage, from subject and body, will be presumed solo for both stages. Full specification is encouraged.
11. P-1 reservations omitting identification of joint effort participants other than the poster may be treated as a single-user effort. Coordination and ensuring completion and result reporting in a joint effort is nominally the responsibility of the poster of the reservation.
12. Use P-1 bounds corresponding to the GPU72 row for the exponent at mersenne.ca. https://www.mersenne.ca/exponent/3321928307 shows B1=17,000,000, B2=1,000,000,000.
13. Reporting P-1 progress in this thread roughly monthly to quarterly is requested. (4-12 times per year.) Include exponent, stage, % complete, ETA. For joint efforts, update stage 2 progress by participant and system also. Dialogue by PM among joint participants for a specific reservation is recommended to establish participation, settle details such as stage 2 bounds split, etc (rather than chatty posting in this thread)
14. Reservations expire after any of the following occur:
    1. One year passes after the last progress report that showed progress on that reservation.
    2. Two years pass after initial reservation.
    3. A factor is found and reported here.
    4. The user posts surrender of a reservation here.
    5. The factoring attempt is completed without finding a factor and is reported here. (In a joint attempt, that would require completion and reporting of all the reservation's participants' portions of the split stage 2.)
15. If a new factor is found, or a reservation completed without finding a factor, it should be reported here. It should also get added to the mersenne.ca database. Put the result(s) in a text file and upload at https://www.mersenne.ca/bulk-factors.php
16. Reservation and other status for exponents 3321928171 through 3321929987 will be maintained in a Libre Office spreadsheet by kriesel, and periodic snapshots posted in this thread.
17. Revision of reservations are to be avoided, but may be performed by posting clearly in this thread, for justifiable reasons such as time deadlines, equipment failure, etc, and must include the exponent and change. It might involve converting a solo reservation to joint, or shortening from both stages to stage 1. Revisions ought meet the same information content requirements as initial reservation.
18. For reservations for only stage 1, the user should plan on preserving the stage 1 files until such time as a user reserves stage 2 on the same exponent, or a central repository for such files is established, and make all reasonable and necessary effort to transfer the stage 1 files for reuse, and ensure their correctness.

Constructive civil comments are invited.

[kriesel]

Current TF status is shown at https://www.mersenne.ca/obd. Current indicated Level is 22.14.
3321928171 completed to the full 92 bits, 2022-02-05 by mersenne.ca user johnny_jack
3321928307 78% complete from 91 to 92, ETA 2022-02-23 by kriesel
3321928319 71% complete from 91 to 92, ETA 2021-02-26 by kriesel
3321938373 complete to 90, available for reservation to higher
There are 15 others in the exponent range up to 3321939987, in progress at bit levels 86-87 to 88-89 by kriesel, as part of an effort to go expeditiously to OBD Level 24 or higher. Anyone with a sufficiently fast GPU is welcome to help reach OBD level 24.

At the moment:
OBD TF completed to 92, ready for reservation or release to others for P-1: 1
OBD TF completed to 91, reserved to 92 bits: 2

Systems with qualification(s) completed & posted for OBD P-1: 3 (1 is unconditionally)
Exponents reserved for P-1: 0
Exponents completed thru stage 1 P-1: 0
Exponents completed thru stage 2 P-1: 0

It's now up to the user who completed TF to 92 on 3321928171, to reserve it for P-1, or reserve it as a joint effort, or release it for reservation for P-1 by others, by 2022-03-05. No action results in delay until by default it becomes available to all for reservation after 2022-03-05.