mersenneforum.org  

Go Back   mersenneforum.org > Prime Search Projects > And now for something completely different

Reply
 
Thread Tools
Old 2022-05-09, 18:20   #34
paulunderwood
 
paulunderwood's Avatar
 
Sep 2002
Database er0rr

22×1,063 Posts
Default

Quote:
Originally Posted by xilman View Post
The step after that is to learn how to drive mpi properly.

I am running mpi-ecpp on a 6-core machine. Without additional information it fired up one master process and 5 workers. Because of hyperthreading two processes could be interleaved at relatively little cost on each physical processor.

That said, each process will grab its own share of memory physical and virtual and you may not be able to run as many processes as processors, let alone twice that.

I a'm not going to fiddle with a(n apparently) working invocation of mpi-ecpp but now wonder whether "mpi -np 7" might be a suitable incantation for a similar job, on the grounds that there may (just) be enough RAM available.

The above assumes that the remaining machines on the local net are not set up to run mpi jobs. This is a valid assumption right now. Whether it is worth the effort of setting them up remains to be seen.

Perhaps EdH can give us a synopsis on how to run client boxes.

Last fiddled with by paulunderwood on 2022-05-09 at 18:23
paulunderwood is offline   Reply With Quote
Old 2022-05-09, 20:04   #35
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

472810 Posts
Default

Quote:
Originally Posted by paulunderwood View Post
Perhaps EdH can give us a synopsis on how to run client boxes.
I need to read over the rest of the thread, but all my details for ecmpi are in my, "How I Install and Run ecmpi Across Several Ubuntu Machines" thread. I use openmpi, in case you're using something different.

Basically, I have an exact same user account on all the machines, with an exact same directory tree, and all machines are key linked via ssh with every other machine. This linking of all machines to all others is important. ecmpi is compiled on each machine, and the hostfile lists the localhost slots= and the user@IP slots= for each client.

I can get more detailed later. Let me look over the rest of the thread. Or, I'll try to answer more questions.
EdH is offline   Reply With Quote
Old 2022-05-09, 20:35   #36
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

23·3·197 Posts
Default

I'm not familiar enough with all the required libraries to implement this, yet. But, it appears that others are running openmpi and xilman knows how to incorporate LAN connected clients (which I already suspected). I'll address further questions, if any arise.
EdH is offline   Reply With Quote
Old 2022-05-09, 20:41   #37
Batalov
 
Batalov's Avatar
 
"Serge"
Mar 2008
Phi(4,2^7658614+1)/2

100110101101002 Posts
Default

I found an interesting feature in this implementation - it doesn't back-track. Could be something that is left for later work. This has a downside of sometimes taking more time in one single step than all the others combined.

In proving a 12k digit number, one step took 10 hrs (already near 10k digit size, i.e. similar to Greg's number above which took 20hrs in all, and on similar node) and then proceeded fast. This is deterministic, too - when repeated again from that step, it took ~10 hours again.

I sent the debug case input to Andreas.
Batalov is offline   Reply With Quote
Old 2022-05-10, 14:10   #38
frmky
 
frmky's Avatar
 
Jul 2003
So Cal

2×3×11×37 Posts
Default

Certification of a 15k digit prime, 2^49831-3637, took 29.7 hours using 12x Xeon E5-2670 v2 cpus with a total of 120 cores and 240 hyperthreads. The longest step of the second phase took 6.3 hours to complete.

It appears that idle processes near the end of the second phase continue to use processor time. I will next try a run with one process per core rather than two.
frmky is online now   Reply With Quote
Old 2022-05-10, 15:24   #39
Cybertronic
 
Cybertronic's Avatar
 
Jan 2007
Germany

22×3×41 Posts
Default

Wow , 15k in 30h ..that is fast !


So I have 3 milestone 20k numbers for testing !


Smallest 20000 digit prime and prime twins.


10^19999+110949
10^19999+1514722609+d, d=0,2


Should be done, the prover extended this page...
https://pzktupel.de/smx000.html


regards
Cybertronic is offline   Reply With Quote
Old 2022-05-10, 15:57   #40
paulunderwood
 
paulunderwood's Avatar
 
Sep 2002
Database er0rr

425210 Posts
Default questions about mpirun ecpp-mpi

Quote:
Originally Posted by EdH View Post
I'm not familiar enough with all the required libraries to implement this, yet. But, it appears that others are running openmpi and xilman knows how to incorporate LAN connected clients (which I already suspected). I'll address further questions, if any arise.
Regarding mpi, can ecpp-mpi be run on non-uniform processors on a network. Can clients drop out without affecting the server? Likewise, can clients be added at runtime?

As far as I can tell, the clients require passwordless ssh keys along with identical project directories and an installation of mpi. Do the clients require ecpp-mpi too?

Last fiddled with by paulunderwood on 2022-05-10 at 16:04
paulunderwood is offline   Reply With Quote
Old 2022-05-10, 16:52   #41
frmky
 
frmky's Avatar
 
Jul 2003
So Cal

2·3·11·37 Posts
Default

Quote:
Originally Posted by paulunderwood View Post
Regarding mpi, can ecpp-mpi be run on non-uniform processors on a network.
Yes
Quote:
Originally Posted by paulunderwood View Post
Can clients drop out without affecting the server?
No
Quote:
Originally Posted by paulunderwood View Post
Likewise, can clients be added at runtime?
No
Quote:
Originally Posted by paulunderwood View Post
As far as I can tell, the clients require passwordless ssh keys along with identical project directories and an installation of mpi.
That's right. In a homogenous cluster one typically uses a shared file system to host the binaries and input files, but identical directories on separate file systems will work.
Quote:
Originally Posted by paulunderwood View Post
Do the clients require ecpp-mpi too?
Yes, each compiled for the processor on which it will be run.
frmky is online now   Reply With Quote
Old 2022-05-10, 18:16   #42
frmky
 
frmky's Avatar
 
Jul 2003
So Cal

2×3×11×37 Posts
Default

Quote:
Originally Posted by frmky View Post
Certification of a 15k digit prime, 2^49831-3637, took 29.7 hours using 12x Xeon E5-2670 v2 cpus with a total of 120 cores and 240 hyperthreads. The longest step of the second phase took 6.3 hours to complete.

It appears that idle processes near the end of the second phase continue to use processor time. I will next try a run with one process per core rather than two.
Running one process per core reduced the time for the second phase to 3.7 hours (the time of the longest step), and the total time to 27 hours.
frmky is online now   Reply With Quote
Old 2022-05-10, 18:36   #43
xilman
Bamboozled!
 
xilman's Avatar
 
"๐’‰บ๐’ŒŒ๐’‡ท๐’†ท๐’€ญ"
May 2003
Down not across

262458 Posts
Default

Quote:
Originally Posted by frmky View Post
Running one process per core reduced the time for the second phase to 3.7 hours (the time of the longest step), and the total time to 27 hours.
Interesting.

Looks like my choice of 1 process per core was (accidentally) correct. Still wondering whether n+1 on a n-core machine might be a better choice to soak up some of the hyperthreading and have n workers alongside the single master. Next time, perhaps.
xilman is offline   Reply With Quote
Old 2022-05-11, 08:07   #44
kruoli
 
kruoli's Avatar
 
"Oliver"
Sep 2017
Porta Westfalica, DE

26×17 Posts
Default

On a fixed system, fastECPP was around double the speed of Primo for a ~10k digit prime.
kruoli is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
For which types of primes is GPU primality test software available? bur GPU Computing 6 2020-08-28 06:20
Fastest software for Mersenne primality test? JonathanM Information & Answers 25 2020-06-16 02:47
APR-CL as primality proof f1pokerspeed FactorDB 14 2014-01-09 21:06
Proof of Primality Test for Fermat Numbers princeps Math 15 2012-04-02 21:49
PRIMALITY PROOF for Wagstaff numbers! AntonVrba Math 96 2009-02-25 10:37

All times are UTC. The time now is 22:41.


Wed Aug 17 22:41:15 UTC 2022 up 41 days, 17:28, 1 user, load averages: 0.83, 0.89, 1.05

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.

โ‰  ยฑ โˆ“ รท ร— ยท โˆ’ โˆš โ€ฐ โŠ— โŠ• โŠ– โŠ˜ โŠ™ โ‰ค โ‰ฅ โ‰ฆ โ‰ง โ‰จ โ‰ฉ โ‰บ โ‰ป โ‰ผ โ‰ฝ โŠ โŠ โŠ‘ โŠ’ ยฒ ยณ ยฐ
โˆ  โˆŸ ยฐ โ‰… ~ โ€– โŸ‚ โซ›
โ‰ก โ‰œ โ‰ˆ โˆ โˆž โ‰ช โ‰ซ โŒŠโŒ‹ โŒˆโŒ‰ โˆ˜ โˆ โˆ โˆ‘ โˆง โˆจ โˆฉ โˆช โจ€ โŠ• โŠ— ๐–• ๐–– ๐–— โŠฒ โŠณ
โˆ… โˆ– โˆ โ†ฆ โ†ฃ โˆฉ โˆช โŠ† โŠ‚ โŠ„ โŠŠ โŠ‡ โŠƒ โŠ… โŠ‹ โŠ– โˆˆ โˆ‰ โˆ‹ โˆŒ โ„• โ„ค โ„š โ„ โ„‚ โ„ต โ„ถ โ„ท โ„ธ ๐“Ÿ
ยฌ โˆจ โˆง โŠ• โ†’ โ† โ‡’ โ‡ โ‡” โˆ€ โˆƒ โˆ„ โˆด โˆต โŠค โŠฅ โŠข โŠจ โซค โŠฃ โ€ฆ โ‹ฏ โ‹ฎ โ‹ฐ โ‹ฑ
โˆซ โˆฌ โˆญ โˆฎ โˆฏ โˆฐ โˆ‡ โˆ† ฮด โˆ‚ โ„ฑ โ„’ โ„“
๐›ข๐›ผ ๐›ฃ๐›ฝ ๐›ค๐›พ ๐›ฅ๐›ฟ ๐›ฆ๐œ€๐œ– ๐›ง๐œ ๐›จ๐œ‚ ๐›ฉ๐œƒ๐œ— ๐›ช๐œ„ ๐›ซ๐œ… ๐›ฌ๐œ† ๐›ญ๐œ‡ ๐›ฎ๐œˆ ๐›ฏ๐œ‰ ๐›ฐ๐œŠ ๐›ฑ๐œ‹ ๐›ฒ๐œŒ ๐›ด๐œŽ๐œ ๐›ต๐œ ๐›ถ๐œ ๐›ท๐œ™๐œ‘ ๐›ธ๐œ’ ๐›น๐œ“ ๐›บ๐œ”