mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > GMP-ECM

Reply
 
Thread Tools
Old 2020-06-17, 20:21   #111
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

2×2,617 Posts
Default

Sorry if this is a same query I might have asked before.

Has anyone expanded ecm.py to work across more than one machine. I am trying to do so, but if someone else has already, I'd be interested in their work, since I'm sure it would be better than anything I cobble together.

I'd prefer to stay away from an mpi implementation, since I'd be using machines with different capabilities and openmpi was still broken in Ubuntu 18.04, last I checked. I have run ecmpi and may even use it again on my 16.04 machines, but would prefer another route.

The direction I'm currently leaning toward is a controlling script (probably bash) that works as a server holding the basic info and keeping track of returned curve count. Then, clients grab the basic info, perform the curves and report back to the server. When the count is complete the server reports success (factors) or failure.
EdH is offline   Reply With Quote
Old 2020-06-17, 21:16   #112
RichD
 
RichD's Avatar
 
Sep 2008
Kansas

5×761 Posts
Default

This sort of sounds like ECMnet...
RichD is offline   Reply With Quote
Old 2020-06-18, 02:15   #113
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

121628 Posts
Default

Quote:
Originally Posted by RichD View Post
This sort of sounds like ECMnet...
I'm going to have to look into that again. I looked at it a few years ago, but don't remember why I didn't continue with it. Thanks. For now, I actually did go with ecmpi, but only with some of the similar machines. There are several more I'd like to be able to add, but they would slow down all the others. ECMnet might just allow adding them at their own pace.
EdH is offline   Reply With Quote
Old 2020-06-18, 08:08   #114
xilman
Bamboozled!
 
xilman's Avatar
 
"๐’‰บ๐’ŒŒ๐’‡ท๐’†ท๐’€ญ"
May 2003
Down not across

2D8A16 Posts
Default

Quote:
Originally Posted by EdH View Post
I'm going to have to look into that again. I looked at it a few years ago, but don't remember why I didn't continue with it. Thanks. For now, I actually did go with ecmpi, but only with some of the similar machines. There are several more I'd like to be able to add, but they would slow down all the others. ECMnet might just allow adding them at their own pace.
I've been running and ECMNET server for years. Email me if you think I may be able to help.
xilman is offline   Reply With Quote
Old 2020-06-18, 12:05   #115
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

2·2,617 Posts
Default

Quote:
Originally Posted by xilman View Post
I've been running and ECMNET server for years. Email me if you think I may be able to help.
Thanks! I'm looking it all over, but in the meantime, I've reconfigured ecmpi with a select set of better machines. I'll see where I get and contact you with any questions.
EdH is offline   Reply With Quote
Old 2021-08-30, 22:54   #116
SethTro
 
SethTro's Avatar
 
"Seth"
Apr 2019

1DD16 Posts
Default

@WraithX

I added ecm.py to my misc-scripts github repo (let me know if you're not happy with this).

I also made a number of changes: cleaning up unused functions, adding some tests, simplifying functions in places

Years ago I planned to make a local db with all existing B1:param:sigma to keep track of what had finished and what was queued.

Last fiddled with by SethTro on 2021-08-30 at 22:56
SethTro is offline   Reply With Quote
Old 2022-11-04, 16:59   #117
SuikaPredator
 
Aug 2022
China

408 Posts
Default Found a bug

Bug: when ecm.py receives -c XXX -threads YYY as params and 0<XXX<YYY, it will run curves indefinitely.

Potential fix: change line 2094 from
Code:
ecm_args1 += ' -c {0:d}'.format(ecm_c//intNumThreads)
into
Code:
ecm_args1 += ' -c {0:d}'.format(max(ecm_c//intNumThreads,1))
I am not sure about whether to modify variable actual_num_threads or not.
SuikaPredator is offline   Reply With Quote
Old 2022-11-04, 17:06   #118
swellman
 
swellman's Avatar
 
Jun 2012

23×13×37 Posts
Default

I must ask why would one request more threads than curves to be run? User error I suppose but never by intention, surely?
swellman is offline   Reply With Quote
Old 2022-11-04, 17:18   #119
SuikaPredator
 
Aug 2022
China

2016 Posts
Default

Quote:
Originally Posted by swellman View Post
I must ask why would one request more threads than curves to be run? User error I suppose but never by intention, surely?
When the -one option is not set and the number of remaining curves is less than the number of threads, ecm.py will enqueue that many curves to the remaining composite and cause that bug. Actually I just met that bug in this case and found out that it had ran thousands of unneeded curves.
SuikaPredator is offline   Reply With Quote
Old 2022-11-05, 03:30   #120
WraithX
 
WraithX's Avatar
 
Mar 2006

10000101012 Posts
Default ecm.py version 0.46

That line 2094 is from Seth's version of ecm.py. I just tried to download and run his version, but it gave me errors with both python 2 and 3, so I couldn't confirm the error.

When I tested my version of ecm.py (v0.44), with -c 3 -threads 4, it only ran 3 curves and finished as expected. However, just to be sure another code path doesn't trigger this issue I updated the referenced line to:
ecm_args1 += ' -c {0:d}'.format(max(ecm_c//intNumThreads,1))

This is version 0.46.
Attached Files
File Type: zip ecm-py_v0.46.zip (21.6 KB, 66 views)
WraithX is offline   Reply With Quote
Old 2022-11-06, 15:50   #121
SuikaPredator
 
Aug 2022
China

25 Posts
Default

Found more bugs on max memory setting... (Seth's version on github, with possible modifications)

When the program is doing extra curves after finding the first factor, it will decrease maxmem on each number by a factor of #threads. I haven't figured out any possible fix yet.

Logs:
Code:
->=============================================================================
-> Working on number: 994165605052757388...557428370002287411 (798 digits)
-> Currently working on: job0360.txt
-> Starting 12 instances of GMP-ECM...
-> ecm -maxmem 341 -c 10 25e4 < job0360.txt > job0360_t00.txt
-> ecm -maxmem 341 -c 9 25e4 < job0360.txt > job0360_t01.txt
-> ecm -maxmem 341 -c 9 25e4 < job0360.txt > job0360_t02.txt
-> ecm -maxmem 341 -c 9 25e4 < job0360.txt > job0360_t03.txt
-> ecm -maxmem 341 -c 9 25e4 < job0360.txt > job0360_t04.txt
-> ecm -maxmem 341 -c 9 25e4 < job0360.txt > job0360_t05.txt
-> ecm -maxmem 341 -c 9 25e4 < job0360.txt > job0360_t06.txt
-> ecm -maxmem 341 -c 9 25e4 < job0360.txt > job0360_t07.txt
-> ecm -maxmem 341 -c 9 25e4 < job0360.txt > job0360_t08.txt
-> ecm -maxmem 341 -c 9 25e4 < job0360.txt > job0360_t09.txt
-> ecm -maxmem 341 -c 9 25e4 < job0360.txt > job0360_t10.txt
-> ecm -maxmem 341 -c 9 25e4 < job0360.txt > job0360_t11.txt

Using B1=250000, B2=183032866, polynomial Dickson(3), 12 threads
GMP-ECM 7.0.4 [configured with GMP 6.1.2, --enable-asm-redc] [ECM]
____________________________________________________________________________
 Curves Complete |   Average seconds/curve   |    Runtime    |      ETA
-----------------|---------------------------|---------------|--------------
   109 of    109 | Stg1 23.16s | Stg2 7.248s |   0d 00:04:54 |   0d 00:00:10

factor_data: Using B1=250000, B2=183032866, polynomial Dickson(3), sigma=1:3888514963
-> *** No factor found.


->=============================================================================
-> Working on number: 448142163283738247...836780873159362451 (803 digits)
-> Currently working on: job1983.txt
-> Starting 12 instances of GMP-ECM...
-> ecm -maxmem 28 -c 16 25e4 < job1983.txt > job1983_t00.txt
-> ecm -maxmem 28 -c 16 25e4 < job1983.txt > job1983_t01.txt
-> ecm -maxmem 28 -c 16 25e4 < job1983.txt > job1983_t02.txt
-> ecm -maxmem 28 -c 15 25e4 < job1983.txt > job1983_t03.txt
-> ecm -maxmem 28 -c 15 25e4 < job1983.txt > job1983_t04.txt
-> ecm -maxmem 28 -c 15 25e4 < job1983.txt > job1983_t05.txt
-> ecm -maxmem 28 -c 15 25e4 < job1983.txt > job1983_t06.txt
-> ecm -maxmem 28 -c 15 25e4 < job1983.txt > job1983_t07.txt
-> ecm -maxmem 28 -c 15 25e4 < job1983.txt > job1983_t08.txt
-> ecm -maxmem 28 -c 15 25e4 < job1983.txt > job1983_t09.txt
-> ecm -maxmem 28 -c 15 25e4 < job1983.txt > job1983_t10.txt
-> ecm -maxmem 28 -c 15 25e4 < job1983.txt > job1983_t11.txt

Using B1=250000, B2=146985312, polynomial Dickson(3), 12 threads
GMP-ECM 7.0.4 [configured with GMP 6.1.2, --enable-asm-redc] [ECM]
____________________________________________________________________________
 Curves Complete |   Average seconds/curve   |    Runtime    |      ETA
-----------------|---------------------------|---------------|--------------
   183 of    183 | Stg1 22.36s | Stg2 7.534s |   0d 00:08:01 |   0d 00:00:03

factor_data: Using B1=250000, B2=146985312, polynomial Dickson(3), sigma=1:1882742718
-> *** No factor found.


->=============================================================================
-> Working on number: 114127662372551733...201108166477746793 (811 digits)
-> Currently working on: job7908.txt
-> Starting 12 instances of GMP-ECM...
-> ecm -maxmem 2 -c 36 25e4 < job7908.txt > job7908_t00.txt
-> ecm -maxmem 2 -c 36 25e4 < job7908.txt > job7908_t01.txt
-> ecm -maxmem 2 -c 36 25e4 < job7908.txt > job7908_t02.txt
-> ecm -maxmem 2 -c 36 25e4 < job7908.txt > job7908_t03.txt
-> ecm -maxmem 2 -c 35 25e4 < job7908.txt > job7908_t04.txt
-> ecm -maxmem 2 -c 35 25e4 < job7908.txt > job7908_t05.txt
-> ecm -maxmem 2 -c 35 25e4 < job7908.txt > job7908_t06.txt
-> ecm -maxmem 2 -c 35 25e4 < job7908.txt > job7908_t07.txt
-> ecm -maxmem 2 -c 35 25e4 < job7908.txt > job7908_t08.txt
-> ecm -maxmem 2 -c 35 25e4 < job7908.txt > job7908_t09.txt
-> ecm -maxmem 2 -c 35 25e4 < job7908.txt > job7908_t10.txt
-> ecm -maxmem 2 -c 35 25e4 < job7908.txt > job7908_t11.txt

Using B1=250000, B2=124690660, polynomial Dickson(3), 12 threads
GMP-ECM 7.0.4 [configured with GMP 6.1.2, --enable-asm-redc] [ECM]
____________________________________________________________________________
 Curves Complete |   Average seconds/curve   |    Runtime    |      ETA
-----------------|---------------------------|---------------|--------------
   424 of    424 | Stg1 23.70s | Stg2 54.71s |   0d 00:47:22 |   0d 00:00:04

factor_data: Using B1=250000, B2=124690660, polynomial Dickson(3), sigma=1:2818859778
-> *** No factor found.


->=============================================================================
-> Working on number: 401539325668293637...329189871599519489 (807 digits)
-> Currently working on: job9196.txt
-> Starting 12 instances of GMP-ECM...
-> ecm -maxmem 0 -c 34 25e4 < job9196.txt > job9196_t00.txt
-> ecm -maxmem 0 -c 33 25e4 < job9196.txt > job9196_t01.txt
-> ecm -maxmem 0 -c 33 25e4 < job9196.txt > job9196_t02.txt
-> ecm -maxmem 0 -c 33 25e4 < job9196.txt > job9196_t03.txt
-> ecm -maxmem 0 -c 33 25e4 < job9196.txt > job9196_t04.txt
-> ecm -maxmem 0 -c 33 25e4 < job9196.txt > job9196_t05.txt
-> ecm -maxmem 0 -c 33 25e4 < job9196.txt > job9196_t06.txt
-> ecm -maxmem 0 -c 33 25e4 < job9196.txt > job9196_t07.txt
-> ecm -maxmem 0 -c 33 25e4 < job9196.txt > job9196_t08.txt
-> ecm -maxmem 0 -c 33 25e4 < job9196.txt > job9196_t09.txt
-> ecm -maxmem 0 -c 33 25e4 < job9196.txt > job9196_t10.txt
-> ecm -maxmem 0 -c 33 25e4 < job9196.txt > job9196_t11.txt

Using B1=250000, B2=183032866, polynomial Dickson(3), 12 threads
GMP-ECM 7.0.4 [configured with GMP 6.1.2, --enable-asm-redc] [ECM]
____________________________________________________________________________
 Curves Complete |   Average seconds/curve   |    Runtime    |      ETA
-----------------|---------------------------|---------------|--------------
   397 of    397 | Stg1 22.55s | Stg2 7.073s |   0d 00:16:59 |   0d 00:00:03

factor_data: Using B1=250000, B2=183032866, polynomial Dickson(3), sigma=1:1550168464
-> *** No factor found.
Also, it is not confirmed as a bug of v0.46 because I am not running that version. Maybe I will try to use v0.46 from today.
SuikaPredator is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Python Coding Help? kelzo Programming 3 2016-11-27 05:16
PHP vs. Python vs. C (all with GMP) daxmick Programming 2 2014-02-10 01:45
Python... Xyzzy Programming 20 2009-09-08 15:51
using libecm from python yqiang GMP-ECM 2 2007-04-22 00:14
Help w/ python. a216vcti Programming 7 2005-10-30 00:37

All times are UTC. The time now is 07:32.


Thu Feb 2 07:32:04 UTC 2023 up 168 days, 5 hrs, 1 user, load averages: 1.04, 0.88, 0.84

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2023, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.

โ‰  ยฑ โˆ“ รท ร— ยท โˆ’ โˆš โ€ฐ โŠ— โŠ• โŠ– โŠ˜ โŠ™ โ‰ค โ‰ฅ โ‰ฆ โ‰ง โ‰จ โ‰ฉ โ‰บ โ‰ป โ‰ผ โ‰ฝ โŠ โŠ โŠ‘ โŠ’ ยฒ ยณ ยฐ
โˆ  โˆŸ ยฐ โ‰… ~ โ€– โŸ‚ โซ›
โ‰ก โ‰œ โ‰ˆ โˆ โˆž โ‰ช โ‰ซ โŒŠโŒ‹ โŒˆโŒ‰ โˆ˜ โˆ โˆ โˆ‘ โˆง โˆจ โˆฉ โˆช โจ€ โŠ• โŠ— ๐–• ๐–– ๐–— โŠฒ โŠณ
โˆ… โˆ– โˆ โ†ฆ โ†ฃ โˆฉ โˆช โŠ† โŠ‚ โŠ„ โŠŠ โŠ‡ โŠƒ โŠ… โŠ‹ โŠ– โˆˆ โˆ‰ โˆ‹ โˆŒ โ„• โ„ค โ„š โ„ โ„‚ โ„ต โ„ถ โ„ท โ„ธ ๐“Ÿ
ยฌ โˆจ โˆง โŠ• โ†’ โ† โ‡’ โ‡ โ‡” โˆ€ โˆƒ โˆ„ โˆด โˆต โŠค โŠฅ โŠข โŠจ โซค โŠฃ โ€ฆ โ‹ฏ โ‹ฎ โ‹ฐ โ‹ฑ
โˆซ โˆฌ โˆญ โˆฎ โˆฏ โˆฐ โˆ‡ โˆ† ฮด โˆ‚ โ„ฑ โ„’ โ„“
๐›ข๐›ผ ๐›ฃ๐›ฝ ๐›ค๐›พ ๐›ฅ๐›ฟ ๐›ฆ๐œ€๐œ– ๐›ง๐œ ๐›จ๐œ‚ ๐›ฉ๐œƒ๐œ— ๐›ช๐œ„ ๐›ซ๐œ… ๐›ฌ๐œ† ๐›ญ๐œ‡ ๐›ฎ๐œˆ ๐›ฏ๐œ‰ ๐›ฐ๐œŠ ๐›ฑ๐œ‹ ๐›ฒ๐œŒ ๐›ด๐œŽ๐œ ๐›ต๐œ ๐›ถ๐œ ๐›ท๐œ™๐œ‘ ๐›ธ๐œ’ ๐›น๐œ“ ๐›บ๐œ”