A Python Diver for GMP-ECM... - Page 2

WraithX · 2011-05-08, 18:21

Announcing ecm.py version 0.05, the changes are:

New Feature:
- Now prints the GMP-ECM configuration line in the console window and in the output file. (ie, GMP-ECM 6.3 [configured with GMP 5.0.1] [ECM])

Bug Fix:
- The -pollfiles command line option wasn't actually being applied, meaning only the default value was used during runtime. This has been fixed.

P.S. Can a Mod remove the zip file (v0.04) from post number 7? Thank you.

WraithX · 2011-06-10, 01:27

Announcing ecm.py version 0.06, the changes are:

New Features:
- Updated output to console and files to be more like GMP-ECM
- Print number of digits for each input number
- Print curve number that found the factor
- Print stage 1 and stage 2 timing for the curve that found a factor

P.S. Can a Mod please remove the zip file v0.04 from post #7 and v0.05 from post #12? Thank you.

WraithX · 2011-09-08, 01:51

Announcing ecm.py version 0.07, the changes are:

Minor Bug Fixes:
- When the input to GMP-ECM was an equation, the number of digits reported by the script was wrong. This has been corrected.
- If saving of output is requested, save the output from previous work that had been completed on this job. It was previously counted and then discarded.
- The previous command line parsing didn't pass the '-one' flag to GMP-ECM. This has been corrected.

New Feature:
- Added some basic input checking before passing an input line to GMP-ECM. This allows us to skip comment lines in a file (lines starting with #), to skip lines with 'invalid characters', and it helps us to count the number of digits for an input that is an equation.

Mini-Geek · 2011-09-22, 23:39

Quote:

Originally Posted by mklasson

Hi,

Mini-Geek pointed me to this thread and asked for a small fix to aliqueit to make it work with ecm.py. Could someone try it out and see if it seems ok?

Change use_ecmpy to true in aliqueit.ini and make sure ecmpy_cmd is set correctly.

http://mklasson.com/aliqueit110a.zip

I finally found some time to try this and ecm-py out, and they seem to work great together! They ECMd a c109 on my quad in no time, and I ran a smaller (~70 digits) sequence a little to make sure the factors were detected correctly.

Mini-Geek · 2011-09-25, 01:47

Quote:

Originally Posted by WraithX

-r <file> Resume the specified job file. After finishing this job, the script will stop.

If you use -r without resending the number to ECM, the script exits with no messages in non-verbose mode, or with the following in verbose mode: (which says it will send "job9096.txt" to GMP-ECM)

Code:

C:\Files\Prime\aliquot>ecm.py -r job9096.txt
-> ___________________________________________________________________
-> | Running ecm.py, a Python driver for distributing GMP-ECM work   |
-> | on a single machine.  It is Copyright, 2011, David Cleaver and  |
-> | is a conversion of factmsieve.py that is Copyright, 2010, Brian |
-> | Gladman.   Version 0.07 (Python 2.6 or later)  7th Sept 2011.   |
-> |_________________________________________________________________|

-> Original command line was:
->  job9096.txt
-> New command line(s) will be either:
->  job9096.txt
->  job9096.txt

The "for ecm_n in number_list:" loop is never entered because number_list is empty. If I echo in the number, I get one of the following errors, depending on if I call ecm.py directly, or call python and give it ecm.py (.py files are associated with python.exe, but some weird things with inputs mean it can be a little different):

Code:

C:\Files\Prime\aliquot\ecm>echo 3327046125926007069399831196172582927914127361035798047985
95753750302739930552267248499144293948801482976343539281 | ecm.py -r job9096.txt
Traceback (most recent call last):
  File "C:\Files\Prime\bin\ecm.py", line 1124, in <module>
    parse_ecm_options(sys.argv, set_args = True)
  File "C:\Files\Prime\bin\ecm.py", line 1041, in parse_ecm_options
    if not sys.stdin.isatty() and set_args and intResume != 1:
AttributeError: 'NoneType' object has no attribute 'isatty'
The process tried to write to a nonexistent pipe.

C:\Files\Prime\aliquot\ecm>echo 3327046125926007069399831196172582927914127361035798047985
95753750302739930552267248499144293948801482976343539281 | python /files/prime/bin/ecm.py
-r job9096.txt
The process tried to write to a nonexistent pipe.

Auto-resuming works fine. Unfortunately this isn't too much help when running through Aliqueit, since it starts from the lowest level of ECM, so all that's saved is the portion of the current level that you've done.

Mini-Geek · 2011-09-25, 19:40

I found the non-verbose level of output to be too high for my tastes while running Aliqueit, so I added a quiet level to the VERBOSE user flag. The modified script is attached. The output looks like this: Edit: adjusted the number abbreviation (which is quiet-mode-only) to take up as much space as possible without wrapping to two lines on a standard 80 character line, e.g. "-> Working on number: 267894106254354723342199974696983903161332... (68 digits)" (a character under in this case; assumed 3 digit length instead of bothering to figure it exactly).

Code:

c102: running 74 ecm curves at B1=11e3...
-> Working on number: 111...471 (102 digits)

GMP-ECM 6.3 [configured with GMP 5.0.1 and --enable-asm-redc] [ECM]
Using B1=11000, B2=1873422, polynomial x^1, 3 threads
Finished 74/74; avg s/curve: stg1 0.056s, stg2 0.049s; runtime: 4s

Or when resuming:

Code:

c102: running 214 ecm curves at B1=5e4...
-> Working on number: 111...471 (102 digits)
-> Found previous job file job1153.txt, will resume work...
-> *** Already completed 0 curves on this number...
-> *** Will run 214 more curves.

GMP-ECM 6.3 [configured with GMP 5.0.1 and --enable-asm-redc] [ECM]
Using B1=50000, B2=12746592, polynomial x^2, 3 threads
Finished 6/214; avg s/curve: stg1 0.246s, stg2 0.174s; runtime: 1s

Or when finding a factor:

Code:

c71: running 84 ecm curves at B1=5e4...
-> Working on number: 205...329 (71 digits)

GMP-ECM 6.3 [configured with GMP 5.0.1 and --enable-asm-redc] [ECM]
Using B1=50000, B2=12746592, polynomial x^2, 3 threads
Finished 16/84; avg s/curve: stg1 0.173s, stg2 0.124s; runtime: 4s

********** Factor found in step 2: 11273477395685202763693
Found probable prime factor of 23 digits: 11273477395685202763693
Composite cofactor 1821672576763466122668846803768085979236448803253 has 49 digits

Instead of this for no factor:

Code:

-> ___________________________________________________________________
-> | Running ecm.py, a Python driver for distributing GMP-ECM work   |
-> | on a single machine.  It is Copyright, 2011, David Cleaver and  |
-> | is a conversion of factmsieve.py that is Copyright, 2010, Brian |
-> | Gladman.   Version 0.07 (Python 2.6 or later)  7th Sept 2011.   |
-> |_________________________________________________________________|

-> Number(s) to factor:
-> 1110042291838354308811195059079495973007266473263641255266850395399993578389209519659321426138548
51471 (102 digits)
->=============================================================================
-> Working on number: 111004229183835430881119505907949597300726647326364125526685039539999357838920
951965932142613854851471 (102 digits)
-> Currently working on: job7476.txt
-> Starting 3 instances of GMP-ECM...

GMP-ECM 6.3 [configured with GMP 5.0.1 and --enable-asm-redc] [ECM]
Using B1=11000, B2=1873422, polynomial x^1, 3 threads
Finished 74/74; avg s/curve: stg1 0.054s, stg2 0.049s; runtime: 4s

-> *** No factor found.

WraithX · 2011-09-26, 02:35

Quote:

Originally Posted by Mini-Geek

If you use -r without resending the number to ECM, the script exits with no messages in non-verbose mode, or with the following in verbose mode: (which says it will send "job9096.txt" to GMP-ECM)

Auto-resuming works fine. Unfortunately this isn't too much help when running through Aliqueit, since it starts from the lowest level of ECM, so all that's saved is the portion of the current level that you've done.

Thank you for letting me know about this. I've never used the -r to resume and so never caught that it was broken. I've been trying to fix it, but I think it may require a big rewrite to get it correct.

I like what you have done with the changes to verbose mode. I will include your changes in my future releases.

I have a question. Did you try "ecm.py -r jobNNNN.txt" manually, or does aliquiet try that? I guess in either case, since it is broken it won't work, I was just curious.

And speaking of aliquiet resuming, is there a way for aliquiet to check for a job file for a given number, and then re-issue the curves for that number at the ecm level specified in the job file? Basically, instead of aliquiet starting ecm from scratch, it could trigger the auto-resume feature of ecm.py (which we know works). However, this will probably require a change to aliquiet.

I will keep working on fixing the -r option of ecm.py and will hopefully have a fix sometime this week.

Mini-Geek · 2011-09-26, 11:56

Quote:

Originally Posted by WraithX

Thank you for letting me know about this. I've never used the -r to resume and so never caught that it was broken. I've been trying to fix it, but I think it may require a big rewrite to get it correct.
...
I have a question. Did you try "ecm.py -r jobNNNN.txt" manually, or does aliquiet try that? I guess in either case, since it is broken it won't work, I was just curious.
...
I will keep working on fixing the -r option of ecm.py and will hopefully have a fix sometime this week.

I did it manually. Aliqueit doesn't use the -r command, since it doesn't know in advance that there's a job file to resume for the number; it just calls ecm.py with the same parameters as last time, and so ecm.py finds the file and autoresumes it.
Considering I don't do a whole lot of resuming, it's not too hard to copy/paste the old parameters saved in the job file to the command line; don't bother to fix it if you feel it's for only my sake.

Quote:

Originally Posted by WraithX

And speaking of aliquiet resuming, is there a way for aliquiet to check for a job file for a given number, and then re-issue the curves for that number at the ecm level specified in the job file? Basically, instead of aliquiet starting ecm from scratch, it could trigger the auto-resume feature of ecm.py (which we know works). However, this will probably require a change to aliquiet.

I'm sure it would require a change to aliqueit, but it strikes me as being entirely possible (how much work? I don't know). Its impact to performance would be tiny if it was only checked for on the first iteration if small factors were already known, instead of checking every number every time. (probably a trivial performance hit anyway, especially if it looked for them once and cached the results)

Quote:

Originally Posted by WraithX

I like what you have done with the changes to verbose mode. I will include your changes in my future releases.

Thank you, glad to hear that.

WraithX · 2011-10-20, 02:16

Announcing ecm.py version 0.08, the changes are:

Bug Fix:
Fixed resuming a specified file when using the -r option. Thank you to Mini-Geek for finding and reporting this.

New Feature:
New VERBOSE levels to give better control over how much information the script will output while running. You can set a default level in the script at the following line:

Code:

VERBOSE = 1 #set to 0, 1, or 2 for quiet, normal, or verbose

Thank you to Mini-Geek for writing and sharing this.

Dubslow · 2012-09-16, 19:08

Code:

∰∂ taskset f nice -n 19 ./ecm.py -c 7553 -one -threads 4 -out EMS.job 43000000 < num
-> ___________________________________________________________________
-> | Running ecm.py, a Python driver for distributing GMP-ECM work   |
-> | on a single machine.  It is Copyright, 2011, David Cleaver and  |
-> | is a conversion of factmsieve.py that is Copyright, 2010, Brian |
-> | Gladman.   Version 0.08 (Python 2.6 or later) 19th Oct 2011.    |
-> |_________________________________________________________________|
 
-> Number(s) to factor:
-> 96829488818499592481168771836336683023181156945795350980834458372199490598743221067775290195641203125439681639536219726888871822435629511515837059837171813128663335953886175536897367740550240372528813404899458874513057418332695709006061299277468749241875966062032012477732299909160292749026996368849279816035027111164073836173908645011 (335 digits)
->=============================================================================
-> Working on number: 96829488818499592481168771836336683023181156945795350980834458372199490598743221067775290195641203125439681639536219726888871822435629511515837059837171813128663335953886175536897367740550240372528813404899458874513057418332695709006061299277468749241875966062032012477732299909160292749026996368849279816035027111164073836173908645011 (335 digits)
-> Found previous job file job3362.txt, will resume work...
-> *** Already completed 0 curves on this number...
-> *** Will run 7553 more curves.
-> Currently working on: job3362.txt
-> Starting 4 instances of GMP-ECM...
-> ./ecm -one -c 1889 43000000 < job3362.txt > job3362_t00.txt
Traceback (most recent call last):
  File "./ecm.py", line 1224, in <module>
    start_ecm_threads()
  File "./ecm.py", line 521, in start_ecm_threads
    procs.append(run_exe(ECM, ecm_args2, in_file = ecm_job, out_file = file_name, wait = False))
  File "./ecm.py", line 325, in run_exe
    p = subprocess.Popen(args.split(' '), **al)
  File "/usr/lib/python2.7/subprocess.py", line 672, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1213, in _execute_child
    raise child_exception
OSError: [Errno 13] Permission denied

The binary is executable, i.e. `./ecm` works fine, and it's the same binary YAFU uses. This is Ubuntu 11.04, with "#! /usr/bin/python" which is version 2.7.1. (To be sure, I did chmod +x ./ecm, but that didn't do anything.)

Dubslow · 2012-09-16, 20:52

Quote:

Originally Posted by Dubslow

The binary is executable, i.e. `./ecm` works fine, and it's the same binary YAFU uses. This is Ubuntu 11.04, with "#! /usr/bin/python" which is version 2.7.1. (To be sure, I did chmod +x ./ecm, but that didn't do anything.)

For kicks I tried "#! /usr/bin/python3" (v 3.2).

Code:

-> Starting 4 instances of GMP-ECM...
-> ./ecm -one -c 1889 43000000 < job3362.txt > job3362_t00.txt
Traceback (most recent call last):
  File "./ecm.py", line 1224, in <module>
    start_ecm_threads()
  File "./ecm.py", line 521, in start_ecm_threads
    procs.append(run_exe(ECM, ecm_args2, in_file = ecm_job, out_file = file_name, wait = False))
  File "./ecm.py", line 325, in run_exe
    p = subprocess.Popen(args.split(' '), **al)
  File "/usr/lib/python3.2/subprocess.py", line 736, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.2/subprocess.py", line 1331, in _execute_child
    raise child_exception_type(err_msg)
RuntimeError: Exception occurred in preexec_fn.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Python Coding Help?	kelzo	Programming	3	2016-11-27 05:16
PHP vs. Python vs. C (all with GMP)	daxmick	Programming	2	2014-02-10 01:45
Python...	Xyzzy	Programming	20	2009-09-08 15:51
using libecm from python	yqiang	GMP-ECM	2	2007-04-22 00:14
Help w/ python.	a216vcti	Programming	7	2005-10-30 00:37