mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   GMP-ECM (https://www.mersenneforum.org/forumdisplay.php?f=55)
-   -   A Python Diver for GMP-ECM... (https://www.mersenneforum.org/showthread.php?t=15508)

WraithX 2011-05-08 18:21

1 Attachment(s)
Announcing ecm.py version 0.05, the changes are:

[B]New Feature[/B]:
- Now prints the GMP-ECM configuration line in the console window and in the output file. (ie, GMP-ECM 6.3 [configured with GMP 5.0.1] [ECM])

[B]Bug Fix[/B]:
- The -pollfiles command line option wasn't actually being applied, meaning only the default value was used during runtime. This has been fixed.

P.S. Can a Mod remove the zip file (v0.04) from post number 7? Thank you.

WraithX 2011-06-10 01:27

1 Attachment(s)
Announcing ecm.py version 0.06, the changes are:

[B]New Features[/B]:
- Updated output to console and files to be more like GMP-ECM
- Print number of digits for each input number
- Print curve number that found the factor
- Print stage 1 and stage 2 timing for the curve that found a factor

P.S. Can a Mod please remove the zip file v0.04 from post #7 and v0.05 from post #12? Thank you.

WraithX 2011-09-08 01:51

1 Attachment(s)
Announcing ecm.py version 0.07, the changes are:

[B]Minor Bug Fixes[/B]:
- When the input to GMP-ECM was an equation, the number of digits reported by the script was wrong. This has been corrected.
- If saving of output is requested, save the output from previous work that had been completed on this job. It was previously counted and then discarded.
- The previous command line parsing didn't pass the '-one' flag to GMP-ECM. This has been corrected.

[B]New Feature[/B]:
- Added some basic input checking before passing an input line to GMP-ECM. This allows us to skip comment lines in a file (lines starting with #), to skip lines with 'invalid characters', and it helps us to count the number of digits for an input that is an equation.

Mini-Geek 2011-09-22 23:39

[QUOTE=mklasson;260148]Hi,

Mini-Geek pointed me to this thread and asked for a small fix to aliqueit to make it work with ecm.py. Could someone try it out and see if it seems ok?

Change use_ecmpy to true in aliqueit.ini and make sure ecmpy_cmd is set correctly.

[url]http://mklasson.com/aliqueit110a.zip[/url][/QUOTE]

I finally found some time to try this and ecm-py out, and they seem to work great together! They ECMd a c109 on my quad in no time, and I ran a smaller (~70 digits) sequence a little to make sure the factors were detected correctly.

Mini-Geek 2011-09-25 01:47

[QUOTE=WraithX;258976]-r <file> Resume the specified job file. After finishing this job, the script will stop.[/QUOTE]

If you use -r without resending the number to ECM, the script exits with no messages in non-verbose mode, or with the following in verbose mode: (which says it will send "job9096.txt" to GMP-ECM)
[code]C:\Files\Prime\aliquot>ecm.py -r job9096.txt
-> ___________________________________________________________________
-> | Running ecm.py, a Python driver for distributing GMP-ECM work |
-> | on a single machine. It is Copyright, 2011, David Cleaver and |
-> | is a conversion of factmsieve.py that is Copyright, 2010, Brian |
-> | Gladman. Version 0.07 (Python 2.6 or later) 7th Sept 2011. |
-> |_________________________________________________________________|

-> Original command line was:
-> job9096.txt
-> New command line(s) will be either:
-> job9096.txt
-> job9096.txt[/code]The "for ecm_n in number_list:" loop is never entered because number_list is empty. If I echo in the number, I get one of the following errors, depending on if I call ecm.py directly, or call python and give it ecm.py (.py files are associated with python.exe, but some weird things with inputs mean it can be a little different):
[code]C:\Files\Prime\aliquot\ecm>echo 3327046125926007069399831196172582927914127361035798047985
95753750302739930552267248499144293948801482976343539281 | ecm.py -r job9096.txt
Traceback (most recent call last):
File "C:\Files\Prime\bin\ecm.py", line 1124, in <module>
parse_ecm_options(sys.argv, set_args = True)
File "C:\Files\Prime\bin\ecm.py", line 1041, in parse_ecm_options
if not sys.stdin.isatty() and set_args and intResume != 1:
AttributeError: 'NoneType' object has no attribute 'isatty'
The process tried to write to a nonexistent pipe.

C:\Files\Prime\aliquot\ecm>echo 3327046125926007069399831196172582927914127361035798047985
95753750302739930552267248499144293948801482976343539281 | python /files/prime/bin/ecm.py
-r job9096.txt
The process tried to write to a nonexistent pipe.[/code]Auto-resuming works fine. Unfortunately this isn't too much help when running through Aliqueit, since it starts from the lowest level of ECM, so all that's saved is the portion of the current level that you've done.

Mini-Geek 2011-09-25 19:40

1 Attachment(s)
I found the non-verbose level of output to be too high for my tastes while running Aliqueit, so I added a quiet level to the VERBOSE user flag. The modified script is attached. The output looks like this: Edit: adjusted the number abbreviation (which is quiet-mode-only) to take up as much space as possible without wrapping to two lines on a standard 80 character line, e.g. "-> Working on number: 267894106254354723342199974696983903161332... (68 digits)" (a character under in this case; assumed 3 digit length instead of bothering to figure it exactly).
[code]c102: running 74 ecm curves at B1=11e3...
-> Working on number: 111...471 (102 digits)

GMP-ECM 6.3 [configured with GMP 5.0.1 and --enable-asm-redc] [ECM]
Using B1=11000, B2=1873422, polynomial x^1, 3 threads
Finished 74/74; avg s/curve: stg1 0.056s, stg2 0.049s; runtime: 4s[/code]
Or when resuming:
[code]c102: running 214 ecm curves at B1=5e4...
-> Working on number: 111...471 (102 digits)
-> Found previous job file job1153.txt, will resume work...
-> *** Already completed 0 curves on this number...
-> *** Will run 214 more curves.

GMP-ECM 6.3 [configured with GMP 5.0.1 and --enable-asm-redc] [ECM]
Using B1=50000, B2=12746592, polynomial x^2, 3 threads
Finished 6/214; avg s/curve: stg1 0.246s, stg2 0.174s; runtime: 1s[/code]
Or when finding a factor:
[code]c71: running 84 ecm curves at B1=5e4...
-> Working on number: 205...329 (71 digits)

GMP-ECM 6.3 [configured with GMP 5.0.1 and --enable-asm-redc] [ECM]
Using B1=50000, B2=12746592, polynomial x^2, 3 threads
Finished 16/84; avg s/curve: stg1 0.173s, stg2 0.124s; runtime: 4s

********** Factor found in step 2: 11273477395685202763693
Found probable prime factor of 23 digits: 11273477395685202763693
Composite cofactor 1821672576763466122668846803768085979236448803253 has 49 digits
[/code]
Instead of this for no factor:
[code]-> ___________________________________________________________________
-> | Running ecm.py, a Python driver for distributing GMP-ECM work |
-> | on a single machine. It is Copyright, 2011, David Cleaver and |
-> | is a conversion of factmsieve.py that is Copyright, 2010, Brian |
-> | Gladman. Version 0.07 (Python 2.6 or later) 7th Sept 2011. |
-> |_________________________________________________________________|

-> Number(s) to factor:
-> 1110042291838354308811195059079495973007266473263641255266850395399993578389209519659321426138548
51471 (102 digits)
->=============================================================================
-> Working on number: 111004229183835430881119505907949597300726647326364125526685039539999357838920
951965932142613854851471 (102 digits)
-> Currently working on: job7476.txt
-> Starting 3 instances of GMP-ECM...

GMP-ECM 6.3 [configured with GMP 5.0.1 and --enable-asm-redc] [ECM]
Using B1=11000, B2=1873422, polynomial x^1, 3 threads
Finished 74/74; avg s/curve: stg1 0.054s, stg2 0.049s; runtime: 4s

-> *** No factor found.[/code]

WraithX 2011-09-26 02:35

[QUOTE=Mini-Geek;272664]If you use -r without resending the number to ECM, the script exits with no messages in non-verbose mode, or with the following in verbose mode: (which says it will send "job9096.txt" to GMP-ECM)

Auto-resuming works fine. Unfortunately this isn't too much help when running through Aliqueit, since it starts from the lowest level of ECM, so all that's saved is the portion of the current level that you've done.[/QUOTE]

Thank you for letting me know about this. I've never used the -r to resume and so never caught that it was broken. I've been trying to fix it, but I think it may require a big rewrite to get it correct.

I like what you have done with the changes to verbose mode. I will include your changes in my future releases.

I have a question. Did you try "ecm.py -r jobNNNN.txt" manually, or does aliquiet try that? I guess in either case, since it is broken it won't work, I was just curious.

And speaking of aliquiet resuming, is there a way for aliquiet to check for a job file for a given number, and then re-issue the curves for that number at the ecm level specified in the job file? Basically, instead of aliquiet starting ecm from scratch, it could trigger the auto-resume feature of ecm.py (which we know works). However, this will probably require a change to aliquiet.

I will keep working on fixing the -r option of ecm.py and will hopefully have a fix sometime this week.

Mini-Geek 2011-09-26 11:56

[QUOTE=WraithX;272736]Thank you for letting me know about this. I've never used the -r to resume and so never caught that it was broken. I've been trying to fix it, but I think it may require a big rewrite to get it correct.
...
I have a question. Did you try "ecm.py -r jobNNNN.txt" manually, or does aliquiet try that? I guess in either case, since it is broken it won't work, I was just curious.
...
I will keep working on fixing the -r option of ecm.py and will hopefully have a fix sometime this week.[/QUOTE]
I did it manually. Aliqueit doesn't use the -r command, since it doesn't know in advance that there's a job file to resume for the number; it just calls ecm.py with the same parameters as last time, and so ecm.py finds the file and autoresumes it.
Considering I don't do a whole lot of resuming, it's not too hard to copy/paste the old parameters saved in the job file to the command line; don't bother to fix it if you feel it's for only my sake.
[QUOTE=WraithX;272736]And speaking of aliquiet resuming, is there a way for aliquiet to check for a job file for a given number, and then re-issue the curves for that number at the ecm level specified in the job file? Basically, instead of aliquiet starting ecm from scratch, it could trigger the auto-resume feature of ecm.py (which we know works). However, this will probably require a change to aliquiet.[/QUOTE]
I'm sure it would require a change to aliqueit, but it strikes me as being entirely possible (how much work? I don't know). Its impact to performance would be tiny if it was only checked for on the first iteration if small factors were already known, instead of checking every number every time. (probably a trivial performance hit anyway, especially if it looked for them once and cached the results)
[QUOTE=WraithX;272736]I like what you have done with the changes to verbose mode. I will include your changes in my future releases.[/QUOTE]
Thank you, glad to hear that. :smile:

WraithX 2011-10-20 02:16

1 Attachment(s)
Announcing ecm.py version 0.08, the changes are:

[B]Bug Fix[/B]:
Fixed resuming a specified file when using the -r option. Thank you to Mini-Geek for finding and reporting this.

[B]New Feature[/B]:
New VERBOSE levels to give better control over how much information the script will output while running. You can set a default level in the script at the following line:
[CODE]VERBOSE = 1 #set to 0, 1, or 2 for quiet, normal, or verbose[/CODE]
Thank you to Mini-Geek for writing and sharing this.

Dubslow 2012-09-16 19:08

:huh:
[code]∰∂ taskset f nice -n 19 ./ecm.py -c 7553 -one -threads 4 -out EMS.job 43000000 < num
-> ___________________________________________________________________
-> | Running ecm.py, a Python driver for distributing GMP-ECM work |
-> | on a single machine. It is Copyright, 2011, David Cleaver and |
-> | is a conversion of factmsieve.py that is Copyright, 2010, Brian |
-> | Gladman. Version 0.08 (Python 2.6 or later) 19th Oct 2011. |
-> |_________________________________________________________________|

-> Number(s) to factor:
-> 96829488818499592481168771836336683023181156945795350980834458372199490598743221067775290195641203125439681639536219726888871822435629511515837059837171813128663335953886175536897367740550240372528813404899458874513057418332695709006061299277468749241875966062032012477732299909160292749026996368849279816035027111164073836173908645011 (335 digits)
->=============================================================================
-> Working on number: 96829488818499592481168771836336683023181156945795350980834458372199490598743221067775290195641203125439681639536219726888871822435629511515837059837171813128663335953886175536897367740550240372528813404899458874513057418332695709006061299277468749241875966062032012477732299909160292749026996368849279816035027111164073836173908645011 (335 digits)
-> Found previous job file job3362.txt, will resume work...
-> *** Already completed 0 curves on this number...
-> *** Will run 7553 more curves.
-> Currently working on: job3362.txt
-> Starting 4 instances of GMP-ECM...
-> ./ecm -one -c 1889 43000000 < job3362.txt > job3362_t00.txt
Traceback (most recent call last):
File "./ecm.py", line 1224, in <module>
start_ecm_threads()
File "./ecm.py", line 521, in start_ecm_threads
procs.append(run_exe(ECM, ecm_args2, in_file = ecm_job, out_file = file_name, wait = False))
File "./ecm.py", line 325, in run_exe
p = subprocess.Popen(args.split(' '), **al)
File "/usr/lib/python2.7/subprocess.py", line 672, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1213, in _execute_child
raise child_exception
OSError: [Errno 13] Permission denied[/code]
The binary is executable, i.e. `./ecm` works fine, and it's the same binary YAFU uses. This is Ubuntu 11.04, with "#! /usr/bin/python" which is version 2.7.1. (To be sure, I did chmod +x ./ecm, but that didn't do anything.)

Dubslow 2012-09-16 20:52

[QUOTE=Dubslow;311872]:huh:

The binary is executable, i.e. `./ecm` works fine, and it's the same binary YAFU uses. This is Ubuntu 11.04, with "#! /usr/bin/python" which is version 2.7.1. (To be sure, I did chmod +x ./ecm, but that didn't do anything.)[/QUOTE]

For kicks I tried "#! /usr/bin/python3" (v 3.2).
[code]-> Starting 4 instances of GMP-ECM...
-> ./ecm -one -c 1889 43000000 < job3362.txt > job3362_t00.txt
Traceback (most recent call last):
File "./ecm.py", line 1224, in <module>
start_ecm_threads()
File "./ecm.py", line 521, in start_ecm_threads
procs.append(run_exe(ECM, ecm_args2, in_file = ecm_job, out_file = file_name, wait = False))
File "./ecm.py", line 325, in run_exe
p = subprocess.Popen(args.split(' '), **al)
File "/usr/lib/python3.2/subprocess.py", line 736, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.2/subprocess.py", line 1331, in _execute_child
raise child_exception_type(err_msg)
RuntimeError: Exception occurred in preexec_fn.[/code]


All times are UTC. The time now is 15:57.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.