mersenneforum.org A Python Diver for GMP-ECM...
 Register FAQ Search Today's Posts Mark Forums Read

 2015-01-01, 16:31 #56 wombatman I moo ablest echo power!     May 2013 32·199 Posts Thanks! I'll try it out and let you know!
2015-02-01, 17:11   #57
Antonio

"Antonio Key"
Sep 2011
UK

10238 Posts

Quote:
 Originally Posted by WraithX Announcing ecm.py version 0.30, the changes are: Code: -New feature that shows ETA (estimated time to job completion) * Since it is based on the average runtime of the job so far, the estimate can be a little early or late. But, in general gives a good idea of how long the current job will take. -New format for the running output. And you can select between different displays of Runtime and ETA like so: # Examples of how you can mix and match Runtime and ETA outputs: #____________________________________________________________________________ # Curves Complete | Average seconds/curve | Runtime | ETA #-----------------|---------------------------|---------------|-------------- # 2114 of 6000 | Stg1 2983s | Stg2 693.5s | 1945628s | 41d 08:37:51 # 2114 of 6000 | Stg1 2983s | Stg2 693.5s | 22.518d | 41d 08:37:51 # 2114 of 6000 | Stg1 2983s | Stg2 693.5s | 22d 12:27:08 | 3573471s # 2114 of 6000 | Stg1 2983s | Stg2 693.5s | 22d 12:27:08 | 41.359d # 2114 of 6000 | Stg1 2983s | Stg2 693.5s | 22d 12:27:08 | 41d 08:37:51 -New email feature. * This option will send out an email when the job completes, either when it finds a factor or when all curves have finished with no factor found. The email will let you know which event happened. * This option can also email out progress reports at regular intervals, so you can have a record of how far along remote machines are in their job. -New log file feature. * This option keeps a record of the work that ecm.py has done. Each line in the log file gets a time stamp so you can know when jobs were started and when they finished. * This option can also record the above "runtime information" at regular intervals.
Great work thanks, works like a charm. Couple of suggestions: -

1. Can it be made to accept the ecm -inp command line option rather than redirected stdin?

currently I stop and restart to achieve this but it would be nice if I could leave it completely unattended.

2015-02-02, 14:23   #58
xilman
Bamboozled!

"𒉺𒌌𒇷𒆷𒀭"
May 2003
Down not across

71×157 Posts

Quote:
 Originally Posted by Antonio 1. Can it be made to accept the ecm -inp command line option rather than redirected stdin?
Just curious, but why?

Either "ecm < foo" or "cat foo | ecm" do exactly the same thing (though 'cat' is spelled 'type' on Windoze systems).

I ask because Paul Z. (i.e. the other Paul) has recently suggested doing away with -inp and several other options in order to simplify the code base and, as the examples show above, it is trivially implemented anyway.

Last fiddled with by xilman on 2015-02-02 at 14:24

2015-02-02, 19:03   #59
Antonio

"Antonio Key"
Sep 2011
UK

32·59 Posts

Quote:
 Originally Posted by xilman Just curious, but why? Either "ecm < foo" or "cat foo | ecm" do exactly the same thing (though 'cat' is spelled 'type' on Windoze systems). I ask because Paul Z. (i.e. the other Paul) has recently suggested doing away with -inp and several other options in order to simplify the code base and, as the examples show above, it is trivially implemented anyway.
No real reason, other than personal preference I suppose.

2015-02-14, 15:22   #60
WraithX

Mar 2006

5·101 Posts
ecm.py v0.32

Quote:
 Originally Posted by Antonio Great work thanks, works like a charm. Couple of suggestions: - 1. Can it be made to accept the ecm -inp command line option rather than redirected stdin? 2. Could it automatically rebalance the thread load if {(curves completed by fastest thread) - (curves completed by slowest thread)} > (number of threads), currently I stop and restart to achieve this but it would be nice if I could leave it completely unattended.
Announcing ecm.py version 0.32. The only change was to add the -inp feature request from above. Starting the program can look like either of the following now:
Code:
python.exe ecm.py [gmp-ecm options] [ecm.py options] B1 < <in_file>
ecm.py -inp <in_file> [gmp-ecm options] [ecm.py options] B1
Implementing number 1 was simple enough.

Implementing number 2 would be really difficult, and potentially not beneficial. Currently I start different ecm.exe processes each with num_curves/num_threads curves of work to do. To "rebalance the load", I'd have to kill all running instances of ecm.exe and then start them up again. Each time you rebalance, you will probably lose between (num_threads/2) to (num_threads) number of curves of work (due to killing ecm.exe when it is partially through its current curve). And, starting ecm.exe incurs some initialization costs (extra time), which will build up with each rebalance. It was really interesting to hear this suggestion and think about the pros/cons, but I've decided not to implement it.
Attached Files
 ecm-py_v0.32.zip (14.4 KB, 164 views)

2015-03-05, 10:41   #61
Antonio

"Antonio Key"
Sep 2011
UK

32·59 Posts

Quote:
 Originally Posted by WraithX Announcing ecm.py version 0.32. The only change was to add the -inp feature request from above. Starting the program can look like either of the following now: Code: python.exe ecm.py [gmp-ecm options] [ecm.py options] B1 < ecm.py -inp [gmp-ecm options] [ecm.py options] B1 Implementing number 1 was simple enough. Implementing number 2 would be really difficult, and potentially not beneficial. Currently I start different ecm.exe processes each with num_curves/num_threads curves of work to do. To "rebalance the load", I'd have to kill all running instances of ecm.exe and then start them up again. Each time you rebalance, you will probably lose between (num_threads/2) to (num_threads) number of curves of work (due to killing ecm.exe when it is partially through its current curve). And, starting ecm.exe incurs some initialization costs (extra time), which will build up with each rebalance. It was really interesting to hear this suggestion and think about the pros/cons, but I've decided not to implement it.
Thanks for implementing (1), I can understand the reason for rejecting (2), however, have you considered the impact of only rebalancing the load if a thread runs out of work (and there are more than, say, 2*threads number of curves remaining to be done?

Now for something more serious:-

As an exercise I was trying out ecm.py on 23*10^20371+7 and found a composite factor, 570237 (3*67*2837) so I tried
(23*10^20371+7)/(3*67*2837) and this produced the following:

Code:
F:\ECM>rem t20
F:\ECM>ecm.py -inp num2ecm.txt -c 80 11000
-> ___________________________________________________________________
-> | Running ecm.py, a Python driver for distributing GMP-ECM work   |
-> | on a single machine.  It is copyright, 2011-2015, David Cleaver |
-> | and is a conversion of factmsieve.py that is Copyright, 2010,   |
-> | Brian Gladman. Version 0.32 (Python 2.6 or later) 14th Feb 2015 |
-> |_________________________________________________________________|
inp_file = num2ecm.txt
-> Number(s) to factor:
Traceback (most recent call last):
File "F:\ECM\ecm.py", line 1493, in <module>
parse_ecm_options(sys.argv, set_args = True, first = True)
File "F:\ECM\ecm.py", line 1397, in parse_ecm_options
print('-> {0:s} ({1:d} digits)'.format(line, num_digits(eval(line.replace('^', '**')))))
File "<string>", line 1, in <module>
OverflowError: integer division result too large for a float
This is on Windows 7 sp1 64bit with Python version: -
Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:44:40)

Any ideas?

Last fiddled with by Antonio on 2015-03-05 at 10:43

 2015-03-05, 13:58 #62 casevh   Dec 2005 23 Posts In Python 2, the division operator performs integer division, but in Python 3 was changed to always return a float. Both versions of Python support "//" to perform integer division. Can you try changing your input to use "//"?
 2015-03-05, 19:26 #63 Antonio     "Antonio Key" Sep 2011 UK 32×59 Posts Changing from / to // allows the python script to work, but when the number is passed to ecm.exe it doesn't parse the // correctly and ignores it and all that follows. So it just finds the 570237 composite factor again. Last fiddled with by Antonio on 2015-03-05 at 19:43
 2015-03-05, 20:46 #64 casevh   Dec 2005 23 Posts Untested, but try changing the offending line (1397 ??) from Code: print('-> {0:s} ({1:d} digits)'.format(line, num_digits(eval(line.replace('^', '**'))))) to Code: print('-> {0:s} ({1:d} digits)'.format(line, num_digits(eval(line.replace('^', '**').replace('/', '//')))))
2015-03-05, 22:58   #65
Antonio

"Antonio Key"
Sep 2011
UK

32·59 Posts

Quote:
 Originally Posted by casevh Untested, but try changing the offending line (1397 ??) from Code: print('-> {0:s} ({1:d} digits)'.format(line, num_digits(eval(line.replace('^', '**'))))) to Code: print('-> {0:s} ({1:d} digits)'.format(line, num_digits(eval(line.replace('^', '**').replace('/', '//')))))

I applied the suggested change to lines
987
1384
1397
1523
1573
and
1634
all now seems to be working ok
many thanks.

Last fiddled with by Antonio on 2015-03-05 at 23:23

2015-03-06, 03:35   #66
WraithX

Mar 2006

5×101 Posts
Announcing ecm.py v0.33...

Quote:
 Originally Posted by casevh Code: print('-> {0:s} ({1:d} digits)'.format(line, num_digits(eval(line.replace('^', '**').replace('/', '//')))))
Quote:
 Originally Posted by Antonio I applied the suggested change to lines 987, 1384, 1397, 1523, 1573, and 1634 all now seems to be working ok many thanks.
A big thanks to casevh for suggesting and Antonio for verifying this solution. I've decided to simplify the code a bit and have put the eval(n.replace('^', '**').replace('/', '//')) in the num_digits function, so now the use of num_digits looks like: format(ecm_n, num_digits(ecm_n)). This looks a lot cleaner, and any future changes can be made in one place.

Here is the updated version, ecm.py v0.33.
Attached Files
 ecm-py_v0.33.zip (14.4 KB, 149 views)

 Similar Threads Thread Thread Starter Forum Replies Last Post kelzo Programming 3 2016-11-27 05:16 daxmick Programming 2 2014-02-10 01:45 Xyzzy Programming 20 2009-09-08 15:51 yqiang GMP-ECM 2 2007-04-22 00:14 a216vcti Programming 7 2005-10-30 00:37

All times are UTC. The time now is 08:24.

Sat Jan 29 08:24:10 UTC 2022 up 190 days, 2:53, 1 user, load averages: 2.53, 1.52, 1.37