mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > GMP-ECM

Reply
 
Thread Tools
Old 2015-01-01, 16:31   #56
wombatman
I moo ablest echo power!
 
wombatman's Avatar
 
May 2013

178510 Posts
Default

Thanks! I'll try it out and let you know!
wombatman is offline   Reply With Quote
Old 2015-02-01, 17:11   #57
Antonio
 
Antonio's Avatar
 
"Antonio Key"
Sep 2011
UK

32×59 Posts
Default

Quote:
Originally Posted by WraithX View Post
Announcing ecm.py version 0.30, the changes are:
Code:
-New feature that shows ETA (estimated time to job completion)
 * Since it is based on the average runtime of the job so far, the estimate can be a little early or late.  But, in general gives a good idea of how long the current job will take.  
 
-New format for the running output.  And you can select between different displays of Runtime and ETA like so:
# Examples of how you can mix and match Runtime and ETA outputs:
#____________________________________________________________________________
# Curves Complete |   Average seconds/curve   |    Runtime    |      ETA
#-----------------|---------------------------|---------------|--------------
#  2114 of   6000 | Stg1  2983s | Stg2 693.5s |      1945628s |  41d 08:37:51
#  2114 of   6000 | Stg1  2983s | Stg2 693.5s |       22.518d |  41d 08:37:51
#  2114 of   6000 | Stg1  2983s | Stg2 693.5s |  22d 12:27:08 |      3573471s
#  2114 of   6000 | Stg1  2983s | Stg2 693.5s |  22d 12:27:08 |       41.359d
#  2114 of   6000 | Stg1  2983s | Stg2 693.5s |  22d 12:27:08 |  41d 08:37:51
 
-New email feature.
 * This option will send out an email when the job completes, either when it finds a factor or when all curves have finished with no factor found.  The email will let you know which event happened.
 * This option can also email out progress reports at regular intervals, so you can have a record of how far along remote machines are in their job.
 
-New log file feature.
 * This option keeps a record of the work that ecm.py has done.  Each line in the log file gets a time stamp so you can know when jobs were started and when they finished.
 * This option can also record the above "runtime information" at regular intervals.
Great work thanks, works like a charm. Couple of suggestions: -

1. Can it be made to accept the ecm -inp command line option rather than redirected stdin?

2. Could it automatically rebalance the thread load if {(curves completed by fastest thread) - (curves completed by slowest thread)} > (number of threads),
currently I stop and restart to achieve this but it would be nice if I could leave it completely unattended.
Antonio is offline   Reply With Quote
Old 2015-02-02, 14:23   #58
xilman
Bamboozled!
 
xilman's Avatar
 
"π’‰Ίπ’ŒŒπ’‡·π’†·π’€­"
May 2003
Down not across

24·13·53 Posts
Default

Quote:
Originally Posted by Antonio View Post
1. Can it be made to accept the ecm -inp command line option rather than redirected stdin?
Just curious, but why?

Either "ecm < foo" or "cat foo | ecm" do exactly the same thing (though 'cat' is spelled 'type' on Windoze systems).

I ask because Paul Z. (i.e. the other Paul) has recently suggested doing away with -inp and several other options in order to simplify the code base and, as the examples show above, it is trivially implemented anyway.

Last fiddled with by xilman on 2015-02-02 at 14:24
xilman is offline   Reply With Quote
Old 2015-02-02, 19:03   #59
Antonio
 
Antonio's Avatar
 
"Antonio Key"
Sep 2011
UK

10000100112 Posts
Default

Quote:
Originally Posted by xilman View Post
Just curious, but why?

Either "ecm < foo" or "cat foo | ecm" do exactly the same thing (though 'cat' is spelled 'type' on Windoze systems).

I ask because Paul Z. (i.e. the other Paul) has recently suggested doing away with -inp and several other options in order to simplify the code base and, as the examples show above, it is trivially implemented anyway.
No real reason, other than personal preference I suppose.
Antonio is offline   Reply With Quote
Old 2015-02-14, 15:22   #60
WraithX
 
WraithX's Avatar
 
Mar 2006

48610 Posts
Default ecm.py v0.32

Quote:
Originally Posted by Antonio View Post
Great work thanks, works like a charm. Couple of suggestions: -

1. Can it be made to accept the ecm -inp command line option rather than redirected stdin?

2. Could it automatically rebalance the thread load if {(curves completed by fastest thread) - (curves completed by slowest thread)} > (number of threads),
currently I stop and restart to achieve this but it would be nice if I could leave it completely unattended.
Announcing ecm.py version 0.32. The only change was to add the -inp feature request from above. Starting the program can look like either of the following now:
Code:
python.exe ecm.py [gmp-ecm options] [ecm.py options] B1 < <in_file>
ecm.py -inp <in_file> [gmp-ecm options] [ecm.py options] B1
Implementing number 1 was simple enough.

Implementing number 2 would be really difficult, and potentially not beneficial. Currently I start different ecm.exe processes each with num_curves/num_threads curves of work to do. To "rebalance the load", I'd have to kill all running instances of ecm.exe and then start them up again. Each time you rebalance, you will probably lose between (num_threads/2) to (num_threads) number of curves of work (due to killing ecm.exe when it is partially through its current curve). And, starting ecm.exe incurs some initialization costs (extra time), which will build up with each rebalance. It was really interesting to hear this suggestion and think about the pros/cons, but I've decided not to implement it.
Attached Files
File Type: zip ecm-py_v0.32.zip (14.4 KB, 154 views)
WraithX is offline   Reply With Quote
Old 2015-03-05, 10:41   #61
Antonio
 
Antonio's Avatar
 
"Antonio Key"
Sep 2011
UK

32·59 Posts
Default

Quote:
Originally Posted by WraithX View Post
Announcing ecm.py version 0.32. The only change was to add the -inp feature request from above. Starting the program can look like either of the following now:
Code:
python.exe ecm.py [gmp-ecm options] [ecm.py options] B1 < <in_file>
ecm.py -inp <in_file> [gmp-ecm options] [ecm.py options] B1
Implementing number 1 was simple enough.

Implementing number 2 would be really difficult, and potentially not beneficial. Currently I start different ecm.exe processes each with num_curves/num_threads curves of work to do. To "rebalance the load", I'd have to kill all running instances of ecm.exe and then start them up again. Each time you rebalance, you will probably lose between (num_threads/2) to (num_threads) number of curves of work (due to killing ecm.exe when it is partially through its current curve). And, starting ecm.exe incurs some initialization costs (extra time), which will build up with each rebalance. It was really interesting to hear this suggestion and think about the pros/cons, but I've decided not to implement it.
Thanks for implementing (1), I can understand the reason for rejecting (2), however, have you considered the impact of only rebalancing the load if a thread runs out of work (and there are more than, say, 2*threads number of curves remaining to be done?

Now for something more serious:-

As an exercise I was trying out ecm.py on 23*10^20371+7 and found a composite factor, 570237 (3*67*2837) so I tried
(23*10^20371+7)/(3*67*2837) and this produced the following:

Code:
F:\ECM>rem t20
F:\ECM>ecm.py -inp num2ecm.txt -c 80 11000
-> ___________________________________________________________________
-> | Running ecm.py, a Python driver for distributing GMP-ECM work   |
-> | on a single machine.  It is copyright, 2011-2015, David Cleaver |
-> | and is a conversion of factmsieve.py that is Copyright, 2010,   |
-> | Brian Gladman. Version 0.32 (Python 2.6 or later) 14th Feb 2015 |
-> |_________________________________________________________________|
inp_file = num2ecm.txt
-> Number(s) to factor:
Traceback (most recent call last):
  File "F:\ECM\ecm.py", line 1493, in <module>
    parse_ecm_options(sys.argv, set_args = True, first = True)
  File "F:\ECM\ecm.py", line 1397, in parse_ecm_options
    print('-> {0:s} ({1:d} digits)'.format(line, num_digits(eval(line.replace('^', '**')))))
  File "<string>", line 1, in <module>
OverflowError: integer division result too large for a float
This is on Windows 7 sp1 64bit with Python version: -
Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:44:40)

Any ideas?

Last fiddled with by Antonio on 2015-03-05 at 10:43
Antonio is offline   Reply With Quote
Old 2015-03-05, 13:58   #62
casevh
 
Dec 2005

23 Posts
Default

In Python 2, the division operator performs integer division, but in Python 3 was changed to always return a float. Both versions of Python support "//" to perform integer division. Can you try changing your input to use "//"?
casevh is offline   Reply With Quote
Old 2015-03-05, 19:26   #63
Antonio
 
Antonio's Avatar
 
"Antonio Key"
Sep 2011
UK

32·59 Posts
Default

Changing from / to // allows the python script to work, but when the number is passed to ecm.exe it doesn't parse the // correctly and ignores it and all that follows. So it just finds the 570237 composite factor again.

Last fiddled with by Antonio on 2015-03-05 at 19:43
Antonio is offline   Reply With Quote
Old 2015-03-05, 20:46   #64
casevh
 
Dec 2005

23 Posts
Default

Untested, but try changing the offending line (1397 ??) from

Code:
print('-> {0:s} ({1:d} digits)'.format(line, num_digits(eval(line.replace('^', '**')))))
to

Code:
print('-> {0:s} ({1:d} digits)'.format(line, num_digits(eval(line.replace('^', '**').replace('/', '//')))))
casevh is offline   Reply With Quote
Old 2015-03-05, 22:58   #65
Antonio
 
Antonio's Avatar
 
"Antonio Key"
Sep 2011
UK

32·59 Posts
Default

Quote:
Originally Posted by casevh View Post
Untested, but try changing the offending line (1397 ??) from

Code:
print('-> {0:s} ({1:d} digits)'.format(line, num_digits(eval(line.replace('^', '**')))))
to

Code:
print('-> {0:s} ({1:d} digits)'.format(line, num_digits(eval(line.replace('^', '**').replace('/', '//')))))

I applied the suggested change to lines
987
1384
1397
1523
1573
and
1634
all now seems to be working ok
many thanks.

Last fiddled with by Antonio on 2015-03-05 at 23:23
Antonio is offline   Reply With Quote
Old 2015-03-06, 03:35   #66
WraithX
 
WraithX's Avatar
 
Mar 2006

2×35 Posts
Default Announcing ecm.py v0.33...

Quote:
Originally Posted by casevh View Post
Code:
print('-> {0:s} ({1:d} digits)'.format(line, num_digits(eval(line.replace('^', '**').replace('/', '//')))))
Quote:
Originally Posted by Antonio View Post
I applied the suggested change to lines
987, 1384, 1397, 1523, 1573, and 1634
all now seems to be working ok
many thanks.
A big thanks to casevh for suggesting and Antonio for verifying this solution. I've decided to simplify the code a bit and have put the eval(n.replace('^', '**').replace('/', '//')) in the num_digits function, so now the use of num_digits looks like: format(ecm_n, num_digits(ecm_n)). This looks a lot cleaner, and any future changes can be made in one place.

Here is the updated version, ecm.py v0.33.
Attached Files
File Type: zip ecm-py_v0.33.zip (14.4 KB, 140 views)
WraithX is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Python Coding Help? kelzo Programming 3 2016-11-27 05:16
PHP vs. Python vs. C (all with GMP) daxmick Programming 2 2014-02-10 01:45
Python... Xyzzy Programming 20 2009-09-08 15:51
using libecm from python yqiang GMP-ECM 2 2007-04-22 00:14
Help w/ python. a216vcti Programming 7 2005-10-30 00:37

All times are UTC. The time now is 03:00.


Mon Nov 29 03:00:24 UTC 2021 up 128 days, 21:29, 0 users, load averages: 1.07, 1.12, 1.13

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.