A Python Diver for GMP-ECM...

[wombatman]

This line:

 ./usr/local/bin./ecm

would seem to be the issue, no? Can you run "./ecm" from the command line? If so, set the BIN_PATH to "" and it should work.

[Dubslow]

ECM_BIN_PATH = /usr/local/bin/ecm

[cgy606]

Originally Posted by wombatman

This line:

 ./usr/local/bin./ecm

would seem to be the issue, no? Can you run "./ecm" from the command line? If so, set the BIN_PATH to "" and it should work.

So I tried that and I get an error:

 bash-3.2$ ./ecm
Invalid arguments. See ./ecm --help.

I am concerned that maybe my ecm installation might be wrong...

[cgy606]

Originally Posted by Dubslow

ECM_BIN_PATH = /usr/local/bin/ecm

So I looked in the python script and you do not set "ECM_BIN_PATH". Rather one sets "ECM_PATH"

 # ###############################################################
# User Defined Variables, set before using...
# ###############################################################
# Set binary directory paths
ECM_PATH = './usr/local/bin'

And then the script prints out "ECM_BIN_PATH" (if their is an error in locating the binaries):

# check that an executable file exists

def check_binary(exe):
  if CHECK_BINARIES:
    if not os.path.exists(ECM_PATH + exe + EXE_SUFFIX):
      print('-> Could not find the program: {0:s}'.format(ECM_PATH + exe + EXE_SUFFIX))
      print('-> Did you set the path properly in this script?')
      print('-> It is currently set to:')
      print('-> ECM_BIN_PATH = {0:s}'.format(ECM_PATH))
      sys.exit(-1)
    if not os.path.isfile(ECM_PATH + exe + EXE_SUFFIX):
      print('-> The following is not a file: {0:s}'.format(ECM_PATH + exe + EXE_SUFFIX))
      print('-> Did you set the path properly in this script?')
      print('-> It is currently set to:')
      print('-> ECM_BIN_PATH = {0:s}'.format(ECM_PATH))
      sys.exit(-1)

I searched the entire script to make sure their are no other instances of "ECM_BIN_PATH", I couldn't find any (BTW I am using v 034 of the python script). Perhaps I am missing something?

[wombatman]

Originally Posted by cgy606

So I tried that and I get an error:

 bash-3.2$ ./ecm
Invalid arguments. See ./ecm --help.

I am concerned that maybe my ecm installation might be wrong...

No, that's actually good. Try changing the ECM_PATH to "" and run the script.

[cgy606]

Originally Posted by wombatman

No, that's actually good. Try changing the ECM_PATH to "" and run the script.

It works!!! I knew I was doing something boneheaded

bash-3.2$ echo "16592218505877252817595384320726519331245298560877363179444333539424579615734146863694149619172215730904092217599971446347242578938012841662398382297100491858685120782396088019559902200489" | python ecm.py -c 16 -threads 4 1000000
-> ___________________________________________________________________
-> | Running ecm.py, a Python driver for distributing GMP-ECM work   |
-> | on a single machine.  It is copyright, 2011-2015, David Cleaver |
-> | and is a conversion of factmsieve.py that is Copyright, 2010,   |
-> | Brian Gladman. Version 0.34 (Python 2.6 or later) 14th Mar 2015 |
-> |_________________________________________________________________|
 
-> Number(s) to factor:
-> 16592218505877252817595384320726519331245298560877363179444333539424579615734146863694149619172215730904092217599971446347242578938012841662398382297100491858685120782396088019559902200489 (188 digits)
->=============================================================================
-> Working on number: 165922185058772528...088019559902200489 (188 digits)
-> Currently working on: job9870.txt
-> Starting 4 instances of GMP-ECM...
-> ./ecm -c 4 1000000 < job9870.txt > job9870_t00.txt
-> ./ecm -c 4 1000000 < job9870.txt > job9870_t01.txt
-> ./ecm -c 4 1000000 < job9870.txt > job9870_t02.txt
-> ./ecm -c 4 1000000 < job9870.txt > job9870_t03.txt
 
GMP-ECM 7.0-rc2 [configured with GMP 6.0.0, --enable-asm-redc] [ECM]
Using B1=1000000, B2=1045563762, polynomial Dickson(6), 4 threads
____________________________________________________________________________
 Curves Complete |   Average seconds/curve   |    Runtime    |      ETA
-----------------|---------------------------|---------------|--------------
     8 of     16 | Stg1 16.90s | Stg2 8.497s |   0d 00:01:16 |   0d 00:02:15

Run 8 out of 16:
Using B1=1000000, B2=1045563762, polynomial Dickson(6), sigma=1:1972960141
Step 1 took 16815ms
********** Factor found in step 1: 99966551448308840149
Found prime factor of 20 digits: 99966551448308840149
Composite cofactor 165977702196287452743884566421762233309052750989118251380454425629289120250968365614593763832099318096182205447167223938283861744093049940708376659370742242048186838661 has 168 digits

It even found a factor. Now quick question, it finds the factor after 8 curves, and then quits. Is their a way to force the script to keep running until all the curves are done or all factors are prime (i.e. the number is fully factored)?

[VBCurtis]

The problem with having it keep running is that only the thread that found the factor would "know" about it; the other threads would continue to try to factor the original number. This is slower, often by a fair bit, than running curves on the new cofactor.

I usually run up to t40 with regular single-threaded ECM (which *does* continue after finding a factor), then use the python script for higher bounds; this reduces the incidence of ecm.py stopping its work due to a found factor (both because there are lower chances per day to find a factor once t40 has been run, and higher chance the cofactor is composite when I do find a 43+ digit factor).

[cgy606]

Well, I guess I could modify the script so that it does this. What I had in mind is that if one of the threads finds a factor, it first tests the primality of the large cofactor (ecm already does this), and if it is still composite, the script could kill all the other process, recalculate the input number (ecm should spit this out), and then reschedule the remains curves. I know yafu does this (or a process similar to this), but I was never able to get yafu to work on my macbook pro. Anyways, this should be a good exercise for me to do since I should learn python scripting anyways, for future jobs in the financial sector...

[VBCurtis]

If you do produce this mod, please post your work! I'd be happy to use that.

[WraithX]

Originally Posted by cgy606

It works!!! I knew I was doing something boneheaded

It even found a factor. Now quick question, it finds the factor after 8 curves, and then quits. Is their a way to force the script to keep running until all the curves are done or all factors are prime (i.e. the number is fully factored)?

I believe I can add this functionality. However, there are some questions about how to proceed. There are several cases that need to be considered:
1) If the factor is prime and the cofactor is prime, I think it's obvious we can stop.
2) If the factor is prime and the cofactor is composite
3) If the factor is composite and the cofactor is prime
4) If the factor is composite and the cofactor is composite

Should we start running the original number of curves on the new composite? Or only run the remaining number of curves on the composite? My guess from what you are asking is to "run the remaining number of curves" on the new composite. And then also, if two (or more) composites are found, should we run the remaining number of curves on each new composite, or run the original number of curves on each composite?

The second part of your question (keep running until the number is completely factored) I think is outside the scope of ecm.py. Much like it is outside the scope of ecm.exe. There is no logic (in ecm.py or ecm.exe) that can keep running curves, and keep choosing bounds to find potentially larger and larger factors that can adequately take into account: the amount of ram a system has available (or will have available in the future), and when the best time to switch over to different factoring methods, like gnfs/snfs, would be. That is more in the realm of yafu.

I can implement the first part of your question, but won't be implementing the second part. Also, since I'm now using the gpu capabilities of ecm, I need a way to start multiple ecm's to resume those saved stage1 residues. So, I'll be adding in the ability for ecm.py to recognize and work with the "-resume" option.

Also, one thing you should know about how paths work. In a previous post you wrote that you set:
ECM_PATH = './usr/local/bin'
That would not point to the /usr/local/bin directory, because you have a "." in front of it. The "." means start from the current working directory and look into these sub folders for what you want. So, if you ran the ecm.py script from the (made up) directory
/Users/cgy606/Documents/ecmpy/ and had set ECM_PATH = './usr/local/bin', it would look for the ecm executable in the directory:
/Users/cgy606/Documents/ecmpy/usr/local/bin/
Which proabably doesn't exist, and so will definitely fail. You could have set:
ECM_PATH = '/usr/local/bin'
Without the leading "." and that would have worked because it is an absolute path, and not a relative path. Since you've got it working now, you don't need to change it, but I wanted to let you know about the difference between absolute paths and relative paths.

[WraithX]

Recently I noticed that the ecm.py script would crash if it was given factorial or primorial input strings. This is because the python "eval" function can't handle these characters. So, instead of writing my own equation parser to figure out how many digits are in these input strings, I've just grabbed the output from ecm.exe to see how many digits it reports are in the input number. So,

Announcing ecm.py v0.35:

Fixed:
  - ecm.py no longer calculates number of digits on its own, it reads this information from the ecm executable.  This fixed a problem where the python "eval" function would crash when it encountered factorial or primorial characters.
    ie, you can now do: echo "140!+1" | python ecm.py 1e6 and it will work correctly, without crashing.
  - also fixed the output when the ecm binary is not found.  It will no longer print out the misleading "ECM_BIN_PATH", it will print out "ECM_PATH" to match the variable name in the python code.