mersenneforum.org Why can't I start factmsieve.py with a poly file, but no factor base?
 Register FAQ Search Today's Posts Mark Forums Read

 2018-03-17, 23:29 #1 EdH     "Ed Hall" Dec 2009 Adirondack Mtns 5,101 Posts Why can't I start factmsieve.py with a poly file, but no factor base? OK, I thought this worked, but it won't work again. The code looks like it should run, but it tries to create a factor base file and crashes. Particulars: --factmsieve won't run poly select on clients. --msieve won't run the full poly select across more than one machine. If I use more than one machine, each makes its own factor base (.fb), even if I combine all the *.p files. --if I separate the poly steps, msieve goes off for ages to do the final root phase. --I can use a separate poly select routine that makes a final poly for the gnfs sievers, but factmsieve.py won't run from there without a fuss. --If I build a factor base file from the poly, factmsieve.py scolds me. Code: \$ python ../factmsieve.py comp94.n 1 3 Code: -> Running Python 2.7 -> This is client 1 of 3 -> Running on 4 Cores with 2 hyper-threads per Core -> Working with NAME = comp94 -> Selected default factorization parameters for 95 digit level. -> Selected lattice siever: gnfs-lasieve4I11e -> Parameter change detected, dumping relations... -> procrels -fb comp94.fb -prel rels.bin -dump __________________________________________________________ | This is the procrels program for GGNFS. | | Version: 0.77.1-20060722-nocona | | This program is copyright 2004, Chris Monico, and subject| | to the terms of the GNU General Public License version 2.| |__________________________________________________________| loadFB(): Could not open comp94.fb for read! Could not load FB from comp94.fb! Return value 255. Terminating... frustration mounts...
 2018-03-17, 23:41 #2 Dubslow Basketry That Evening!     "Bunslow the Bold" Jun 2011 40
2018-03-18, 02:57   #3
EdH

"Ed Hall"
Dec 2009

5,101 Posts

Quote:
 Originally Posted by Dubslow Having read but not truly understood your particulars, lets at least examine that factmsieve.py output: Code: loadFB(): Could not open comp94.fb for read! That looks like an OS/FS problem more than an NFS/msieve problem (though I could certainly be wrong, never having used the script). Are you sure the file exists with the right name?
The .fb file does not exist, on purpose. the .fb file is created by each unique machine. Each of those files contains no score reference, so there is no way to determine which is better. factmsieve.py will create a poly file from the named .fb file, but I am trying to skip the .fb file by providing a .poly file. Perhaps I am misreading the following snippet, or more probably not following its relation to other code:
Code:
# Is there a poly file already, or do we need to create one?
if not os.path.exists(NAME + '.poly'):
if os.path.exists(NAME + '.fb'):
output('-> Converting msieve polynomial (*.fb) to ggnfs (*.poly) format.')
fb_to_poly()
...
I am pursuing something else more convoluted and I could always go back to my own sieving work distribution scripts, but this has already been done and I've already had success with factmsieve.py. I just don't think now that I was using the correct poly in my previous runs. I think one of the .fb files was somehow dragged into the mix rather than the optimum poly.

2018-03-18, 05:37   #4
hyramgraff

Jan 2018

418 Posts

Quote:
 Originally Posted by EdH Code: -> procrels -fb comp94.fb -prel rels.bin -dump
Where is the .fb file that procrels is expecting supposed to be created? This looks like where you're having issues with the file not being found.

 2018-03-18, 14:36 #5 EdH     "Ed Hall" Dec 2009 Adirondack Mtns 5,101 Posts I tried the following steps: --I ran three poly select processes on three separate machines. Each created a factor base, but of course, each was different. --I ran the perl script to find the best poly from the three .p files. --I found the .fb file that matched the poly and renamed it to match the input number file. --I removed all the extra files, so there would be no contamination. --I ran: Code: python ../factmsieve.py comp94.n 1 3 Failure!: Code: -> | Monico. Version 0.86 (Python 2.6 or later) 20th June 2011. | -> |______________________________________________________________| -> Running Python 2.7 -> This is client 1 of 3 -> Running on 4 Cores with 2 hyper-threads per Core -> Working with NAME = comp94 -> Converting msieve polynomial (*.fb) to ggnfs (*.poly) format. -> Selected default factorization parameters for 95 digit level. -> Selected lattice siever: gnfs-lasieve4I11e -> Parameter change detected, dumping relations... -> procrels -fb comp94.fb -prel rels.bin -dump __________________________________________________________ | This is the procrels program for GGNFS. | | Version: 0.77.1-20060722-nocona | | This program is copyright 2004, Chris Monico, and subject| | to the terms of the GNU General Public License version 2.| |__________________________________________________________| readPoly(): Did not find a valid 'n' value! loadFB(): readPoly() reported an error! Could not load FB from comp94.fb! Return value 255. Terminating... comp94.fb: Code: N 1975636228803860706131861386351317508435774072460176838764200263234956507563682801432890234281 SKEW 1256916.79 R0 -48400675083113392109375 R1 80840080969 A0 -61641819524944283968862064 A1 1061875853900591730890 A2 79968233798293 A3 -593456406 A4 360 comp94.poly: Code: n: 1975636228803860706131861386351317508435774072460176838764200263234956507563682801432890234281 Y0: -48400675083113392109375 Y1: 80840080969 c0: -61641819524944283968862064 c1: 1061875853900591730890 c2: 79968233798293 c3: -593456406 c4: 360 skew: 1256916.79 type: gnfs comp94.log: Code:  -> factmsieve.py (v0.86) -> This is client 1 of 3 -> Running on 4 Cores with 2 hyper-threads per Core -> Working with NAME = comp94 -> Converting msieve polynomial (*.fb) to ggnfs (*.poly) format. -> Selected lattice siever: gnfs-lasieve4I11e -> Parameter change detected, dumping relations... -> procrels -fb comp94.fb -prel rels.bin -dump comp94.n: Code: n: 1975636228803860706131861386351317508435774072460176838764200263234956507563682801432890234281 Maybe I need to rename and leave the appropriate .p file...
 2018-03-18, 14:46 #6 RichD     Sep 2008 Kansas 2×1,879 Posts FWIW, on the Perl side, the R0/R1 follow the A's.
2018-03-18, 15:21   #7
EdH

"Ed Hall"
Dec 2009

10011111011012 Posts

Quote:
 Originally Posted by RichD FWIW, on the Perl side, the R0/R1 follow the A's.
Yeah, and I used that order when I manually constructed a factor base file in one of my previous efforts. I moved them again just now and tried, with no change.

I suppose I'll just have to write my own routine or modify factmsieve.py as I did to get it to run my mpi stuff way back when I knew remembered more about python...

 2018-03-18, 15:45 #8 RichD     Sep 2008 Kansas 2·1,879 Posts Again FWIW, on the Perl side, all I need is the .poly file to get it started. Code: perl ./factmsieve.pl comp94.poly
2018-03-18, 16:11   #9
EdH

"Ed Hall"
Dec 2009

5,101 Posts

Quote:
 Originally Posted by RichD Again FWIW, on the Perl side, all I need is the .poly file to get it started. Code: perl ./factmsieve.pl comp94.poly
You bring up a very good point, but the reason I'm trying to use the python script is because it already takes care of dividing the sieving ranges up for all my subsequent machines. I haven't looked at the perl script in a number of years, but I guess I should.

I have discovered that part of the reason it is failing appears to be that I need more info in the .fb file to calculate the sieving parameters. Until I can incorporate the extra machines into the poly selection, I have created a work-around by suppressing the abort code when I start factmsieve.py with client info. I then need to wait until the primary machine (client 1) generates the poly, to start subsequent machines. Not ideal, but quite workable for now (other than the lost poly time from the other machines).

Thanks for all the help.

 2018-03-18, 18:20 #10 schickel     "Frank <^>" Dec 2004 CDP Janesville 2·1,061 Posts You can comment out the code that calls procrels. That is a hold over from the original ggnfs suite. procrels was use to accumulate the sieved relations; it makes sure the relations are completely factored and does a de-dupe of the new relations against the already found relations. There is/was a .params file created at the start of a job that contains the fb bound and the *lp settings. If there is a change in any of the values, procrels is called with "-dump" to dump all the relations in raw format and reread them to make sure they're still valid with the new bounds. It looks like the script isn't finding the .params file (or if it's there, it doesn't match the .poly) so procrels gets called to dump/reprocess any accumulated relations but procrels doesn't find a "proper" .fb file.
2018-03-18, 19:15   #11
EdH

"Ed Hall"
Dec 2009

5,101 Posts

Quote:
 Originally Posted by schickel You can comment out the code that calls procrels. That is a hold over from the original ggnfs suite. procrels was use to accumulate the sieved relations; it makes sure the relations are completely factored and does a de-dupe of the new relations against the already found relations. There is/was a .params file created at the start of a job that contains the fb bound and the *lp settings. If there is a change in any of the values, procrels is called with "-dump" to dump all the relations in raw format and reread them to make sure they're still valid with the new bounds. It looks like the script isn't finding the .params file (or if it's there, it doesn't match the .poly) so procrels gets called to dump/reprocess any accumulated relations but procrels doesn't find a "proper" .fb file.
Thanks! That helps me understand a little bit more. If I can understand enough, I'll build distribution into the poly select part of the script. My level of ignorance at what I am seeing leads me to believe this can be done without too difficult a time. I'll see...

 Similar Threads Thread Thread Starter Forum Replies Last Post jasong Conjectures 'R Us 36 2010-08-03 06:25 Peter Hackman Factoring 7 2009-10-26 18:27 michaf Conjectures 'R Us 18 2008-05-21 10:08 Golem86 Factoring 11 2007-07-17 18:08 hallstei Factoring 5 2005-04-19 11:58

All times are UTC. The time now is 09:54.

Sun Dec 4 09:54:30 UTC 2022 up 108 days, 7:23, 0 users, load averages: 1.14, 1.09, 1.02