mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > Factoring

Reply
 
Thread Tools
Old 2018-03-17, 23:29   #1
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

3×1,151 Posts
Default Why can't I start factmsieve.py with a poly file, but no factor base?

OK, I thought this worked, but it won't work again.

The code looks like it should run, but it tries to create a factor base file and crashes.

Particulars:
--factmsieve won't run poly select on clients.
--msieve won't run the full poly select across more than one machine. If I use more than one machine, each makes its own factor base (.fb), even if I combine all the *.p files.
--if I separate the poly steps, msieve goes off for ages to do the final root phase.
--I can use a separate poly select routine that makes a final poly for the gnfs sievers, but factmsieve.py won't run from there without a fuss.
--If I build a factor base file from the poly, factmsieve.py scolds me.
Code:
$ python ../factmsieve.py comp94.n 1 3
Code:
-> Running Python 2.7
-> This is client 1 of 3
-> Running on 4 Cores with 2 hyper-threads per Core
-> Working with NAME = comp94
-> Selected default factorization parameters for 95 digit level.
-> Selected lattice siever: gnfs-lasieve4I11e
-> Parameter change detected, dumping relations... 
-> procrels -fb comp94.fb -prel rels.bin -dump

 __________________________________________________________ 
|        This is the procrels program for GGNFS.           |
| Version: 0.77.1-20060722-nocona                          |
| This program is copyright 2004, Chris Monico, and subject|
| to the terms of the GNU General Public License version 2.|
|__________________________________________________________|
loadFB(): Could not open comp94.fb for read!
Could not load FB from comp94.fb!
Return value 255. Terminating...
frustration mounts...
EdH is offline   Reply With Quote
Old 2018-03-17, 23:41   #2
Dubslow
Basketry That Evening!
 
Dubslow's Avatar
 
"Bunslow the Bold"
Jun 2011
40<A<43 -89<O<-88

3·2,399 Posts
Default

Having read but not truly understood your particulars, lets at least examine that factmsieve.py output:

Code:
loadFB(): Could not open comp94.fb for read!
That looks like an OS/FS problem more than an NFS/msieve problem (though I could certainly be wrong, never having used the script). Are you sure the file exists with the right name?
Dubslow is offline   Reply With Quote
Old 2018-03-18, 02:57   #3
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

3·1,151 Posts
Default

Quote:
Originally Posted by Dubslow View Post
Having read but not truly understood your particulars, lets at least examine that factmsieve.py output:

Code:
loadFB(): Could not open comp94.fb for read!
That looks like an OS/FS problem more than an NFS/msieve problem (though I could certainly be wrong, never having used the script). Are you sure the file exists with the right name?
The .fb file does not exist, on purpose. the .fb file is created by each unique machine. Each of those files contains no score reference, so there is no way to determine which is better. factmsieve.py will create a poly file from the named .fb file, but I am trying to skip the .fb file by providing a .poly file. Perhaps I am misreading the following snippet, or more probably not following its relation to other code:
Code:
# Is there a poly file already, or do we need to create one?
if not os.path.exists(NAME + '.poly'):
  if os.path.exists(NAME + '.fb'):
    output('-> Converting msieve polynomial (*.fb) to ggnfs (*.poly) format.')
    fb_to_poly()
    ...
I am pursuing something else more convoluted and I could always go back to my own sieving work distribution scripts, but this has already been done and I've already had success with factmsieve.py. I just don't think now that I was using the correct poly in my previous runs. I think one of the .fb files was somehow dragged into the mix rather than the optimum poly.

Thanks for your reply.
EdH is offline   Reply With Quote
Old 2018-03-18, 05:37   #4
hyramgraff
 
Jan 2018

3·11 Posts
Default

Quote:
Originally Posted by EdH View Post
Code:
-> procrels -fb comp94.fb -prel rels.bin -dump
Where is the .fb file that procrels is expecting supposed to be created? This looks like where you're having issues with the file not being found.
hyramgraff is offline   Reply With Quote
Old 2018-03-18, 14:36   #5
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

3·1,151 Posts
Default

I tried the following steps:

--I ran three poly select processes on three separate machines. Each created a factor base, but of course, each was different.

--I ran the perl script to find the best poly from the three .p files.

--I found the .fb file that matched the poly and renamed it to match the input number file.

--I removed all the extra files, so there would be no contamination.

--I ran:
Code:
python ../factmsieve.py comp94.n 1 3
Failure!:
Code:
-> | Monico.   Version 0.86 (Python 2.6 or later) 20th June 2011. |
-> |______________________________________________________________|
-> Running Python 2.7
-> This is client 1 of 3
-> Running on 4 Cores with 2 hyper-threads per Core
-> Working with NAME = comp94
-> Converting msieve polynomial (*.fb) to ggnfs (*.poly) format.
-> Selected default factorization parameters for 95 digit level.
-> Selected lattice siever: gnfs-lasieve4I11e
-> Parameter change detected, dumping relations... 
-> procrels -fb comp94.fb -prel rels.bin -dump

 __________________________________________________________ 
|        This is the procrels program for GGNFS.           |
| Version: 0.77.1-20060722-nocona                          |
| This program is copyright 2004, Chris Monico, and subject|
| to the terms of the GNU General Public License version 2.|
|__________________________________________________________|
readPoly(): Did not find a valid 'n' value!
loadFB(): readPoly() reported an error!
Could not load FB from comp94.fb!
Return value 255. Terminating...
comp94.fb:
Code:
N 1975636228803860706131861386351317508435774072460176838764200263234956507563682801432890234281
SKEW 1256916.79
R0 -48400675083113392109375
R1 80840080969
A0 -61641819524944283968862064
A1 1061875853900591730890
A2 79968233798293
A3 -593456406
A4 360
comp94.poly:
Code:
n: 1975636228803860706131861386351317508435774072460176838764200263234956507563682801432890234281
Y0: -48400675083113392109375
Y1: 80840080969
c0: -61641819524944283968862064
c1: 1061875853900591730890
c2: 79968233798293
c3: -593456406
c4: 360
skew: 1256916.79
type: gnfs
comp94.log:
Code:
 -> factmsieve.py (v0.86)
 -> This is client 1 of 3
 -> Running on 4 Cores with 2 hyper-threads per Core
 -> Working with NAME = comp94
 -> Converting msieve polynomial (*.fb) to ggnfs (*.poly) format.
 -> Selected lattice siever: gnfs-lasieve4I11e
 -> Parameter change detected, dumping relations... 
 -> procrels -fb comp94.fb -prel rels.bin -dump
comp94.n:
Code:
n: 1975636228803860706131861386351317508435774072460176838764200263234956507563682801432890234281
Maybe I need to rename and leave the appropriate .p file...
EdH is offline   Reply With Quote
Old 2018-03-18, 14:46   #6
RichD
 
RichD's Avatar
 
Sep 2008
Kansas

C6A16 Posts
Default

FWIW, on the Perl side, the R0/R1 follow the A's.
RichD is offline   Reply With Quote
Old 2018-03-18, 15:21   #7
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

3×1,151 Posts
Default

Quote:
Originally Posted by RichD View Post
FWIW, on the Perl side, the R0/R1 follow the A's.
Yeah, and I used that order when I manually constructed a factor base file in one of my previous efforts. I moved them again just now and tried, with no change.

I suppose I'll just have to write my own routine or modify factmsieve.py as I did to get it to run my mpi stuff way back when I knew remembered more about python...
EdH is offline   Reply With Quote
Old 2018-03-18, 15:45   #8
RichD
 
RichD's Avatar
 
Sep 2008
Kansas

2×7×227 Posts
Default

Again FWIW, on the Perl side, all I need is the .poly file to get it started.
Code:
perl ./factmsieve.pl comp94.poly
RichD is offline   Reply With Quote
Old 2018-03-18, 16:11   #9
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

3·1,151 Posts
Default

Quote:
Originally Posted by RichD View Post
Again FWIW, on the Perl side, all I need is the .poly file to get it started.
Code:
perl ./factmsieve.pl comp94.poly
You bring up a very good point, but the reason I'm trying to use the python script is because it already takes care of dividing the sieving ranges up for all my subsequent machines. I haven't looked at the perl script in a number of years, but I guess I should.

I have discovered that part of the reason it is failing appears to be that I need more info in the .fb file to calculate the sieving parameters. Until I can incorporate the extra machines into the poly selection, I have created a work-around by suppressing the abort code when I start factmsieve.py with client info. I then need to wait until the primary machine (client 1) generates the poly, to start subsequent machines. Not ideal, but quite workable for now (other than the lost poly time from the other machines).

Thanks for all the help.
EdH is offline   Reply With Quote
Old 2018-03-18, 18:20   #10
schickel
 
schickel's Avatar
 
"Frank <^>"
Dec 2004
CDP Janesville

83216 Posts
Default

You can comment out the code that calls procrels. That is a hold over from the original ggnfs suite. procrels was use to accumulate the sieved relations; it makes sure the relations are completely factored and does a de-dupe of the new relations against the already found relations.

There is/was a .params file created at the start of a job that contains the fb bound and the *lp settings. If there is a change in any of the values, procrels is called with "-dump" to dump all the relations in raw format and reread them to make sure they're still valid with the new bounds.

It looks like the script isn't finding the .params file (or if it's there, it doesn't match the .poly) so procrels gets called to dump/reprocess any accumulated relations but procrels doesn't find a "proper" .fb file.
schickel is offline   Reply With Quote
Old 2018-03-18, 19:15   #11
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

3·1,151 Posts
Default

Quote:
Originally Posted by schickel View Post
You can comment out the code that calls procrels. That is a hold over from the original ggnfs suite. procrels was use to accumulate the sieved relations; it makes sure the relations are completely factored and does a de-dupe of the new relations against the already found relations.

There is/was a .params file created at the start of a job that contains the fb bound and the *lp settings. If there is a change in any of the values, procrels is called with "-dump" to dump all the relations in raw format and reread them to make sure they're still valid with the new bounds.

It looks like the script isn't finding the .params file (or if it's there, it doesn't match the .poly) so procrels gets called to dump/reprocess any accumulated relations but procrels doesn't find a "proper" .fb file.
Thanks! That helps me understand a little bit more. If I can understand enough, I'll build distribution into the poly select part of the script. My level of ignorance at what I am seeing leads me to believe this can be done without too difficult a time. I'll see...
EdH is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Base-6 speed for prime testing vs. base-2 jasong Conjectures 'R Us 36 2010-08-03 06:25
trial division over a factor base Peter Hackman Factoring 7 2009-10-26 18:27
Algebraic factor issues base 24 michaf Conjectures 'R Us 18 2008-05-21 10:08
GNFS - Factor Base Golem86 Factoring 11 2007-07-17 18:08
Quadratic Sieve - How large should the factor base be? hallstei Factoring 5 2005-04-19 11:58

All times are UTC. The time now is 02:43.

Sun Nov 29 02:43:32 UTC 2020 up 79 days, 23:54, 3 users, load averages: 0.93, 1.18, 1.21

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.