mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   Factoring (https://www.mersenneforum.org/forumdisplay.php?f=19)
-   -   Why can't I start factmsieve.py with a poly file, but no factor base? (https://www.mersenneforum.org/showthread.php?t=23167)

EdH 2018-03-17 23:29

Why can't I start factmsieve.py with a poly file, but no factor base?
 
OK, I thought this worked, but it won't work again.

The code looks like it should run, but it tries to create a factor base file and crashes.

Particulars:
--factmsieve won't run poly select on clients.
--msieve won't run the full poly select across more than one machine. If I use more than one machine, each makes its own factor base (.fb), even if I combine all the *.p files.
--if I separate the poly steps, msieve goes off for ages to do the final root phase.
--I can use a separate poly select routine that makes a final poly for the gnfs sievers, but factmsieve.py won't run from there without a fuss.
--If I build a factor base file from the poly, factmsieve.py scolds me.
[code]
$ python ../factmsieve.py comp94.n 1 3
[/code][code]
-> Running Python 2.7
-> This is client 1 of 3
-> Running on 4 Cores with 2 hyper-threads per Core
-> Working with NAME = comp94
-> Selected default factorization parameters for 95 digit level.
-> Selected lattice siever: gnfs-lasieve4I11e
-> Parameter change detected, dumping relations...
-> procrels -fb comp94.fb -prel rels.bin -dump

__________________________________________________________
| This is the procrels program for GGNFS. |
| Version: 0.77.1-20060722-nocona |
| This program is copyright 2004, Chris Monico, and subject|
| to the terms of the GNU General Public License version 2.|
|__________________________________________________________|
loadFB(): Could not open comp94.fb for read!
Could not load FB from comp94.fb!
Return value 255. Terminating...
[/code]frustration mounts...

Dubslow 2018-03-17 23:41

Having read but not truly understood your particulars, lets at least examine that factmsieve.py output:

[code]loadFB(): Could not open comp94.fb for read![/code]

That looks like an OS/FS problem more than an NFS/msieve problem (though I could certainly be wrong, never having used the script). Are you sure the file exists with the right name?

EdH 2018-03-18 02:57

[QUOTE=Dubslow;482654]Having read but not truly understood your particulars, lets at least examine that factmsieve.py output:

[code]loadFB(): Could not open comp94.fb for read![/code]That looks like an OS/FS problem more than an NFS/msieve problem (though I could certainly be wrong, never having used the script). Are you sure the file exists with the right name?[/QUOTE]
The .fb file does not exist, on purpose. the .fb file is created by each unique machine. Each of those files contains no score reference, so there is no way to determine which is better. factmsieve.py will create a poly file from the named .fb file, but I am trying to skip the .fb file by providing a .poly file. Perhaps I am misreading the following snippet, or more probably not following its relation to other code:
[code]
# Is there a poly file already, or do we need to create one?
if not os.path.exists(NAME + '.poly'):
if os.path.exists(NAME + '.fb'):
output('-> Converting msieve polynomial (*.fb) to ggnfs (*.poly) format.')
fb_to_poly()
...
[/code]I am pursuing something else more convoluted and I could always go back to my own sieving work distribution scripts, but this has already been done and I've already had success with factmsieve.py. I just don't think now that I was using the correct poly in my previous runs. I think one of the .fb files was somehow dragged into the mix rather than the optimum poly.

Thanks for your reply.

hyramgraff 2018-03-18 05:37

[QUOTE=EdH;482653][code]
-> procrels -fb comp94.fb -prel rels.bin -dump
[/code][/QUOTE]

Where is the .fb file that procrels is expecting supposed to be created? This looks like where you're having issues with the file not being found.

EdH 2018-03-18 14:36

I tried the following steps:

--I ran three poly select processes on three separate machines. Each created a factor base, but of course, each was different.

--I ran the perl script to find the best poly from the three .p files.

--I found the .fb file that matched the poly and renamed it to match the input number file.

--I removed all the extra files, so there would be no contamination.

--I ran:
[code]
python ../factmsieve.py comp94.n 1 3
[/code]Failure!:
[code]
-> | Monico. Version 0.86 (Python 2.6 or later) 20th June 2011. |
-> |______________________________________________________________|
-> Running Python 2.7
-> This is client 1 of 3
-> Running on 4 Cores with 2 hyper-threads per Core
-> Working with NAME = comp94
-> Converting msieve polynomial (*.fb) to ggnfs (*.poly) format.
-> Selected default factorization parameters for 95 digit level.
-> Selected lattice siever: gnfs-lasieve4I11e
-> Parameter change detected, dumping relations...
-> procrels -fb comp94.fb -prel rels.bin -dump

__________________________________________________________
| This is the procrels program for GGNFS. |
| Version: 0.77.1-20060722-nocona |
| This program is copyright 2004, Chris Monico, and subject|
| to the terms of the GNU General Public License version 2.|
|__________________________________________________________|
readPoly(): Did not find a valid 'n' value!
loadFB(): readPoly() reported an error!
Could not load FB from comp94.fb!
Return value 255. Terminating...
[/code]comp94.fb:
[code]
N 1975636228803860706131861386351317508435774072460176838764200263234956507563682801432890234281
SKEW 1256916.79
R0 -48400675083113392109375
R1 80840080969
A0 -61641819524944283968862064
A1 1061875853900591730890
A2 79968233798293
A3 -593456406
A4 360
[/code]comp94.poly:
[code]
n: 1975636228803860706131861386351317508435774072460176838764200263234956507563682801432890234281
Y0: -48400675083113392109375
Y1: 80840080969
c0: -61641819524944283968862064
c1: 1061875853900591730890
c2: 79968233798293
c3: -593456406
c4: 360
skew: 1256916.79
type: gnfs
[/code]comp94.log:
[code]
-> factmsieve.py (v0.86)
-> This is client 1 of 3
-> Running on 4 Cores with 2 hyper-threads per Core
-> Working with NAME = comp94
-> Converting msieve polynomial (*.fb) to ggnfs (*.poly) format.
-> Selected lattice siever: gnfs-lasieve4I11e
-> Parameter change detected, dumping relations...
-> procrels -fb comp94.fb -prel rels.bin -dump
[/code]comp94.n:
[code]
n: 1975636228803860706131861386351317508435774072460176838764200263234956507563682801432890234281
[/code]Maybe I need to rename and leave the appropriate .p file...

RichD 2018-03-18 14:46

FWIW, on the Perl side, the R0/R1 follow the A's.

EdH 2018-03-18 15:21

[QUOTE=RichD;482696]FWIW, on the Perl side, the R0/R1 follow the A's.[/QUOTE]
Yeah, and I used that order when I manually constructed a factor base file in one of my previous efforts. I moved them again just now and tried, with no change.

I suppose I'll just have to write my own routine or modify factmsieve.py as I did to get it to run my mpi stuff way back when I [STRIKE]knew[/STRIKE] remembered more about python...

RichD 2018-03-18 15:45

Again FWIW, on the Perl side, all I need is the .poly file to get it started.
[CODE]perl ./factmsieve.pl comp94.poly[/CODE]

EdH 2018-03-18 16:11

[QUOTE=RichD;482700]Again FWIW, on the Perl side, all I need is the .poly file to get it started.
[CODE]perl ./factmsieve.pl comp94.poly[/CODE][/QUOTE]
You bring up a very good point, but the reason I'm trying to use the python script is because it already takes care of dividing the sieving ranges up for all my subsequent machines. I haven't looked at the perl script in a number of years, but I guess I should.

I have discovered that part of the reason it is failing appears to be that I need more info in the .fb file to calculate the sieving parameters. Until I can incorporate the extra machines into the poly selection, I have created a work-around by suppressing the abort code when I start factmsieve.py with client info. I then need to wait until the primary machine (client 1) generates the poly, to start subsequent machines. Not ideal, but quite workable for now (other than the lost poly time from the other machines).

Thanks for all the help.

schickel 2018-03-18 18:20

You can comment out the code that calls procrels. That is a hold over from the original ggnfs suite. procrels was use to accumulate the sieved relations; it makes sure the relations are completely factored and does a de-dupe of the new relations against the already found relations.

There is/was a .params file created at the start of a job that contains the fb bound and the *lp settings. If there is a change in any of the values, procrels is called with "-dump" to dump all the relations in raw format and reread them to make sure they're still valid with the new bounds.

It looks like the script isn't finding the .params file (or if it's there, it doesn't match the .poly) so procrels gets called to dump/reprocess any accumulated relations but procrels doesn't find a "proper" .fb file.

EdH 2018-03-18 19:15

[QUOTE=schickel;482717]You can comment out the code that calls procrels. That is a hold over from the original ggnfs suite. procrels was use to accumulate the sieved relations; it makes sure the relations are completely factored and does a de-dupe of the new relations against the already found relations.

There is/was a .params file created at the start of a job that contains the fb bound and the *lp settings. If there is a change in any of the values, procrels is called with "-dump" to dump all the relations in raw format and reread them to make sure they're still valid with the new bounds.

It looks like the script isn't finding the .params file (or if it's there, it doesn't match the .poly) so procrels gets called to dump/reprocess any accumulated relations but procrels doesn't find a "proper" .fb file.[/QUOTE]Thanks! That helps me understand a little bit more. If I can understand enough, I'll build distribution into the poly select part of the script. My level of ignorance at what I am seeing leads me to believe this can be done without too difficult a time. I'll see...


All times are UTC. The time now is 06:49.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.