mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   Factoring (https://www.mersenneforum.org/forumdisplay.php?f=19)
-   -   Advice for large SNFS jobs? (https://www.mersenneforum.org/showthread.php?t=17690)

ryanp 2013-01-21 19:41

Advice for large SNFS jobs?
 
Hi all,

I'm working on factoring (2801^83-1)/2800 using a modified factmsieve.py -- and yes, I have a cluster available to me... :)

factmsieve tells me at the get-go:

Fri Jan 18 09:58:41 2013 -> Estimated minimum relations needed: 5.53168e+08

I'm able to make it up to about 200M relations with the default parameters (starting at rational q from 238450000, FAMAX = 476900000) before hitting GGNFS' limit: it can't handle special q >= 2^30 - 1.

Does anyone have any advice for what to do with jobs this big? Try to change the sieving window somehow? (And if so -- to what)?

Here's the polynomial I'm using:

[code]n: 4784427753962229503583191777575386925462640502543527013793934480234680863804447852383959785408791045459809147067083157248015897910382151758867576620242257524246139326208569043470479714282260046673050230392057658284742406595942226610043596316622243579005395853667131475327572196568483
m: 1829715316371090533839726975772594414416841479201
deg: 6
skew: 0
type: snfs
c6: 1
c0: -2801[/code]

which gives the factor base:

[code]N 4784427753962229503583191777575386925462640502543527013793934480234680863804447852383959785408791045459809147067083157248015897910382151758867576620242257524246139326208569043470479714282260046673050230392057658284742406595942226610043596316622243579005395853667131475327572196568483
SKEW 3.75
A6 1
A0 -2801
R1 1
R0 -1829715316371090533839726975772594414416841479201
FAMAX 476900000
FRMAX 476900000
SALPMAX 4294967296
SRLPMAX 4294967296[/code]

fivemack 2013-01-21 19:50

I'm impressed by the scale of your cluster, but factmsieve is not designed for jobs this big.

The polynomial is right, and the alim and lp look reasonable, but you're clearly using the wrong sieving binary since you're getting 0.25 relations per Q. I think you should be using 16e, and you should be using three large primes on the rational side (lpbr=32 mfbr=96 rlambda=3.6)

For things this large I tend to start from small Q (eg Q=1e7) rather than Q=Qmax/2.

debrouxl 2013-01-21 19:56

What siever did factmsieve.py choose for such a job ? NFS@Home would probably choose ggnfs-lasieve4I16e, if not the corresponding lasieve5.

ryanp 2013-01-21 22:19

debrouxl: It's using gnfs-lasieve4I16e, as I expected.

fivemack: Thanks, I'll try starting with Q=1e7 and the mfbr/rlambda values you suggested... With those values, do you think there's a shot that GGNFS/msieve will be able to finish this thing? :)

henryzz 2013-01-21 22:33

[QUOTE=ryanp;325395]debrouxl: It's using gnfs-lasieve4I16e, as I expected.

fivemack: Thanks, I'll try starting with Q=1e7 and the mfbr/rlambda values you suggested... With those values, do you think there's a shot that GGNFS/msieve will be able to finish this thing? :)[/QUOTE]

Even if it doesn't assuming you are on linux you should be able to run the later version of the siever that will sieve higher Qs.

ryanp 2013-01-21 22:58

[QUOTE=henryzz;325396]Even if it doesn't assuming you are on linux you should be able to run the later version of the siever that will sieve higher Qs.[/QUOTE]

Is that in the latest released version of GGNFS, or the version from head in SVN?

henryzz 2013-01-22 00:27

Here is a link to the newer siever. There shouldn't be much speed difference unless you can get ecm working helpfully.
[url]http://mersenneforum.org/showpost.php?p=308178&postcount=15[/url]

I don't think the source is in the svn.

Batalov 2013-01-22 01:11

It's definitely been [URL="http://ggnfs.svn.sourceforge.net/viewvc/ggnfs/trunk/src/experimental/lasieve4_64/"]there[/URL] for two years and is patched for many found problems. [URL="http://ggnfs.svn.sourceforge.net/viewvc/ggnfs/trunk/contrib/remdups/"]remdups[/URL] is contrib/ directory.

I thought that the lasieve5 source was also there but it is not maintained; you'd have to figure everything for yourself. If it is not there, you can find a zip [URL="http://www.mersenneforum.org/showthread.php?p=298833&highlight=lasieve5#post298833"]somewhere[/URL] on this forum.

ryanp 2013-01-22 07:34

[QUOTE=henryzz;325408]Here is a link to the newer siever. There shouldn't be much speed difference unless you can get ecm working helpfully.
[url]http://mersenneforum.org/showpost.php?p=308178&postcount=15[/url]

I don't think the source is in the svn.[/QUOTE]

Hmm, after downloading/untarring that version I still get:

[code]./gnfs-lasieve4I16e -k -o spairs.out.test -v -n0 -r input.job.test
gnfs-lasieve4I16e (with asm64): L1_BITS=15, SVN $Revision: 399 $
Cannot handle special q >= 1073741823[/code]

Same with building from source (in the src/experimental/lasieve4_64 tree, using the instructions in INSTALL). Am I doing something wrong, or is there a newer version somewhere that can handle large q?

ryanp 2013-01-22 07:36

Erm, perhaps I missed this in INSTALL:

[code]NOTE for Phenom/K8 users: replace in athlon64/ls-defs.asm
define(l1_bits,15)dnl
=>
define(l1_bits,16)dnl

and in athlon64/siever-config.h
#define L1_BITS 15
=>
#define L1_BITS 16[/code]

though that's not my CPU type... Is it still "safe"/advised?

Batalov 2013-01-22 08:00

No, this is only for the CPUs that have 64Kb L1 cache, i.e. AMD CPUs. (log[SUB]2[/SUB]64Kb = 16) Don't change L1 bits for Intel CPUs.

How much of the q area have you already sieved? What side have you sieved on? There's no need really for a project of this size to go over q>2^30. Try to cover the area from q=10^7 to your current lower limit (where you started, 238450000). Even if you go over 2^30, the yield will be less and less. You may get a better yield by repeating some of the most productive (lower q) areas with the parameters that Tom (fivemack) suggested earlier. Have you used 3LP? Like -
[code]lpbr: 33
lpba: 33
mfba: 66
mfbr: 96
alambda: 2.55
rlambda: 3.7
[/code]


Have you tried to filter your existing set of relations?
Last but not the least, do you have a computer (set of computers) to solve the resulting >40M matrix?
(As the saying goes, take no offense, - it's not the size (of the cluster), it's how you use it that matters. Have you done a snfs-~270-280 before doing this snfs-290?)

If you really want to go to very high q values, use the link to lasieve5 message.


All times are UTC. The time now is 21:36.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.