GNFS estimates

[Jeff Gilchrist]

Originally Posted by fivemack

The large-prime bound is the major influence on the number of relations needed. At a given large-prime bound, larger numbers tend to need more relations, but this is a much less substantial effect.

With all the SNFS experience you have, would it be possible for you to post your ggnfs "def-par.txt" file for those of us without as much experience can at least have some decent parameters to start from?

Jeff.

[fivemack]

I don't have a def-par.txt file, nor do I use the ggnfs scripts for large numbers. I have a couple of perl scripts, but they tend to have comments like 'now put the polynomial into count.gp and copy the results into this array'; not very fire-and-forget.

I start by picking the large-prime bound: I then try various small-prime bounds, and one or two sievers, and do test sieving over small ranges, estimate from those how long the job would take to get enough relations given the large-prime bound, and pick the small-prime bound and siever that gives the quickest runtime. I don't think you can avoid doing something at least that fiddly if you're working outside the bounds of experience.

A rule of thumb which seems reasonable at least up to 140 digits is to pick the small-prime bound such that you can expect to get all your relations by sieving from spb/2 up to spb; small Q are at least slightly higher-yielding than large Q.

However, I'll post a few example points - I've tended to do individual very-large jobs, I don't have a statistically significant collection of jobs above 130 digits.

GNFS 120-130 digits: 27-bit large primes, siever 13e, small primes 6000000 works quite well for me towards the top end of the range, small primes 4000000 works OK towards the lower end.

GNFS, 138 digits: probably 28-bit large primes, 13e and 14e pretty comparable, small prime 10 million works for me

By 150 digits you'll want 29-bit large primes and siever 14e, small prime bound 20 million seems reasonable. Don't change over too early; 28-bit LP was faster than 29-bit LP at the 144-digit point.

I'm currently doing a 159-digit GNFS with 29-bit large primes and siever 15e, small prime bound 50 million.

SNFS

difficulty 180-ish: 28-bit large primes are better than 27-bit; siever 13e; small prime bound around 10 million

crossover 28-29 is somewhere around 200; 30 is too big at difficulty 200.

SNFS 210-240: 30-bit large primes, siever 14e, small prime bound somewhere around 30 million

SNFS 240-270: 31-bit large primes, siever 15e, small prime bound around 100 million for larger numbers

Not much above SNFS 270, you'll want to switch to siever 16e; but the current version of this horribly over-estimates the memory usage (wants ~3.5G per process), and fixing that will be a serious development effort.

[Batalov]

I have added a small patch to the factMsieve.pl in the GGNFS repository. It is based on personal experience as well as collective experience of near-rep-unit participants (which was model-fitted by M.Kamada).
The ten lines of perl code should extend def-par.txt virtually.

Now that I've read Tom's comments, I am very glad that the perl hack agrees very much with his ideas. The SVN version is 344.

[Jeff Gilchrist]

Originally Posted by Batalov

I have added a small patch to the factMsieve.pl in the GGNFS repository. It is based on personal experience as well as collective experience of near-rep-unit participants (which was model-fitted by M.Kamada).
The ten lines of perl code should extend def-par.txt virtually.

The new feature is very nice, automatically selected parameters for an SNFS difficulty 202 number with the default def-par.txt distributed with ggnfs so this is a great feature for newbies!

You can download the latest factMsieve.pl here:
http://ggnfs.svn.sourceforge.net/vie.../factMsieve.pl

[Batalov]

Here is a reference to M.Kamada's statistics:
http://homepage2.nifty.com/m_kamada/...ions.htm#howto
(the five graphs).
The first thing for everyone to consider (regardless of parameters) is how much time one is willing to spend. See the first graph - expected time in CPU hours. If you don't have a 1000 CPU-hours for a complete job, then please do not consider going above 205-210 digits snfs (145 digits gnfs).

The perl script will do better than before, but do keep in mind that there are certain factorizations that would still benefit from careful manual .poly file crafting (see Tom's message above for guidance); here's a short list that is not meant to be complete:
1. Quartics for larger numbers (when quartic is necessary): at the very least, switch the side to -a and ponder uneven limits,
2. Sextics (converted quartics) for numbers with difficulty ~160-190, same as #1, but lopsided to the other side.
3. All numbers above, say 200-210: test-sieving (deciding quintic vs. sextic, fine-tuning the limits, lambdas), custom scripts (that would avoid idle CPUs and not even try to filter before a certain number of relations), etc, etc...

The script should behave better under 200-210.

[Batalov]

Yet another hack: adding a file to the working directory called "MINRELS.txt". You can create/edit it while the perl script is already running. The script will read it before every filtering stage, so if you have got a good idea how many relations msieve's filtering "wants" - put that number in that file. If you later get a better idea - edit it!

You will save a ton of time from needless filtering; it also allows you to deliberately oversieve (for a smaller matrix).

You need the 345 version of the
http://ggnfs.svn.sourceforge.net/vie.../factMsieve.pl

[Jeff Gilchrist]

Originally Posted by Batalov

Yet another hack: adding a file to the working directory called "MINRELS.txt". You can create/edit it while the perl script is already running. The script will read it before every filtering stage, so if you have got a good idea how many relations msieve's filtering "wants" - put that number in that file. If you later get a better idea - edit it!

That is cool, I have been adding my own change to the built-in estimate in the perl script, for C133-C135 gnfs jobs, I multiply by 2.5 and get almost an exact amount but that means I have to remember to change it before I run the script, etc...

This way is much better because I can tweak things on the fly and not have to worry about editing the script itself. Good job!

[Andi47]

Originally Posted by Batalov

I have added a small patch to the factMsieve.pl in the GGNFS repository. It is based on personal experience as well as collective experience of near-rep-unit participants (which was model-fitted by M.Kamada).
The ten lines of perl code should extend def-par.txt virtually.

Now that I've read Tom's comments, I am very glad that the perl hack agrees very much with his ideas. The SVN version is 344.

I just recognized that the new FactMsieve.pl script beeps 5 time when it writes the time scale file - this would be quite annoying if that happens at night. How can I switch the beeps off? (I am not very familiar with perl)

[smh]

Originally Posted by Andi47

I just recognized that the new FactMsieve.pl script beeps 5 time when it writes the time scale file - this would be quite annoying if that happens at night. How can I switch the beeps off? (I am not very familiar with perl)

Change the last line of the script to:

# for($i = 0; $i < 5; $i++){sleep 1;print "\a";}

[10metreh]

Originally Posted by Andi47

I just recognized that the new FactMsieve.pl script beeps 5 time when it writes the time scale file - this would be quite annoying if that happens at night. How can I switch the beeps off? (I am not very familiar with perl)

I would guess those beeps are there to alert you trhat the factorization has finished. It was quite useful when I had a job that took 4 dependencies and I was not in the room as I usually am.

[Batalov]

Originally Posted by Andi47

I just recognized that the new FactMsieve.pl script beeps 5 time when it writes the time scale file - this would be quite annoying if that happens at night. How can I switch the beeps off? (I am not very familiar with perl)

Oh, it's not new; it was always there, and in factLat.pl as well (it's inheretied from factLat.pl). The answer is above and it's correct. It's "after" not "when", so it can surely be removed.

There's a lot of interesting "decorations" there, in the end of the script, including calling procrels (and only to gauge the speed of the computer; if it fails - never mind); the dmesg calls (btw, they also fail sometimes, not working as expected after the computer had been up for a while -- the kernel ring buffer messages about the bogomips and CPUs get "ring"ed out.)