msieve 1.52 with GPU polynomial selection

[cgy606]

I have been using yafu to do nfs on inputs around 150 digits and would like to to now use a gpu to help speed up polynomial selection on inputs larger than 150 digits (I have read a numerous places that it isn't practical to use poly selection for inputs less than 140 digits with a gpu). Anyways, since I have never used msieve directly and the readme doesn't say much about what kind of line command to use for stage 1 gpu polynomial selection, I looked around on this form and am using the following command:

>msieve.exe -v -g 0 -np1


Msieve v. 1.52 (SVN unknown)
Wed Oct 05 02:23:33 2016
random seeds: fbb158e0 8b921750
factoring 14983833053669267073388859640749386761592128691535474632716887403740027331275486495085918514582693799340373342068801035123016311177538173489337402480577483 (155 digits)
searching for 15-digit factors
commencing number field sieve (155-digit input)
commencing number field sieve polynomial selection
polynomial degree: 5
max stage 1 norm: 8.32e+023
max stage 2 norm: 5.19e+021
min E-value: 2.32e-012
poly select deadline: 1020287
time limit set to 283.41 CPU-hours
expecting poly E from 2.83e-012 to > 3.26e-012
searching leading coefficients from 1 to 17047468
using GPU 0 (GeForce GTX 980)
selected card has CUDA arch 5.2
cannot load library 'sort_engine_sm20.dll', error 126
error: failed to load GPU sorting engine

Although it doesn't say the SVN number, I downloaded the latest version I could find, which is 942 which claims to have gpu support. Obviously, I don't want to spend 280 hours searching for a polynomial, and thus I am going to have to play around with the parameters (no doubt I will have some questions on this), but as a first step I would like to get things working on the gpu. It claims that a file "sort_engine_sm20.dll" can't be loaded, but its in my directory:

Directory of C:\Users\Documents\Factoring\msieve_gpu

10/05/2016 02:29 AM <DIR> .
10/05/2016 02:29 AM <DIR> ..
08/25/2013 08:25 PM 1,570 AUTHORS.GMPECM
08/25/2013 08:25 PM 3,795 AUTHORS.MPIR
08/25/2013 08:25 PM 96,586 Changes
08/25/2013 08:25 PM 35,147 COPYING.GMPECM
08/25/2013 08:25 PM 35,147 COPYING.MPIR
10/05/2016 02:07 AM 0 msieve.dat.m
08/17/2013 02:26 PM 1,307,136 msieve.exe
10/05/2016 02:29 AM 1,182 msieve.log
11/14/2014 11:54 AM 119,888 pthreadGC2.dll
08/25/2013 08:25 PM 25,290 Readme
08/25/2013 08:25 PM 34,686 README.GMPECM
08/25/2013 08:25 PM 3,889 README.MPIR
08/25/2013 08:25 PM 57,203 Readme.nfs
08/25/2013 08:25 PM 7,987 Readme.qs
08/17/2013 02:26 PM 380,928 sort_engine_sm11.dll
08/17/2013 02:27 PM 458,752 sort_engine_sm13.dll
08/17/2013 02:26 PM 459,776 sort_engine_sm20.dll
08/17/2013 02:26 PM 134,747 stage1_core_sm11.ptx
08/17/2013 02:27 PM 134,731 stage1_core_sm13.ptx
08/17/2013 02:27 PM 129,841 stage1_core_sm20.ptx
10/05/2016 02:23 AM 155 worktodo.ini
10/05/2016 02:23 AM 155 worktodo.txt
22 File(s) 3,428,591 bytes
2 Dir(s) 388,388,360,192 bytes free

I tried to do cpu only poly selection using svn 942 but it fails for the same reason. Using svn 958, cpu poly selection works:

msieve152.exe -v -np1


Msieve v. 1.52 (SVN 958)
Wed Oct 05 02:29:48 2016
random seeds: e7c648cc 2929e0a1
factoring 453900508707227117088044239531189329962427502081603308969082793654958205298131757286256493811103457282903828743904586118273728781849349264159173919 (147 digits)
searching for 15-digit factors
commencing number field sieve (147-digit input)
commencing number field sieve polynomial selection
polynomial degree: 5
max stage 1 norm: 2.96e+022
max stage 2 norm: 4.52e+020
min E-value: 6.73e-012
poly select deadline: 478041
time limit set to 132.79 CPU-hours
expecting poly E from 7.05e-012 to > 8.10e-012
searching leading coefficients from 1 to 7173428
deadline: 400 CPU-seconds per coefficient
coeff 12 specialq 1 - 642311 other 42546 - 102110
aprogs: 6679 entries, 8791 roots
12 30473492865977 130483586313074599578258471154
12 391108512956581 130483586313428536359648699855
12 190036387843211 130483586313104264552255614163
12 497903426792239 130483586312969950410530939056
12 99555052674679 130483586313102856028358736905
12 600506580821749 130483586312706903525049241402
12 1221048763726757 130483586318845356291945943661
12 2148660442996951 130483586300814908091187560627
12 87480910558181 130483586313063800465946947163
12 1436859739291417 130483586321040833931721147487
12 1213171586284541 130483586318594367511176389436
12 131262263274587 130483586312817142862671690680
12 565369239538831 130483586312326311257545617485
12 16429244447641 130483586313059619825238059161
12 211253239565317 130483586313016451909772556149
12 16020807683731 130483586313059861562225900477
12 92548280928131 130483586313079469319119183091
12 1817425292632033 130483586321008646147001995250
12 1367724755956723 130483586309277352903604946221

received signal 2; shutting down
hashtable: 8192 entries, 0.13 MB
polynomial selection complete
error generating or reading NFS polynomials
elapsed time 00:00:05

any ideas about what is going wrong or how to fix this. Thanks...

[Brian Gladman]

Originally Posted by cgy606

Although it doesn't say the SVN number, I downloaded the latest version I could find, which is 942 which claims to have gpu support. Obviously, I don't want to spend 280 hours searching for a polynomial, and thus I am going to have to play around with the parameters (no doubt I will have some questions on this), but as a first step I would like to get things working on the gpu. It claims that a file "sort_engine_sm20.dll" can't be loaded, but its in my directory:

Directory of C:\Users\Documents\Factoring\msieve_gpu

08/17/2013 02:26 PM 380,928 sort_engine_sm11.dll
08/17/2013 02:27 PM 458,752 sort_engine_sm13.dll
08/17/2013 02:26 PM 459,776 sort_engine_sm20.dll

any ideas about what is going wrong or how to fix this. Thanks...

You need to add the sub-directory 'cub' to your 'msieve_gpu' directory and move the three files listed above into it. The latest Visual Studio build puts them in the right relative location but you have to do it manually for earlier builds.

[cgy606]

That's not it. I didn't build this using VS2015. I downloaded the zip from http://gilchrist.ca/jeff/factoring/ (the 1.52dev SVN 942 (compiled by Brian Gladman)) and simple unpacked it and put it into a folder.

[Brian Gladman]

Originally Posted by cgy606

That's not it. I didn't build this using VS2015. I downloaded the zip from http://gilchrist.ca/jeff/factoring/ (the 1.52dev SVN 942 (compiled by Brian Gladman)) and simple unpacked it and put it into a folder.

It doesn't matter how the binary was built, it expects to see the sort engines in a sub-directory of the directory where the other executables are, one named 'cub' (in any event, your binary was built with an earlier version of Visual Studio).

[cgy606]

Originally Posted by Brian Gladman

It doesn't matter how the binary was built, it expects to see the sort engines in a sub-directory of the directory where the other executables are, one named 'cub' (in any event, your binary was built with an earlier version of Visual Studio).

I moved all the msieve files for the gpu to the same directory as msieve152. Here is what the directory looks like:

C:\Users\Documents\Factoring\msieve>dir
Volume in drive C has no label.
Volume Serial Number is 588F-04AB

Directory of C:\Users\Documents\Factoring\msieve

10/05/2016 01:15 PM <DIR> .
10/05/2016 01:15 PM <DIR> ..
08/25/2013 08:25 PM 1,570 AUTHORS.GMPECM
08/25/2013 08:25 PM 3,795 AUTHORS.MPIR
08/25/2013 08:25 PM 96,586 Changes
08/25/2013 08:25 PM 35,147 COPYING.GMPECM
08/25/2013 08:25 PM 35,147 COPYING.MPIR
10/05/2016 01:13 PM <DIR> cub
10/05/2016 02:07 AM 0 msieve.dat.m
08/17/2013 02:26 PM 1,307,136 msieve.exe
10/05/2016 01:15 PM 5,910 msieve.log
08/19/2016 02:11 AM 1,363,644 msieve152.exe
10/04/2016 09:25 AM <DIR> pthreadgc2
11/14/2014 11:54 AM 119,888 pthreadGC2.dll
08/25/2013 08:25 PM 25,290 Readme
08/25/2013 08:25 PM 34,686 README.GMPECM
08/25/2013 08:25 PM 3,889 README.MPIR
08/25/2013 08:25 PM 57,203 Readme.nfs
08/25/2013 08:25 PM 7,987 Readme.qs
08/17/2013 02:26 PM 134,747 stage1_core_sm11.ptx
08/17/2013 02:27 PM 134,731 stage1_core_sm13.ptx
08/17/2013 02:27 PM 129,841 stage1_core_sm20.ptx
10/05/2016 02:23 AM 155 worktodo.ini
10/05/2016 02:23 AM 155 worktodo.txt
20 File(s) 3,497,507 bytes
4 Dir(s) 388,363,792,384 bytes free

I tried to rerun gpu msieve and I am still getting the same error:

C:\Users\Documents\Factoring\msieve>msieve.exe -v -np1


Msieve v. 1.52 (SVN unknown)
Wed Oct 05 13:15:37 2016
random seeds: 45ee0db8 6bfdcdd7
factoring 14983833053669267073388859640749386761592128691535474632716887403740027331275486495085918514582693799340373342068801035123016311177538173489337402480577483 (155 digits)
searching for 15-digit factors
commencing number field sieve (155-digit input)
commencing number field sieve polynomial selection
polynomial degree: 5
max stage 1 norm: 8.32e+023
max stage 2 norm: 5.19e+021
min E-value: 2.32e-012
poly select deadline: 1020287
time limit set to 283.41 CPU-hours
expecting poly E from 2.83e-012 to > 3.26e-012
searching leading coefficients from 1 to 17047468
using GPU 0 (GeForce GTX 980)
selected card has CUDA arch 5.2
cannot load library 'sort_engine_sm20.dll', error 126
error: failed to load GPU sorting engine

[pinhodecarlos]

Please try these binaries:

http://www.mersenneforum.org/showpos...0&postcount=45
http://www.mersenneforum.org/showpos...3&postcount=50

[cgy606]

Originally Posted by pinhodecarlos

Please try these binaries:

http://www.mersenneforum.org/showpos...0&postcount=45
http://www.mersenneforum.org/showpos...3&postcount=50

Thank you Carlos, the one with CUDA7 works (at least is seems to generate a ton of number triplets on the screen which I think is stage 1). Now, without asking a ton of questions, is there something I can read that will tell me how to configure the parameters of stage 1 poly search given a particular input size (the readme isn't too informative in this regard). What I mean is, what should the bounds be for the stage 1/2 norms, min E-Value, leading coeffs, poly search ranges and time limits per coeff and for the entire process. Also, can gpu do anything other than a degree 5 poly? Thanks...

[jasonp]

GPU poly selection works for degree 4, 5, or 6. For a 150-digit input anything other than degree 5 is a waste of effort. I'll leave it to others who have much more practice than I do to help with parameter selection; most of the time you make the stage 1 bound more and more stringent until hits arrive at a rate of just a few per second. Then you can do size optimization at the same rate that stage 1 hits are produced. Buffer up a few thousand to a few million of those, then run the root optimization on the 100-10000 size-optimized results whose size is best.

[cgy606]

I have a quick question. I am running a stage 1 polynomial selection on a 155 digit number. I set a deadline for searching of 100 seconds ("poly_deadline=100"), while not changing anything else. msieve appears to be running, however it took about 16 minutes. Why doesn't it enforce the deadline I specified? Furthermore, I get the following message at the end:

156 13734560102344961 2491722452247316394300991682441
156 28347894540434953 2491722449832725012667605005815
156 38998963368318367 2491722449473816650196253487469
156 17772432379479701 2491722451500964249800099105531
polynomial selection complete
error generating or reading NFS polynomials
elapsed time 00:16:09

Any ideas on what went wrong (or if something is wrong?)

[jasonp]

The error message is just because you didn't produce any complete polynomials, only stage 1 hits. If you ran size or root optimization as well then you would get a best polynomial that would get saved to file.

The time limit stuff has not been well tested; the limit is on CPU time, which is zero while the GPU is running. Experience has shown that nothing I can put in code will satisfy everyone who wants to limit by time. CPU or wall clock time? Does time in low-power or sleep mode count differently? How does GPU time contribute to that?

[cgy606]

Originally Posted by jasonp

The error message is just because you didn't produce any complete polynomials, only stage 1 hits. If you ran size or root optimization as well then you would get a best polynomial that would get saved to file.

The time limit stuff has not been well tested; the limit is on CPU time, which is zero while the GPU is running. Experience has shown that nothing I can put in code will satisfy everyone who wants to limit by time. CPU or wall clock time? Does time in low-power or sleep mode count differently? How does GPU time contribute to that?

I actually ran the size opt and root opt and got a polynomial that yafu was able to use, so that works.

In regards to timing, I guess I would like to know how many stage 1 hits are enough (based on the size of the number being factored) such that I have reasonable confidence that a satisfactory polynomial can be found in the remaining stages. I produced 760K stage 1 hits in 970 seconds which subsequently lead to a polynomial being found after the other two stages in a little less than 2 hours (20 min for size and 95 min for root opt). Do you think this is enough effort for a c155 gnfs? What benefit would I get if I were to let it spend more time doing polynomial search. I know these are tough questions to answer (and many of you are performing 'alpha' research in these areas), but in order to do a differential analysis, usually one has to quantify the marginal benefits from continuing their efforts beyond the point they are at and compare to what they have already achieved.

On a technical note, I read in the readme that np1 and nps can be done at the same time. If I do this, will msieve do np1 on the GPU and than nps on the CPU (once np1 has finished) in series, or will it do then in parallel as hits are being generated from stage 1?