mersenneforum.org Low CPU/GPU usage?
 User Name Remember Me? Password
 Register FAQ Search Today's Posts Mark Forums Read

 2017-02-02, 10:48 #12 GeoffreyY   "Geoffrey Yeung" Feb 2017 London 2·32 Posts I stand corrected, after running for an hour, I have the following output in RSA.ms: http://pastebin.com/Avj6B01B Still, this is different from what I saw previously. How can I check if this is valid? Edit: good news, with invoking msieve from command line, my GPU is currently on full load. Thanks guys Last fiddled with by GeoffreyY on 2017-02-02 at 10:51
 2017-02-02, 17:12 #13 chris2be8     Sep 2009 5·389 Posts GNFS has several stages: 1: Searching for small factors with ECM. If you know it doesn't have any small factors you can skip this (RSA numbers usually don't). 2: Searching for a polynomial. This can be aided by your GPU. And split over several systems. 3: Sieving for relations. This can be split over several systems, each core on each system sieving a separate range. Be careful to ensure you don't sieve the same range more than once, this produces duplicate relations which are useless. 4: Building a matrix (by running msieve -nc1). It may fail saying it needs more relations, if so sieve some more. But don't take the message saying how many more it needs seriously. This is single threaded. 5: Solving the matrix (msieve -nc2). This may take several days for RSA170. It can use multiple cores on 1 system. 6: The square root stage (msieve -nc3) which finds the factors. This is single threaded. You might do better to start by factoring RSA100 first (In a different directory to the one you are working in). That should not take long and seeing it all the way through will tell you what to expect in larger runs. Then RSA110, RSA120, etc. Each will take about 4 times as long as the last. Also look at the NFS@Home forum thread http://mersenneforum.org/showthread....20023&page=162 on linear algebra. You can read the msieve logs for similar sized jobs which might be helpful. HTH Chris
 2017-02-02, 21:52 #15 GeoffreyY   "Geoffrey Yeung" Feb 2017 London 2×32 Posts Thank you very much for all the info! I don't think I'll have any more problems from here going forwards. I sieved through 5.5 to 6.2 mil this afternoon, but I didn't find any poly above 3.2e-13. I only have a laptop with a 950m, which's the only machine I've been sieving on. There are some technical difficulties with uni's machine. One more question, is the polynomial searching non-deterministic? I think I got different results passing through the same range twice. Also, are there any particular aspect / coefficient of the polynomial I should be looking for, besides the Murphy E-score? Or are they simply data points that don't carry much weight?
2017-02-02, 23:12   #16
VBCurtis

"Curtis"
Feb 2005
Riverside, CA

22·19·59 Posts

Quote:
 Originally Posted by GeoffreyY One more question, is the polynomial searching non-deterministic? I think I got different results passing through the same range twice. Also, are there any particular aspect / coefficient of the polynomial I should be looking for, besides the Murphy E-score? Or are they simply data points that don't carry much weight?
1. msieve searches a subset of the search space for each leading coefficient, randomizing which piece each run to minimize repeated work if multiple people run the same range. It was my impression that my tight stage1-norm reduced the search space so much that msieve didn't do this, but your results suggest otherwise. So, sort-of deterministic? :)

2. Score is an estimate of poly effectiveness; it is imperfect but within 3-5% of accurate; that is, two polys of the same score can differ by up to 5% speed in use. For jobs this size, folks usually test any poly that scores within 3-4% of the best poly (say, within 0.15 in your case) in case the best one happens to run poorly and 2nd or 3rd place runs well. If you find nothing better than 3.30, I wouldn't bother with test sieving; just use the 3.4699 poly I posted.

Best wishes on your effort- post if you have further questions, such as what parameters to select. The choices are made around line 520 in factmsieve.py, and formulas are easily edited to nudge the script to pick a different siever or large-prime bound. Hopefully, it picks 15e rather than 14e (part of the name of the lasieve executable), and chooses 2^31 as large prime bound (in a .job file created by the running script, you'll see lbpr and lpba; those should be 31 for a job this size, with 32 acceptable but 30 probly bad).

Last fiddled with by VBCurtis on 2017-02-02 at 23:17

 2017-02-03, 10:35 #17 GeoffreyY   "Geoffrey Yeung" Feb 2017 London 100102 Posts I still haven't found any better poly, so I've moved on to finding relations using your poly. I think this RSA.poly file should be alright: http://pastebin.com/w15j4PDx I'm able to use uni's computer for this step, but I have to manually create the .job files so that there's no overlap, because I'm not sure how to do otherwise. (example: http://pastebin.com/jKdcRigw) I guess I can make my own .resume files? One problem though, sometimes after invoking the command, the uni computer reports the .exe is not responding. I think it took a bit too long to create the .afb.0 file. They all have the same size (14,951,800 bytes), are they all identical? If I simply copy, paste and rename the files, I no longer have the occasional crash at start-up, but I'm not sure if I should do that. Another thing, do I simply transfer all the spairs.out.t_ files from my multiple computers and truncate them? I'm just zipping them up for now. Sorry if my questions are getting bothersome. I'm partly recording the process for myself for when I come back in the future. Thanks!
 2017-02-03, 13:34 #18 GeoffreyY   "Geoffrey Yeung" Feb 2017 London 2×32 Posts quick update: currently I have the following files in \gfns\RSA: http://pastebin.com/sWYD4t5Q And from there using "..\factmsieve.py RSA" seems to be working. I think I have a total of 100 relations/sec utilizing uni's computers. This easy setup means I can easily set up more computers if I want to / if I can.
 2017-02-03, 14:47 #19 jasonp Tribal Bullet     Oct 2004 DA116 Posts Your error is an FAQ; if you only run poly selection in stages then unless you run the stage 2 root sieve you will not assemble a complete polynomial. By default the NFS driver in Msieve assumes you want to do the whole factorization, so not having a complete polynomial is treated as an error. It actually is silly to do that, since even if you did have a complete polynomial the sieve code in Msieve is probably an order of magnitude slower than the high-performance lattice sieve that a script calls an external binary to perform. In the past we have had students complete big factorizations (i.e. 160 digits) in a remarkably short time, by taking over idle computer labs. On modern machines a 170-digit factorization starts to get into the range where the linear algebra is a major headache to perform in a reasonable time, i.e. it will take weeks unless you use MPI and have a few machines with a high-speed interconnect (gigabit ethernet isn't enough).
2017-02-03, 16:48   #20
VBCurtis

"Curtis"
Feb 2005
Riverside, CA

22×19×59 Posts

Quote:
 Originally Posted by GeoffreyY One problem though, sometimes after invoking the command, the uni computer reports the .exe is not responding. I think it took a bit too long to create the .afb.0 file. They all have the same size (14,951,800 bytes), are they all identical? If I simply copy, paste and rename the files, I no longer have the occasional crash at start-up, but I'm not sure if I should do that. Sorry if my questions are getting bothersome. I'm partly recording the process for myself for when I come back in the future. Thanks!
No, the .afb files do change from range to range, and it's not wise to move/rename them.

Threads like this become reference material when the next person comes along, so your idea of recording the process for yourself is wise.

(I don't know the answer to spairs.add, so I didn't address that question)

2017-02-03, 21:34   #21
EdH

"Ed Hall"
Dec 2009

1101011110012 Posts

Quote:
 Originally Posted by GeoffreyY ... Another thing, do I simply transfer all the spairs.out.t_ files from my multiple computers and truncate them? I'm just zipping them up for now. ...
You should not truncate the files. You should concatenate all of the unzipped spairs.out.t_ files into spairs.add:
Code:
cat spairs.out.t_* >> spairs.add
From there, factmsieve.py will concatenate spairs.add to the RSA.dat file. By using the spairs.add file, you allow factmsieve to modify the RSA.dat file instead of risking a manual modification causing a crash.

2017-02-17, 11:47   #22
GeoffreyY

"Geoffrey Yeung"
Feb 2017
London

2×32 Posts

I have successfully factorized: C170 = P85 * P85. For various reasons I won't post the exact result until 27th Feb. (deadline for this bonus challenge) Thank you everyone for helping.

Quote:
 In the past we have had students complete big factorizations (i.e. 160 digits) in a remarkably short time, by taking over idle computer labs.

To be safe, I got up to ~120% of minimum relations needed, and got 30 nontrivial dependencies at the end. Dependency number 1 worked anyway. Oh well.

Apparently I had to convert my .poly into .fb to start the linear algebra phase with factmsieve.py.

During the -nc2 phase my windows crashed (BSOD), and the .mat file became 0 bytes for some reason. The .dat file was also re-written. but luckily we have .gz backup. I think it's a RAM issue, as I had chrome with multiple tabs opened.

I'll try and see if I can reproduce the low CPU/GPU usage issue that started this thread.

Besides that, I'm probably going to grab an aliquot sequence to work on. I was looking into helping factordb too, but it doesn't seem 100% stable at the moment.

 Similar Threads Thread Thread Starter Forum Replies Last Post Brain GPU Computing 9 2011-04-12 22:25 lavalamp Software 0 2010-10-11 20:46 Primix Hardware 2 2008-07-20 23:44 ECMFreak Factoring 13 2007-07-20 17:34 Unregistered Software 6 2003-11-19 07:05

All times are UTC. The time now is 08:34.

Sat Nov 28 08:34:31 UTC 2020 up 79 days, 5:45, 3 users, load averages: 1.57, 1.68, 1.65