Stopping and restarting factmsieve

ThomRuley · 2021-12-30, 14:25

I was thinking about pausing a factorization in msieve to finish a few high-priority jobs in prime95. If I stop gracefully with Control+c, will the program pick up where it left off when I restart it or is this likely to introduce errors into the data file?

VBCurtis · 2021-12-30, 17:56

How to pause and restart depends on what phase of the job you're in.
Your thread title says "factmsieve", but your question says "msieve". Those are two different things.
If you're in the sieving part of factmsieve, then your plan works just fine, and factmsieve resumes the job by restarting the current block of Q-ranges.

If you're in the matrix-solving part of msieve, you'll want to restart msieve directly rather than through factmsieve, and you'll want to use -ncr flag to tell msieve "there is a matrix to resume, please resume it". Without -ncr, msieve will start the matrix over from 0 dimensions. I don't recall how to pass -ncr to factmsieve, which is why I call msieve directly in these circumstances (or I just don't cancel a factmsieve job when msieve is running the matrix).

swellman · 2021-12-30, 18:06

Code:

msieve -v -ncr -nc3 -t 4

Is how I would restart a matrix that had Cntl-c out of on a four threaded machine.

The -nc3 tells msieve to move on the square root phase once LA is complete.

If not, msieve will finish the linear algebra (-nc2) and then terminate without finding factors. You could then restart msieve again, this time with -nc3 but why bother?

VBCurtis · 2021-12-30, 18:11

The catch with restarting a matrix job that was created by factmsieve is that factmsieve has created file names different from msieve default.
I remember copying the msieve invocation line from factmsieve's screen display and adding in -ncr, so that msieve would find all the files. But it has been quite a few years since I've used factmsieve, so... you know.

swellman · 2021-12-30, 18:42

And I have never used factmsieve, so take my comments from that perspective.

Thom - play with it a bit on smaller jobs is the best advice. Losing 4 hours of work is annoying - 2 months a disaster.

ThomRuley · 2021-12-30, 19:02

As of this morning it was still sieving with about 70% of minimum relations. If I get back home and fid it doing matrix steps, i would just let it finish.

charybdis · 2021-12-30, 19:35

Quote:

Originally Posted by swellman

Thom - play with it a bit on smaller jobs is the best advice. Losing 4 hours of work is annoying - 2 months a disaster.

Isn't accidentally deleting all the relations and having to restart from scratch an important character-building experience?

In all seriousness, if you're worried about things going wrong when you restart the script, make sure to back up the relations file (the big .dat file), and the .dat.cyc and .dat.mat files if they exist. Nothing should happen to these files, but it's better to be safe than sorry.

swellman · 2021-12-30, 20:08

Quote:

Originally Posted by charybdis

Isn't accidentally deleting all the relations and having to restart from scratch an important character-building experience?

In all seriousness, if you're worried about things going wrong when you restart the script, make sure to back up the relations file (the big .dat file), and the .dat.cyc and .dat.mat files if they exist. Nothing should happen to these files, but it's better to be safe than sorry.

Backups? Life’s an adventure - no net!!

VBCurtis · 2021-12-30, 20:23

Quote:

Originally Posted by charybdis

Isn't accidentally deleting all the relations and having to restart from scratch an important character-building experience?

This. Happens. To. Everyone.

charybdis · 2021-12-30, 20:28

Quote:

Originally Posted by swellman

Backups? Life’s an adventure - no net!!

OK, maybe I should get round to deleting my ~75GB of accumulated CADO relation files from now-finished SNFS jobs. Should be fine as long as I don't mix them up with unfinished jobs...

ThomRuley · 2021-12-30, 22:08

UPDATE:

Thanks for letting me know that it is safe to pause msieve.

I found another workaround for the GIMPS jobs. Since the jobs are small, I simply opted to run them on another computer that has not been running any recent high-priority jobs. The first of these will finish in about 20 minutes. Now I am curious whether Primenet will take the job directly off the machine that was originally assigned the jobs, or whether they will disappear on the next completion estimate update, or I will have to manually remove them.

2021-12-30, 14:25	#1
ThomRuley May 2003 3×97 Posts	Stopping and restarting factmsieve I was thinking about pausing a factorization in msieve to finish a few high-priority jobs in prime95. If I stop gracefully with Control+c, will the program pick up where it left off when I restart it or is this likely to introduce errors into the data file?

2021-12-30, 18:06	#3
swellman Jun 2012 DDB₁₆ Posts	Code: msieve -v -ncr -nc3 -t 4 Is how I would restart a matrix that had Cntl-c out of on a four threaded machine. The -nc3 tells msieve to move on the square root phase once LA is complete. If not, msieve will finish the linear algebra (-nc2) and then terminate without finding factors. You could then restart msieve again, this time with -nc3 but why bother?

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Stopping a primenet work fetch loop	SELROC	GPU Computing	2	2019-03-31 10:40
Automatically stopping P-1 to benchmark? Why?	Uncwilly	Software	2	2018-02-23 05:24
Stopping New Work	dmaine	Information & Answers	1	2011-07-05 22:30
PrimeGrid PSP (Sieve) is stopping soon!	Joe O	Prime Sierpinski Project	1	2010-11-04 23:05
Stopping on a dime?	R.D. Silverman	NFSNET Discussion	1	2006-03-20 23:46

2021-12-30, 17:56	#2
VBCurtis "Curtis" Feb 2005 Riverside, CA 1010011001101₂ Posts	How to pause and restart depends on what phase of the job you're in. Your thread title says "factmsieve", but your question says "msieve". Those are two different things. If you're in the sieving part of factmsieve, then your plan works just fine, and factmsieve resumes the job by restarting the current block of Q-ranges. If you're in the matrix-solving part of msieve, you'll want to restart msieve directly rather than through factmsieve, and you'll want to use -ncr flag to tell msieve "there is a matrix to resume, please resume it". Without -ncr, msieve will start the matrix over from 0 dimensions. I don't recall how to pass -ncr to factmsieve, which is why I call msieve directly in these circumstances (or I just don't cancel a factmsieve job when msieve is running the matrix).

2021-12-30, 18:11	#4
VBCurtis "Curtis" Feb 2005 Riverside, CA 3×5²×71 Posts	The catch with restarting a matrix job that was created by factmsieve is that factmsieve has created file names different from msieve default. I remember copying the msieve invocation line from factmsieve's screen display and adding in -ncr, so that msieve would find all the files. But it has been quite a few years since I've used factmsieve, so... you know.

2021-12-30, 18:42	#5
swellman Jun 2012 6733₈ Posts	And I have never used factmsieve, so take my comments from that perspective. Thom - play with it a bit on smaller jobs is the best advice. Losing 4 hours of work is annoying - 2 months a disaster.

2021-12-30, 19:02	#6
ThomRuley May 2003 3·97 Posts	As of this morning it was still sieving with about 70% of minimum relations. If I get back home and fid it doing matrix steps, i would just let it finish.

2021-12-30, 22:08	#11
ThomRuley May 2003 3×97 Posts	UPDATE: Thanks for letting me know that it is safe to pause msieve. I found another workaround for the GIMPS jobs. Since the jobs are small, I simply opted to run them on another computer that has not been running any recent high-priority jobs. The first of these will finish in about 20 minutes. Now I am curious whether Primenet will take the job directly off the machine that was originally assigned the jobs, or whether they will disappear on the next completion estimate update, or I will have to manually remove them.