Lots of roundoff errors

[TheMawn]

Hey, folks.

I've been getting a number of roundoff errors in a DC. I wouldn't be heartbroken if the test came out bad since it's a smaller exponent (less wasted time) but I'd still like to get to the bottom of this.

I've attached a screenshot of the worker in question.

Here's the bits of results.txt relevant to the exponent in question.

[Tue Aug 12 00:32:29 2014]

Trying 1000 iterations for exponent 32562559 using 1680K FFT.
If average roundoff error is above 0.24236, then a larger FFT will be used.
Final average roundoff error is 0.23838, using 1680K FFT for exponent 32562559.

[Tue Aug 12 07:50:20 2014]

Iteration: 6557188/32562559, Possible error: round off (0.4375) > 0.40
Continuing from last save file.
Disregard last error.  Result is reproducible and thus not a hardware problem.
For added safety, redoing iteration using a slower, more reliable method.
Continuing from last save file.

[Tue Aug 12 22:27:40 2014]

Iteration: 7679973/32562559, Possible error: round off (0.4375) > 0.40
Continuing from last save file.

[Wed Aug 13 01:03:36 2014]

Iteration: 7679973/32562559, Possible error: round off (0.4375) > 0.40
Continuing from last save file.
Disregard last error.  Result is reproducible and thus not a hardware problem.
For added safety, redoing iteration using a slower, more reliable method.
Continuing from last save file.

I was having some strange stuttering problems (audio and video) during a Civ5 round with friends. I was going to stop a Prime95 worker, but when I saw all of this crap, I stopped everything altogether and rebooted. All technical issues vanished.

When I started my worker up again, I took the screenshot.

7:50 on Tuesday is some random time of the morning.

22:27 is when we started the round.

1:03 on Wednesday is when I started everything up again.


The part that scares me is "2 roundoff errors of which 1 is repeatable"

EDIT: Now that I think of it, the 22:27 and 1:03 errors are probably the same ones. I think I stopped the program before it had the chance to double-check the questionable iteration.

[axn]

Everything is consistent with an exponent right on the crossover point between FFT sizes. Things should be fine the way they are.

EDIT:- Increase your checkpointing frequency so that you lose fewer iterations when restarting.

[TheMawn]

Or should I just start over and force a larger FFT...?

[axn]

Originally Posted by TheMawn

Or should I just start over and force a larger FFT...?

Perhaps that is better. At least you wouldn't have to worry this much.

[kladner]

Originally Posted by axn

Perhaps that is better. At least you wouldn't have to worry this much.

I have recently restarted a couple of assignments with a higher FFT, though I did not have the "reproducible error" message. Part of my problem was that there were voltage stability problems, I think. I have worked on addressing these. I am trying the next higher FFT on the rationale that if the hardware is still causing errors, it will deliver a hard round off error, even with the higher FFT.

The above is a follow up to trying to balance my set voltage against Load Line Calibration so that it does not dip below a threshold voltage which I consider safe, but does not end up too high under load. This is somewhat complicated, because the load varies widely between idle, P95 running, and P95 plus two hungry GTX 500 series GPUs. I think I have it under control now after considerable testing with various combinations of GPUs and the different Torture Test modes.

I will have to complete a few more assignments (DC) to be really confident about the situation.

[Prime95]

Originally Posted by TheMawn

Or should I just start over and force a larger FFT...?

Your roundoff errors of 0.4375 are nowhere near 0.5. Relax.

[TObject]

Maybe these should be called roundoff warnings. LOL

[ewmayer]

Originally Posted by Prime95

Your roundoff errors of 0.4375 are nowhere near 0.5. Relax.

While 0.4375 is mostly safe (~99% in my experience), if you get multiple such during a test, don't get too relaxed.

George, do you have any large-dataset stats on ROEs at the above level, vs bad-results? A histogram of "number of 0.4375 errors during test vs % of such tests which failed" would be really useful.

[TheMawn]

I think Misters Lucas and Lehmer would be pretty proud of what we're doing. Needing a special method using Sine and Cosine just to square a (big) integer.

[Prime95]

Originally Posted by ewmayer

George, do you have any large-dataset stats on ROEs at the above level, vs bad-results? A histogram of "number of 0.4375 errors during test vs % of such tests which failed" would be really useful.

No I don't. However, prime95 uses a special method to redo any iteration with an ROE above 0.40625. In effect, prime95 can tolerate ROE up to 0.59375.

[ewmayer]

Originally Posted by Prime95

No I don't. However, prime95 uses a special method to redo any iteration with an ROE above 0.40625. In effect, prime95 can tolerate ROE up to 0.59375.

You can reliably determine if e.g. a 0.4375 is really a 0.5625 which has been NINT-aliased? Do tell - something based on an FFT checksum?