View Single Post
Old 2004-07-31, 17:34   #11
Prime95
P90 years forever!
 
Prime95's Avatar
 
Aug 2002
Yeehaw, FL

22·1,783 Posts
Default

Further study of this naughty case reveals no problem in the FFT code. Unfortunately, the FFT data number being squared does not exhibit "random" properties that generates low round-off errors.

I'm talking to the PFGW folks about possible workarounds. They have seen this many times in the past. Since prp3 does the last 50 squarings with round-off checking enabled, then I think it is safe to use. If you get this error in the last few iterations, I'll bet you have a probable prime and the next rev of prp3 will backtrack and handle properly.
Prime95 is online now