View Single Post
Old 2017-11-30, 00:58   #5
danaj
 
"Dana Jacobsen"
Feb 2011
Bangkok, TH

22×227 Posts
Default

Seems like a new thread for this makes sense. This is going far off topic.

I was thinking just check around windows of the gaps < 4e18. Our program should find the gap. I'm unsure as to the value but it is cheap as long as the windows are reasonably sized.

For the spot check of *our* results, we could do the same (but I see no point), but what we probably want is checking that the recheck finds the same gaps as the original.

Alternately as you suggest just do random ranges. What you want is random selections without duplication, which can be done using Perl/ntheory's randperm, Mathematica's RandomSample, numpy's choice (though it can easily blow up in time and memory), and more. Ensuring no double check of own work is probably manual with our current non-automated methods.

In theory it'd seem like we'd also want more like "a random 10% of each partipants work" rather than completely random, or we will likely do no double checking of some people's work.

While it'd be nice to have full results from everything already done, I'm not sure how many people sent that info. What I believe we do have is the "largest gap" for reasonably sized subranges, so one could go through all the results and turn them into [version, parameters, expected value(s), participant id] tuples for small chunks. Version here is distinguishing between the "A,B" ranges vs. "A,B,M1,M2" modulo ranges. Then it would be reasonable to do the sampling from that set and assign the double checks easily enough.
danaj is offline   Reply With Quote