Abuse (?) of the manual assignment system

[kriesel]

Or P+1:

New features in Version 30.6 of prime95.exe
-------------------------------------------

1) P+1 factoring.  A worktodo.txt entry looks like this:
	Pplus1=k,b,n,c,B1,B2,nth_run[,how_far_factored][,"known_factors"]
   Unlike P-1, the fact that factors of Mersenne numbers is 1 mod 2p is of no value.
   Thus, P-1 is vastly more effective at finding factors.  A P+1 run is about as
   valuable as running one ECM curve.  P+1 stage 1 is 50% slower than P-1 stage 1
   but several times faster than ECM stage 1.  P+1 stage 2 is a little faster than
   P-1 stage 2 which in turn is a little faster than ECM stage 2.
   Unlike P-1, P+1 has only a 50% chance of finding a factor if factor+1 is B1/B2 smooth.
   Thus, it makes sense to do 1 or 2 (maybe 3) runs.  That is what the nth_run argument is for.
   There are two special starting values for P+1 that have a slightly higher chance of
   finding a factor.  These special starting values correspond to nth_run=1 and nth_run=2.

https://mersenneforum.org/showpost.p...60&postcount=4

[Andrew Usher]

Even granting that more than one assignment is needed in those cases, it doesn't apply to LL/PRP (as self-DCs are discouraged). Surely no one (unless attemping such a DC) _intends_ to have two LL assignments on the same exponent; and if the server does that, it's undesirable.

[Uncwilly]

Originally Posted by Andrew Usher

it doesn't apply to LL/PRP (as self-DCs are discouraged). Surely no one (unless attemping such a DC) _intends_ to have two LL assignments on the same exponent; and if the server does that, it's undesirable.

Your initial post did not specify on LL/PRP. If you meant that, say that, or admit that you are wrong. Doing parallel LL runs for testing has been a common enough thing. Ken has done some larger exponents for testing. Doing a second run and cross checking the residues also runs to be saved if they deviate from each other. While these might not have 2 separate assignments, they do form a self-DC. But, if one is on Prime95 and the other is CUDAlucas or GPUowl, then no harm.

[Madpoo]

Hopefully this weekend I'll have a chance to come up with some set of criteria for expiring these old assignments.

Just to get the ball rolling, I was looking at this as a start:
- assigned over 2 years ago, last updated > 1 year ago, and the "next expected" date is in the past (it's overdue even by its own estimate)

With that, I get 1372 assignments I can expire.

That was part of the wonderful mystery of these is that clients can tell the server when it will next check in, and sometimes that can be a LONG way off. The assignment can still be expired depending on what category it's in, but if it's way out past that category 4 range, then who cares? But that's when people doing other kinds of factoring work get a little antsy... If there are individual exponents out there that someone really wanted to do TF on, but it was assigned and had little progress, then my opinion is, go ahead and do the TF on it (assuming in this case that it's a LL/PRP assignment and not someone else's TF). But that's just me...

Meanwhile I'll do what I can to clear up some stuff, but I will probably lean towards being cautious about making sure assignments are kept around if their existence doesn't really seem to be hurting anyone. That might mean anything < 1 year old is safe... exceptions might be made if individual users have a lot of work they were assigned and haven't done any of it, but we'll see.

[Uncwilly]

Originally Posted by Madpoo

Just to get the ball rolling, I was looking at this as a start:
- assigned over 2 years ago, last updated > 1 year ago, and the "next expected" date is in the past (it's overdue even by its own estimate)

With that, I get 1372 assignments I can expire.

There are a bunch in the 332M range that have not checked in in more than a year and are 2 or more years old.

[Mark Rose]

I follow that one year rule, with two exceptions:

1. Some exponents with an LL result by a user still show an LL assignment to the same user.
2. A user has a large number of exponents reserved for years with no progress reported. If there has been zero progress reported for a year, I think it's fair game to TF, even if the assignment is actively being maintained. One user in the 332M range has over 100 exponents reserved like this.

[Andrew Usher]

Without sidetracking things, yes, I was thinking only about LL/PRP, and specifically about the kind relevant here, where a user receives more assignment(s) automatically.

The conclusion that the result of this server behavior is undesirable seems to be indisputable - there should not be 'dead' assignments remaining when the user completed the work he meant to do and has no intent of coming back to it.

A new one just was created yesterday, which Mark has already found - M213455533, on which the user got a manual LL assignment on Jan. 16 (FTC - I guess it's still possible through worktodo) and turned in the result without an AID, resulting in his assignment converting to a non-expiring DC; he already has created 5 such, probably the same way, has 3 more yet to complete, and probably doesn't know there's anything wrong with his results reporting - if he notices at all, he probably assumes it's a server bug. And for practical purposes it is - those assignments should be expired just as if they'd been reported with the correct AID, as one user's mistake should not affect the whole system.

As for cleaning up old assignments in general, the specifics aren't terribly important, but there needs to be a time limit on _all_ assignments; none should stand indefinitely without activity. And for individual users I'll repeat Bill and kirkland - their assignments are, though less seriously, distorting ECM and DC respectively in the same way that the first user was P-1, and if they show no activity they have no defense for >1000 assignments.

I fully agree with Mark Rose (and might go farther) - his first case is the type I'm talking about, his second the type Aaron is talking about expiring, and both should be.

[Madpoo]

Originally Posted by Mark Rose

I follow that one year rule, with two exceptions:

1. Some exponents with an LL result by a user still show an LL assignment to the same user.
2. A user has a large number of exponents reserved for years with no progress reported. If there has been zero progress reported for a year, I think it's fair game to TF, even if the assignment is actively being maintained. One user in the 332M range has over 100 exponents reserved like this.

I looked for specifically LL assignments (I'll check PRP next though I expect far less, if any) where a user has an assignment for the same exponent they've turned in a result for.

214 of them, and I see the 100 by user "patgie" alone. 24 of them are from the anonymous users so that's expected.

Some are absolutely on purpose, like Kriesel assigning himself a triple-check for M74270929 which he already did one result for but mismatched the original. Ideally someone else would do that because if he matches his own result, I'll see that as a self-verified exponent and do an independent test (yeah, I still do those, although LaurV has done a bunch of 100M digit first & second tests and I just threw up my hands at that... not sure why he's not doing PRP instead, but whatever...)

Kriesel is also doing a *3rd* test on M340830817 because neither of his previous two matched the first one.

If I further exclude any such assignments where the assignment was made *after* the LL result was turned in, there are only 190. Of those, 98 are patgie, 16 from a user "msubaroda_jainam", 11 from the anonymous user (which I should exclude from any cleanup attempt), and then gets into smaller counts:

Alpha2378	7
Galamo Monkam	7
LoveYouTenderly	7
Tenderly_D	5
Zaron3	2
usagi	2
doshin	2
peterstrasser	2
Ma XiangRu	2

After that are another 20 users with just a single case.

All that to say, it's not a huge problem, at any rate. The users with a lot of them must have something peculiar going on that's causing it... something they're doing.

[Madpoo]

I'm looking a little more at the official assignment expiration process in the code. Besides the super complicated rules based expirations, there are some interesting things to be seen in this one:

Non LL/PRP work is expired if:

• the date next expected is empty *and* the assignment is over a day old (a client will typically check in at least once after getting an assignment... if it doesn't after a day, it's a very good indicator they've moved on)
• OR the assignment is over 1 day old and it's been over 60 days since it last checked in
• and it's over 10 days past the date next expected

Other expirations that happen there are for first time LL/PRP tests specifically in the range of 330e6 to 350e6 where:

• the current progress is empty or < 2% done AND at least one of the following:
  • the date next expected is empty *and* the assignment is over a day old
  • the expected completion date has some value (doesn't matter what) and the "next expected" date is a year ago
  • the expected completion is empty and the next expected date is anytime in the past

Another set of rules specifically for P-1 looks like:

• Assigned 30 days ago
• expected completion date is empty
• exponent is somewhere between 101e6 (must have been the first-check range at the time) and a dynamic value that currently sets that top range to 112,523,820... I guess this makes sure P-1 isn't hogging the leading edge of first time checks

Another blob looks at PRP-cofactor assignments specifically and expires them if:

• it's over 60 days old and at some point it was manually extended via the website
• over 100 days old otherwise

One final thing it does is for PRP certiification assignments that expires them if:

• it was assigned over (4 + exponent/10e6) days ago... larger exponents get more time, but for example an exponent around 100e6 gets 14 days, an exponent around 330e6 gets 37 days, etc.

I hope that provides a little transparency. There are notes in the code (that I may have put in) about those exponents in the 330-350e6 range where it's < 2% done... at the time this cleaned up a lot of really old and stagnant assignments but not so much anymore - it was more about cleaning things up (I think) before the new rules went into effect several years ago.

Bear in mind that there's still the much more complicated rules expirations for LL and PRP work, these are just the non-PRP/LL and a few exceptions that were thrown into the mix.

[kriesel]

Originally Posted by Madpoo

Some are absolutely on purpose, like Kriesel assigning himself a triple-check for M74270929 which he already did one result for but mismatched the original. Ideally someone else would do that because if he matches his own result, I'll see that as a self-verified exponent and do an independent test (yeah, I still do those, although LaurV has done a bunch of 100M digit first & second tests and I just threw up my hands at that... not sure why he's not doing PRP instead, but whatever...)

Kriesel is also doing a *3rd* test on M340830817 because neither of his previous two matched the first one.

... or each other.
There are two things going on here, in my runs mentioned by Madpoo.
One: trying to clean up my own messes.
Two: on a small set of ~100Mdigit exponents, doing iterated LL to evaluate LL unreliability at 100Mdigit. The sample set on LLDC at 100Mdigit is still quite small. (And I suppose, should remain small indefinitely. PRP/GEC/proof is that superior.) But to strengthen the case for PRP with GEC & proof generation, for newbies & doubters, a little more sampling may help.
Five exponents remain to be completed from the 12 selected in https://mersenneforum.org/showthread.php?t=26871, and some of them are on old slow dual-Xeon systems with ECC ram.

When practical, if a previous run by me is done by gpuowl and does not provide a match, I'll follow with prime95, which will report interim residues and use the security code and apply entirely different code on different hardware (CPU vs. GPU). A proper prime95 security code should allay doubts as to whether they were really independent runs.
I run LL DC on gpuowl v6.11-380 on only one GPU currently, that has proven highly reliable at DC wavefront and somewhat higher. (Zero bad final res64 to date since May 2021 on that GPU, only two on that GPU ID since I began logging bad res64 by source in 2018, and those might have been for a different physical GPU.)
But that GPU reliability does not prevent the first LL by someone else done years earlier being bad on occasion, yielding a mismatch between the first two LLs on the exponent. A previous example is https://www.mersenne.org/report_expo...l=1&expired=1; another is M333044087
The third run on M74270929 is on prime95 v30.8b14 on a Xeon Phi with ECC MCDRAM & ETA 1 day, to verify or refute the M74270929 LLDC on the very reliable GPU.

[Madpoo]

I've cleaned up some old assignments, about 1850-1900 total. Around 1600 that were old assignments in general, and then those 270'ish or whatever where the user had already turned in a result for the same exponent and got converted a a double-check.