mersenne.ca server migration and probable downtime (Nov 2020)

[James Heinrich]

Originally Posted by lycorn

Hmm,... If it is working, don't fix it. 😁 Or is it not? Good luck, anyway.

Sometimes that's very good advice.

It's working, but it could work better (more efficiently). Hence today's change(s).
I haven't yet started the changes, will post back when I do.

[James Heinrich]

I have put a temporary hold on the changes, I have found another major structure change I want to make at the same time and it will take a while to prepare and test that, so I won't get around to fiddling with the live server until much later today, possibly tomorrow.

[James Heinrich]

OK, revisiting this finally, I plan to take mersenne.ca offline around 12h from now, for an unknown length of time, probably somewhere between 4-6h, to apply database modifications. Apologies in advance for the downtime.

[James Heinrich]

I'm ready to start the process, I'll be taking the site down in the next 5 mins or so.

edits:
10:40 - starting offline table compression
11:30 - lost internet connection during compression, had to redo some of it, compression finished, starting known_factors table migration
12:30 - copying to new known_factors tables up to about 3G
13:30 - finished copying to new known_factors tables, starting associating k index with external table (this will probably take a while)
16:00 - k-index association is proceeding, but slower than I had hoped I'm debating the possibility of bringing the site back online before this is complete
17:30 - still making (slow) progress, still considering how to bring the site up sooner rather than later
20:00 - still not done updating, but the last 20% are the slowest. I'm tentatively bringing the site up to see what errors I've introduced while the update should continue in the background.
20:04 - temporarily taking it down again while I investigate some unexpectedness
21:30 - was making good progress, fixed some bugs, but somehow one of the compressed tables crashed and erased itself. Fortunately it's easily restored from backup, but I'm not sure how or why that happened.
23:20 - I'll leave the site up overnight, there are still some background updates going on but that shouldn't interfere. However, I will need to take the site down, briefly, again tomorrow to remove fields and rebuild large tables. And it turns out my clever idea to compress the read-only archived-results tables isn't going to work, I found that I do in fact need write access in some cases to those tables. Which is a shame, cause it saves 10GB or so of space.

[James Heinrich]

Sorry guys, time to take the site down again for a few hours (hopefully not more than that, but we'll see).
10:00 - well, unfortunately it seems that table compression is not reliable, had that corrupted table yesterday and trying to uncompress them today resulted in all those tables being corrupted, so I need to re-import them all (which means more downtime, sadly). Lesson learned.
12:00 - corrupted tables re-imported, still changing structure on known-factors tables. Maybe an hour(?) remaining
12:48 - putting site back online

I think that's it for downtime (I hope).

[Viliam Furik]

Originally Posted by James Heinrich

Sorry guys, time to take the site down again for a few hours (hopefully not more than that, but we'll see).

Few minutes before you turned it off, I have uploaded a bunch of TF1G results, about 850 factors. Will they be counted? Because I have already emptied the results file for mfaktc.

It wouldn't hurt much if they were gone, it was only about 1,5 days of TF.

---

But I have noticed one funny thing. One of the factors was from a different range than it should be... M1915293893 was being TFed from 68 to 69 bits. k_max was reported to be 154100575576, by mfaktc. Factor popped out at 77.7% of the run. It was 590311681940496271463 = 2 * 154104718957 * 1915293893 + 1. Exponent is only a tad bigger than 2^69, and its k is bigger than the reported k_max.

How is that even possible?

[James Heinrich]

Originally Posted by Viliam Furik

Few minutes before you turned it off, I have uploaded a bunch of TF1G results, about 850 factors. Will they be counted? Because I have already emptied the results file for mfaktc.

As long as they were accepted by the server at the time then yes they'll be processed.

Originally Posted by Viliam Furik

M1915293893 M1915293983 was being TFed from 68 to 69 bits. k_max was reported to be 154100575576, by mfaktc. Factor popped out at 77.7% of the run. It was 590311681940496271463 = 2 * 154104718957 * 1915293893 + 1. Exponent is only a tad bigger than 2^69, and its k is bigger than the reported k_max. How is that even possible?

That's perhaps a question for the mfaktc thread, but certainly the bitlevels, and possibly the k_min/max values, are more approximate targets than absolute values. You say "factor from 68-69", so mfaktc calculates what 2^68 and 2^69 are in terms of k for that exponent and sets those targets, but when it comes to actually walking through the classes it may need to go slightly below-in and/or above-max to ensure it gets the entire range covered, so you can occasionally get "lucky" and find a factor just outside your target range (69.000038789949 bits in this case).

[James Heinrich]

Server should be back online for the foreseeable future, thank you for your patience.