mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   mersenne.ca (https://www.mersenneforum.org/forumdisplay.php?f=173)
-   -   mersenne.ca (https://www.mersenneforum.org/showthread.php?t=23051)

henryzz 2018-04-15 17:02

[QUOTE=GP2;485301]Disk storage is cheap. PrimeNet could start storing this data too.[/QUOTE]

Hosting bandwidth can be more expensive though.

James Heinrich 2018-04-15 17:25

I usually see it the other way -- you have to pay for additional storage, whereas often [i]bandwidth[/i] (e.g. 250Mbps) is fixed but data transfer is uncapped.
Not that the demand for factor tables should be tremendously high, at least I don't think so.

henryzz 2018-04-15 19:55

[QUOTE=James Heinrich;485365]I usually see it the other way -- you have to pay for additional storage, whereas often [i]bandwidth[/i] (e.g. 250Mbps) is fixed but data transfer is uncapped.
Not that the demand for factor tables should be tremendously high, at least I don't think so.[/QUOTE]

The forum has a data transfer cap. Currently it is a non issue but it was before we upgraded plan.

snme2pm1 2018-04-16 11:37

I recollect that Aaron has provisioned a greater server then ever before, and anticipated it to be under-utilised.
I also recall his long ago comments about lots of spinners! Lovely!
Is not some kind of amalgamation of forces due?
I also believe that Aaron provisioned matters such that the primenet sever was a virtual machine.
As far as I am aware, James is a Linux inclined bloke, due to past head bash emoticons in relation to Widows facilities regarding database nuisances.
So would it be not decent to create a minor related linux virtual machine for James to configure?
Thereby more forces would be combined.

Uncwilly 2018-04-16 14:14

I am of the type that thinks that there is value to having the data and websites spread across multiple different companies resources. What if the company that host PrimeNet has a court action or bankruptcy. Or the data center has a natural disaster or disgruntled employee with plan to do as much damage as possible.

Mirroring the data and custom software that makes up GIMPS should be on at least 3 physical devices, at least 2 wide spread locations, and on different hardware platforms (with at least 2 different storage media [a couple of thumb drives in addition to the SSD/HDD's at the server].

Mersenne.ca, PrimeNet, MF, and GPU72 are all on different companies hardware. Mirroring all datasets and code on to each makes sense. Syncing once a week from the mother site should not max anyone's bandwidth.

Madpoo 2018-04-18 04:51

[QUOTE=snme2pm1;485439]I recollect that Aaron has provisioned a greater server then ever before, and anticipated it to be under-utilised.
I also recall his long ago comments about lots of spinners! Lovely!
Is not some kind of amalgamation of forces due?
I also believe that Aaron provisioned matters such that the primenet sever was a virtual machine.
As far as I am aware, James is a Linux inclined bloke, due to past head bash emoticons in relation to Widows facilities regarding database nuisances.
So would it be not decent to create a minor related linux virtual machine for James to configure?
Thereby more forces would be combined.[/QUOTE]

Reading these posts, the thought did occur to me that maybe a consolidation would be possible. The last time I updated the hardware (when the server was physically moved from Seattle to Dallas) it gained even more CPU/memory/drive space to the point where I'm able to clone Primenet entirely and run a backup copy for testing. We used it to test some of the PRP code and database changes before it went live.

Ultimately I'd defer to George and James on whether either of them would actually want to, but technically it seems possible. The colo provider does give us a good amount of bandwidth and I'm pretty sure we have plenty to spare on a monthly basic. As far as CPU goes, Primenet typically doesn't use much although sometimes the SQL queries may spike it for a bit during more complicated maintenance, but otherwise not too bad.

[QUOTE=Uncwilly;485449]I am of the type that thinks that there is value to having the data and websites spread across multiple different companies resources. What if the company that host PrimeNet has a court action or bankruptcy. Or the data center has a natural disaster or disgruntled employee with plan to do as much damage as possible.
...[/QUOTE]

Speaking just for Primenet, it uses a cloud backup provider with weekly full SQL backups and I think the incremental SQL backups are every 4 hours or something like that (haven't checked lately). The cloud provider I just switched to has an option to backup the entire virtual machine, and I do have the space for it on the account, so maybe I'd look at doing that as well on some quarterly basis. Even though all the data/code is backed up, it would still take some time to rebuild the VM and get it all setup and running so maybe having a full VM snapshot would be a nice-to-have.

Uncwilly 2018-04-18 06:03

[QUOTE=Madpoo;485610]Even though all the data/code is backed up, it would still take some time to rebuild the VM and get it all setup and running so maybe having a full VM snapshot would be a nice-to-have.[/QUOTE]James and Chris have some code that should be preserved. There are some other odds and ends around, some loosely associated with GIMPS and some independent, that would be good to have duplicates (like some of Luigi's things, all of the GPU code sets, a quarterly snap shot of mlucas, etc.)

I know PrimeNet is well set up, but considering that none of the data are on the wayback machine, a well handled second back scheme (mirroring on a different company's platform) might still be wise.

LaurV 2018-04-19 05:49

If external storage space needed, I can offer a corner on my shelves too. If you provide a link, I can download and store what I found there (possibly periodically). 50G is not much, can keep (say) the last 4-5 generations in a 250-300G dedicated backpack...

(edit: assuming it is zipped/encrypted already, otherwise the space to store it will be lower when zipped by myself)

James Heinrich 2018-04-19 10:23

See post #40 for download links, ~3.5GB zipped.

snme2pm1 2018-04-20 13:37

I recall sometime ago there was occasion to speak about the Bus Factor, in regard to PrimeNet.
Well I wonder how that might play into the way that people think in near discussion for mersenne.ca.
Having resources distinctly mirrored, might offer a distinct advantage in safety terms, and I would prefer that James doesn't fall under a bus any time soon (they exhibit too many tons).

kladner 2018-04-20 14:39

My first thought was to wonder which (electronic) bus was in question. I realize now that it is the Bus of Unexpected Fatality. :sad:


All times are UTC. The time now is 04:34.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.