mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   Cloud Computing (https://www.mersenneforum.org/forumdisplay.php?f=134)
-   -   Which Cloud Service is the best for the Mersenne Prime Search? (https://www.mersenneforum.org/showthread.php?t=26445)

mersenne1588 2021-01-26 14:22

Which Cloud Service is the best for the Mersenne Prime Search?
 
Hi,

If I want to run the LLR test on a cloud computing platform or a cloud-based environment, when I am looking for a new record prime, which service is the best in terms of price-performance ratio? Amazon EC2, Google Compute Engine (GCE), Microsoft Azure, Google Colab?

Haven't used any of these services yet. I've heard that Google Colab is free, but math programs can only be written with Python, the connection to the Google Cloud Server is interrupted at regular intervals, and sometimes a GPU and TPU are not available. The question is whether the use of Google Colab makes here sense at all. Or whether it is better to use paid services that sometimes also offer free services, for example when I book capacities in the lower segment, such as Amazon EC2. It would have the advantage that I could first test smaller numbers and thus become more familiar with the handling.

I'm not looking for the cheapest provider here, but the provider who offers the best price-performance ratio in terms of price and processing speed. Which provider would you recommend me here? Which provider has here the best price-performance ratio?

chalsall 2021-01-26 16:42

[QUOTE=mersenne1588;570144]I'm not looking for the cheapest provider here, but the provider who offers the best price-performance ratio in terms of price and processing speed. Which provider would you recommend me here? Which provider has here the best price-performance ratio?[/QUOTE]

Since money doesn't seem to be a problem, why don't you spin up a few instances on all the Cloud Compute providers and measure the throughput per credit spent?

Then you can empirically determine where the economic curves cross for your particular use-case (read: the type of work you most enjoy contributing towards), and use that knowledge moving forward.

Hint: A certain Ben is using AWS. Perhaps a heuristic.

mersenne1588 2021-01-26 17:54

[QUOTE=chalsall;570158]Since money doesn't seem to be a problem, why don't you spin up a few instances on all the Cloud Compute providers and measure the throughput per credit spent?[/QUOTE]


I only mentioned the money to make it clear that the price is not my only concern. I just wanted to hear the opinion of people who have already gained experience with Cloud Computing, which provider is recommended in terms of price-performance ratio. That's all. I think, someone can certainly tell me which provider is recommended here.

chalsall 2021-01-26 18:23

[QUOTE=mersenne1588;570163]I think, someone can certainly tell me which provider is recommended here.[/QUOTE]

Based solely on heuristics, AWS is your best bet (at this current moment in time) if you're willing to put money behind your compute asks.

Your KM/s delta-V might be different than this model, depending on your particular situation. (Sorry for the different frames of reference.)

LaurV 2021-01-27 04:02

[QUOTE=mersenne1588;570144]I've heard that [...] math programs can only be written with Python[/QUOTE]
That is false, you are confused there. We run mprime and gpuowl on colab, none of which are written in python, one is written is C and assembler, the other one uses openCL too, to force a graphic card (GPU) to crunch numbers instead of CPU. Python is kinda "shell", or "command interpreter" there, you may use python to launch your program, write your scripts, but don't confuse that with the real applications that you can run.

gLauss 2021-01-27 07:35

AWS is too expensive (for me)
 
I have done some experimenting with AWS (and tested a DC number), but even in Germany with high power costs (30ct/kWh) it is usually cheaper to run it on the hardware one already owns if you have the hardware anyway. If you want to try, here is my crude approach how I did it: [url=https://klaus.hohenpoelz.de/running-gimps-mprime-on-amazon-web-services.html]https://klaus.hohenpoelz.de/running-gimps-mprime-on-amazon-web-services.html[/url] I did it mainly for experimenting with EC2 API.

retina 2021-01-27 08:30

[QUOTE=gLauss;570210]... with high power costs (30ct/kWh) it is usually cheaper to run it on the hardware one already owns ...[/QUOTE]I cut off the last part about if-you-already-have-the-hardware because I don't think it matters. Unless one only plans to run for a few weeks, or maybe a few months at most, and then quit. After the initial payback time running your own setup will cost less overall.

It makes perfect sense because a commercial company wants to make a profit, so you aren't going to be able to pay less for cycles, and also pay them a premium towards their profit.

It is a fallacy about how since they pay lower power prices because of scale, then that somehow means you can benefit from that. You would be treating them as simply a power reseller. And power resellers also have to make a profit. Which is why when we buy power from resellers it ends up costing the same anyway. And add in the extra hardware capital costs and worker's salaries and whatnot, and that surpasses any possible power savings.

axn 2021-01-27 08:51

Except when they're selling surplus compute power using "spot" prices. They're already burning electricity by keeping the physical hardware on, they might as well try to minimize losses / turn a small profit.

gLauss 2021-01-27 09:51

[QUOTE=axn;570219]Except when they're selling surplus compute power using "spot" prices.[/QUOTE]
Yes, but I was comparing "spot" prices to my power costs and still twice as much as I pay for my power. Don't forget all those virtualization AWS does has overhead, too. The most ridiculous overpriced AWS stuff is the 9ct / GB outgoing traffic fee.

Back to topic: I have not tried GCP or Azure instead of AWS. However, I have heard that they are less expensive, so it might be worthy to try those platforms for GIMPS.

mersenne1588 2021-01-27 10:10

[QUOTE=LaurV;570202]That is false, you are confused there.[/QUOTE]

English is not my mother tongue, but I think you wanted to say: "You get confused there" or "You mix up something there". But as a person, I am certainly not "confused".:lol:

LaurV 2021-01-27 10:30

Thanks. That's what I wanted to say, and I won't contradict you on an English subject. Till some native weights in (or not), I still believe my sentence was right. Not a native speaker here either.

:confus:


All times are UTC. The time now is 18:26.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.