mersenneforum.org  

Go Back   mersenneforum.org > Great Internet Mersenne Prime Search > Hardware > Cloud Computing

Reply
 
Thread Tools
Old 2016-07-10, 21:27   #1
GP2
 
GP2's Avatar
 
Sep 2003

5×11×47 Posts
Default Google Compute Engine

Amazon EC2 vs Google Compute Engine

Here is my comparison of Amazon's EC2 (Elastic Compute Cloud) and Google Compute Engine for the purpose of Lucas-Lehmer testing of Mersenne numbers. (Original posts here and here)

Google offers instance types comparable to Amazon's. For LL testing, the instance types we want are designated "compute optimized" in Amazon's terminology and "High CPU" in Google's terminology.

As of August 2016, here are the respective instance types:

Code:
 Amazon          Google         "Virtual      Actual           Memory     Memory
  EC2         Compute Engine      CPUs"        cores            (Amz)     (Goog)
----------    --------------    --------      ------           ------     -------
c4.large      n1-highcpu-2          2            1             3.75 GiB   1.80 GB
c4.xlarge     n1-highcpu-4          4            2             7.5  GiB   3.60 GB
c4.2xlarge    n1-highcpu-8          8            4             15   GiB   7.20 GB
c4.4xlarge    n1-highcpu-16        16            8             30   GiB  14.40 GB
c4.8xlarge    n1-highcpu-32   Amz=36, Goo=32  Amz=18, Goo=16   60   GiB  28.80 GB
Amazon also has GPU instances, but these are not cost-effective at all at the moment. Google does not have GPU instances.

Amazon and Google both offer Linux or Windows, but Windows is several times more expensive.

Cores and memory

Both Amazon and Google mention "Virtual CPUs", but this is marketing obfuscation. It just means the cores use hyperthreading, which is not that useful for LL testing. Only the number of cores matters.

Amazon offers more memory, but that's irrelevant for LL testing which uses a tiny amount of memory. Memory only matters for P−1 testing.

Google lets you customize machine types, so you can select quantities of cores and memory different from the standard offerings. However, in practice for LL testing there is no pricing benefit at all from doing so. This does not save any money for one-core instances (n1-highcpu-2), but by configuring a multi-core machine with a minimal amount of memory there can be some savings.

Regions and zones

Both Amazon and Google have many different "regions" within the US and across the world, where their servers are physically located. Within each region there are multiple "availability zones" (Amazon terminology) or "available zones" (Google terminology). Conceptually, this is something similar to having multiple datacenter buildings ("zones") within the same city or county ("region").

This matters because prices can be different in different availability zones. In particular, for LL testing using Amazon EC2, the prices can vary wildly and drastically.

Processors

Amazon's hardware has gone through various generations (c1, c2, c3, and now c4). The c3 was introduced in November 2013 and the c4 in January 2015. Although c3 instances are still available, they should never be used for LL testing because c4 instances benchmark 50% faster and the price is virtually identical.

Amazon's c4 instances use 2.9 GHz Intel Xeon E5-2666 v3 (Haswell) processors, a custom chip "optimized specifically for EC2". If you use the largest instance (c4.8xlarge), then you have all the cores at your disposal and can "control C-state and P-state configuration" (i.e., maybe turn on turbo or whatever).

Google takes a different approach. The "n1-highcpu" instance type doesn't refer to a specific processor type, rather they have different processor architectures in different zones, ranging from Sandy Bridge to Ivy Bridge to Haswell. Obviously Haswell is the only one you want for LL testing. There is no information about the clock rate in GHz.

Kinds of instances (interruptible and non-interruptible)

As mentioned above, instances come in various named types, according to how many cores and how much memory they have. But instances can also vary in terms of their duration and interruptibility, with big price discounts available if you are willing to let your instances get interrupted whenever higher-paying customers show up.

Some kinds of cloud applications require 24/7 availability (for instance, websites), and some compute jobs for business have strict real-world deadlines (for instance, simulations and modeling); those customers can't tolerate interruption of services. However LL testing does not have these constraints, so we will always choose the cheapest option.

Kinds of instances (Amazon)

Amazon offers three kinds: on-demand instances, reserved instances, and spot instances. On-demand instances are the most expensive: you pay a fixed rate per hour, for as long or as short a duration as you wish. Reserved instances give you a substantial discount if you commit to a term of either one year or three years. However, in practice, both of these are irrelevant and we will never use them, they are too expensive.

The third kind is spot instances, which is the only kind that makes sense for LL testing. Unlike the other two kinds, spot instances can be terminated (permanently) at any time with only two minutes notice. Spot instances have no fixed price: their hourly prices fluctuate according to market demand. Usually they are up to 90% cheaper than on-demand instances, but sometimes they have drastic price spikes to levels up to ten times higher than on-demand prices. At any given time the spot prices may be drastically different in each availability zone in each region.

When you launch a spot instance, you set a limit price for the most you're willing to pay per hour; if the spot price goes higher than that, your instance is terminated. Although termination is permanent, your storage volumes (i.e., virtual hard drives) can be configured to persist even when the instance they are attached to is terminated, and they can then be reattached to some other instance. You can also have a networked file system (EFS, Elastic File System) that allows multiple instances to share the same filesystem; this filesystem is permanently available and unaffected by the termination of any instance (as of July 2016 this is a newly introduced feature only available in a few regions).

You can set up a "fleet" for automatic launching of spot instances. If your instances are terminated when the spot price rises above your limit price, you can choose to have new instances launched automatically whenever the spot price once again falls below your limit price.

There is no hard time limit for spot instances, they continue to run as long as the spot price remains below your limit price. In practice, however, extreme drastic price spikes do occasionally occur, so setting a high limit price does not guarantee that your instance will run indefinitely. It will not.

Kinds of instances (Google)

Google offers normal instances and preemptible instances. Normal instances pay a fixed rate per hour, but if "sustained usage" is maintained then discounts are automatically applied; unlike Amazon, there is no need to reserve a term of one or more years in order to get a discount. However, in practice, normal instances are irrelevant for our purposes and we will never use them, they are too expensive.

Google's preemptible instances are the only kind that makes sense for LL testing. They are far cheaper than even the lowest discounted price available with full sustained usage of a normal instance. Preemptible instances can be terminated (non-permanently) at any time, and they always terminate regardless after at most 24 hours.

Preemptible instances have a fixed hourly price set by Google. This is different from Amazon spot instances, whose prices fluctuate according to market forces.

PS, note that in Google's terminology, "termination" of an instance is the same as "stopping" of an instance in Amazon terminology. It is the equivalent of doing a shutdown of the virtual machine, which can later be rebooted. Whereas in Amazon's terminology, "termination" of an instance means the same as "deletion" of an instance in Google's terminology: the virtual machine is gone but its hard drive storage can be configured to persist, and can later be attached and mounted on some other instance.

Price comparison

The prices mentioned below are for instances running Linux; Windows instances are several times more expensive.

As of July 2016, Google's fixed price for preemptible instances of instance type n1-highcpu-2 is 2.0 cents per hour for US regions and 2.2 cents per hour for Europe/Asia regions. As of August 2016, Google's fixed price for preemptible instances of instance type n1-highcpu-2 is 1.5 cents per hour for US regions and 1.65 cents per hour for Europe/Asia regions. It is the same price for Sandy Bridge and Ivy Bridge and Haswell processors, so you would have to take care to choose an available zone that uses Haswell.

Running Windows instead of Linux costs an extra 4.0 cents per hour per core, or perhaps per CPU (the online documentation contradicts itself). The n1-highcpu-2 has one core, but two virtual CPUs. Either way, Windows is not a viable option.

The prices for preemptible instances of the larger instance types go up in exact proportion: 3.0 or 3.3 cents per hour for n1-highcpu-4 for US or Europe/Asia respectively, all the way up to 24.0 or 26.4 cents per hour for n1-highcpu-32.

Google lets you configure custom machine types with nonstandard quantities of cores and memory. The pricing for preemptible instances is 1.002 or 1.103 cents per hour per virtual CPU in US or Europe/Asia respectively, plus 0.156 or 0.172 cents per hour for each GB of memory in US or Europe/Asia respectively 0.698 or 0.768 cents per hour per virtual CPU in US or Europe/Asia respectively, plus 0.094 or 0.103 cents per hour for each GB of memory in US or Europe/Asia respectively. For the one core instances there are no savings, since a custom virtual machine with 2 virtual CPUs (i.e., one core) and 1 GB of memory would come to 1.49 or 1.639 cents per hour in US or Europe/Asia respectively, almost identical to the price of 1.5 or 1.65 cents per hour in US or Europe/Asia respectively for an n1-highcpu-2 (2 virtual CPUs, i.e., one core) which has 1.8 GB of memory. However, since Lucas-Lehmer testing requires very little memory, you could configure a multi-core custom virtual machine with a much-smaller-than-standard amount of memory and save money that way.

Amazon's spot prices can fluctuate considerably over time, and vary drastically from one availability zone to another. It is not at all uncommon, for instance, for the spot price of a c4.large to be 1.7 cents per hour at some point in time in one availability zone and 2.5 cents an hour in another availability zone. Or you could have a c4.large costing 2.2 cents an hour while simultaneously a c4.2xlarge in the same availability zone is costing 5.5 cents an hour; the price is only two-and-a-half times as high even though the latter has four times more cores than the former. (The above are actual real examples, observed at the time I am writing these words). These incongruous price disparities often persist for days or longer and are not arbitraged away quickly or at all.

Spot instance prices for Windows are much higher than for Linux, typically roughly five times higher. Windows is not a viable option.

These fluctuations and inconsistencies make it hard to determine exactly what average rate you will pay with Amazon, but in practice you can do a bit better than 2 cents an hour for c4.large, and even more so if you are willing to hunt for bargains. For instance, you can mix and match instance types and migrate to other regions and availability zones in order to chase the most cost-effective spot pricing. You can stop all your instances when spot prices are high (either manually or by setting a low limit price) and then compensate by running twice as many instances when prices are low.

If you don't spend time actively finding and exploiting temporary spot pricing bargains, then the pricing of Amazon spot instances and Google preemptible instances will be pretty much comparable. As of August 2016, Amazon's spot prices rarely go as low as Google's fixed pricing for preemptible instances.

Benchmarks

I haven't tried using Google Compute Engine yet, so no benchmarks are available. Presumably the Haswell processors used by Google are roughly comparable to those used by Amazon.

Note that one factor that can affect benchmarking is the fact that multiple cores usually share the same L2 or L3 cache. Since mprime is extremely sensitively tuned for optimal cache usage, the presence of other Amazon customers running applications on the other cores of the same server can occasionally impact the L2 and L3 caches and affect the benchmarks. In practice this is relatively low impact.

Conclusion

For Lucas-Lehmer testing, as of July 2016, Amazon probably has the edge because August 2016:

1) Amazon is generally no more expensive than Google, and probably a bit cheaper. It can even be significantly cheaper if you are willing to spend a bit of time chasing short-term bargains. Google's August 2016 price cut for preemptible instances gives it a price advantage in most cases, at least for the time being, and the possibility of custom-configuring multi-core instances with a relatively small amount of memory adds to the savings.

2) The hard limit of 24 hours for a preemptible Google instance seems rather inconvenient. Presumably you have to restart them on a daily basis. However Google has "instance groups" which seem to be comparable to Amazon's spot fleet requests, so perhaps it might be possible to automate this.

3) Amazon's recently introduced Elastic File System (EFS), a type of NFS 4.1 filesystem that lets different instances share the same filesystem, is a pretty major convenience. However, EFS is only available in three regions so far, and in those regions the spot pricing is generally average to high as of August 2016.

In general there doesn't seem to be a compelling reason right now to switch to Google, at least for LL testing, although perhaps someone with a different use case might reach a different conclusion (e.g., a website). Of course, as both companies compete the situation continues to evolve. Amazon offers more convenience, but Google's pricing is fairly compelling. The situation may continue to evolve, since Amazon is opening new regions (data centers) in Ohio and in Montreal in late 2016 or early 2017.

Last fiddled with by GP2 on 2016-08-11 at 03:27 Reason: August 2016, Google cut prices for preemptible instances by 25%
GP2 is offline   Reply With Quote
Old 2016-07-10, 21:51   #2
science_man_88
 
science_man_88's Avatar
 
"Forget I exist"
Jul 2009
Dumbassville

26·131 Posts
Default

Quote:
Originally Posted by GP2 View Post
These fluctuations and inconsistencies make it hard to determine exactly what average rate you will pay with Amazon, but in practice you can do a bit better than 2 cents an hour for c4.large, and even more so especially if you are willing to occasionally mix and match instance types and migrate to other regions and availability zones in order to chase the most cost-effective spot pricing.
Have you tried using : EMA's

and plotting the MACD against the signal line ?
science_man_88 is offline   Reply With Quote
Old 2016-07-11, 13:23   #3
LaurV
Romulan Interpreter
 
LaurV's Avatar
 
"name field"
Jun 2011
Thailand

35×41 Posts
Default

Quote:
Originally Posted by GP2 View Post
Amazon EC2 vs Google Compute Engine.
Outstanding! This is the kind of posts I like. Thanks for putting it together.
You spent a lot of time testing all those things and then decided of sharing your experience with us
LaurV is offline   Reply With Quote
Old 2016-07-11, 15:26   #4
Mark Rose
 
Mark Rose's Avatar
 
"/X\(‘-‘)/X\"
Jan 2013

23·32·41 Posts
Default

Yes, thank you for this!
Mark Rose is offline   Reply With Quote
Old 2016-07-12, 21:31   #5
Madpoo
Serpentine Vermin Jar
 
Madpoo's Avatar
 
Jul 2014

52·7·19 Posts
Default

Any chance of comparing the equivalent Azure platforms? It is very helpful to have the existing comparisons you put together.

I was just wondering whether Azure counts hyperthreads as cores and fortunately it looks like they do NOT enable HT.

I also just searched on whether Azure has GPU instances and the answer is "not yet"... Looks like they're planning a rollout of those later this year as an "N-Series" option:
http://www.techradar.com/us/news/int...-boost-1320090
Madpoo is offline   Reply With Quote
Old 2016-07-13, 00:28   #6
GP2
 
GP2's Avatar
 
Sep 2003

5×11×47 Posts
Default

Quote:
Originally Posted by Madpoo View Post
Any chance of comparing the equivalent Azure platforms? It is very helpful to have the existing comparisons you put together.
I'll create a Microsoft Azure thread, because in the long run it makes sense to have threads for each cloud platform. The main problem though, I don't think Azure has anything comparable to Amazon EC2's spot instances or Google Compute Engine's preemptible instances. So that's a showstopper.
GP2 is offline   Reply With Quote
Old 2016-07-13, 01:27   #7
Madpoo
Serpentine Vermin Jar
 
Madpoo's Avatar
 
Jul 2014

52·7·19 Posts
Default

Quote:
Originally Posted by GP2 View Post
I'll create a Microsoft Azure thread, because in the long run it makes sense to have threads for each cloud platform. The main problem though, I don't think Azure has anything comparable to Amazon EC2's spot instances or Google Compute Engine's preemptible instances. So that's a showstopper.
I wonder if Microsoft uses spare cycles for their own features in some creative way. Anyway, those GPU compute instances looked interesting... up to a dual K80 system (at some unholy price, I'm sure).

From time to time when I've been looking at weird exponents, I've had moments where I wished I had a good GPU to do some extra factoring first. I don't need to do full time factoring so I'd have to run the #'s on what it would take to get a low end thing like a 480 or 1070 (1060 when it's out?) and use it sometimes, compared to the cost of paying by the hour for a high end M60 or K80. Depends greatly on how much factoring I think I'd be doing, whether it'll work in my current desktop or if I'd need to update the PSU, etc. Azure or AWS GPU compute sure seems attractive, all things considered, in my particular case.
Madpoo is offline   Reply With Quote
Old 2016-07-13, 03:51   #8
Mark Rose
 
Mark Rose's Avatar
 
"/X\(‘-‘)/X\"
Jan 2013

23·32·41 Posts
Default

Quote:
Originally Posted by Madpoo View Post
I wonder if Microsoft uses spare cycles for their own features in some creative way. Anyway, those GPU compute instances looked interesting... up to a dual K80 system (at some unholy price, I'm sure).

From time to time when I've been looking at weird exponents, I've had moments where I wished I had a good GPU to do some extra factoring first. I don't need to do full time factoring so I'd have to run the #'s on what it would take to get a low end thing like a 480 or 1070 (1060 when it's out?) and use it sometimes, compared to the cost of paying by the hour for a high end M60 or K80. Depends greatly on how much factoring I think I'd be doing, whether it'll work in my current desktop or if I'd need to update the PSU, etc. Azure or AWS GPU compute sure seems attractive, all things considered, in my particular case.
I would look at the RX 470 instead of the RX 480. 85% of the TF performance for 75% of the price using 73% of the power. I would say it's the sweet spot for dipping your toes into GPU computing. The GTX 1060 won't match either of those for performance or price.
Mark Rose is offline   Reply With Quote
Old 2016-07-13, 05:57   #9
LaurV
Romulan Interpreter
 
LaurV's Avatar
 
"name field"
Jun 2011
Thailand

35×41 Posts
Default

Quote:
Originally Posted by Mark Rose View Post
85% of the TF performance for 75% of the price using 73% of the power
One hen and half lay an egg and half in a day and half. How many eggs lay nine hens in nine days... Or to stay on topic, What's your gain in a month, running 10 of those, 24/7? (arbitrary numbers, but I couldn't stop myself trolling, sorry! The tripple-enumeration really looked like the hens problem, haha. Maybe this is good for the puzzle thread?)
LaurV is offline   Reply With Quote
Old 2016-07-13, 08:49   #10
Mark Rose
 
Mark Rose's Avatar
 
"/X\(‘-‘)/X\"
Jan 2013

23·32·41 Posts
Default

Quote:
Originally Posted by LaurV View Post
One hen and half lay an egg and half in a day and half. How many eggs lay nine hens in nine days... Or to stay on topic, What's your gain in a month, running 10 of those, 24/7? (arbitrary numbers, but I couldn't stop myself trolling, sorry! The tripple-enumeration really looked like the hens problem, haha. Maybe this is good for the puzzle thread?)
According to James' numbers, an RX 470 should put out 453.2 GHzd/d of TF. An RX 470 draws 110 watts (not counting power supply inefficiencies). So you get 172 GHzd/kWh.

An RX 480 should yield 535.2 drawing 150 watts, giving 149 GHzd/kWh.

10 470's for 30 days would give you 136.0 THzd while 10 480's for 30 days would give you 160.6 THzd.

Heh.
Mark Rose is offline   Reply With Quote
Old 2016-07-14, 00:08   #11
GP2
 
GP2's Avatar
 
Sep 2003

5·11·47 Posts
Default

Quote:
Originally Posted by Madpoo View Post
I wonder if Microsoft uses spare cycles for their own features in some creative way.
ALL the big players consume a lot of GPU cycles for their own features, for machine learning and neural networks.

Microsoft has Cortana, Google has Google Now, and Amazon has their Echo device for the home, all of which have to be smart to do speech recognition and figuring out verbal input from users. Google tries to be smart when processing your search engine queries, fixing typos and making inferences, and maybe Microsoft Bing is moving in the same direction. Google recognizes the faces and places in your Google Photos, Microsoft has their Captionbot that tries to recognize what a photo represents. Facebook and Apple are doing the same (e.g. Siri), although they don't have public clouds.

There's a huge push for practical applications of artificial intelligence and GPU farms are the backbone of a lot of the processing that is needed for that. In the short term that massive internal demand probably keeps prices high for GPU instances on the public cloud, but maybe in the longer term it will mean a massive investment in more GPUs that might eventually benefit us.

Last fiddled with by GP2 on 2016-07-14 at 00:09
GP2 is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
How-to guide for running LL tests on Google Compute Engine cloud GP2 Cloud Computing 4 2020-08-03 11:21
New GPU Compute System airsquirrels GPU Computing 90 2017-12-08 00:13
Cost to compute prime Unregistered Information & Answers 30 2013-12-18 03:34
New Compute Box Christenson Hardware 0 2011-01-15 04:44
My throughput does not compute... petrw1 Hardware 9 2007-08-13 14:38

All times are UTC. The time now is 16:01.


Sat May 28 16:01:54 UTC 2022 up 44 days, 14:03, 0 users, load averages: 1.55, 1.46, 1.57

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.

≠ ± ∓ ÷ × · − √ ‰ ⊗ ⊕ ⊖ ⊘ ⊙ ≤ ≥ ≦ ≧ ≨ ≩ ≺ ≻ ≼ ≽ ⊏ ⊐ ⊑ ⊒ ² ³ °
∠ ∟ ° ≅ ~ ‖ ⟂ ⫛
≡ ≜ ≈ ∝ ∞ ≪ ≫ ⌊⌋ ⌈⌉ ∘ ∏ ∐ ∑ ∧ ∨ ∩ ∪ ⨀ ⊕ ⊗ 𝖕 𝖖 𝖗 ⊲ ⊳
∅ ∖ ∁ ↦ ↣ ∩ ∪ ⊆ ⊂ ⊄ ⊊ ⊇ ⊃ ⊅ ⊋ ⊖ ∈ ∉ ∋ ∌ ℕ ℤ ℚ ℝ ℂ ℵ ℶ ℷ ℸ 𝓟
¬ ∨ ∧ ⊕ → ← ⇒ ⇐ ⇔ ∀ ∃ ∄ ∴ ∵ ⊤ ⊥ ⊢ ⊨ ⫤ ⊣ … ⋯ ⋮ ⋰ ⋱
∫ ∬ ∭ ∮ ∯ ∰ ∇ ∆ δ ∂ ℱ ℒ ℓ
𝛢𝛼 𝛣𝛽 𝛤𝛾 𝛥𝛿 𝛦𝜀𝜖 𝛧𝜁 𝛨𝜂 𝛩𝜃𝜗 𝛪𝜄 𝛫𝜅 𝛬𝜆 𝛭𝜇 𝛮𝜈 𝛯𝜉 𝛰𝜊 𝛱𝜋 𝛲𝜌 𝛴𝜎𝜍 𝛵𝜏 𝛶𝜐 𝛷𝜙𝜑 𝛸𝜒 𝛹𝜓 𝛺𝜔