mersenneforum.org  

Go Back   mersenneforum.org > Great Internet Mersenne Prime Search > Hardware > Cloud Computing

Reply
 
Thread Tools
Old 2016-07-13, 15:10   #1
GP2
 
GP2's Avatar
 
Sep 2003

3×863 Posts
Default Microsoft Azure

Microsoft Azure vs other cloud platforms

Microsoft's Azure competes head-to-head with Amazon's EC2 (Elastic Compute Cloud) and Google's Compute Engine as one of the three leading cloud platforms. Compared to the other two platforms, how suitable is Azure for the purpose of Lucas-Lehmer testing of Mersenne numbers?

Edit: Azure is introducing low-priority VMs, which correspond to Amazon's spot instances or Google's preemptible virtual machines. However, these are currently only available in preview, in conjunction with Azure Batch. When they are generally available, the paragraphs below will need to be revised.

The answer is: not at all, for the simple reason that Azure does not offer any kind of cheap interruptible instances comparable to Amazon's spot instances or Google's preemptible instances. The normal non-interruptible instances are far too expensive.

You can stop reading here, but I will still try to give some description of Azure.

A direct comparison between Amazon EC2 and Google Compute Engine is given in this thread.

Instance types

For LL testing, the instance types we want are designated "compute optimized" in Amazon's terminology and "High CPU" in Google's terminology; Azure uses "compute optimized" and "optimized compute" interchangeably.

As of July 2016, here are the respective instance types:

Code:
 Amazon          Google       Microsoft   Actual        Memory     Memory    Memory
  EC2         Compute Engine    Azure      cores         (EC2)      (GCE)    (Azure)
----------    --------------  --------    ------        -------    -------   -------
c4.large      n1-highcpu-2       F1         1          3.75 GiB    1.80 GB     2 GB
c4.xlarge     n1-highcpu-4       F2         2          7.5  GiB    3.60 GB     4 GB
c4.2xlarge    n1-highcpu-8       F4         4          15   GiB    7.20 GB     8 GB
c4.4xlarge    n1-highcpu-16      F8         8          30   GiB   14.40 GB    16 GB
c4.8xlarge    n1-highcpu-32      F16       16          60   GiB   28.80 GB    32 GB
                                      (but EC2 has 18)
Amazon also has GPU instances, but these are not cost-effective at all at the moment. Google does not have GPU instances. Microsoft will be introducing some in the near future. (source from Madpoo)

Cores and memory

Both Amazon and Google prefer to talk about "virtual CPUs" rather than cores, but this is marketing obfuscation. Only the number of cores matters, and there is one core for every two virtual CPUs. Azure is more honest and just states the number of cores, and names their instance types accordingly.

Amazon offers more memory, but that's irrelevant for LL testing which uses a tiny amount of memory. Memory only matters for P−1 testing.

Google lets you customize machine types, so you can select quantities of cores and memory different from the standard offerings. However, in practice for LL testing there is no pricing benefit at all from doing so.

Processors

Amazon's c4 instance types use 2.9 GHz Intel Xeon E5-2666 v3 (Haswell) processors. Do not use the older c3 instance type for LL testing because c4 instances benchmark 50% faster and the price is virtually identical.

Google also provides Haswell processors, but they also have some older Sandy Bridge and Ivy Bridge processors in some of their "available zones" (info here). Obviously Haswell is the only one you want for LL testing. There is no information about the clock rate in GHz.

Azure's F-series instance types use 2.4 GHz Intel Xeon E5-2673 v3 (Haswell) processors, which have a slower clock rate than Amazon's.

Note that Azure actually has various other offerings: the A-series for "general purpose compute", the D-series which is 60% faster than the A-series, and the Dv2-series which is 35% faster than the D-series and also uses the same Xeon E5-2673 v3 processor as the F-series.

Clearly only the Dv2-series and F-series are roughly comparable (Haswell architecture) to what Amazon and Google offer, while the others are a lot slower. No information is given about whether the Dv2 and F differ in performance. The F-series is newer and is less expensive, presumably because of the fact that it has less memory and smaller hard drive storage (which are both irrelevant for LL testing), so that would be the logical choice between the two.

Note that the Dv2-series and F-series only provide local SSD storage, which is only temporary and is lost when the instance terminates. For persistent storage (aka "Premium Storage") you have to choose the DSv2-series or FS-series, whose base price is the same as the non-S variants but then you have to pay an unspecified extra amount to actually add on the persistent storage. Persistent storage is a must for LL testing on interruptible instances like Amazon's spot instances and Google's preemptible instances, otherwise your worktodo and save files would evaporate when the instance is abruptly terminated, but then again Azure doesn't offer interruptible instances anyway.

There is also a G-series based on the Intel Xeon E5-4600 v4 architecture, but its prices are literally five times higher than the F-series counterpart with the same number of cores.

Azure's instances can run on either Windows or Linux. For all but the (very slow) A-series, the pricing of the Windows instances is about twice as high as the Linux versions. With Google's preemptible instances or Amazon's spot instance, the Windows pricing is typically four or five times higher than the Linux pricing. Windows is not a viable option for LL testing in the cloud.

No interruptible kinds of instances

Edit: Azure is in the process of introducing low-priority VMs, currently only available as part of Azure Batch. When they are generally available, this section will need to be revised.

Both Amazon and Google offer interruptible instances, where you get big price discounts in exchange for being willing to let your instances get interrupted whenever higher-paying customers show up. Amazon's version are called "spot instances" and Google's version are called "preemptible instances".

However Azure does not offer this. Their pricing overview page mentions only pay-as-you-go subscriptions and prepaid subscriptions, which correspond to Amazon's on-demand instances and reserved instances.

With Google preemptible instances the cheapest n1-highcpu-2 instances cost a fixed rate of 2 cents an hour, and with Amazon spot instances the cheapest c4.large instances average roughly the same or a bit cheaper (however spot prices fluctuate, sometimes drastically). But since Azure does not offer interruptible instances the cheapest F1 instances are only available at the pay-as-you-go rate of 5.6 cents an hour.

Conclusion

Perhaps in the future Microsoft might offer cheap interruptible instances as Amazon and Google already do. Until then, Azure isn't a viable option.

Meanwhile, here is a head-to-head comparison of Amazon vs. Google.

Last fiddled with by GP2 on 2017-07-29 at 16:08
GP2 is offline   Reply With Quote
Old 2016-07-14, 02:43   #2
Madpoo
Serpentine Vermin Jar
 
Madpoo's Avatar
 
Jul 2014

37·89 Posts
Default

Good info, very helpful.

Seems that Azure is after the enterprise crowd with their lack of transient instances. I seem to recall that with an MSDN subscription, you get a certain # of Azure hours each month or something like that. Might be a nice bonus for anyone in that situation.

Oh, here we go:
https://azure.microsoft.com/en-us/pr...efits-details/

I might take advantage of that myself since I have the subscription.
Madpoo is offline   Reply With Quote
Old 2016-07-15, 08:23   #3
LaurV
Romulan Interpreter
 
LaurV's Avatar
 
Jun 2011
Thailand

5×1,889 Posts
Default

Outstanding GP2!
Good job!

Last fiddled with by LaurV on 2016-07-15 at 08:26 Reason: frogot my speillng
LaurV is offline   Reply With Quote
Old 2016-08-07, 18:04   #4
GP2
 
GP2's Avatar
 
Sep 2003

3·863 Posts
Default Azure GPU instances now available in preview

Microsoft is previewing their new NC-series instances for Azure (compute-focused GPU instances) using the Tesla K80. The CPU is Xeon E5-2690 v3.

Azure blog announcement

Nvidia blog announcement

Just a reminder though, Azure doesn't have any interruptible instances like Amazon's spot instances or Google's preemptible instances, so the pricing is probably impractical.
GP2 is offline   Reply With Quote
Old 2016-11-18, 11:40   #5
GP2
 
GP2's Avatar
 
Sep 2003

50358 Posts
Default

Microsoft announced general availability of GPU instances starting on December 1. They will offer both Tesla K80 and Tesla M60.

Along with Google's recent announcement, this means that all three big cloud providers will soon offer virtual machine instances with available GPUs.
GP2 is offline   Reply With Quote
Old 2016-11-18, 18:42   #6
henryzz
Just call me Henry
 
henryzz's Avatar
 
"David"
Sep 2007
Cambridge (GMT/BST)

16EE16 Posts
Default

Quote:
Originally Posted by GP2 View Post
Microsoft announced general availability of GPU instances starting on December 1. They will offer both Tesla K80 and Tesla M60.

Along with Google's recent announcement, this means that all three big cloud providers will soon offer virtual machine instances with available GPUs.
Hopefully this will bring the prices down at amazon.
henryzz is online now   Reply With Quote
Old 2017-07-29, 15:59   #7
GP2
 
GP2's Avatar
 
Sep 2003

50358 Posts
Default

Azure will soon offer low-priority VMs, they are in preview. These correspond to AWS spot instances and GCE preemptible VMs.

Low-priority VMs are currently available only within Azure Batch.

The pricing on low-priority VMs is $0.01 /hr for the 1 core F1, $0.02 /hr for the 2 core F2, etc. (for Linux)

However: "The prices for the low-priority VMs are for the preview. Prices may change when the service is generally available."

The one-cent-an-hour price point certainly looks attractive, assuming it doesn't change when the preview ends. However, the F1, F2, ..., F16 virtual machines don't include persistent storage. You would need to get the F1S at the same price as the F1, and purchase Premium Storage separately.

But Premium Managed Disks are $5.28 /month for a P4 (32 GB), which works out to 0.73 cents an hour for a 30-day month. The Standard Managed Disks (HDD rather than SSD) are cheaper, but it's not clear if they're compatible with the FS virtual machines. There doesn't seem to be anything smaller than the 32 GB size, which is way way more than needed for mprime worktodo and configuration files.

There are also Premium Unmanaged Disks, but they don't come in any size smaller than 128 GB, for $19.71 a month.

Azure has a thing called Blob storage, which seems to corresponds to Amazon S3 or Google Cloud Storage. But legacy applications like mprime want to write files to disk. There's also a thing called File Storage, which uses SMB network file shares that can be shared by multiple VMs, so maybe this is a viable option too. More details about Azure storage in this link.

Last fiddled with by GP2 on 2017-07-29 at 16:02
GP2 is offline   Reply With Quote
Old 2017-07-30, 08:19   #8
henryzz
Just call me Henry
 
henryzz's Avatar
 
"David"
Sep 2007
Cambridge (GMT/BST)

587010 Posts
Default

Is there any particular reason why storage can't be based upon a network drive(possibly hosted at home)? Speeds would be based upon internet connection speed and latency wouldn't be great but prime95 doesn't need much I/O.
henryzz is online now   Reply With Quote
Old 2017-08-24, 01:27   #9
storm5510
Random Account
 
storm5510's Avatar
 
Aug 2009

2×971 Posts
Default

Quote:
Originally Posted by Madpoo View Post
Good info, very helpful.

Seems that Azure is after the enterprise crowd with their lack of transient instances. I seem to recall that with an MSDN subscription, you get a certain # of Azure hours each month or something like that. Might be a nice bonus for anyone in that situation.

Oh, here we go:
https://azure.microsoft.com/en-us/pr...efits-details/

I might take advantage of that myself since I have the subscription.
Simply out of curiosity, how difficult is it to set one of these up and use it?
storm5510 is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Microsoft CryptGenRandom R.D. Silverman Programming 14 2012-03-04 22:23
Microsoft ripoff Prime95 Soap Box 14 2009-01-24 10:45
Ghag!... Microsoft... is... killing... me! E_tron Soap Box 10 2006-12-28 12:00
Microsoft xna moo Hardware 0 2006-12-12 01:07
Microsoft Plus XP clowns789 Software 2 2004-08-04 14:45

All times are UTC. The time now is 07:45.

Tue May 11 07:45:14 UTC 2021 up 33 days, 2:26, 1 user, load averages: 2.07, 1.84, 1.60

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.