Unfavorable price trends for AWS EC2 spot instances?

[GP2]

In conjunction with their annual re:Invent conference held in late November, Amazon recently announced a bunch of changes in the way spot instances work. There are more options for interruption behavior, spot price fluctuations have been greatly smoothed, and it is easier to launch spot instances from the command line.

This seems to be causing spot instance prices to increase steadily.

First I'll describe the changes, then explain why they are affecting prices.

Spot prices now fluctuate a lot less. It used to be that they could move up and down at any time, but now they rise and fall only very gradually, based on long-term trends rather than instantaneous demand.

Also, it used to be that when your spot instances were interrupted (usually due to the spot market price moving above your limit price), the instance would always be terminated. A couple of months before the conference, they added the option that spot instances could also be stopped. During the conference itself, they announced that spot instances could also hibernate.

Reminder of what these terms mean:

• Terminated: all your state is lost, however the contents of your "local hard drive" (EBS storage) can optionally be saved, and obviously anything stored on a network file system (EFS) is unaffected. This is like your laptop or desktop computer disappearing, and nothing is left except (optionally) a USB thumb drive, which could be plugged into a new computer so you can copy your files.
• Stopped: Your "local hard drive" (EBS storage) persists, with the same root directory tree, but the contents of RAM are lost. This is like your laptop or desktop computer being powered off.
• Hibernating: The contents of your "local hard drive" (EBS storage) and your RAM both persist. This is like your laptop or desktop computer going into sleep mode, or the way most smartphones work when you turn them off.

A minor disadvantage of stopped and hibernating instances is that they continue to run up storage charges. Each instance uses a minimum of 8 GB "hard drive size", and on a monthly basis this costs about 80 cents, which is not too onerous, but does add about 0.1 cents per hour to the spot price.


All of this makes spot instances friendlier and easier to use, with costs that are much more predictable. It is a lot easier now to adapt existing applications which have been running on on-demand or reserved instances and have them run on spot instances instead.

This may be very bad for number crunching on the cloud however. The difference between on-demand pricing and spot prices has traditionally been nearly an order of magnitude, and that gap will probably narrow now, as spot instances see greater usage by corporations that have been paying full price until now. Also, you can't set a low limit price to let mprime run intermittently during price dips, because there are no more price dips anymore. It's just a flatline with just a very tiny slope (so far, only upward).

Ohio (us-east-2) consistently has the cheapest spot prices. In the last ten days or so since the re:Invent announcements and change to the new smoother pricing model, the hourly cost for a one-core c4.large instances has risen from about 1.6 cents to about 1.8 cents, and for the two-core c4.xlarge it's steadily risen from about 2.5 cents to about 3.2 cents (or about 1.25 cents to 1.6 cents per core). This slow steady rise shows no sign of stopping. (Note that the larger multicore instances are not cost effective at all, since the spot prices usually increase at a higher-than-linear rate and performance scales at a less-than-linear rate.)

Of course, we are in the post-Black-Friday end-of-year holiday shopping season, where a lot of companies including Amazon itself are ramping up their use of computing resources, and there is also an end-of-quarter effect which has been seen before in late June and late September. So the increases could be attribuable to that. Maybe.

We might have a better idea what's happening when we see what prices look like in mid-January. However I'm pessimistic and will start scaling back as existing assignments complete.

[chalsall]

Originally Posted by GP2

We might have a better idea what's happening when we see what prices look like in mid-January. However I'm pessimistic and will start scaling back as existing assignments complete.

I'm not terribly surprised by this.

Anyone who used Google and Microsoft's clouds in addition to Amazon's were a little surprised by how inexpensive Amazon's spot prices were.

[Dubslow]

Originally Posted by chalsall

I'm not terribly surprised by this.

Anyone who used Google and Microsoft's clouds in addition to Amazon's were a little surprised by how inexpensive Amazon's spot prices were.

The other possible reading of it is "no one was successfully competing with Amazon, so they had no incentive to not increase their own margins at no external penalty (or internal cost)".

[chalsall]

Originally Posted by Dubslow

The other possible reading of it is "no one was successfully competing with Amazon, so they had no incentive to not increase their own margins at no external penalty (or internal cost)".

I absolutely agree. Bezos isn't known to leave much money on the table.

[GP2]

Originally Posted by chalsall

I absolutely agree. Bezos isn't known to leave much money on the table.

I actually disagree with this. Amazon has been notorious for plowing profits back into growth and expanding services, and so far their shareholders have been extremely patient with them.

The latest moves will probably cut costs considerably for the overwhelming majority of business customers, who'll be able to make greater use of spot instances instead of the far more expensive on-demand and reserved instances. Those are the folks whose spend is measured with six figures, not three. Numbercrunching hobbyists are surely a truly minuscule and insignificant market.

Google Compute Engine at 1.5 cents an hour per core (a fixed and non-fluctuating price) will always be an alternative. Unfortunately, the hardware Google uses is significantly slower than Amazon's, so the true apples-to-apples cost equivalence point on AWS is higher than that, at least 1.8 or 1.9 cents I think.

[chalsall]

Originally Posted by GP2

Unfortunately, the hardware Google uses is significantly slower than Amazon's, so the true apples-to-apples cost equivalence point on AWS is higher than that, at least 1.8 or 1.9 cents I think.

For GPUs, Google and M$ Azure all offer the same kit as Amazon. Tesla K80s and P100s.

Having run all of them, I actually find that Google and Azure are pretty good. Before this price change Amazon spot instances were still the best bang for the buck, but perhaps that has now changed.

[GP2]

Originally Posted by chalsall

For GPUs, Google and M$ Azure all offer the same kit as Amazon. Tesla K80s and P100s.

Having run all of them, I actually find that Google and Azure are pretty good. Before this price change Amazon spot instances were still the best bang for the buck, but perhaps that has now changed.

You can't attach GPUs to a preemptible virtual machine (equivalent of spot instance) on Google Compute Engine. You can on AWS EC2, but it's just not cost effective, the demand for machine learning keeps the prices too high. As for Azure, I'm not even sure if they offer preemptible/spot instances yet.

If AWS c4 instances stay at 1.6 to 1.7 cents per hour per core in us-east-2, Amazon will still have the best bang for the buck, but in the last 10 days ever since the recent changes, the spot price has been climbing steadily and I'm worried about the coming days and weeks.

[Mark Rose]

Originally Posted by GP2

I actually disagree with this. Amazon has been notorious for plowing profits back into growth and expanding services, and so far their shareholders have been extremely patient with them.

Amazon makes a good chunk of their profit off AWS: https://www.forbes.com/sites/greatsp.../#1f86be564745

The latest moves will probably cut costs considerably for the overwhelming majority of business customers, who'll be able to make greater use of spot instances instead of the far more expensive on-demand and reserved instances. Those are the folks whose spend is measured with six figures, not three. Numbercrunching hobbyists are surely a truly minuscule and insignificant market.

If the spot prices are really so smooth now, I'll take another look at them.

[GP2]

Originally Posted by Mark Rose

If the spot prices are really so smooth now, I'll take another look at them.

Why is it necessary for them to be smooth? (and yes they are now, very). That just means there are no price dips to take advantage of.

LL testing is very tolerant of interruptions and downtime. It uses very little memory and the savefiles are small. I would set low limit prices, and sometimes a single LL test would end up running on dozens of different physical machines, maybe with only a few minutes or a few hours worth of iterations on each one, between each spot relaunch and termination.

You might think that's living dangerously, since just one bad machine will spoil the end result, but AWS servers use ECC memory and Amazon retires them as soon as they detect that the hardware is acting up, and in any case I only do strategic double checks, and all of the residues have been verified good so far (either right away, or on the triple check where necessary).

Maybe it's time to look at AWS Batch, although the underlying infrastructure there is spot instances (and containers), so there's probably no cost savings to be had, for the time being anyway.

[retina]

With the nature of long term LL testing over many years I'm not so convinced that using "cloud" servers gives any cost advantage. For one-off highly compute intensive jobs I can see the advantage of not having to purchase dedicated hardware for a job of only a few weeks/months. But for something that runs continuously with no end (or at least until the prime exponents run out) buying your own machine(s) just feels like the better way IMO. There are fewer uncertainties when having your own: 24/7 always, known costs, private data, doesn't need always-on Internet, real CPU (not virtualised)

Summary:
Long-term: BYO
Short-term: Cloud (if privacy is not an issue)

[Dubslow]

Originally Posted by retina

With the nature of long term LL testing over many years I'm not so convinced that using "cloud" servers gives any cost advantage.

I believe the main factors that can make a cloud cheaper are 1) cheaper power at the datacenter's physical location (compared to the user), and 2) economies of scale in cooling. Either is certainly capable of dropping the recurring cost below that of home-operated computers, nevermind the economies of scale in capital investment (or time spent on maintenance and upkeep of home computers). Rest assured that the people here who make extensive and continued use of GIMPS-on-the-cloud have performed the price analysis before so quickly jumping on the bandwagon (and often it's a supplement to their home power, not a replacement).