Amazon EC2

[patrik]

I've been trying out the Amazon EC2 linux sites the last few days after noticing chalsall's post last week. (The noticing took place last week.) I've attached an image of the benchmarks I got.

I also ran some real iterations of double checks and first time tests. (I.e. only double checks with FFT size 1728K running, or only first time tests (possibly of a few different sizes, since I got them at different times) running.) Lines with those timings are given just below the well-known lines.

I ran the first tests on their site in Europe, and then I moved to Oregon, since that is slightly cheaper (at least right now).

The names follow the same naming convention as Amazon do, or an abbreviation thereof. I have appended "-oreg" to the computer names of the tests in Oregon.

The benchmark of t1.micro is somewhat misleading, since you only get those times for a short time. For a longer time you may get a sixth of that throughput.

I did not notice any slowing down of the first time tests when running first time tests on all cores versus running first time tests on most cores but double checks on the rest.

Spot instances of "Cluster Compute Eight Extra Large" (cc2.8xlarge) in Oregon seems to give the best bang for the buck at present.

Edit: I meant to post this in the Hardware forum (not in the GPU subforum). Can someone with mod rights move it, please?

[chalsall]

Originally Posted by patrik

The benchmark of t1.micro is somewhat misleading, since you only get those times for a short time. For a longer time you may get a sixth of that throughput.

I did not notice any slowing down of the first time tests when running first time tests on all cores versus running first time tests on most cores but double checks on the rest.

Interesting analysis. Thanks for sharing.

It is known that the t1.micro instances are not really intended for CPU bound jobs -- they can be used for such, but they're slow. They are best for things like low-load web serving, DNS serving, and file transfers into a ECS volume for processing under a larger instance later. (I use them for the latter two applications.)

Originally Posted by patrik

Spot instances of "Cluster Compute Eight Extra Large" (cc2.8xlarge) in Oregon seems to give the best bang for the buck at present.

It is also known that the larger VM instances give you better performance, since you tend to get most of the "real" machine.

Also, for this type of work, "Spot" instances are definitely the way to go. For one of my projects (computer vision), it is almost cheaper to rent a EC2 virtual machine than to pay the electricity to run an equivalent machine here in Bimshire. Factor in (no pun intended) the fact that there's no capital expenditure, and I can spin them up only when I need them (and as many as I need), it's a big win for me.

[Manpowre]

This is very interesting, I also see EC2 has GPU cloud with 2x Tesla boards.. any test on this with MfaktC ?

[Karl M Johnson]

No point, mfaktc does not use dp fp calculations.

[Manpowre]

Originally Posted by Karl M Johnson

No point, mfaktc does not use dp fp calculations.

Ahh, Only CudaLucas that does that.. gotcha.

[TheJudger]

Originally Posted by Karl M Johnson

No point, mfaktc does not use dp fp calculations.

At least no in performance relevant sections. Some screen outputs are calculated as double and sieve initialization and worktodo parsing (checking limits) use very few DP instructions. But again: not performance relevant!

Oliver

[patrik]

As a follow-up of this old thread, I just want to say that I have just benchmarked the new servers Amazon provide, especially their c3.8xlarge machine. Compared to the old cc2.8xlarge, iteration times are about 13% lower for the new c3.8xlarge, while being just under 1.2% more expensive per hour with the spot instance prices at their lowest level. (This is while testing 16 exponents with 3840K FFT.)

However, it is important to select the correct virtualization type: hvm. You select that when you choose the so called AMI (Amazon Machine Image), containing OS etc. for your instance. Otherwise the iteration times get much worse.

File	Setup					Iterations	Avg. time (ms)
==============================================================================
c3.8xlarge:
bench1	16 workers, 32 threads, paravirtual	 1180000	49.6707
bench2	16 workers, 32 threads, paravirtual	  130000	49.0276
bench3	16 workers, 32 threads, hvm		 3160000	31.7276
bench4	16 workers, 16 threads, hvm		34140000	30.8819
bench5	16 workers, 16 threads, hvm		51190000	30.895

Compare old cluster cc2.8xlarge:
bench1	16 workers, 32 threads, hvm		  480000	37.5672
bench2	16 workers, 16 threads, hvm		  810000	35.5468

[ET_]

Originally Posted by patrik

As a follow-up of this old thread, I just want to say that I have just benchmarked the new servers Amazon provide, especially their c3.8xlarge machine. Compared to the old cc2.8xlarge, iteration times are about 13% lower for the new c3.8xlarge, while being just under 1.2% more expensive per hour with the spot instance prices at their lowest level. (This is while testing 16 exponents with 3840K FFT.)

However, it is important to select the correct virtualization type: hvm. You select that when you choose the so called AMI (Amazon Machine Image), containing OS etc. for your instance. Otherwise the iteration times get much worse.

File	Setup					Iterations	Avg. time (ms)
==============================================================================
c3.8xlarge:
bench1	16 workers, 32 threads, paravirtual	 1180000	49.6707
bench2	16 workers, 32 threads, paravirtual	  130000	49.0276
bench3	16 workers, 32 threads, hvm		 3160000	31.7276
bench4	16 workers, 16 threads, hvm		34140000	30.8819
bench5	16 workers, 16 threads, hvm		51190000	30.895

Compare old cluster cc2.8xlarge:
bench1	16 workers, 32 threads, hvm		  480000	37.5672
bench2	16 workers, 16 threads, hvm		  810000	35.5468

Are there price differences between hvm(AMI) and paravirtual?

Luigi
---

[patrik]

As I understand there are free AMI's for either paravirtual or HVM, and you pay the time you have the instance (and for storage and I/O). However, I can't access the details of my bill yet.

I'm actually not sure that it is the virtualization that matters, but that is the only difference in the description that I can find between the two images I've tried.

[ET_]

Originally Posted by patrik

As I understand there are free AMI's for either paravirtual or HVM, and you pay the time you have the instance (and for storage and I/O). However, I can't access the details of my bill yet.

I'm actually not sure that it is the virtualization that matters, but that is the only difference in the description that I can find between the two images I've tried.

I see. Thanks!

luihgi

[Mark Rose]

Originally Posted by ET_

Are there price differences between hvm(AMI) and paravirtual?

Luigi
---

Not directly. The newer generation HVM hardware (m3/c3/r3/i2/g2) is about 40% cheaper than the older PVM hardware. The m3 instances can also run PVM, but I would always run HVM if possible as there's way less jitter with disk and network IO.

On-demand pricing is cheaper than buying if you only need it for a short time. If you are running hardware for more than a few months per year, you're way better off buying a reservation than paying on-demand pricing (can be up to 70% cheaper). Reserved pricing is competitive with running your own hardware if you have to pay someone to maintain that hardware, network, etc.

Amazon already runs mprime/prime95.

If you're looking to rent hardware for crunching primes Digital Ocean is a far better deal. For $5/month, you'll get a VPS that will give performance of about half a c3.large that would cost you $75/month on-demand.

AMIs are usually free, but they will charge more per hour if you run Windows or other paid AMIs.