How-to guide for running LL tests on the Amazon EC2 cloud

[chalsall]

Originally Posted by GP2

Note that anything bigger, such as the four-core c4.2xlarge all the way up to the 18-core (not 16-core) c4.8xlarge is rarely worthwhile, because the price of an N-core instance for larger N will usually be a lot more than N times the cost of a one-core c4.large instance, and the throughput of an N-core instance running mprime will usually be significantly less than the total throughput of N one-core c4.large instances.

It depends on what you are trying to do. Not everyone here is using AWS / EC2 only for mprime.

[GP2]

Originally Posted by chalsall

It depends on what you are trying to do. Not everyone here is using AWS / EC2 only for mprime.

The User Data script that is included with this guide (as an attachment to one of the posts) lets you run CUDALucas or mfaktc on the GPU of a p2.xlarge instance, while simultaneously running mprime on the CPU. But in general using GPUs on the AWS cloud is not at all cost-effective because the great demand for GPUs in machine learning keeps the spot prices high for instances with GPUs.

I'm curious what other CPU-based programs people are running on AWS and if they adapted this framework (EFS + the User Data script) or use something else.

By the way, the User Data script is now at version 1.11, with various bug fixes, and you can now reboot the machine and mprime will resume automatically. Normally a User Data script only runs at instance launch time, but now the script puts itself into crontab so that it will run at reboot time as well. Anyone using an older version should upgrade.

[chalsall]

Originally Posted by GP2

Anyone using an older version should upgrade.

Extreme coolness.

That's sincere.

[ET_]

Originally Posted by GP2

I'm curious what other CPU-based programs people are running on AWS and if they adapted this framework (EFS + the User Data script) or use something else.

You would be amazed!
I am running it with llr, did it with primegaps, Fermat searcching programs and with my sieve. That's because a top-level FMA3-aware server or desktop would still cost more to me than the AWS system.

Originally Posted by GP2

By the way, the User Data script is now at version 1.11, with various bug fixes, and you can now reboot the machine and mprime will resume automatically. Normally a User Data script only runs at instance launch time, but now the script puts itself into crontab so that it will run at reboot time as well. Anyone using an older version should upgrade.

Cool! Thanks! bow:

[GP2]

There was a bug introduced in v 1.10 and v 1.11 that prevented CUDALucas/mfaktc from starting on p2.xlarge instances (two shell variables were needlessly and incorrectly made "readonly"). This didn't have any effect on mprime.

Version 1.12 is now attached to this earlier post.

[amaurymartiny]

Thanks GP2 for this excellent tutorial, and totally up to date.

I ran the tutorial against a c4.8xlarge, and after some time, the CPU utilization is 50 percent in the monitoring tab in AWS console, as shown in the picture:

http://i.imgur.com/Yykn0sZ.png

If I do a top in the instance, I see a CPU usage of 1800%, which makes sense with the fact that there are 18 cores. I just want to make sure that seeing 50% in AWS console is normal.

Thanks.

[GP2]

Originally Posted by amaurymartiny

I ran the tutorial against a c4.8xlarge, and after some time, the CPU utilization is 50 percent in the monitoring tab in AWS console, as shown in the picture:

http://i.imgur.com/Yykn0sZ.png

If I do a top in the instance, I see a CPU usage of 1800%, which makes sense with the fact that there are 18 cores. I just want to make sure that seeing 50% in AWS console is normal.

I don't know if it's "normal", but I do get the same thing. I'm running c4.large and c4.xlarge instances, and I get %CPU = 100% and 200% respectively in top, so I think that's what really matters. I'm not sure what that Monitoring tab is measuring.

If you're using an instance with lots of cores like c4.8xlarge, it's probably best to run version 29.2 of mprime, because it uses the hwloc library to automatically figure out the topology. The tutorial still mentions version 28.10 because that's what the official download page still uses.

In terms of pricing, us-east-2 (Ohio) has the lowest spot prices, and the spot prices for the 18-core c4.8xlarge are consistently well above 18 times the cost of a 1-core c4.large, by about maybe 50%. The 8-core c4.4xlarge and 4-core c4.2xlarge also have a premium, though less pronounced, so I stick to the c4.large and c4.xlarge for my own usage.

[GP2]

A small change to the script (now version 1.13), to take into account the fact that the new feature in AWS that spot instances can now be stopped.

You can choose the behavior at launch time (stopping or termination). See [the documentation](http://docs.aws.amazon.com/AWSEC2/la...le-differences) for an explanation of the difference. Resuming a stopped instance vs. relaunching a new instance to replace a terminated instance is like the difference between rebooting a computer and buying a new computer to replace an old computer.

It still makes sense for "Interruption behavior" to be "Terminated" rather than "Stopped", since this avoids incurring unnecessary EBS storage charges (at 8 GB for the root volume of each instance) while instances aren't running. However, now the script should now still work even if spot instances are stopped rather than terminated.

[GP2]

The user-data script (linked in this post) is now version 1.15

The EFS filesystem is now mounted with dirsync. Previously it was possible for instances to end up pointing to the wrong work directory due to race conditions.

[Cheetahgod]

I followed your steps but I can't seem to get this working with the lastest version of prime95. It worked with the version you used in your examples. I want to use AVX512 of the c5 intstances.

[Uncwilly]

Originally Posted by Cheetahgod

I followed your steps but I can't seem to get this working with the lastest version of prime95.

Make sure you are using mprime not Prime95.