How-to guide for running LL tests on Google Compute Engine cloud

[GP2]

How to search for Mersenne primes using the Google Compute Engine cloud platform

Introduction

The GIMPS project (at http://mersenne.org ) aims to find very large prime numbers; the primes it finds are usually record-breaking. The prime numbers it searches for are of a particular kind known as Mersenne primes.

A program called Prime95 runs on Windows computers and performs a so-called Lucas-Lehmer test ("LL test") on candidates known as Mersenne numbers to see if they are Mersenne primes — the overwhelming majority are not, in fact the odds are maybe one in several hundred thousand. The Linux version of Prime95 is called mprime; it's the same program, but with a command-line interface rather than a GUI. The Prime95/mprime program can be downloaded from http://www.mersenne.org/download/

Some people run Prime95/mprime in the background on their desktop computer or laptop. It does its calculations and communicates the results to a server on the Internet known as PrimeNet. However it uses a computer's computing capabilities rather fully: your electricity bill will go up, your computer fan might start running faster and louder, and the extra heat generated might heat up your room a bit or make your air conditioner work a bit harder.

An alternative to running Prime95/mprime on your own computer is to run it on a server. Some companies have a large array of server computers (aka "the cloud"), and some run a "public cloud" business that allows anyone to log in remotely over the Internet and use their cloud for various purposes (for a fee, of course). Examples of such cloud platforms include Amazon EC2, Google Compute Engine (GCE) and Microsoft Azure.

This thread explains how to use Google Compute Engine to do Lucas-Lehmer testing in the cloud. There is another thread that explains how to use Amazon EC2 for the same purpose.

Costs

Costs can fluctuate and available offers and hardware change over time. If that information was included here it would quickly go out of date. Instead see [this thread... to be added later] for an up-to-date discussion of cost comparisons and price competitiveness among the rival clouds.

Nevertheless, as of June 2017, Amazon EC2 runs mprime version 29.1 considerably faster than GCE on comparable servers (i.e., with the same number of cores), by 25% or more. It might still be more economical to run on GCE whenever GCE's fixed prices are considerably lower than EC2's fluctuating (spot market) prices, but as of June 2017, the cheapest spot market prices among the EC2 regions have been consistently lower than GCE's price over a period of several months.

PrimeNet account

You can optionally create a PrimeNet account at http://www.mersenne.org/gettingstarted/

You can contribute to the GIMPS project anonymously if you prefer, but having a PrimeNet account lets you accumulate "GigaHertz-days" credits and keep track of where you rank on a leaderboard compared to other project participants; you can also keep track of your results and your pending work more conveniently.


Next section: Using Google Compute Engine for Lucas-Lehmer testing

[GP2]

Regions and Zones

Google Compute Engine (GCE) offers servers that are located in various different regions around the world, and within each region there are various "zones". The Regions and Zones page gives some information about them.

As of June 2017, there are GCE regions in the United States, Europe, and Asia.

However, it makes no difference what part of the world you yourself live in, you can use servers in GCE regions around the world without restriction.

Within each region, there are different "zones", which have a letter designation. For instance, the us-central1 region has zones named us-central1-a, us-central1-b, us-central1-c, us-central1-f.

Prices

For the type of work we will be doing, we will incur hourly charges and use so-called "preemptible virtual machines" (preemptible VMs).

Prices can vary by region.

Price information can be found on the Google Compute Engine Pricing page; you can use the drop-down menu to view prices in the different regions: Iowa, Northern Virginia, Oregon, South Carolina, and the various regions in Europe and Asia. Look at the section titled High-CPU machine types and look at the column labeled Preemptible price, and in particular, the line for "n1-highcpu-2".

If you are familiar with Amazon EC2 spot prices, note that there are a few differences: on GCE the preemptible prices are fixed and remain stable for the long term, unless Google makes a manual price adjustment. Prices vary by region, but prices are the same for every zone within a region. Also Google bills by the minute rather than rounding up time used to the nearest whole hour.

As of June 2017, Oregon (us-west1), Iowa (us-central1), and South Carolina (us-east1) have the (same) lowest price; Northern Virginia is a bit more expensive. Europe and Asia are consistently more expensive than the US regions, by 10% or more, so we will only use US regions for our purposes. However prices and availability can change over time, so check the pricing page for yourself from time to time.

Different micro-architectures

Different zones, even within the same region, can offer different server types with different micro-architectures: Sandy Bridge, Ivy Bridge, Haswell, Broadwell, and Skylake (the latter with AVX-512). Newer micro-architectures offer advanced machine instructions that can run certain types of computing jobs (like LL tests) considerably faster than older micro-architectures can, so it is important to select your zones and micro-architectures with care. Note: Google does not keep this information secret, but they don't exactly present it in an obvious and transparent way on their page for launching VM Instances. Detailed instructions will be given in the later sections of this guide.

As of June 2017, GCE charges exactly the same for servers running different micro-architectures, despite their varying capabilities. Within a couple of months, however, they intend to start charging a premium of 6.07% to run highcpu servers on Skylake.

Prerequisites

GCE offers server computers with either Windows or Linux pre-installed. However, running Windows incurs an hourly surcharge that will make it more than three-and-a-half times as expensive as Linux for our purposes. Therefore we will run Linux on our GCE servers.

Some basic familiarity with the Linux command-line interface (the "shell" known as "bash") will be very helpful. However, you don't need any familiarity with SSH software, because the GCE console window lets you run SSH sessions directly in a browser window.

Various Linux distributions are available, including Debian, CentOS, CoreOS and Ubuntu, and various different versions of each are often available (for instance Ubuntu 14.04, 16.04 and 17.04). Red Hat Enterprise Linux and SUSE Linux Enterprise Server are also available, but they incur hourly surcharges which makes SUSE more than twice as expensive as the free Linux distributions for our purposes, and Red Hat about five times more expensive.

However, your own computer (the one that you will use to log into GCE) does not have to run Linux, it can run Windows or MacOS or anything else.

Your Google Cloud account

You can access your Google Cloud account (or sign up for one) at https://cloud.google.com/ . You can log in with the same username and password you use for Gmail or other Google services online.

You will need to provide a payment method (e.g. credit card) when you sign up for Google Cloud. However, as of June 2017 and for some time now, Google is offering $300 worth of credit as a free-trial promotion.


Next section: to be announced

[stb37]

Any chance you could give details of how once a linux server has been setup, how to get prime95 running on it? Struggling with this part.


Thanks

[lukerichards]

In my experience, initiating a Google VM instance with vCPU cores will only result in mprime allowing me to run workers.

Is this because of the way that Google advertises vCPU cores that are actually hyperthreads?

Can I utilise cores effectively without any loss in throughput, or is it simply that a 10 core VM in Google only actually gives me access to 5 physical cores?

[ATH]

Yes I think so, at least on Amazon EC2: n vCPU is with hyperthreading, so it is n/2 real cores. Usually on EC2 it is faster not using the hyperthreading (not sure on Google), so in local.txt:
CoresPerTest=<n/2> (number of real cores) assuming you are only running WorkerThreads=1.