mersenneforum.org How-to guide for running LL tests on the Amazon EC2 cloud
 Register FAQ Search Today's Posts Mark Forums Read

 2016-07-15, 06:28 #12 GP2     Sep 2003 258510 Posts Setting up an EFS filesystem: terminating the instance you created After you finish creating all the prime-init.txt and local-init.txt files in each of the subdirectories in the previous step, you can stay logged in to observe what happens when you launch the instances that will do the actual LL testing. For instance you can run the following commands: Code: cd /mnt-efs/mprime/instances ls -lrt * (or instead of "mnt-efs", choose the same name you chose in the previous set of commands) You should see only the subdirectories and files you created as part of the setup, namely: c4.large c4.xlarge c4.2xlarge c4.4xlarge c4.8xlarge and their associated local-init.txt and prime-init.txt files, however this will change when you launch some new instances to start running mprime. If you logged in to an existing instance in the previous sections in order to do the setup and configuration, then you can just keep it running for its initial purpose. However, if you launched a new instance solely for the purpose of carrying out the initial setup and configuration of the EFS file system, then do remember to terminate it eventually, otherwise it will keep running up a bill even if it is sitting idle. Merely logging out of the ssh client sesssion will not terminate the instance. To terminate the instance: Go to the EC2 console at http://console.aws.amazon.com/ec2/, then click on the "Instances" link in the left-hand-side menu. Make sure you are in the same AWS region where you launched the instance in the previous steps, and change it if necessary. The region name is indicated at the top right part of the page. Click on the instance you want to terminate with the left mouse button to select it, then click on it again with the right mouse button to bring up a popup menu. In the popup menu, select "Instance State" and "Terminate". In the resulting confirmation window, click on the blue "Yes, Terminate" button. Next section: Using "spot instances" to run mprime Last fiddled with by GP2 on 2016-07-27 at 07:03
2016-07-15, 06:44   #13
GP2

Sep 2003

258510 Posts
Using "spot instances" to run mprime

At last we are done with the one-time setup and configuration, and we're ready to actually start number crunching some LL tests. We will do so using Amazon's "spot instances".

Some applications require 24/7 availability: for example, websites should never be down. However, an application like mprime/Prime95 does not have strict deadlines or uptime requirements, and can tolerate being interrupted.

Cloud platforms like Amazon EC2 and Google Compute Engine (but not Microsoft Azure) offer big discounts to customer who are willing to let their cloud applications get interrupted whenever a higher-paying customer shows up. Amazon's version of interruptible instances are called "spot instances". We will now launch one.

Go to the EC2 console at http://console.aws.amazon.com/ec2/, then click on the "Spot Requests" link in the left-hand-side menu.

Make sure you are in the same AWS region where you created the EFS filesystem in the previous steps, and change it if necessary. The region name is indicated at the top right part of the page.

You might see a "Pricing History" button at the top (if not, don't worry). Make note of it, it can be useful in the future. Its use is described below.

Launching a spot instance to run mprime

Click on the blue "Request Spot Instances" button (or the blue "Get started" button)

In the next page, set the following:

Request type: Request and Maintain

Target capacity: 1 instances

AMI: Amazon Linux AMI

(Advanced note: unfortunately you can't choose an "instance store" AMI
instead of an EBS-backed AMI in order to save EBS storage charges,
because the former isn't compatible with c4 instances)

Instance type(s): It should say "c4.large". Note: if you see "c3.large" then click on the black "x" circle to get rid of it, then click on the "Select" button and choose "c4.large". Note: NEVER run instances in the "c3" family they are roughly the same cost as the more modern "c4" family but considerably slower.

Allocation strategy: Lowest price

Network: Keep this at the default, it will say something like "vpc-........ (default)"

Availability zone: No preference (launch in cheapest Availability

Maximum price: Set your max price (per instance/hour)

When "Maximum price" is set to "Set your max price", a "Pricing History" button appears next to it. If you click on it, a window pops up, and you can change the header settings:

Code:
Product: Linux/UNIX   Instance type: c4.large   Date range: 1 day   Availability zone: All Zones
By looking at the graphs in the Spot Instance Pricing History window, you can get a feel for what the cost trends are. In this window, change the Instance type in the header settings if necessary to "c4.large" (not "c3.large" or any of the others).

Note that spot instance prices fluctuate according to market forces, and different regions can have quite different prices, however prices should average roughly 2 cents (per hour) or so for instance type c4.large in a region located in the US. Make a mental note of what level you want to set your limit price at; if the spot price graph rises above that level, your spot instances will terminate. New spot instances will launch automatically when the price drops back down, and they will resume the work left behind by the old ones.

Click the blue "Close" button to close the Spot Instance Pricing History window.

Now that you have formed some idea of what you want your limit price to be, enter it into the "\$" field next to the "Set your max price" selection. For instance, enter 0.0175 if you want to pay no more than 1.75 cents per hour for your spot instance.

Click on the blue "Next" button

In the next page,

Instance store: ignore this, we don't need it.

EBS volumes: ignore this, we are using the EFS filesystem now instead.

EBS-optimized: ignore this

Monitoring enabled: ignore this, note that CloudWatch detailed monitoring would incur additional charges

Health check: select the checkbox for "Replace unhealthy instances"

Tenancy: keep this at "Default - run a shared hardware instance". Note: selecting "Dedicated" instead would incur large minimum charges.

Interruption behavior: keep this at "Terminate". If you set it to "Stop", it should still work, but if your AWS account is more than one year old, you will unnecessarily incur storage charges for the 8 GB EBS root volumes for each instance, even while your instances are not running.

User data: skip over this for the moment, we'll get back to it shortly

Instance Tags: you probably don't need these, but define some if you want.

Key pair name: choose the key pair name that was mentioned (or created) in the "Make sure that you have a key pair for ssh logins" section earlier

IAM instance profile: choose the IAM role (IAM instance role) that was mentioned (or created) in the "Make sure your IAM instance role exists and it has the right permissions" section, in other words mprime-instance-role or whatever you named it.

IAM fleet role: keep the "aws-ec2-spot-fleet-tagging-role"

Security groups: select "default"

Auto-assign IPv4 Public IP: keep this at "Use subnet setting (Enable)"

Request valid from: keep this at "Now"
Request valid until: you can keep this at "1 year from now"

Also, although it is fairly unimportant, you should unselect the "Terminate instances at expiration" box.

Now go back to the "User data" field.

Select the "As file" option. A "Choose File" button will appear, but before we can choose a file we have to create it first.

Download the "user_data_mprime" file that is an attachment to this forum post. Its exact name will vary depending on the version, for instance "user_data_mprime_1.10.txt".

That is the file you will use, but before using it, you must make one change.

In the Setting up an EFS filesystem: create the filesystem section, you made note of the File System ID of the newly-created EFS filesystem, which is of the form fs-xxxxxxxx (where each "x" is a hexadecimal digit). The exact value will be unique to you, and will be different for each AWS region where you created an EFS filesystem.

For each AWS region where you created an EFS filesystem, there should be a pair of lines of the form:

Code:
region-xxxx-x)
readonly FILE_SYSTEM_ID=fs-xxxxxxxx;;
Change the "region-xxxx-x" to the AWS region where you created the EFS filesystem (for instance, "us-east-1" for N. Virginia, "us-east-2" for Ohio, "us-west-2" for Oregon, "eu-west-1" for Ireland, etc.)

Change the "fs-xxxxxxxx" to the correct value for the EFS filesystem, as described above; this value will be different for each AWS region, and it will be unique to you and no other user.

If you don't use a particular region, and haven't created an EFS filesystem in that region, then just delete the corresponding lines.

After you downloaded the "user_data_mprime" attached file and made the above edits to set the correct FILE_SYSTEM_ID values, click on the "Choose File" button and select that file.

Now that you've filled in the User Data field and everything else, click on the blue "Review" button.

Check the "Max price" is what you wanted it to be. Make sure that "Key pair name" and "IAM instance profile" are what you wanted them to be.

Click on the blue "Launch" button.

Next section: Monitoring the progress of the spot instance(s)
Attached Files
 user_data_mprime_1.16.txt (15.8 KB, 526 views)

Last fiddled with by GP2 on 2018-04-04 at 07:07 Reason: updating user data script (runs itself at reboot time too, not just instance launch time; take into account that spot instances now stoppable; dirsync)

 2016-07-15, 07:06 #14 GP2     Sep 2003 A1916 Posts Monitoring the progress of the spot instance(s) EC2 console Go to the EC2 console at http://console.aws.amazon.com/ec2/, then click on the "Spot Requests" link in the left-hand-side menu. Make sure you are in the AWS region where you launched the spot instance in the previous step. The region name is indicated at the top right part of the page. Select "Request type: fleet" and "State: all". You should see a line for your newly-created spot request. Click on it. As long as the current spot price in at least one of the "availability zones" in the current AWS region was lower than the limit price you specified when you created your spot request, an instance will launch within a minute or two. In the bottom half of the page, click on the "Instances" tab. When your instance launches, it will be visible there. It will have an instance ID of the form i-................. (where each "." is a hexadecimal digit, and there are 17 hexadecimal digits). Now click on the "Instances" link in the left-hand-side menu. You will see the newly launched instance, and the "Instance State" will be "running", with a green ball next to it. The "Instance Type" will be c4.large (assuming that's what you selected when you made the spot request), and the "Availability Zone" name will be the same as the region, but with an extra letter added (such as "a", "b", etc.) Click on the "Spot Requests" link in the left-hand-side menu. You can click on the "Pricing History" button again. You can change the "Availability Zone" setting to show only one graph, the one for the availability zone where your instance actually launched. This graph will tell you the actual hourly rate that you'll be paying in each passing hour. SSH client terminal window Presumably, you left the ssh client program terminal window open for the time being. If that window is still open, you can enter the commands: Code: cd /mnt-efs/mprime/instances ls -lrt * (or instead of "mnt-efs", use whatever name you chose earlier) At first, you should see only the subdirectories you created as part of the setup, namely: c4.large c4.xlarge c4.2xlarge c4.4xlarge c4.8xlarge; however after the instance launches, you will notice an extra subdirectory underneath c4.large, which will have a name identical to the instance ID. You can go to that subdirectory: Code: cd c4.large/ Now type cd i-, but instead of hitting the Enter button, hit the Tab button. The command line should then auto-complete with the full name of the subdirectory, assuming it is the only subdirectory there. Now you can hit Enter. You are now in the work directory where mprime is running. Enter the command Code: ls -lrt * for a directory listing, you will notice a worktodo.txt file and a file whose name is of the form p....... which is the save file. Enter the command Code: cat worktodo.txt to view the contents of the worktodo.txt file. There will be lines of the form Test= or DoubleCheck= , which list the exponents of the Mersenne numbers you are testing. Rerun the previous commands: Code: cd /mnt-efs/mprime/instances ls -lrt * (or instead of "mnt-efs", use whatever name you chose earlier) The "ls" commands shows the time-and-date for last modification of the i-................. subdirectory. As time goes on, if you repeat the ls -lrt * command, you will see that time-and-date change, because the p....... save files get written at regular intervals. If this time-and-date ever lags behind the current clock time and calendar date, then it almost certainly means that mprime has stopped working for some reason (usually because the spot instance was terminated due to a rise in the spot price above your limit price, and the spot price hasn't fallen back down again yet). mersenne.org pages If you have a PrimeNet user account (you can create one at http://www.mersenne.org/gettingstarted/ ), and if you entered that account name in the V5UserID= field in the prime-init.txt file in the Setting up an EFS filesystem: initial setup and configuration section, then you can log into that account at the mersenne.org website. You can go to http://www.mersenne.org/cpus/ to see the list of your computers that are currently doing LL testing on Mersenne numbers. You should see an entry for your newly-launched instance, along with the Mersenne exponent that it is working on. Then go to http://www.mersenne.org/workload/ to see the complete list of Mersenne exponents assigned to you. One of them should be the one from the newly-launched instance. The "Stage, %" field will be blank for now, but if you return tomorrow and take another look, it should show what percentage of completion has been achieved so far. When the percentage of completion indicates that the LL test is nearly finished, you can wait a while and then go to http://www.mersenne.org/results/ to see the completed result. Unless you are very very lucky it will not be a Mersenne prime, but rather it will display a "Residue" of the form XXXXXXXXXXXXXX__, which is a sort of "certificate" indicating that this Mersenne number is not prime. The website http://mersenneforum.org/ contains discussions about Mersenne prime hunting and the Prime95/mprime program itself. Changing the number of simultaneous instances to more than one, or none The spot request you created will keep running indefinitely. If spot instances are terminated due to the spot price getting too high, it will keep respawning new spot instance whenever the spot price falls sufficiently. It will keep doing more and more Lucas-Lehmer tests, and continue to run up a bill. You might eventually choose to run more than one instance simultaneously, or you might choose to stop running them altogether. See the next section. Next section: Running two or more Lucas-Lehmer testing instances at the same time, or quitting completely Last fiddled with by GP2 on 2017-07-30 at 17:42
2016-07-15, 14:14   #16
chalsall
If I May

"Chris Halsall"
Sep 2002

22·2,551 Posts

Quote:
 Originally Posted by GP2 The end

2016-07-15, 15:16   #17
Serpentine Vermin Jar

Jul 2014

22×3×277 Posts

Quote:
 Originally Posted by GP2 ...I also strongly recommend lowering the DiskWriteTime from the default 30 minutes to something smaller like 10 minutes. There are some circumstances where save files do not get written when instances are terminated, in particular when you yourself terminate the instance from the EC2 console. This modified setting helps to ensure that no more than 10 minutes' work maximum is lost under those circumstances.
Good info on all of this. For the disk write, may as well do it every couple of minutes. Back when it actually took a while to write a file of a few MB to disk (or floppy?) it made sense to do it every 30 minutes, but that was so 1990s... LOL

 2016-07-15, 18:10 #18 GP2     Sep 2003 5×11×47 Posts An up-to-date list of AWS regions that support EFS (Elastic File System) is maintained by Amazon at this page. PS, sorry for the length, if I spent more time I might have tried to make it more concise. I just wanted to document everything before I forgot what I did to set it up. I'm not familiar enough with AWS CloudFormation or similar tools, probably there's some way that the initial setup and configuration part could be automated a lot more.
2016-07-15, 18:54   #19

"Kieren"
Jul 2011
In My Own Galaxy!

27AE16 Posts

Quote:
 Originally Posted by GP2 An up-to-date list of AWS regions that support EFS (Elastic File System) is maintained by Amazon at this page. PS, sorry for the length, if I spent more time I might have tried to make it more concise. I just wanted to document everything before I forgot what I did to set it up. I'm not familiar enough with AWS CloudFormation or similar tools, probably there's some way that the initial setup and configuration part could be automated a lot more.
Please don't be sorry. Completeness counts at least as much as conciseness.

 2016-07-23, 21:31 #20 GP2     Sep 2003 5×11×47 Posts How to monitor the progress of a p??????? save file When you run mprime in the background on a server, you don't see output lines showing the progress: Code: [July 09 15:23] Iteration: 21631169 / 40000003 [54.08%] However, this information can be extracted from the p??????? savefiles. The exponent is stored at offset 20 and the iteration count is stored at offset 56; both are four bytes in little-endian order. Here is a Python script that reads one or more p??????? savefiles and outputs one line for each, showing its state of progress: (name the script showprogress.py , or whatever you want: Code: #! /usr/bin/env python """ This program takes one or more arguments, which are filenames of save files created by various programs related to Mersenne prime testing. Filenames are of the following forms (the ? represent alphanumberic characters0: p??????? : created by mprime/Prime95, doing Lucas-Lehmer testing c??????? : created by CUDALucas, doing Lucas-Lehmer testing e??????? : created by mprime/Prime95, doing ECM testing m??????? : created by mprime/Prime95, doing P-1 testing For each filename, a line is output, similar to: [July 09 15:23] LL (mprime): 21631169 / 40000003 [54.08%] or if the -s (--show_filename) option was set, the output is like: p0F00003: [July 09 15:23] LL (mprime): 21631169 / 40000003 [54.08%] or if the -n (--no_timestamp) option was set, the output is like: LL (mprime): 21631169 / 40000003 [54.08%] """ from __future__ import print_function, division import sys import os import os.path import datetime import struct class p_file: def __init__(self, f): self.f = f def read(self): self.s = self.f.read(60) def unpack(self): self.p = struct.unpack_from("
 2016-10-10, 00:53 #21 GP2     Sep 2003 5·11·47 Posts The startup script (in an earlier message in this thread) for launching mprime has now been updated to also work with the new p2.xlarge instance, which has a GPU and four CPU cores. The updated script will run CUDALucas simultaneously with mprime. Note: only a single instance of CUDALucas is run, so the script isn't (yet) suitable for the larger p2.8xlarge and p2.16xlarge instance type, which have 8 GPUs and 16 GPUs respectively. In principle, the script could be readily modified to launch multiple CUDALucas instances, each with a different -d on the command line, but I haven't tried this yet (mostly because the spot prices for these larger instance types are impractically high at the moment). To make this work, you need to create a p2.xlarge directory alongside the c4.large, c4.xlarge, etc. directories mentioned in the earlier messages. Sample local-init.txt file for p2.xlarge subdirectory: Code: OldCpuSpeed=2600 NewCpuSpeedCount=0 NewCpuSpeed=0 RollingAverage=1000 RollingAverageIsFromV27=1 ComputerID=P2_XL Memory=58000 during 7:30-23:30 else 58000 Affinity=100 WorkerThreads=1 ThreadsPerTest=2` The prime-init.txt file for the p2.xlarge subdirectory is the same as for the other subdirectories. You also need to create a parallel subdirectory structure for CUDALucas. Download the CUDALucas source code and compile it, and place the executable at: /mnt-efs/CUDALucas/prog/CUDALucas Create a directory /mnt-efs/CUDALucas/instances/p2.xlarge and copy the CUDALucas.ini file (from the source code distribution) into that subdirectory.
 2016-10-17, 23:08 #22 GP2     Sep 2003 5×11×47 Posts AWS us-east-2 region (Ohio) is now online, has Elastic File System A new AWS region, Ohio (us-east-2) is now available. The Elastic File System (EFS) is available for this new region, so the scripts and methods mentioned in this thread will work for it too. In addition to Ohio, EFS is also available in us-west-2 (Oregon), us-east-1 (Northern Virginia) and eu-west-1 (Ireland). Note that EFS is still not available in any other regions, including Northern California (us-west-1) and Frankfurt (eu-central-1). Other new regions coming soon include London, Paris and Montreal.

 Similar Threads Thread Thread Starter Forum Replies Last Post GP2 Cloud Computing 4 2020-08-03 11:21 ZFR Software 4 2018-02-02 20:18 kladner Science & Technology 7 2017-03-02 14:18 dragonbud20 Information & Answers 12 2015-09-26 21:40 GARYP166 Information & Answers 11 2009-07-13 19:39

All times are UTC. The time now is 08:22.

Sat Jan 29 08:22:41 UTC 2022 up 190 days, 2:51, 1 user, load averages: 0.98, 1.18, 1.26