mersenneforum.org  

Go Back   mersenneforum.org > Great Internet Mersenne Prime Search > Hardware > Cloud Computing

Reply
 
Thread Tools
Old 2016-07-30, 05:14   #1
GP2
 
GP2's Avatar
 
Sep 2003

5·11·47 Posts
Default Major projects using the cloud

I dug up some old threads involving a major cloud computing project by WraithX, who is an active user in these forums. Since the Cloud Computing sub-forum was established recently, it seems reasonable to at least mention and summarize it here.

The project involved making progress on the home prime HP(49), and involved considerable expenditures of both time and money.

Here is the first thread from 2014, describing factoring a 210-digit number using GPU instances on the Amazon EC2 cloud. A later post mentions that the total cost ran to $7600.

A previous post from 2012 described some earlier progress, it's not clear if cloud computing was involved in that.



I don't know if such a project could be undertaken today, for various reasons.

Unfortunately, using GPU instances on Amazon is insanely expensive at the moment. There is no competition, since Google and Microsoft Azure do not yet offer GPU instances, although Azure is planning to introduce some. The reason for the high prices is not completely clear, but it certainly seems that the demand exceeds the supply.

There is tremendous internal demand at all these giant companies since GPUs are used extensively in machine learning, so they use them to power things like Amazon Echo, Microsoft's Cortana, Google's intelligent search, and so forth. There is also considerable external demand in the public cloud for video transcoding batch jobs.

Also, knowledgeable users here have described Amazon's current generation of g2 instances as a step backward in terms of bang for the buck, from the old cc instances, which have only limited availability now.

Still, it would be fun to idly speculate what sort of prime-related project might be the highest or best priority, if you could throw a bunch of money at it to conjure up serious cloud computing resources.
GP2 is offline   Reply With Quote
Old 2016-07-30, 05:34   #2
Mark Rose
 
Mark Rose's Avatar
 
"/X\(‘-‘)/X\"
Jan 2013

55538 Posts
Default

Amazon's GPU instances aren't that expensive, if you consider you're getting Tesla hardware. If you were getting consumer grade hardware, then yes, the prices would be insane.
Mark Rose is offline   Reply With Quote
Old 2016-07-30, 05:57   #3
GP2
 
GP2's Avatar
 
Sep 2003

1010000110012 Posts
Default

Quote:
Originally Posted by Mark Rose View Post
Amazon's GPU instances aren't that expensive, if you consider you're getting Tesla hardware. If you were getting consumer grade hardware, then yes, the prices would be insane.
I guess I've spent too much time marveling at the step-function spot price graphs for the g2.2xlarge instances, which spend a not insignificant amount of time in the stratosphere at about $6.50 an hour, or g2.8xlarge at about $28 an hour (compared to the on-demand price of $2.60 for the latter).

The prices for cc2.8xlarge do seem more reasonable, but still between $0.30 to $0.40 an hour. It never dips as low as the [I]average[/I] of $0.254 reported by WraithX for the work done a few years ago using that same instance type. And as a "previous generation" device (in Amazon's terms), you never know how long it will even remain available.

Edit: oops I mixed up the cc2.8xlarge with the cg1.4xlarge (Tesla "Fermi" M2050). Never mind.

Last fiddled with by GP2 on 2016-07-30 at 06:26
GP2 is offline   Reply With Quote
Old 2016-07-30, 06:03   #4
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

22×32×139 Posts
Default

The NFS factoring job that WraithX did uses GPU only for polynomial selection, the first phase and 3-4% of the total project effort. The two lengthy tasks (sieving and linear algebra) are CPU-only.
If such a task were attempted by someone now, there are sufficient GPU resources among the forum members to not need to use AWS GPU's.

Last fiddled with by VBCurtis on 2016-07-30 at 06:04
VBCurtis is offline   Reply With Quote
Old 2016-07-31, 21:25   #5
GP2
 
GP2's Avatar
 
Sep 2003

258510 Posts
Default

Quote:
Originally Posted by VBCurtis View Post
The two lengthy tasks (sieving and linear algebra) are CPU-only.
Well, even better if a computation project is CPU-based, the cloud is good for that.

There are other possibilities. The discovery just the other day that M5240707 is semiprime (or semiPRPrime) makes me think there is some low hanging fruit there. PRP testing doesn't go through PrimeNet, rather it's manually assigned and manually reported at mersenne.ca, so there's only a handful of people doing it, yet this latest PRP factor of M5240707 ranks number 4 on the list of largest PRPs. Each PRP test takes only a few hours in that range, and in the "wavefront" of 3.6M the PRP tests take less than an hour each with the smallest c4.large instance on Amazon EC2.

If a PRP factor was found for an exponent in the range of 13.3M or higher, it would currently very likely rank number 1. Maybe that's worth aiming for.

Last fiddled with by GP2 on 2016-07-31 at 21:27
GP2 is offline   Reply With Quote
Old 2016-07-31, 23:27   #6
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

7·1,423 Posts
Default

Quote:
Originally Posted by GP2 View Post
Well, even better if a computation project is CPU-based, the cloud is good for that.
Indeed.

Just to put on the table, I've spent the last few weeks "making friends" with Google's Cloud Computing (AKA: Google Cloud Platform :: Compute Engine).

I took advantage of Google's offering of $300 of free credit to evaluate. While somewhat constrained, I was still able to spin up 10*8 vcpus associated with hundreds of GBs of storage. With Amazon's EC2 you are seriously constrained with regards to how much CPU and storage you can bring online (or in standby) for evaluation (basically: nothing).

My takeaway?

1. Google doesn't offer any GPU enabled instances, so when I need to use GPUs I'll continue to use Amazon.

2. Google doesn't offer any form of POSIX compliment shared read-write storage (such as Amazon's EFS).

2.1. Read-only storage from multiple instances to a single volume is possible, but one has to first mount the volume in read-write mode under a single instance to load the data, and then unmount. Only then can one mount the volume in read-only mode under multiple instances.

3. Having said the above, Google's instances have the advantage that their "/" (root) file system survives stops, halts and restarts.

3.1. Amazon's instances' "/" file system goes away upon any form of a halt or restart. One can ask (and pay for) the file system not being deleted, and then recover the data manually or programmatically, but it's a cost and extras steps.

I hope the above might be helpful to anyone exploring the Cloud Computing space.

To anyone who hasn't already taken advantage of the free offerings of the providers, I would encourage you to do so. Worth the effort to "make friends" with the offerings.
chalsall is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Cloud Computing sites rdick Cloud Computing 1 2016-12-02 01:27
Cloud computing Unregistered Information & Answers 10 2011-05-10 00:57
Major overhaul of the DB 10metreh Aliquot Sequences 5 2010-08-29 01:10
Major forum upgrade Xyzzy Forum Feedback 24 2008-10-30 00:23
Major software upgrade. Xyzzy Forum Feedback 37 2008-07-06 17:56

All times are UTC. The time now is 23:45.


Mon Oct 25 23:45:26 UTC 2021 up 94 days, 18:14, 0 users, load averages: 2.73, 3.00, 3.21

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.