mersenneforum.org Major projects using the cloud
 Register FAQ Search Today's Posts Mark Forums Read

 2016-07-30, 05:14 #1 GP2     Sep 2003 5·11·47 Posts Major projects using the cloud I dug up some old threads involving a major cloud computing project by WraithX, who is an active user in these forums. Since the Cloud Computing sub-forum was established recently, it seems reasonable to at least mention and summarize it here. The project involved making progress on the home prime HP(49), and involved considerable expenditures of both time and money. Here is the first thread from 2014, describing factoring a 210-digit number using GPU instances on the Amazon EC2 cloud. A later post mentions that the total cost ran to $7600. A previous post from 2012 described some earlier progress, it's not clear if cloud computing was involved in that. I don't know if such a project could be undertaken today, for various reasons. Unfortunately, using GPU instances on Amazon is insanely expensive at the moment. There is no competition, since Google and Microsoft Azure do not yet offer GPU instances, although Azure is planning to introduce some. The reason for the high prices is not completely clear, but it certainly seems that the demand exceeds the supply. There is tremendous internal demand at all these giant companies since GPUs are used extensively in machine learning, so they use them to power things like Amazon Echo, Microsoft's Cortana, Google's intelligent search, and so forth. There is also considerable external demand in the public cloud for video transcoding batch jobs. Also, knowledgeable users here have described Amazon's current generation of g2 instances as a step backward in terms of bang for the buck, from the old cc instances, which have only limited availability now. Still, it would be fun to idly speculate what sort of prime-related project might be the highest or best priority, if you could throw a bunch of money at it to conjure up serious cloud computing resources.  2016-07-30, 05:34 #2 Mark Rose "/X\(‘-‘)/X\" Jan 2013 55538 Posts Amazon's GPU instances aren't that expensive, if you consider you're getting Tesla hardware. If you were getting consumer grade hardware, then yes, the prices would be insane. 2016-07-30, 05:57 #3 GP2 Sep 2003 1010000110012 Posts Quote:  Originally Posted by Mark Rose Amazon's GPU instances aren't that expensive, if you consider you're getting Tesla hardware. If you were getting consumer grade hardware, then yes, the prices would be insane. I guess I've spent too much time marveling at the step-function spot price graphs for the g2.2xlarge instances, which spend a not insignificant amount of time in the stratosphere at about$6.50 an hour, or g2.8xlarge at about $28 an hour (compared to the on-demand price of$2.60 for the latter).

The prices for cc2.8xlarge do seem more reasonable, but still between $0.30 to$0.40 an hour. It never dips as low as the [I]average[/I] of $0.254 reported by WraithX for the work done a few years ago using that same instance type. And as a "previous generation" device (in Amazon's terms), you never know how long it will even remain available. Edit: oops I mixed up the cc2.8xlarge with the cg1.4xlarge (Tesla "Fermi" M2050). Never mind. Last fiddled with by GP2 on 2016-07-30 at 06:26  2016-07-30, 06:03 #4 VBCurtis "Curtis" Feb 2005 Riverside, CA 22×32×139 Posts The NFS factoring job that WraithX did uses GPU only for polynomial selection, the first phase and 3-4% of the total project effort. The two lengthy tasks (sieving and linear algebra) are CPU-only. If such a task were attempted by someone now, there are sufficient GPU resources among the forum members to not need to use AWS GPU's. Last fiddled with by VBCurtis on 2016-07-30 at 06:04 2016-07-31, 21:25 #5 GP2 Sep 2003 258510 Posts Quote:  Originally Posted by VBCurtis The two lengthy tasks (sieving and linear algebra) are CPU-only. Well, even better if a computation project is CPU-based, the cloud is good for that. There are other possibilities. The discovery just the other day that M5240707 is semiprime (or semiPRPrime) makes me think there is some low hanging fruit there. PRP testing doesn't go through PrimeNet, rather it's manually assigned and manually reported at mersenne.ca, so there's only a handful of people doing it, yet this latest PRP factor of M5240707 ranks number 4 on the list of largest PRPs. Each PRP test takes only a few hours in that range, and in the "wavefront" of 3.6M the PRP tests take less than an hour each with the smallest c4.large instance on Amazon EC2. If a PRP factor was found for an exponent in the range of 13.3M or higher, it would currently very likely rank number 1. Maybe that's worth aiming for. Last fiddled with by GP2 on 2016-07-31 at 21:27 2016-07-31, 23:27 #6 chalsall If I May "Chris Halsall" Sep 2002 Barbados 7·1,423 Posts Quote:  Originally Posted by GP2 Well, even better if a computation project is CPU-based, the cloud is good for that. Indeed. Just to put on the table, I've spent the last few weeks "making friends" with Google's Cloud Computing (AKA: Google Cloud Platform :: Compute Engine). I took advantage of Google's offering of$300 of free credit to evaluate. While somewhat constrained, I was still able to spin up 10*8 vcpus associated with hundreds of GBs of storage. With Amazon's EC2 you are seriously constrained with regards to how much CPU and storage you can bring online (or in standby) for evaluation (basically: nothing).

My takeaway?

1. Google doesn't offer any GPU enabled instances, so when I need to use GPUs I'll continue to use Amazon.

2. Google doesn't offer any form of POSIX compliment shared read-write storage (such as Amazon's EFS).

2.1. Read-only storage from multiple instances to a single volume is possible, but one has to first mount the volume in read-write mode under a single instance to load the data, and then unmount. Only then can one mount the volume in read-only mode under multiple instances.

3. Having said the above, Google's instances have the advantage that their "/" (root) file system survives stops, halts and restarts.

3.1. Amazon's instances' "/" file system goes away upon any form of a halt or restart. One can ask (and pay for) the file system not being deleted, and then recover the data manually or programmatically, but it's a cost and extras steps.

I hope the above might be helpful to anyone exploring the Cloud Computing space.

To anyone who hasn't already taken advantage of the free offerings of the providers, I would encourage you to do so. Worth the effort to "make friends" with the offerings.

 Similar Threads Thread Thread Starter Forum Replies Last Post rdick Cloud Computing 1 2016-12-02 01:27 Unregistered Information & Answers 10 2011-05-10 00:57 10metreh Aliquot Sequences 5 2010-08-29 01:10 Xyzzy Forum Feedback 24 2008-10-30 00:23 Xyzzy Forum Feedback 37 2008-07-06 17:56

All times are UTC. The time now is 23:45.

Mon Oct 25 23:45:26 UTC 2021 up 94 days, 18:14, 0 users, load averages: 2.73, 3.00, 3.21