Register FAQ Search Today's Posts Mark Forums Read

 2018-09-05, 13:23 #1 pepi37     Dec 2011 After milion nines:) 5D316 Posts Google compute engine I has one problem: on machine with 8 vCPU, on first thread I only can get 4 cores per test not 5 as written in file. Second worker works with 3 cores without any problem [Worker #1] CoresPerTest=5 [Worker #2] CoresPerTest=3 Last fiddled with by pepi37 on 2018-09-05 at 13:54 Reason: add more info
 2018-09-05, 13:47 #2 GP2     Sep 2003 258510 Posts On both AWS and GCE, eight vCPU actually means four cores with hyperthreading. So your total CoresPerTest should add up to 4, not 8. Also, have you benchmarked mprime on GCE vs. mprime on AWS EC2? If I recall correctly the Skylakes on GCE are considerably slower than on AWS, and to make matters worse, you have to be careful to specify Skylake or better on GCE, otherwise you might get some other architecture. Make sure your /proc/cpuinfo output contains avx512 elements in the "flags" line. On the other hand GCE has a fixed cost of 1.5 cents/hour per core for preemptible instances while AWS spot prices are variable and currently around 1.9 cents/hour for a one-core instance, so that would compensate for lesser throughput. I forget exactly where the breakeven point is, though.
 2018-09-05, 13:52 #3 pepi37     Dec 2011 After milion nines:) 101110100112 Posts GP2, thanks for reply, but I manage to get 99% utilization with somehow weird setup Yes, I choose Skylake, and find cheapest region and lower memory requirements as low as I can: so with free 300$I can manage about 70 days of work. setup is next Affinity=99 NumCPUs=5 ConfigureHyperthreads=1 CpuNumHyperthreads=2 WorkerThreads=2 Pid=0 OldCpuSpeed=2504 NewCpuSpeedCount=0 NewCpuSpeed=0 [Worker #1] CoresPerTest=5 [Worker #2] CoresPerTest=3 mprime works without any warning , and utilization is 97% Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 8 On-line CPU(s) list: 0-7 Thread(s) per core: 2 Core(s) per socket: 4 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 85 Model name: Intel(R) Xeon(R) CPU @ 2.00GHz Stepping: 3 CPU MHz: 2000.172 BogoMIPS: 4000.34 Hypervisor vendor: KVM Virtualization type: full L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 56320K NUMA node0 CPU(s): 0-7 Flags: ----- avx512cd avx512bw avx512vl fma3 is detected in mprime Last fiddled with by pepi37 on 2018-09-05 at 14:05 Reason: add more info  2018-09-06, 04:20 #4 GP2 Sep 2003 5×11×47 Posts I don't think it ever makes sense to specify more cores in your mprime configuration file than the actual physical cores available. You might get "99% utilization", but that doesn't mean it's performing efficiently. In any case, you are better off running four instances of a one-core (2 vCPU) virtual machine (n1-highcpu-2) rather than one instance of a four-core (8 vCPU) virtual machine. On GCE the cost for this will be exactly the same, but your total throughput will be considerably higher. It's easy to "clone" (Google's terminology) your existing setup to create new virtual machines. Make sure the new ones also use Skylake though. On GCE the pricing is mostly uniform, the only difference is that the non-US-based servers (Europe, Asia, Canada) are about 10% more expensive than the ones in the US, so always use a US server. I hope you are using preemptible instances, they're much, much cheaper. It's annoying because you have to restart them at least once a day, and sometimes even more often than that. You can get the CloudConsole app for your phone, it's convenient for doing the daily restarts. 2018-09-06, 12:02 #5 pepi37 Dec 2011 After milion nines:) 3·7·71 Posts Quote:  Originally Posted by GP2 .but your total throughput will be considerably higher. It's easy to "clone" (Google's terminology) your existing setup to create new virtual machines. Make sure the new ones also use Skylake though. Small test show me increase in speed about 15% if I use 4 cpu2 instance instead 1 cpu8 instance. Thanks for pointing: but since it is free, and since I would like to setup, run and forget, I choose one instance 8vcpu. Also I would not give to myself some extra job to start and stop instances on daily bases :) I total : this instance will give me ( in predicted time of free$) about 1400 workunits from 1.5 to 2.2 milion digits: so it is not bad for free thing!

2018-09-06, 21:08   #6
GP2

Sep 2003

5×11×47 Posts

Quote:
 Originally Posted by pepi37 Small test show me increase in speed about 15% if I use 4 cpu2 instance instead 1 cpu8 instance. Thanks for pointing: but since it is free, and since I would like to setup, run and forget, I choose one instance 8vcpu. Also I would not give to myself some extra job to start and stop instances on daily bases :) I total : this instance will give me ( in predicted time of free$) about 1400 workunits from 1.5 to 2.2 milion digits: so it is not bad for free thing! Preemptible instances cost 1.5 cents an hour, the regular instances cost 7.09 cents an hour. That's an 80% saving!! If you're running a website or other Internet application, it has to keep running without interruption. But what we are doing is essentially a batch job. There is no uptime requirement, no deadline. It doesn't matter if it's down for part of the day. What's your goal? If you're trying to discover a Mersenne prime with that free$300, you can easily quintuple your chances with very little extra effort, just by using preemptible instances and multiple one-core instances.

PS,
GCE lets you specify a startup script. So you can automatically restart mprime whenever the preemptible instance restarts. You literally only need to download the CloudConsole app onto your phone, and check it once a day and restart your preemptible instances as necessary. A few seconds of effort each day, no more than that. You never need to log in to a shell terminal.

Last fiddled with by GP2 on 2018-09-06 at 21:17

2018-09-06, 22:41   #7
pepi37

Dec 2011
After milion nines:)

5D316 Posts

Quote:
 Originally Posted by GP2 What's your goal? If you're trying to discover a Mersenne prime with that free $300, you can easily quintuple your chances with very little extra effort, just by using preemptible instances and multiple one-core instances. This is not my goal, I have much less limits. One prime with 2M digits if good for me! But your point is clear ( thanks for that - I just now stop machine and build new preemptible instance) 6x times longer means many more processed tasks P.S Must find place where to put script for restart mprime Last fiddled with by pepi37 on 2018-09-06 at 22:42 2018-09-07, 01:36 #8 GP2 Sep 2003 5·11·47 Posts Quote:  Originally Posted by pepi37 Must find place where to put script for restart mprime Go to the Compute Engine dashboard, on the page that lists your VM instances. Click on one of those instances and you'll get to a page where you scroll down until you reach "Custom metadata" and "shutdown-script" and "startup-script". Edit that and just start up mprime in there. 2018-09-07, 03:03 #9 retina Undefined "The unspeakable one" Jun 2006 My evil lair 2·3,167 Posts Quote:  Originally Posted by GP2 ...and check it once a day and restart your preemptible instances as necessary. Yeah, this "check it once per day" thing is just awful IMO. The cost of free computing is just too high. It's like those terrible credit cards. Check it once per day to make sure no one has stolen your money. Aaargh, no thanks. Too much hassle for too little gain. 2018-09-07, 07:37 #10 pepi37 Dec 2011 After milion nines:) 3×7×71 Posts Quote:  Originally Posted by retina Yeah, this "check it once per day" thing is just awful IMO. The cost of free computing is just too high. It's like those terrible credit cards. Check it once per day to make sure no one has stolen your money. Aaargh, no thanks. Too much hassle for too little gain. Please just not fight between In our world on this small blue dot, very rare things are free. When I discovered this : quick calculation give me 60 days of 24/7 processing :I stay on one instance and "power" of that 8vCPU instance is nearly same as my I5-3750 K at 3.4 Ghz. Yes, I specifically select Skylake or newer, but those CPU work at 2 GHz, and I5-3570K works at 3.5 Ghz.So they are fast, since on 2 Ghz have same speed as for 3.5 Ghz. But when I get option to get 6x more time on same machine, I know it must be some catch (22) : And yes, instance will be stopped if work 24 hour or less. Download mobile app , and as GP2 say: just start it , and enjoy. Since we here are not on the race , in fact I lost nothing, and got much more. So at the end I am very pleased with new situation: and 6 x more compute "free time" :) P.S Too much hassle for too little gain I will not agree with that: using 8 instances with 1 vCPu will give me gain about 15 % compared to one 8vCPu instance: and that is "little gain" ,but get 6 times more time for same price is not little gain.In one case I can process about 1000 ( my tasks) and on the other 6000 tasks. I thing difference is ( very) noticeable :) Last fiddled with by pepi37 on 2018-09-07 at 07:42 Reason: add more text 2018-09-07, 07:46 #11 ET_ Banned "Luigi" Aug 2002 Team Italia 483810 Posts Quote:  Originally Posted by pepi37 Please just not fight between In our world on this small blue dot, very rare things are free. When I discovered this : quick calculation give me 60 days of 24/7 processing :I stay on one instance and "power" of that 8vCPU instance is nearly same as my I5-3750 K at 3.4 Ghz. Yes, I specifically select Skylake or newer, but those CPU work at 2 GHz, and I5-3570K works at 3.5 Ghz.So they are fast, since on 2 Ghz have same speed as for 3.5 Ghz. But when I get option to get 6x more time on same machine, I know it must be some catch (22) : And yes, instance will be stopped if work 24 hour or less. Download mobile app , and as GP2 say: just start it , and enjoy. Since we here are not on the race , in fact I lost nothing, and got much more. So at the end I am very pleased with new situation: and 6 x more compute "free time" :) P.S Too much hassle for too little gain I will not agree with that: using 8 instances with 1 vCPu will give me gain about 15 % compared to one 8vCPu instance: and that is "little gain" ,but get 6 times more time for same price is not little gain.In one case I can process about 1000 ( my tasks) and on the other 6000 tasks. I thing difference is ( very) noticeable :) BTW, how did you get that$300 bonus?

 Similar Threads Thread Thread Starter Forum Replies Last Post GP2 Cloud Computing 4 2020-08-03 11:21 GP2 Cloud Computing 32 2018-01-23 02:16 airsquirrels GPU Computing 90 2017-12-08 00:13 Christenson Hardware 0 2011-01-15 04:44 petrw1 Hardware 9 2007-08-13 14:38

All times are UTC. The time now is 10:05.

Tue Jan 25 10:05:52 UTC 2022 up 186 days, 4:34, 0 users, load averages: 1.46, 1.70, 1.50