mersenneforum.org  

Go Back   mersenneforum.org > Great Internet Mersenne Prime Search > Hardware > GPU Computing

Reply
 
Thread Tools
Old 2016-08-13, 14:23   #1
airsquirrels
 
airsquirrels's Avatar
 
"David"
Jul 2015
Ohio

11·47 Posts
Default New GPU Compute System

Well I managed to snag 8 Titan Blacks and water blocks for just $2400 on eBay this morning, and I've got an empty SuperMicro 4027TR that has just been running CPUs on air. I might finally be going team green, even with much older hardware....

I really enjoyed building the AMD systems last year with Fury X cards - 8 Fury X cards put out about 8000 GhzDay/Day in TF and averages around 500 GhzDay/Day of LL. Unfortunately AMD's Polaris architecture really disappointed this year and even though NVIDIA Pascal is proving to be extremely performant it would take somewhere around $13,000 for 8x Titan X Pascals on water (650 GhzDay/Day) or $6-7000 for 8x 1080's on water (500 GhzDay/Day).

In both cases the old Titan Blacks actually perform nearly as well in LL, yielding around 700 GhzDay/Day at the 4096K FFT that I've been referencing. $2400 sounds like a great deal!

As must as I wish I could just throw down for 8x P100 PCIe whenever they come out, for > $30,000, I think I will stick to eBay.
airsquirrels is offline   Reply With Quote
Old 2016-08-13, 14:38   #2
Mark Rose
 
Mark Rose's Avatar
 
"/X\(‘-‘)/X\"
Jan 2013

22·36 Posts
Default

That is a good deal!

I'm tempted to pick up more GTX 580s for TF as they're getting really cheap now, but power consumption makes that an unfriendly proposition. I'd also need to build a new system to plug them into and find an electrical circuit with enough spare capacity.

How many GPUs do you have now?
Mark Rose is offline   Reply With Quote
Old 2016-08-13, 15:52   #3
airsquirrels
 
airsquirrels's Avatar
 
"David"
Jul 2015
Ohio

51710 Posts
Default

Quote:
Originally Posted by Mark Rose View Post
...and find an electrical circuit with enough spare capacity.?
That one always seems to be the challenge! Although we've got a larger solar array going in soon that should help with that one...

The number of cards ebbs and flows, I tend to lose about 1 card every other month or so from heat death, liquid/condensation issues, or just general failures. We have stabilized a lot of things so I'm hoping that number goes down. My graveyard currently has:
1x HD7990 - PCIe extender shorted, card smoked
4x Fury X (All lost in the same liquid leak - insurance covered)
3x Titan (infamous memory mosfet failure, technically repairable but I lack the SMD oven)
1x 295X2 - Stock cooler, raw heat death, just let out the smoke one day

We have 81 live GPUs + the 8 I just won and 1 Fury X on order, so that will make 90.

29 NVIDIA and 61 AMD.
airsquirrels is offline   Reply With Quote
Old 2016-08-13, 17:21   #4
VictordeHolland
 
VictordeHolland's Avatar
 
"Victor de Hollander"
Aug 2011
the Netherlands

23·3·72 Posts
Default

You are truly the King of GPUS!
VictordeHolland is offline   Reply With Quote
Old 2016-08-14, 15:01   #5
LaurV
Romulan Interpreter
 
LaurV's Avatar
 
Jun 2011
Thailand

32×5×199 Posts
Default

Quote:
Originally Posted by airsquirrels View Post
3x Titan (infamous memory mosfet failure, technically repairable but I lack the SMD oven)
You can send them to me I have the knowledge and capacity to repair them, and can find a mobo and some intestines to put them to work...
LaurV is online now   Reply With Quote
Old 2016-08-15, 21:23   #6
diep
 
diep's Avatar
 
Sep 2006
The Netherlands

69210 Posts
Default

Quote:
Originally Posted by airsquirrels View Post
That one always seems to be the challenge! Although we've got a larger solar array going in soon that should help with that one...

The number of cards ebbs and flows, I tend to lose about 1 card every other month or so from heat death, liquid/condensation issues, or just general failures. We have stabilized a lot of things so I'm hoping that number goes down. My graveyard currently has:
1x HD7990 - PCIe extender shorted, card smoked
4x Fury X (All lost in the same liquid leak - insurance covered)
3x Titan (infamous memory mosfet failure, technically repairable but I lack the SMD oven)
1x 295X2 - Stock cooler, raw heat death, just let out the smoke one day

We have 81 live GPUs + the 8 I just won and 1 Fury X on order, so that will make 90.

29 NVIDIA and 61 AMD.
Cool man 8 cards Titan Black.

Your cards only get turned on when sun shines?
diep is offline   Reply With Quote
Old 2016-08-15, 22:12   #7
airsquirrels
 
airsquirrels's Avatar
 
"David"
Jul 2015
Ohio

11·47 Posts
Default

Ohio is net-metering, so we run the cards 24/7 and get paid for excess power generation during the day, pay for power at night. It balances out to a net positive.
airsquirrels is offline   Reply With Quote
Old 2016-08-15, 22:39   #8
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

3·55 Posts
Default

Quote:
Originally Posted by airsquirrels View Post
Ohio is net-metering, so we run the cards 24/7 and get paid for excess power generation during the day, pay for power at night. It balances out to a net positive.
If I may please rant just a little bit about this...

The Generation, Transmission and Distribution (GTD) Industry have a bit of a legitimate complaint about this.

You generate power and feed it back into the grid when you can't use it at the same price you buy it for, but you don't pay for any of the infrastructure (cables, poles, generators, management of same). And, possibility, when you are generating power and can't use it others also can't use it.

Have you thought about storing excess energy on your own premises for your own use with a temporal delta?

The provider should not be assumed to be a capacitor to your variable production and demands.
chalsall is offline   Reply With Quote
Old 2016-08-15, 23:16   #9
diep
 
diep's Avatar
 
Sep 2006
The Netherlands

22·173 Posts
Default

Quote:
Originally Posted by airsquirrels View Post
Ohio is net-metering, so we run the cards 24/7 and get paid for excess power generation during the day, pay for power at night. It balances out to a net positive.
Wow those politicians are FUBAR there as well - as buying in power from the net for those electricity companies during surges will soon be 100 dollar a megawatt hour, when the Sun isn't shining in november, whereas when there is surplus power it's worth less than 1 dollar a megawatthour, if you can sell it at all.

Over here the energy companies need to pay a fine when they generate wind or solar panel power when there is a surpluse on the market :)

In short energy companies basically build energy centrals solely for the surges there are on the net.

So this sort of subsidy can't last forever.
diep is offline   Reply With Quote
Old 2016-08-15, 23:23   #10
diep
 
diep's Avatar
 
Sep 2006
The Netherlands

69210 Posts
Default

Quote:
Originally Posted by chalsall View Post
If I may please rant just a little bit about this...

The Generation, Transmission and Distribution (GTD) Industry have a bit of a legitimate complaint about this.

You generate power and feed it back into the grid when you can't use it at the same price you buy it for, but you don't pay for any of the infrastructure (cables, poles, generators, management of same). And, possibility, when you are generating power and can't use it others also can't use it.

Have you thought about storing excess energy on your own premises for your own use with a temporal delta?

The provider should not be assumed to be a capacitor to your variable production and demands.
Yeah well i also have milling machines here and lathe. Those eat major power of course when used (and i don't even have a real good one that eats easily 30+ kilowatt). The entire industry has designed its machines such that they eat a constant amount of power. Only the peak power usage you have a year you pay for. Is that 7 megawatt? then you pay for 7 megawatt all year.

I'm always amazed by the high amounts of cash a megawatt hour government needs to pay for towards the energy companies - just because of so called energy savings - whereas it makes their bill 5x higher.

For companies eating lots of power, you get a constant megawatt for about 25 euro a megawatt hour now.

That's like 2.5 eurocent a kilowatthour?

On other hand for this household the first 10k kilowatthour we consume we pay a lot of taxes. Can easily go to 35 eurocent a kilowatthour because of the 25 eurocent taxes on top of it which hardly are there for industry of course.
diep is offline   Reply With Quote
Old 2016-08-15, 23:39   #11
airsquirrels
 
airsquirrels's Avatar
 
"David"
Jul 2015
Ohio

10058 Posts
Default

Quote:
Originally Posted by chalsall View Post
If I may please rant just a little bit about this

The provider should not be assumed to be a capacitor to your variable production and demands.
We actually still get billed the transmission fee. Our bill is split between metered kWh and generation and transmission.

In the short term it is cheaper to pay the transmission fee than invest in and maintain on-site batteries/inverters etc. I do actually have ambitions of setting that up and running DC systems, it just doesn't make anywhere close to economic sense right now.
airsquirrels is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Cost to compute prime Unregistered Information & Answers 30 2013-12-18 03:34
Doubled compute power for a day? Christenson PrimeNet 19 2011-10-26 08:29
New Compute Box Christenson Hardware 0 2011-01-15 04:44
Compute billions of digits of Pi using GMP M0CZY Miscellaneous Math 5 2010-10-14 09:40
My throughput does not compute... petrw1 Hardware 9 2007-08-13 14:38

All times are UTC. The time now is 04:05.

Wed Dec 2 04:05:20 UTC 2020 up 83 days, 1:16, 1 user, load averages: 2.41, 2.36, 2.10

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.