Building DC farms; Experience, advice, questions and answers

Prime Monster · 2002-08-30, 10:34

I might as well start a thread on this, since this theme seems to be as popular for Prime, as it is for other DC projects. I run both an ordinary DC Farm (13 machines with 17 CPUs) and a Monster (12 CPUs) and I will shortly add another mini Monster (6 CPUs), so I have experience with a lot of different areas of Farm building (accumulation) and Monster building. I'd like this thread to contain advice on all aspects of DC Farming, not only Monster solutions, all operating systems, all CPUs, all motherboards and so on.

One way or another some of us get so bitten by the DC bug that we end up with more than 1 or 2 machines at home doing GIMPS full-time. Some go a bit beyond a few machines and end up with a large collection of machines, so-called DC farms.

There is a special class of DC farms, generically called Monsters (named so after one of the first Monsters, the OcUK Stomp-Monster - a Seti Cruncher). These monsters are special purpose machines built only for this one task alone, with the aim of getting as much bang for the $ as possible. Most of the existing Monsters are crunching Seti.

So if you are into building DC Farms or Monsters; this thread is for you. For building, read accumulating and visa versa.

During the last year or so, several DC Farm projects were started and completed. Valuable experience have been made in areas from how to run a Netboot solution, how to manage a large number of machines, how to share a power-supply between two motherboards and so on. Much of this can be used by others for their own setup, either for ordinary DC Farms or for Monsters.

What areas are important when building a Farm or a Monster? Are you limited by the cost of electricity, or available resources, is space important, what about heat or noise, can your machines run open or closed, are your skills up to the task etc. The last is in my opinion the least important, because we can always learn something new and with the help available here and other places that should not be a limiting factor.

In difference to many other DC projects, GIMPS is special in that the P4 is way much better than any other CPU type. This means that the optimal solution for a GIMPS Farm or Monster is P4 based. I hasten to say that it is not limited to P4s, you can use any CPU you feel comfortable with, but on a pure bang for $ scale, the P4 is king.

Where do you start. If you are like any of us, then you started with one machine, got another after a while and then never looked back. The more, the better. Now has come the time to consolidate and make some decision about where to go next. What new and higher level can you aspire to. Your decision, your call. Ask and you will hopefully get the answers you need. There is no right or wrong way here. There are only more cycles to add. :D

I'll start of with a link to my Monster site which shows one particular solution based on Netboot and LTSP (GNU/Linux). The site also contains links to to other solutions as well.

Lord_Humungous · 2002-08-30, 22:59

In the Prime Monster, are the NIC's affixed to the motherboard in some fashion or is the spring pressure from the PCI slots the only thing holding 'em in?

Prime Monster · 2002-08-31, 00:43

Quote:

In the Prime Monster, are the NIC's affixed to the motherboard in some fashion or is the spring pressure from the PCI slots the only thing holding 'em in?

Only the spring pressure. Never had any problem with it so far....

Tasuke · 2002-08-31, 01:07

a dab of hot/craft glue will fix it if you do find you have a problem of walking PCBs

Prime95 · 2002-08-31, 01:31

Do any motherboards with NICs built in also support a boot rom?

Lord_Humungous · 2002-08-31, 02:08

Quote:

Originally Posted by Tasuke

a dab of hot/craft glue will fix it if you do find you have a problem of walking PCBs

..or RTV. I was looking over the Prime Monster site and saw the mobo's were hanging sideways, so the question just occured to me.

Lord_Humungous · 2002-08-31, 02:24

Quote:

Originally Posted by Prime95

Do any motherboards with NICs built in also support a boot rom?

I second that question. Booting with just the mobo/uP/RAM would be the ultimate. I see on Asus's site that the P4S333M can come with 10/100 and it doesn't look like the price is much different.

I'm downloading the user manual now. I should have the .pdf in a day or two. GOD I hate dial-up.

Lord_Humungous · 2002-08-31, 02:41

Looks like you'd have to hack the bios on the motherboard. This file explains it..

http://www.mersenneforum.org/attachm...pecialBrew.pdf

Prime Monster · 2002-08-31, 09:38

Quote:

Do any motherboards with NICs built in also support a boot rom?

On a standard motherboard, without PXE support, you have to hack the BIOS. Basically what you have to do is to add the bootrom code to the bios. The bootrom code is usually around 32KB +/-, so any modern BIOS should have ample enough space.

The problem is that you can screw up the bios completely by doing this. There are tools out there to make life a bit easier for you, so that you can make backups, fallback etc, but as many have experienced with just tuning the bios. It can bork the system.

The PXE effort is akin to having bootrom code support in the bios. Since I decided not to go that way (mostly for lack of knowledge and time) I have not investigated that option any further.

Paulie · 2002-08-31, 17:41

At work, we have a bunch of RLX server blades. They are headless/floppyless, and use PXE boot to do the software installs. They run a Transmeta chip.

I ran a benchmark on one (Win2k, 640M ram, 3 NICs, 2-40gig IBM TravelStar laptop HD's), 512KB Level 2 cache, 133mhz FSB

Transmeta(tm) Crusoe(tm) Processor TM5600
CPU speed: 665.58 MHz
CPU features: RDTSC, CMOV, MMX
L1 cache size: unknown
L2 cache size: unknown
L1 cache line size: unknown
L2 cache line size: unknown
Prime95 version 22.8, RdtscTiming=1
Best time for 256K FFT length: 105.611 ms.
Best time for 320K FFT length: 139.238 ms.
Best time for 384K FFT length: 165.960 ms.
Best time for 448K FFT length: 199.995 ms.
Best time for 512K FFT length: 221.548 ms.
Best time for 640K FFT length: 293.651 ms.
Best time for 768K FFT length: 355.083 ms.
Best time for 896K FFT length: 425.030 ms.
Best time for 1024K FFT length: 478.481 ms.
Best time for 1280K FFT length: 620.693 ms.
Best time for 1536K FFT length: 745.929 ms.
Best time for 1792K FFT length: 891.290 ms.

I noticed the benchmark doesn't have the cache sizes, so I pulled them off Transmeta's website:

64kb L1 Instruction
64kb L1 Data
256kb L2 WriteBack
128bit native instruction set with code morphing to x86

A 4U rack of 24 blades puts out less heat than a single Compaq DL380 with two hard drives. It's amazing. Plus, each blade is around $1200. 8) 8)

They also make a PentiumIII version, but they require a different chasis and do put out more heat.

JC · 2002-09-01, 06:17

Quote:

Originally Posted by Prime95

Do any motherboards with NICs built in also support a boot rom?

No, I've a brand new ASUS P4B533-V and, as far as I've tried, I cannot boot 'from network'. The Integrated NIC is an Intel PRO/100 VE.
Shame on Asus !

2002-08-30, 10:34	#1
Prime Monster Aug 2002 404₈ Posts	Building DC farms; Experience, advice, questions and answers I might as well start a thread on this, since this theme seems to be as popular for Prime, as it is for other DC projects. I run both an ordinary DC Farm (13 machines with 17 CPUs) and a Monster (12 CPUs) and I will shortly add another mini Monster (6 CPUs), so I have experience with a lot of different areas of Farm building (accumulation) and Monster building. I'd like this thread to contain advice on all aspects of DC Farming, not only Monster solutions, all operating systems, all CPUs, all motherboards and so on. One way or another some of us get so bitten by the DC bug that we end up with more than 1 or 2 machines at home doing GIMPS full-time. Some go a bit beyond a few machines and end up with a large collection of machines, so-called DC farms. There is a special class of DC farms, generically called Monsters (named so after one of the first Monsters, the OcUK Stomp-Monster - a Seti Cruncher). These monsters are special purpose machines built only for this one task alone, with the aim of getting as much bang for the $ as possible. Most of the existing Monsters are crunching Seti. So if you are into building DC Farms or Monsters; this thread is for you. For building, read accumulating and visa versa. During the last year or so, several DC Farm projects were started and completed. Valuable experience have been made in areas from how to run a Netboot solution, how to manage a large number of machines, how to share a power-supply between two motherboards and so on. Much of this can be used by others for their own setup, either for ordinary DC Farms or for Monsters. What areas are important when building a Farm or a Monster? Are you limited by the cost of electricity, or available resources, is space important, what about heat or noise, can your machines run open or closed, are your skills up to the task etc. The last is in my opinion the least important, because we can always learn something new and with the help available here and other places that should not be a limiting factor. In difference to many other DC projects, GIMPS is special in that the P4 is way much better than any other CPU type. This means that the optimal solution for a GIMPS Farm or Monster is P4 based. I hasten to say that it is not limited to P4s, you can use any CPU you feel comfortable with, but on a pure bang for $ scale, the P4 is king. Where do you start. If you are like any of us, then you started with one machine, got another after a while and then never looked back. The more, the better. Now has come the time to consolidate and make some decision about where to go next. What new and higher level can you aspire to. Your decision, your call. Ask and you will hopefully get the answers you need. There is no right or wrong way here. There are only more cycles to add. :D I'll start of with a link to my Monster site which shows one particular solution based on Netboot and LTSP (GNU/Linux). The site also contains links to to other solutions as well.

2002-08-31, 17:41	#10
Paulie Aug 2002 223 Posts	RLX Server Blades At work, we have a bunch of RLX server blades. They are headless/floppyless, and use PXE boot to do the software installs. They run a Transmeta chip. I ran a benchmark on one (Win2k, 640M ram, 3 NICs, 2-40gig IBM TravelStar laptop HD's), 512KB Level 2 cache, 133mhz FSB Transmeta(tm) Crusoe(tm) Processor TM5600 CPU speed: 665.58 MHz CPU features: RDTSC, CMOV, MMX L1 cache size: unknown L2 cache size: unknown L1 cache line size: unknown L2 cache line size: unknown Prime95 version 22.8, RdtscTiming=1 Best time for 256K FFT length: 105.611 ms. Best time for 320K FFT length: 139.238 ms. Best time for 384K FFT length: 165.960 ms. Best time for 448K FFT length: 199.995 ms. Best time for 512K FFT length: 221.548 ms. Best time for 640K FFT length: 293.651 ms. Best time for 768K FFT length: 355.083 ms. Best time for 896K FFT length: 425.030 ms. Best time for 1024K FFT length: 478.481 ms. Best time for 1280K FFT length: 620.693 ms. Best time for 1536K FFT length: 745.929 ms. Best time for 1792K FFT length: 891.290 ms. I noticed the benchmark doesn't have the cache sizes, so I pulled them off Transmeta's website: 64kb L1 Instruction 64kb L1 Data 256kb L2 WriteBack 128bit native instruction set with code morphing to x86 A 4U rack of 24 blades puts out less heat than a single Compaq DL380 with two hard drives. It's amazing. Plus, each blade is around $1200. 8) 8) They also make a PentiumIII version, but they require a different chasis and do put out more heat.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Any answers on missing factors?	schickel	Aliquot Sequences	8	2011-11-29 12:24
How do you track your farms status?	mathemajikian	PrimeNet	2	2007-07-24 21:10
Need advice for building a new computer	garo	Hardware	20	2006-12-31 22:50
GGNFS questions and Answers	VJS	Factoring	100	2006-01-15 08:34
Monsters and Monster farms	Unregistered	Data	6	2004-08-12 00:28

2002-08-30, 22:59	#2
Lord_Humungous Aug 2002 23 Posts	In the Prime Monster, are the NIC's affixed to the motherboard in some fashion or is the spring pressure from the PCI slots the only thing holding 'em in?

2002-08-31, 01:07	#4
Tasuke Aug 2002 2³·3² Posts	a dab of hot/craft glue will fix it if you do find you have a problem of walking PCBs

2002-08-31, 01:31	#5
Prime95 P90 years forever! Aug 2002 Yeehaw, FL 2²×13×157 Posts	Do any motherboards with NICs built in also support a boot rom?

2002-08-31, 02:41	#8
Lord_Humungous Aug 2002 23 Posts	Looks like you'd have to hack the bios on the motherboard. This file explains it.. http://www.mersenneforum.org/attachm...pecialBrew.pdf