Hardware recommendation for a 64-node PC cluster

Hardware recommendation for a 64-node PC cluster

Post by Tony Jacob » Sun, 05 Sep 2004 07:27:04



Need some recommendations on building a 64-node cluster for my school's
Biometrics lab. The funding for the project is limited (<$100K). I am
thinking about utilizing commercially available off-the-self motherboards,
processor, memory and PC chassis for the project. The cluster will be
running under Linux O.S. with Linux-based MPI applications. Any suggestions
on processors, motherboards and chassis will be greatly appreciated. Thanks!

Tony

 
 
 

Hardware recommendation for a 64-node PC cluster

Post by A.zedil » Sun, 05 Sep 2004 12:36:27


You might want to consider using a 1U or specialized chassis for your
64-node PC cluster. A PC chassis might be okay, if you don't have the
budget.  If you do decide to use a 1U or a specialized chassis, you
might want to take a look at ATXBlade from Rackmount.com.  It is a
blade like chassis. You will find more information at following URL:

<http://www.rackmount.com/Rackmt/ATXBladeS100.htm>

Allen


> Need some recommendations on building a 64-node cluster for my school's
> Biometrics lab. The funding for the project is limited (<$100K). I am
> thinking about utilizing commercially available off-the-self motherboards,
> processor, memory and PC chassis for the project. The cluster will be
> running under Linux O.S. with Linux-based MPI applications. Any suggestions
> on processors, motherboards and chassis will be greatly appreciated. Thanks!

> Tony


 
 
 

Hardware recommendation for a 64-node PC cluster

Post by Vladimir Florinsk » Sun, 05 Sep 2004 15:54:34



> Need some recommendations on building a 64-node cluster for my school's
> Biometrics lab. The funding for the project is limited (<$100K). I am
> thinking about utilizing commercially available off-the-self
> motherboards, processor, memory and PC chassis for the project. The
> cluster will be running under Linux O.S. with Linux-based MPI
> applications. Any suggestions on processors, motherboards and chassis
> will be greatly appreciated. Thanks!

If you only need 64 CPUs, build a 32-node cluster of SMP machines. Equip
each node with a Tyan Tiger dual Opteron board, 2 CPUs (Opteron 246 or
248) and sufficient memory (say, 2 GB). You don't need video cards or
disks, except for the master node. Gigabit NICs are included on board the
Tyans. For enclosures, I would recommend 2U rackmount chassis. Each node
should come under $2000 which will leave you with plenty of cash for a
gigabit switch. And remember that while building the cluster is
relatively trivial, maintenance, repair and software support will take a
lot of your time.

--
Vladimir

 
 
 

Hardware recommendation for a 64-node PC cluster

Post by Al Dyk » Sun, 05 Sep 2004 20:56:00




>You might want to consider using a 1U or specialized chassis for your
>64-node PC cluster. A PC chassis might be okay, if you don't have the
>budget.  If you do decide to use a 1U or a specialized chassis, you
>might want to take a look at ATXBlade from Rackmount.com.  It is a
>blade like chassis. You will find more information at following URL:

><http://www.veryComputer.com/;

>Allen



>> Need some recommendations on building a 64-node cluster for my school's
>> Biometrics lab. The funding for the project is limited (<$100K). I am
>> thinking about utilizing commercially available off-the-self motherboards,
>> processor, memory and PC chassis for the project. The cluster will be
>> running under Linux O.S. with Linux-based MPI applications. Any suggestions
>> on processors, motherboards and chassis will be greatly appreciated. Thanks!

>> Tony

Give some thought to amount of shelving you need, which will determine
floor space. Then think about how you are going to do the the power
wiring. You need at least 6 20-AMP circuits for this puppy. Put it in
a room with enough A/C to keep it cool.

You can get cheap grey steel shelving and space the shelves so that
you can use any mini-tower you can get your hands on, or you can use
standard computer racks and shelves with either 1U/2U rack cases or
use micro-atx desktop cases like this and put them on edge of rack
shelves, 4 per shelf (stacking computer boxes makes it really hard to
work on anything but the top box, IMO.)  

http://www.veryComputer.com/
&depa=0

By my calculation you could get 20 of these cases in one 7ft-high
rack; 19inch Rack+5 shelves+rack power strips ; about $600US.  You
need 4 of these.    About 24 square ft of floorspace.

With Micro-ITX route the case/PSU is $50US and there are a bunch of
microatx mobos for about $50. Look at newegg for them.

The cheapest 1U case I've seen, lately:

http://www.veryComputer.com/
&DEPA=0

You can get 42 of these in a rack, and you don't need shelves. I'm
told cooling can be a problem if you max-out the rack.  I'd build one
rack-full (42) and operate it for a while before I decided if I needed
3, 3, or 4 racks to hold the whole cluster.

If you don't get the power amd mechanical stuff right you'll have an
unreliable cluster that's a pain to work on.

There's an O'Rielly book on Beowolf cluster that I'm sure you already
know about.

I know that switch latency is a big deal with high performance clusters
and that low-latency switches are expensive.  

It's possible that inter-box data rate is the bottleneck for your
cluster and that some on-mobo GbE interfaces are much faster than
others, or cheap PCI cards, because they have a direct memory
interface and don't use the PCI bus, which can't fill a GbE
pipe. Newegg lists 33 mobos (AMD) that have on-mobo GbE, from $70US to
$170US.

--
Al*s
-----------
adykes  at p a n i x . c o m

 
 
 

Hardware recommendation for a 64-node PC cluster

Post by Steve Wolf » Mon, 06 Sep 2004 07:37:06


Quote:> If you only need 64 CPUs, build a 32-node cluster of SMP machines. Equip
> each node with a Tyan Tiger dual Opteron board, 2 CPUs (Opteron 246 or
> 248) and sufficient memory (say, 2 GB). You don't need video cards or
> disks, except for the master node. Gigabit NICs are included on board the
> Tyans. For enclosures, I would recommend 2U rackmount chassis. Each node
> should come under $2000 which will leave you with plenty of cash for a
> gigabit switch. And remember that while building the cluster is
> relatively trivial, maintenance, repair and software support will take a
> lot of your time.

  Alternatively, if time isn't a concern, the upcoming Horus-based systems
from Newisys and AMD's dual-core chips will allow 8 4-socket machines with
dual-core chips (64 total CPUs) to be tied together into a single system.
Nifty stuff!

 Info here:

http://mywebpages.comcast.net/davewang202/newisys/HorusHotChips2004.pdf

steve

 
 
 

1. Looking for computer case/chassis recommendations for a 32-node PC cluster

Looking for computer case/chassis recommendations for a 32-node PC cluster.
I am building a 32-node PC cluster for my school's MPI programming lab?  I
am planning to put a single Intel P4 3.2Ghz processor MB, 2GB DDR2 RAM, 40GB
HD (for booting), dual Gigabit LAN and Video in each cluster.  Thanks for
your help!

Dave

2. flashing monitor

3. Converting an existing 32-node PC cluster

4. Looking for less 252

5. Cluster hardware: comments, recommendations, criticisms

6. use library

7. VERITAS cluster server vs. SUN cluster -- Any recommendations ?

8. Trouble with CHAP & PAP

9. Recommendations for Solaris x86 PC hardware?

10. How to recover an application node and an ADSM node into one node?

11. Linux/mips64 on 64 node, 128p, 64G machine

12. Reduce the radix tree nodes to 64 slots

13. Q: What is the minimum Sol7 S/W cluster that will boot 64 bit mode?