LINUX ability to handle large beowulf clusters

LINUX ability to handle large beowulf clusters

Post by Jon Robert » Fri, 21 May 1999 04:00:00



I was talking with a friend who is researching a supercomputer design
that may become one of the top 10 machines in the world. I mentioned
Beaowulf Linux clustering and the response was that Linux is good but
can't support more than about 300 nodes in a cluster (and they plan on
being larger than that). Is this true? If there IS a way around this
limitation I'd like to know so I can make further recommendations for
Linux to him.
 
 
 

LINUX ability to handle large beowulf clusters

Post by Peter Englmaie » Fri, 21 May 1999 04:00:00


It's not that Linux is restricted. It is the PC's. I guess
the 'friend' was refering to the hardware of a baeowulf cluster which
is simply not the ultimate answer to supercomputing. Same true for
any clustered system. The kind of supercomputer that he had in mind
needs a fast memory backbone. And: all supercomputers use Unix
anyway, so why pushing Linux in? After all Linux is Unix. For
supercomputing there are specialised compilers, specialised Unixes,
and specialised memory architectures and they do not run X on it.

If you want to discuss this further: stay in c.o.l.advocacy where
this belongs to.


> What are the reasons for Linux not being able to support more than
> 300?
> I can't think of anything that has that limitation.

> >I was talking with a friend who is researching a supercomputer design
> >that may become one of the top 10 machines in the world. I mentioned
> >Beaowulf Linux clustering and the response was that Linux is good but
> >can't support more than about 300 nodes in a cluster (and they plan on
> >being larger than that). Is this true? If there IS a way around this
> >limitation I'd like to know so I can make further recommendations for
> >Linux to him.


 
 
 

LINUX ability to handle large beowulf clusters

Post by Brian Vicen » Sat, 22 May 1999 04:00:00


What are the reasons for Linux not being able to support more than
300?
I can't think of anything that has that limitation.

>I was talking with a friend who is researching a supercomputer design
>that may become one of the top 10 machines in the world. I mentioned
>Beaowulf Linux clustering and the response was that Linux is good but
>can't support more than about 300 nodes in a cluster (and they plan on
>being larger than that). Is this true? If there IS a way around this
>limitation I'd like to know so I can make further recommendations for
>Linux to him.

 
 
 

LINUX ability to handle large beowulf clusters

Post by Frank Sweetse » Sat, 22 May 1999 04:00:00



> I was talking with a friend who is researching a supercomputer design
> that may become one of the top 10 machines in the world. I mentioned
> Beaowulf Linux clustering and the response was that Linux is good but
> can't support more than about 300 nodes in a cluster (and they plan on
> being larger than that). Is this true? If there IS a way around this
> limitation I'd like to know so I can make further recommendations for
> Linux to him.

try checking out http://www.beowulf.org/

--
Frank Sweetser rasmusin at wpi.edu fsweetser at blee.net  | PGP key available
paramount.ind.wpi.edu RedHat 5.2 kernel 2.2.5        i586 | at public servers

Quote:> This made me wonder, suddenly: can telnet be written in perl?

Of course it can be written in Perl.  Now if you'd said nroff,
that would be more challenging...   -- Larry Wall
 
 
 

LINUX ability to handle large beowulf clusters

Post by Joachim Zob » Tue, 25 May 1999 04:00:00



>I was talking with a friend who is researching a supercomputer design
>that may become one of the top 10 machines in the world. I mentioned
>Beaowulf Linux clustering and the response was that Linux is good but
>can't support more than about 300 nodes in a cluster (and they plan on
>being larger than that). Is this true? If there IS a way around this
>limitation I'd like to know so I can make further recommendations for
>Linux to him.

If you want to run that many nodes on pc hardware availability will be
a problem. How does beowulf handle this? If you have 300 nodes and the
whole system is down if one fails there might not be much uptime.

Hth,
Joachim

 --
"I read the news today oh boy"             - The Beatles - A Day In The Life

Althoug this message has a valid From header, replies

are preferred.

 
 
 

LINUX ability to handle large beowulf clusters

Post by Chris Fros » Tue, 25 May 1999 04:00:00



Quote:> If you want to run that many nodes on pc hardware availability will be
> a problem. How does beowulf handle this? If you have 300 nodes and the
> whole system is down if one fails there might not be much uptime.

For the clusters which I have setup we've always done things so that no
one box is essential (except for one cluster where we only had one box
which controlled everything, but then it would only be that one box which
was essential). Maybe the person who asked is doing it a different way,
but you may want to see what could be done so as to rid whatever makes you
dependent upon every box being up.

--
- Chris
-- Visit Me at http://www.frostnet.advicom.net --

 
 
 

LINUX ability to handle large beowulf clusters

Post by Josef M?ller » Thu, 27 May 1999 04:00:00



> I was talking with a friend who is researching a supercomputer design
> that may become one of the top 10 machines in the world. I mentioned
> Beaowulf Linux clustering and the response was that Linux is good but
> can't support more than about 300 nodes in a cluster (and they plan on
> being larger than that). Is this true? If there IS a way around this
> limitation I'd like to know so I can make further recommendations for
> Linux to him.

There was a "Linux Cluster Event" run by a German TV station (WDR)
during its "ComputerNacht" event last december. See
http://www.uni-paderborn.de/cs/heiss/linux/cluster/artikel/cluster.html
(Sorry, but since the event is so long ago, there is little information
left, so all I could find was this German article)

They connected more than 512 Linux systems to solve some equations and
render a couple of films. Apparently they made it into the Guinness Book
of Records ...

Hope this helps,

Josef
--
PS Die hier dargestellte Meinung ist die persoenliche Meinung des
Autors!
PS This article reflects the autors personal views only!

 
 
 

LINUX ability to handle large beowulf clusters

Post by m.benndor » Fri, 28 May 1999 04:00:00



> There was a "Linux Cluster Event" run by a German TV station (WDR)
> during its "ComputerNacht" event last december. See
> http://www.uni-paderborn.de/cs/heiss/linux/cluster/artikel/cluster.html
> (Sorry, but since the event is so long ago, there is little information
> left, so all I could find was this German article)

the official homepage for that is now:

http://www.linux-cluster.org/

but it's all in german

mike

 
 
 

LINUX ability to handle large beowulf clusters

Post by The Ghost In The Machi » Fri, 28 May 1999 04:00:00


On Thu, 27 May 1999 18:47:21 +0200,



>> There was a "Linux Cluster Event" run by a German TV station (WDR)
>> during its "ComputerNacht" event last december. See
>> http://www.uni-paderborn.de/cs/heiss/linux/cluster/artikel/cluster.html
>> (Sorry, but since the event is so long ago, there is little information
>> left, so all I could find was this German article)

>the official homepage for that is now:

>http://www.linux-cluster.org/

>but it's all in german

Throw it at

http://babelfish.altavista.digital.com

then. :-)

Quote:

>mike

----

 
 
 

1. help with linux clustering (BEOWULF)

Hi Everyone,

I would like to thank everyone for helping me with setting the new
netfilter script for my linux box (Redhat 7 with kernel 2.4.2).  The box
is running very smooth without a glitch.  

Now I know more about linux and its capability, I am even more
interested in learning more about it especially the linux clustering
capability.  Let me describe what I would like to achieve and ask
for everyone advice whether it is feasible.

At the moment, I am running RedHat Linux 7 with kernel 2.4.2 as
a gateway (firewall and NAT) on a Pentium 233Mhz/64MB RAM
for my home network which connects to the Internet via a DSL
connection with a static IP address.  I have 3 windows machines
sitting behind the firewall that connectto the Internet via NAT
from the Linux box.  I am also running ssh server, sendmail, NTP
server and webmin on the gateway.  My brother just gives me his
old PC (200Mhz/64MB RAM) and I would like to use this
opportunity to learn linux clustering and apply this to the gateway
for redudancy and hopefully many more.

I would like to be able to cluster the 200Mhz/64mb RAM machine
with my current 233Mhz machine.  I am willing to reinstall the whole
thing from scractch in order to learn BEOWULF technology.  The
way I understand about BEOWULF is that it is similar to
parallel computing.  Therefore, am I correct in assuming that if one
of the box in a cluster dies, my gateway is still functioning.  Does
it mean that I will have redudancy in the gateway (with the
exception of the DSL connection).  I would like to learn more about
the hand-on with linux cluster.  Could someone please point me
to the right direction?  

Many Thanks....
David

2. WANTED: Desktop Computer Users

3. reinstalling lilo

4. NYC LOCAL: Thursday 15 March 2001 UNIGROUP: Jozef Skvarcek on GNU/Linux Beowulf Clusters

5. Q: Solaris Interrupts

6. help with linux clustering (BEOWULF)

7. PC clients not browsing across FreeBSD PPP link

8. Q: Handling large mail loads....how to distribute mail handling??

9. Beowulf cluster setup

10. beowulf clusters

11. Beowulf clusters

12. "Building Beowulf Clusters"