Web server clustering project

Web server clustering project

Post by not bi » Fri, 17 Jul 1998 04:00:00



I am starting the research into the design and implementation of a 3
node cluster to provide high availability web, database, and support
services to a computer based physics lab. As envisioned, the primary
interface machine will be the web server. The database that provides
the dynamic web pages will be on a separate machine. Some other
processes that accept input from the web process and output to the
database will be on the third machine.

This system will provide computational, storage and interface services
for a daily grade for the students who use it. As such, reliability is
a top priority.  To accommodate that, I am planning on implementing as
much of the High Availability  capabilities as are feasible.

I have been reading all I can find about Linux clusters. Most of the
info I have found concerns parallel processing. This is not a parallel
processing system. All three types of services will be up and running
on all three machine at all times. The hard drive system will need to
be common to all three. Essentially, in the event of a failure of any
node, the remaining nodes will take over the process from the failed
node with as little effect on the user as possible. Each node has a
principle task responsibility. The other tasks are simply idle until
needed due to node failure. The only node that actually access the
(raid 1) hard drives is the database node. Available memory on a
non-database node can be used to cache database writes to facilitate
the recovery of data from a database node failure.

I've opened a HUGE can of worms here ;-)

I guess what I need to do is write a daemon that is identical in
function for each node, but configured through a startup script to
specify what task has priority. It should monitor a system heartbeat,
probably dedicated ethernet port, from each other node to determine
status and also for message passing. It should be able to bring up an
idle process when a node fails and allocate hardware resources for the
new configuration. A potential problem is with the IP address. If the
DB node fails, the user machine will not see the node change addresses
since they never directly access the DB node. Likewise for the node
that handles the support processes (LaTex, GNU plot, others). If the
web server fails, that presents a more serious problem. The client
machines, 2+ dozen mac's, will be on a private subnet, 192.168., and
they can't update their tables without a reboot. So the new web node
_must_ have the same IP as the failed web node. To add even more
complexity, the cluster will also be the gateway for the Macs. It will
also be running a cron job to disable network access beyond the
cluster during normal lab hours.  I don't want my students surfing
instead of working. I have heard of IP shadowing, but have no
information.

So, I am open to suggestions, comments, info, links to sites, book
titles, etc. I have proposed a one year development time for the whole
cluster, with a single machine application prototype of the user
visible/used portion by around the Jan 1999. I love my job!

Jim Kinney M.S.
Educational Technology Specialist
Department of Physics
Emory University

 
 
 

Web server clustering project

Post by Matthew Hannig » Mon, 20 Jul 1998 04:00:00


NCR sell a product called LifeKeeper and other complementary
products to do the sort of things that you want to do.

They developed them for their big iron (unix) machines
and have ported them to NT.  

Not saying that you should try and port or anything but
reading up on it should give you ideas on implementation.

-Matt

 
 
 

1. Web server clustering project

I am starting the research into the design and implementation of a 3
node cluster to provide high availability web, database, and support
services to a computer based physics lab. As envisioned, the primary
interface machine will be the web server. The database that provides
the dynamic web pages will be on a separate machine. Some other
processes that accept input from the web process and output to the
database will be on the third machine.

This system will provide computational, storage and interface services
for a daily grade for the students who use it. As such, reliability is
a top priority.  To accommodate that, I am planning on implementing as
much of the High Availability  capabilities as are feasible.

I have been reading all I can find about Linux clusters. Most of the
info I have found concerns parallel processing. This is not a parallel
processing system. All three types of services will be up and running
on all three machine at all times. The hard drive system will need to
be common to all three. Essentially, in the event of a failure of any
node, the remaining nodes will take over the process from the failed
node with as little effect on the user as possible. Each node has a
principle task responsibility. The other tasks are simply idle until
needed due to node failure. The only node that actually access the
(raid 1) hard drives is the database node. Available memory on a
non-database node can be used to cache database writes to facilitate
the recovery of data from a database node failure.

I've opened a HUGE can of worms here ;-)

I guess what I need to do is write a daemon that is identical in
function for each node, but configured through a startup script to
specify what task has priority. It should monitor a system heartbeat,
probably dedicated ethernet port, from each other node to determine
status and also for message passing. It should be able to bring up an
idle process when a node fails and allocate hardware resources for the
new configuration. A potential problem is with the IP address. If the
DB node fails, the user machine will not see the node change addresses
since they never directly access the DB node. Likewise for the node
that handles the support processes (LaTex, GNU plot, others). If the
web server fails, that presents a more serious problem. The client
machines, 2+ dozen mac's, will be on a private subnet, 192.168., and
they can't update their tables without a reboot. So the new web node
_must_ have the same IP as the failed web node. To add even more
complexity, the cluster will also be the gateway for the Macs. It will
also be running a cron job to disable network access beyond the
cluster during normal lab hours.  I don't want my students surfing
instead of working. I have heard of IP shadowing, but have no
information.

So, I am open to suggestions, comments, info, links to sites, book
titles, etc. I have proposed a one year development time for the whole
cluster, with a single machine application prototype of the user
visible/used portion by around the Jan 1999. I love my job!

Jim Kinney M.S.
Educational Technology Specialist
Department of Physics
Emory University

2. forgot login name - Help RH8

3. Spanish project to build powerful web servers based on Linux Cluster Technology

4. Linux on a NexGen ?

5. session replication for clustered apache web servers

6. ld error with Slackware 3.0

7. Set up high availability web server clusters with Understudy!

8. internet router

9. red hat cluster web server

10. "fallback" web server - How? (without clustering)

11. Announcing the Apache HTTP web server project

12. AUS-Melbourne Full time Network and Server manager for web project

13. Sun Cluster 3.0 vs Veritas Cluster Server