Poor Apache 2 performance (relative to 1)

Poor Apache 2 performance (relative to 1)

Post by Micha » Sat, 05 Jul 2003 12:21:47



Dear All,

We have 4 (identical) machines (Solaris 9 512MB SunV100) that are
behind a BIG-IP load balancer.  Three (3) of the machines run Apache
1.3.27 the fourth runs Apache 2.0.45.  Apache 2 has been configured to
use worker threads.  The issue that we are having is that the
performance of Apache 2 is significantly lower (35%-50%) than Apache
1.
On Apache 1 we currently have MaxClients set to 1024, which is where
the machines start to run out of memory.
On Apache 2 we have the following setup
StartServers        100
MaxClients          4000
MinSpareThreads     100
MaxSpareThreads     100
ThreadsPerChild     100
MaxRequestsPerChild  0
The Apache 2 machine never seems to be in trouble with memory or cpu.
So I am wondering why it does not seem to be able to keep up with
Apache 1.  In theory we should be able to serve more users per MByte
of memory with Apache 2, but this is not proving to be the case.  Can
anyone suggest a configuration change that may help??

Cheers,

Michael.

 
 
 

Poor Apache 2 performance (relative to 1)

Post by Joshua Sli » Sat, 05 Jul 2003 23:38:00



> On Apache 1 we currently have MaxClients set to 1024, which is where
> the machines start to run out of memory.
> On Apache 2 we have the following setup
> StartServers        100
> MaxClients          4000
> MinSpareThreads     100
> MaxSpareThreads     100
> ThreadsPerChild     100
> MaxRequestsPerChild  0
> The Apache 2 machine never seems to be in trouble with memory or cpu.
> So I am wondering why it does not seem to be able to keep up with
> Apache 1.  In theory we should be able to serve more users per MByte
> of memory with Apache 2, but this is not proving to be the case.  Can
> anyone suggest a configuration change that may help??

I don't think that configuration makes any sense.  You should be seeing
complaints in the error_log.

For example, you are starting StartServers*ThreadsPerChild threads
at startup.  That is 10,000! which is greater than MaxClients.

Plus you also have MinSpareThreads=MaxSpareThreads, which means you are
forcing Apache to constantly keep EXACTLY 100 idle threads.

Finally, if your 1.3 machines tend to run out of memory at 1024 processes,
then asking for 4000 clients on 2.0 may be streching it.  Yes, threads are
more efficient than processes, but I'm not sure they are that much more
efficient.

See: http://httpd.apache.org/docs-2.0/mod/worker.html

Try something like this:
StartServers        5
MaxClients          2000
MinSpareThreads     25
MaxSpareThreads     200
ThreadsPerChild     100
MaxRequestsPerChild  0

And make sure you monitor the memory usage to assure you aren't swapping.

Joshua.

 
 
 

Poor Apache 2 performance (relative to 1)

Post by Micha » Wed, 09 Jul 2003 11:00:20




> > On Apache 1 we currently have MaxClients set to 1024, which is where
> > the machines start to run out of memory.
> > On Apache 2 we have the following setup
> > StartServers        100
> > MaxClients          4000
> > MinSpareThreads     100
> > MaxSpareThreads     100
> > ThreadsPerChild     100
> > MaxRequestsPerChild  0
> > The Apache 2 machine never seems to be in trouble with memory or cpu.
> > So I am wondering why it does not seem to be able to keep up with
> > Apache 1.  In theory we should be able to serve more users per MByte
> > of memory with Apache 2, but this is not proving to be the case.  Can
> > anyone suggest a configuration change that may help??

> I don't think that configuration makes any sense.  You should be seeing
> complaints in the error_log.

> For example, you are starting StartServers*ThreadsPerChild threads
> at startup.  That is 10,000! which is greater than MaxClients.

> Plus you also have MinSpareThreads=MaxSpareThreads, which means you are
> forcing Apache to constantly keep EXACTLY 100 idle threads.

> Finally, if your 1.3 machines tend to run out of memory at 1024 processes,
> then asking for 4000 clients on 2.0 may be streching it.  Yes, threads are
> more efficient than processes, but I'm not sure they are that much more
> efficient.

> See: http://httpd.apache.org/docs-2.0/mod/worker.html

> Try something like this:
> StartServers        5
> MaxClients          2000
> MinSpareThreads     25
> MaxSpareThreads     200
> ThreadsPerChild     100
> MaxRequestsPerChild  0

> And make sure you monitor the memory usage to assure you aren't swapping.

> Joshua.

Joshua,

Thanks for the input.  You are right that the values of the parameters
do not make sense.  The correct values are:
StartServers        100
ServerLimit         100
MaxClients          6400
MinSpareThreads     10
MaxSpareThreads     100
ThreadsPerChild     64
MaxRequestsPerChild  0

However the problem is not so much with the configuration of the
worker threads but the performance in general.  Apache 2 performance
is still a long way from Apache 1.  Note that with the above
configuration the machine has around 70MBytes free memory, and CPU
utilisation is low (20-30 %).  I get the feeling that I am missing
something.

Thanks,

Michael.

 
 
 

Poor Apache 2 performance (relative to 1)

Post by Joshua Sli » Wed, 09 Jul 2003 22:53:58



> Thanks for the input.  You are right that the values of the parameters
> do not make sense.  The correct values are:
> StartServers        100
> ServerLimit         100
> MaxClients          6400
> MinSpareThreads     10
> MaxSpareThreads     100
> ThreadsPerChild     64
> MaxRequestsPerChild  0

> However the problem is not so much with the configuration of the
> worker threads but the performance in general.  Apache 2 performance
> is still a long way from Apache 1.  Note that with the above
> configuration the machine has around 70MBytes free memory, and CPU
> utilisation is low (20-30 %).  I get the feeling that I am missing
> something.

I am not an expert in high-performance web serving, but I can make a
couple comments:

1. If apache 1.3 can handle around 1024 clients in your setup, expecting
apache 2 to handle 6400 seems unreasonable.  Just the context switching
alone could be deadly.  Try bringing that back down to the 1024 range
and then increasing it slowly.

2. Unless this server is instantly hit with 6400 clients the moment it
starts, you are asking for a ton of thrashing at start-up.  The
server starts 6400 threads, and then must kill them all until only
100 are free.  I'd suggest setting StartServers much lower and
MaxSpareThreads much higher.

3. To figure out what is really going wrong, you may need to do
substantially more analysis.  Just saying "performance is 35-50% lower"
is practically meaningless.  Are you talking about hits/second?  With
the same number of simultaneous clients?  What kind of load?  What
kind of content?  What kind of network?  What does your server-status
display look like?  etc...

Joshua.

 
 
 

Poor Apache 2 performance (relative to 1)

Post by Micha » Thu, 10 Jul 2003 09:23:16




> > Thanks for the input.  You are right that the values of the parameters
> > do not make sense.  The correct values are:
> > StartServers        100
> > ServerLimit         100
> > MaxClients          6400
> > MinSpareThreads     10
> > MaxSpareThreads     100
> > ThreadsPerChild     64
> > MaxRequestsPerChild  0

> > However the problem is not so much with the configuration of the
> > worker threads but the performance in general.  Apache 2 performance
> > is still a long way from Apache 1.  Note that with the above
> > configuration the machine has around 70MBytes free memory, and CPU
> > utilisation is low (20-30 %).  I get the feeling that I am missing
> > something.

> I am not an expert in high-performance web serving, but I can make a
> couple comments:

> 1. If apache 1.3 can handle around 1024 clients in your setup, expecting
> apache 2 to handle 6400 seems unreasonable.  Just the context switching
> alone could be deadly.  Try bringing that back down to the 1024 range
> and then increasing it slowly.

> 2. Unless this server is instantly hit with 6400 clients the moment it
> starts, you are asking for a ton of thrashing at start-up.  The
> server starts 6400 threads, and then must kill them all until only
> 100 are free.  I'd suggest setting StartServers much lower and
> MaxSpareThreads much higher.

> 3. To figure out what is really going wrong, you may need to do
> substantially more analysis.  Just saying "performance is 35-50% lower"
> is practically meaningless.  Are you talking about hits/second?  With
> the same number of simultaneous clients?  What kind of load?  What
> kind of content?  What kind of network?  What does your server-status
> display look like?  etc...

> Joshua.

Joshua,

Thanks again for your input.  In response to your comments I have the
following to contribute.

The network consists of 4 identical machines that are being used to
host a live sports site, the site provides information about current
games in progress.  The machines only serve up the same static html.
All machines are behind a load balancer.  All machines have been TCP
tuned in the same way.

1. Setting Apache 2 to handle 6400 clients was based on the same logic
as for sizing Apache 1.  Increase MaxClients until almost all of the
physical memory is used up.  We want to be able to handle as many
clients as possible.

2. The machine only serves up information when live games are on.  And
when games are on the load is large, so we start at the maximum.

3. Saying that the load is 35-50% lower is not really meaningless, as
the machine is being load balanced with 3 other identical machines.
The figures are based on log file analysis on the 4 machines.  And the
machine running Apache 2 services 35-50% fewer clients than the Apache
1 machines.  The content being served is identical on all 4 machines.
Memory and CPU usage on the Apache 2 machine do not appear to be an
issue (using 'top' and 'vmstat').  Logic, says that if the machine
(Apache 2) is not swapping and there is spare CPU around then it
should be able to service more clients.  This does not appear to be
the case.  It appears to me that I am missing something?

I hope this is clear enough?

Thanks again,

Michael.

 
 
 

Poor Apache 2 performance (relative to 1)

Post by Paul Rubi » Thu, 10 Jul 2003 09:58:19



> The network consists of 4 identical machines that are being used to
> host a live sports site, the site provides information about current
> games in progress.  The machines only serve up the same static html.

If you're only serving static html, why not use Tux or thttpd or
something like that instead of Apache?
 
 
 

1. Poor Apache performance

I am running Apache 1.1.1 on a sparc 5 (solaris 2.4) model 85 with 128
meg of ram. We currently have 15 Virtural web servers running. The
problem i have experence is the transfer rates from our server ranges
from 100kbs to 900kbs. With a 28.8 i would expect 2.0-3.0kbs rates.
There are other process running, but most are idle.  Is there a config
problem here that i missed in the doc. Why is the server running so
slow. My laptop running NT4.0 with 32 meg ram runs faster then this.
Can anyone offer help in this.

Thanks

Kevin Billings
Augusta.Net, Inc.

2. FTP death spiral

3. Apache 2: poor performance on Solaris

4. NROFF/TROFF for LaserWriter

5. Apache 1.0.5 poor performance

6. A problem: WIN NT (spit) cannot see Linux/SAMBA

7. Poor(?) Linux Apache performance

8. vidio update driver install

9. Poor HTTP based file upload performance to Apache

10. Poor Apache performance on Solaris x86 vs Linux

11. HELP: very poor apache performance....

12. Relative performance between ext2 and ext3 file systems.

13. Relative performance of Solaris/SunMotif vs. SunOS/ICS Motif