Apache: child processes don't exit

Apache: child processes don't exit

Post by Jens Sieckman » Sat, 26 Apr 2003 23:39:05



Hello,

I'm using the following web-system
  - Red Hat Linux release 7.3 (Valhalla)
  - Apache 1.3.27
  - mod_ssl 2.8.12, OpenSSL/0.9.6b (normally I compiled OpenSSL 0.9.7a
    in mod_ssl with --with-ssl on the source dir, but it always compiles
    th standard OpenSSL 0.9.6b which was installed during system
    installation. I can see this in the Apache log:
    mod_ssl/2.8.12 OpenSSL/0.9.6b)
  - PHP/4.3.1

Besides I use MySQL and a template system (APL), that is implemented
as a PHP extension (in C++).

In irregular intervals (nothing during 3 days, sometimes 6 times a day)
the load of the webserver increases endlessly (normally between 0.5 and
1.5) because some httpd child processes don't exit. In the apache error
log messages occur like:

[warn] child process 2629 still did not exit, sending a SIGTERM

and later:

[error] child process 2629 still did not exit, sending a SIGKILL

Before these messages sometimes but not always segmentation fault-
errors occur.

But the SIGKILL signal don't seems to have any impact, because the
httpd processes pile up and absorb cpu time and memory until the
webserver is unusable.

As a workaround I wrote a simple daemon-script that restarts the webserver
if the load becomes too high. But this should'nt be the solution.

Did anyone hear of such problems or knows any solutions: other software
versions or webserver configurations?

Thanks in advance,
Jens

 
 
 

Apache: child processes don't exit

Post by Chris Morri » Sat, 26 Apr 2003 23:49:20



> In irregular intervals (nothing during 3 days, sometimes 6 times a day)
> the load of the webserver increases endlessly (normally between 0.5 and
> 1.5) because some httpd child processes don't exit. In the apache error
> log messages occur like:

> [warn] child process 2629 still did not exit, sending a SIGTERM
> and later:
> [error] child process 2629 still did not exit, sending a SIGKILL

> Before these messages sometimes but not always segmentation fault-
> errors occur.

> But the SIGKILL signal don't seems to have any impact, because the
> httpd processes pile up and absorb cpu time and memory until the
> webserver is unusable.

If SIGKILL isn't working it's a fairly impressively wrong process.
I've seen similar before due to file locking problems, but no doubt
there's other causes.  Does a manual kill -9 as root get rid of them?

http://localhost/server-status (assuming you enable this in
httpd.conf) will tell you the last request served by each child
process.  Is it the same page or lots of different ones?

Quote:> As a workaround I wrote a simple daemon-script that restarts the webserver
> if the load becomes too high. But this should'nt be the solution.

Oh, so restarting the main apache server *does* kill them off?  Even
stranger.

--
Chris

 
 
 

Apache: child processes don't exit

Post by Jens Sieckman » Sat, 26 Apr 2003 23:58:35



> http://localhost/server-status (assuming you enable this in
> httpd.conf) will tell you the last request served by each child
> process.  Is it the same page or lots of different ones?

I've to figure that out.

Quote:> Oh, so restarting the main apache server *does* kill them off?  Even
> stranger.

I make a killall httpd.

Greets,
Jens

 
 
 

Apache: child processes don't exit

Post by Nick K » Sun, 27 Apr 2003 03:31:53




Quote:> In irregular intervals (nothing during 3 days, sometimes 6 times a day)
> the load of the webserver increases endlessly (normally between 0.5 and
> 1.5) because some httpd child processes don't exit. In the apache error
> log messages occur like:

That's a seriously unstable server.  Is it all compiled from clean source?
My guess would be you have a module with a resource (of some kind) leak
that's declining to exit cleanly.  I've had somewhat-similar behaviour
when using modules before they're fully debugged.

Quote:> But the SIGKILL signal don't seems to have any impact, because the
> httpd processes pile up and absorb cpu time and memory until the
> webserver is unusable.

That used to happen quite a lot with some earlier deadrat (I think
5.x or 6.x), but I've not heard of it recently.

--
Nick Kew

Available for contract work - Programming, Unix, Networking, Markup, etc.

 
 
 

Apache: child processes don't exit

Post by Joachim Ri » Tue, 29 Apr 2003 06:26:13


Quote:> I'm using the following web-system
>   - Red Hat Linux release 7.3 (Valhalla)
>   - Apache 1.3.27
>   - mod_ssl 2.8.12, OpenSSL/0.9.6b (normally I compiled OpenSSL 0.9.7a
>     in mod_ssl with --with-ssl on the source dir, but it always compiles
>     th standard OpenSSL 0.9.6b which was installed during system
>     installation. I can see this in the Apache log:
>     mod_ssl/2.8.12 OpenSSL/0.9.6b)

erm, --with-ssl should not point to the openssl 0.9.7a source
directory (btw, b is out already due to some remotely possible
attacks) but to the place you installed it in (you _did_ make
configure, make, make test and make install openssl 0.9.7a, right?).

by default this would be /usr/local/ssl, iirc...

btw, OpenSSL/0.9.6b is _really_ crufty and might well be the source
for your stability woes as well as some unwanted visitors with root
priviledges...

joachim

 
 
 

Apache: child processes don't exit

Post by Jens Sieckman » Tue, 29 Apr 2003 17:00:01



> That's a seriously unstable server.  Is it all compiled from clean source?
> My guess would be you have a module with a resource (of some kind) leak
> that's declining to exit cleanly.  I've had somewhat-similar behaviour
> when using modules before they're fully debugged.

I compliled everything from source. I only use mod_ssl and mod_php.

Greets, Jens

 
 
 

Apache: child processes don't exit

Post by Jens Sieckman » Tue, 29 Apr 2003 17:04:21



>>I'm using the following web-system
>>  - Red Hat Linux release 7.3 (Valhalla)
>>  - Apache 1.3.27
>>  - mod_ssl 2.8.12, OpenSSL/0.9.6b (normally I compiled OpenSSL 0.9.7a
>>    in mod_ssl with --with-ssl on the source dir, but it always compiles
>>    th standard OpenSSL 0.9.6b which was installed during system
>>    installation. I can see this in the Apache log:
>>    mod_ssl/2.8.12 OpenSSL/0.9.6b)

> erm, --with-ssl should not point to the openssl 0.9.7a source
> directory (btw, b is out already due to some remotely possible
> attacks) but to the place you installed it in (you _did_ make
> configure, make, make test and make install openssl 0.9.7a, right?).

Puh, error in the config-help?:

./configure --help
mod_ssl configure options:
   --with-apache=DIR       ...path to Apache 1.3.x source tree
   --with-apxs[=FILE]      ...path to APXS program
   --with-ssl=DIR          ...path to OpenSSL source tree
[..]

Jens

 
 
 

Apache: child processes don't exit

Post by Joachim Ri » Wed, 30 Apr 2003 01:45:40


Quote:> > erm, --with-ssl should not point to the openssl 0.9.7a source
> > directory (btw, b is out already due to some remotely possible
> > attacks) but to the place you installed it in (you _did_ make
> > configure, make, make test and make install openssl 0.9.7a, right?).

> Puh, error in the config-help?:

> ./configure --help
> mod_ssl configure options:
>    --with-apache=DIR       ...path to Apache 1.3.x source tree
>    --with-apxs[=FILE]      ...path to APXS program
>    --with-ssl=DIR          ...path to OpenSSL source tree

ok, seems i didn't remember this one right, sorry.

anyways there remains the question wether openssl 0.9.7a was properly
installed and/or why the modssl configure got the wrong openssl
version...

joachim

 
 
 

Apache: child processes don't exit

Post by Jens Sieckman » Wed, 30 Apr 2003 21:27:40



> ok, seems i didn't remember this one right, sorry.

> anyways there remains the question wether openssl 0.9.7a was properly
> installed and/or why the modssl configure got the wrong openssl
> version...

Yes, at the moment I try to figure that out. I use compile
openssl 0.9.7a and with modssl 2.8.14-1.3.27 (on my PI 133,
*yawn*)

Greets,
Jens

 
 
 

Apache: child processes don't exit

Post by Jens Sieckman » Wed, 30 Apr 2003 23:13:17



> Yes, at the moment I try to figure that out. I use compile
> openssl 0.9.7a and with modssl 2.8.14-1.3.27 (on my PI 133,
> *yawn*)

Now it works with 0.9.7b. When compiling I also set the env
variable SSL-BASE to the source dir of opennsl. Maybe the was
it.

Greets,
Jens

 
 
 

1. Getting a child and child's child process-id

I have a program that forks, creating a child, and this child forks
again, creating a child of its own.  Both child and sub-child continue
running for the life of the program.

I need for the main parent program to have the process-id of its child
(that's easy), and the process-id of the child's child.

The only way I can figure out how to do this is to create an unnamed
pipe befor the first child is created, and have the child's child write
its process-id into the pipe.

This seems like alot of overhead for one integer to be passed.  

Is there another way?  I am running on SVr3.1.

--
Bruce Momjian           | 830 Blythe Avenue               | home: (215)853-3000

2. xxgdb problem?

3. Does 4.x BSD swap text pages ?

4. Apache - child process did not exit

5. Slip link problem

6. Apache and 'couldn't spawn child process'

7. mod_perl or FastCGI?

8. Why Apache says 'reason: couldn't spawn child process'?

9. Apache: child process xxxxxx still did not exit

10. httpd - child processes don't die !!

11. 'Could not spawn child process' problem with Apache 1.26