Overworked DNS stops answering

Overworked DNS stops answering

Post by Arne K. Haaj » Thu, 23 Sep 1999 04:00:00



Hello,

We have an overworked server running both DNS for 15000 zones (both
slave and master) , and Apache for 300 domains.

Kernel is 2.2.12. Server is P2 400Mhz, 192MB RAM with Mylex DAC960 RAID.
Bandwith to the Internet is 1MBs, and LAN is 16Mb Token Ring.

Recently the DNS stopped answering queries from nslookup (UDP), but
zone-transfers (TCP) work fine. We will setup a dedicated DNS on a new
server, but in the meantime I need a solution.

Can this have something to do with running out of filedescriptors ?
Apache consumes a lot for its logfiles, but I am not sure about named.
Is there something I can do to tune the system ?

Any suggestions would be much appreciated!

Arne
--
--------------------------------
Arne K. Haaje   | T: 69 92 04 90
Enebakkveien 2  | F: 69 92 04 91
N-1825 Tomter   | M: 92 88 44 66
--------------------------------
The system needed Windows 95
or better, so I installed Linux!

 
 
 

Overworked DNS stops answering

Post by bill davids » Fri, 24 Sep 1999 04:00:00



| We have an overworked server running both DNS for 15000 zones (both
| slave and master) , and Apache for 300 domains.
|
| Kernel is 2.2.12. Server is P2 400Mhz, 192MB RAM with Mylex DAC960 RAID.
| Bandwith to the Internet is 1MBs, and LAN is 16Mb Token Ring.
|
| Recently the DNS stopped answering queries from nslookup (UDP), but
| zone-transfers (TCP) work fine. We will setup a dedicated DNS on a new
| server, but in the meantime I need a solution.

  If the performance is okay on the existing server, why bother?

| Can this have something to do with running out of filedescriptors ?
| Apache consumes a lot for its logfiles, but I am not sure about named.
| Is there something I can do to tune the system ?

  Look at /proc/sys/fs/file-nr. It should have three numbers, which from
inspection are the max files ever open, the number of unused file
descriptors, and the max files allowed at present. If the first and last
numbers are close or the same, add 50% to the 2nd number and stare it in
the file-max.

This makes it clearer:
  # cd /proc/sys/fs
  # cat file-max
  14782 12004 15000
  # echo 17000 > file-max
  # cat file-nr
  14782 14211 17000

  I find I have to do this more than I would like, so I had the info "in
cache" on this. When you find a value which is good and leaves some room
for growth put the echo in a file run at boot time.

  If you are worried, and you should be on a critical server, a small
perl script will check this and warn if the unused is less than 10% of
the max, or whatever you think the threshhold should be.

--

  When taking small children to a carnival, always have them go potty
*before* you let them go on the rides, and let them eat all the junk
food and candy *after*.

 
 
 

Overworked DNS stops answering

Post by David Schwart » Sun, 26 Sep 1999 04:00:00



> Recently the DNS stopped answering queries from nslookup (UDP), but
> zone-transfers (TCP) work fine. We will setup a dedicated DNS on a new
> server, but in the meantime I need a solution.

        The DNS server needs to bind to each IP address on the machine
individually. It's probably having trouble binding to the 300+ IPs on
it. The best work around is not to run a name server on a machine with
more than 100 IP addresses.

        DS

 
 
 

Overworked DNS stops answering

Post by Bob Hauc » Tue, 28 Sep 1999 04:00:00



> > Recently the DNS stopped answering queries from nslookup (UDP), but
> > zone-transfers (TCP) work fine. We will setup a dedicated DNS on a new
> > server, but in the meantime I need a solution.

>    The DNS server needs to bind to each IP address on the machine
> individually. It's probably having trouble binding to the 300+ IPs

Or add "listen-on" to the options in named.conf (bind 8.x):

options {
        ...
        listen-on {192.168.1.1};

Quote:}

This will prevent it trying to bind 300 IP's.  In most cases this is
what you want as you have specified only one or a few of those 300 to be
a DNS.

--
 -| Bob Hauck
 -| Wasatch Communications Group
 -| http://www.wasatch.com/~bobh

 
 
 

Overworked DNS stops answering

Post by Bob Hauc » Tue, 28 Sep 1999 04:00:00



Quote:> options {
>    ...
>    listen-on {192.168.1.1};
> }

Please add a semicolon (;) after "192.168.1.1" but before the '}'.

--
 -| Bob Hauck
 -| Wasatch Communications Group
 -| http://www.wasatch.com/~bobh

 
 
 

Overworked DNS stops answering

Post by Arne K. Haaj » Wed, 29 Sep 1999 04:00:00




> > > Recently the DNS stopped answering queries from nslookup (UDP), but
> > > zone-transfers (TCP) work fine. We will setup a dedicated DNS on a new
> > > server, but in the meantime I need a solution.

> >       The DNS server needs to bind to each IP address on the machine
> > individually. It's probably having trouble binding to the 300+ IPs

> Or add "listen-on" to the options in named.conf (bind 8.x):

> options {
>         ...
>         listen-on {192.168.1.1};
> }

> This will prevent it trying to bind 300 IP's.  In most cases this is
> what you want as you have specified only one or a few of those 300 to be
> a DNS.

I use NamedVirtualHost for Apache, so named only listens on 3
interfaces.

Arne

 
 
 

Overworked DNS stops answering

Post by bill davids » Thu, 30 Sep 1999 04:00:00





|
| > Recently the DNS stopped answering queries from nslookup (UDP), but
| > zone-transfers (TCP) work fine. We will setup a dedicated DNS on a new
| > server, but in the meantime I need a solution.
|
|       The DNS server needs to bind to each IP address on the machine
| individually. It's probably having trouble binding to the 300+ IPs on
| it. The best work around is not to run a name server on a machine with
| more than 100 IP addresses.

  I'm not disagreeing with you, but how would this effect only nslookup
and not zone transfers? I would expect it to listen or not, but
selective listening seems to suggest another problem.

  There are problems with DNS returning an answer of greater than 512
bytes. Some clients will fail if this is the case. Could this be related
in some way? If the DNS was set up so that any one name lookup returned
an answer with IP addresses totalling more than 512 bytes some clients
would not handle it.

  I don't remember if the packet gets larger than 512 and overflows the
buffer, or if there is some form of continuation packet. In any case
clients do have this problem, I had a fight with that yesterday!

--

  When taking small children to a carnival, always have them go potty
*before* you let them go on the rides, and let them eat all the junk
food and candy *after*.

 
 
 

Overworked DNS stops answering

Post by David Schwart » Sat, 02 Oct 1999 04:00:00






> |
> | > Recently the DNS stopped answering queries from nslookup (UDP), but
> | > zone-transfers (TCP) work fine. We will setup a dedicated DNS on a new
> | > server, but in the meantime I need a solution.
> |
> |       The DNS server needs to bind to each IP address on the machine
> | individually. It's probably having trouble binding to the 300+ IPs on
> | it. The best work around is not to run a name server on a machine with
> | more than 100 IP addresses.

>   I'm not disagreeing with you, but how would this effect only nslookup
> and not zone transfers?

        Because TCP is connection-based, you can use 'getpeername' to find the
other end. So named does not have this same problem with TCP.

Quote:> I would expect it to listen or not, but
> selective listening seems to suggest another problem.

        Try querying it on several different IPs. You will likely find that
some work and some don't.

Quote:> There are problems with DNS returning an answer of greater than 512
> bytes. Some clients will fail if this is the case. Could this be related
> in some way? If the DNS was set up so that any one name lookup returned
> an answer with IP addresses totalling more than 512 bytes some clients
> would not handle it.

        You are overanalyzing a simple problem.

Quote:>   I don't remember if the packet gets larger than 512 and overflows the
> buffer, or if there is some form of continuation packet. In any case
> clients do have this problem, I had a fight with that yesterday!

        The RFC specifies that longer packets not be sent:

"Messages sent using UDP user server port 53 (decimal). Messages carried
by UDP are restricted to 512 bytes (not counting the IP or UDP headers).
Longer messages are truncated and the TC bit is set in the header."

        DS