The NFS slowdown problem ...

The NFS slowdown problem ...

Post by en.. » Fri, 22 May 1998 04:00:00



Hi,

some of you might perhaps have experienced the same problem as we have
here - writing from a Linux machine to a directory that is NFS-mounted
from a Solaris, SunOS or OSF/1 system is _lots_ slower than writing to a
directory that is mounted from another Linux system. We are not able
to achieve NFS write rates to OSF/1 of more than ca. 200 kB/sec. over a
155 MBit/sec ATM line, whereas ftp, rcp etc. transfers run at 8-10 _M_B/sec. !
We did the appropriate rsize/wsize tuning and used 8192 for each on all
NFS mounts. The Linux system in question is running a 2.0.29 kernel with
Werner Almesberger's ATM 0.31, the machine on the other end is a DEC
AlphaServer 2100 running Digital Unix (OSF/1) 4.0d. The problems are also
reproducible when using a UltraSparc running Solaris 2.6 on 100BTX ethernet
as server with a Linux system on the same fast ethernet switch.

I'm now trying to find the reason for the slowdown by analysing tcpdump
output for the NFS transfers on an otherwise idle network. As I'm no
NFS expert, I'd be happy if you lend me a helping hand ...

Some people kindly suggested that this was effectively a problem with
writing to a directory that was mounted from a server running NFS V3,
but the same effects seem to occur when writing to a SunOS 4.1.4 NFS server
which is NFS V2 only.

My test setup is currently a Linux system running Linux 2.1.101 (called
velox) that mounts a directory from
1. a SparcStation IPC (sunbeam) running SunOS 4.1.4 (i.e., NFS V2 only)
2. another Linux/intel system (isis) running 2.0.33
(I'm currently running the tests at home on a 10 MBit network, but I'm
 observing similar slowdowns here ! I'll run the same tests on our ATM network
 on Friday ...)

The first major difference in the tcpdump output I can observe is:

Writes to the SparcStation give me the following output:

03:04:41.538907 velox.unix-ag.org.66760000 > sunbeam.unix-ag.org.nfs: 1472


Writes to the Linux system produce the following:

03:05:42.003120 velox.unix-ag.org.ae7c0000 > isis.unix-ag.org.nfs: 1472


So, the difference is that transfers to the Linux system have an unknown
file handle. Does anyone have an explanation for this behaviour ?

regards,

 
 
 

1. NFS slowdown - Solaris client, Linux server.

We are experiencing an itermitent NFS performance slowdown on a
Solaris 8 Ultra-80 when writing to filesystems mounted from a Redhat 8
NFS server. The write performance is good initially, then after some
period of time (10's of minutes), write performance slows by at least
a factor of 10 and the Solaris system CPU usage jumps to > 40%.
Strangely, if any access to the target filesystem on the NFS server is
done (e.g. ls -l), the client system CPU drops, and the NFS write
performance returns to normal. But, usually after another 10 - 20
minutes, the slowdown re-occurs, requiring another "ls -l" to speed it
up again. The output files are fairly large (> 6 GBytes) and the
writes are purely sequential.

We spent a fair amount of time getting Solaris NFS clients to work
properly with a Linux server, so I'm quite familiar with most of the
config options, but this one has me stumped.

Has anyone seen this behaviour, or have some tips on where to start
looking to try to fix it? The same problem occurred consistently
before applying the Solaris 108528-27 kernel ugrade, but it's still
there, although it's now much harder to reproduce.

System details are:

Solaris 8 NFS client:
  Ultra-80, Kernel version: SunOS 5.8 Generic 108528-27 Nov 2003
  mount options: vers=3,proto=udp,sec=none,hard,intr,link,symlink,acl,
                 rsize=8192,wsize=8192,retrans=5,timeo=11
  Gigabit Ethernet controller.

Redhat 8 NFS server:
  Dual Pentium Xeon 2.8 GHz, Redhat 8, kernel: 2.4.20-20.8smp
  nfs-utils-1.0.5-1
  export options: rw,insecure,sync,no_subtree_check,insecure_locks
  Intel Pro/1000 Gigabit Ethernet, e1000 driver vers. 5.2.20.

Thanks,

Marc Langlois.
Key Seismic Solutions, Calgary, AB, Canada.

2. ncurses

3. NFS slowdown in 2.6; memcntl(SYNC)

4. /etc/hosts.deny & hosts.allow

5. Periodic slowdowns of NFS

6. Patching X: 3.3.1->3.3.2

7. nfs slowdown since 2.5.4

8. 'fix-cua'

9. iptables & nfs slowdowns

10. nfs-server slowdown in 2.4.20-pre10 with client 2.2.19

11. Slowdown problems

12. gnome slowdown problem

13. Problem with major slowdowns