Poor NFS performance from Solaris to Linux

Poor NFS performance from Solaris to Linux

Post by James Hewit » Sat, 12 Dec 1998 04:00:00



This is an old problem, but it doesn't seem to have been fixed yet.
NFS data transfer speed between a server running Solaris 2.6 and a
PC running Linux (Slackware 3.6 with kernel 2.1.130) is 5 to 8
times slower than between two Solaris 2.6 machines.  This is true
even using udp and NFS version 2 between the Solaris machines.

Does anyone have any information on the cause of this?  I hoping
to have some time to put into working on a fix in the next month
or two, and it would be nice not to have to start from scratch.

Thanks,

-James

--
James Hewitt

My real email address is the part in parenthesis:

May the Source be with you

 
 
 

Poor NFS performance from Solaris to Linux

Post by Roland Latou » Sat, 12 Dec 1998 04:00:00




> > This is an old problem, but it doesn't seem to have been fixed yet.
> > NFS data transfer speed between a server running Solaris 2.6 and a
> > PC running Linux (Slackware 3.6 with kernel 2.1.130) is 5 to 8
> > times slower than between two Solaris 2.6 machines.  This is true
> > even using udp and NFS version 2 between the Solaris machines.

> > Does anyone have any information on the cause of this?  I hoping
> > to have some time to put into working on a fix in the next month
> > or two, and it would be nice not to have to start from scratch.

> I know a networking guru who noticed the same thing between IRIX
> boxes. He dug a bit deeper and found that whenever an IRIX box
> was talking to another IRIX box it used 8k packets instead of
> 1.5k packets.

Found this in 'man mount':
       Especially useful options include

       rsize=8192,wsize=8192
              This will make your nfs connection much faster than
              with the default buffer size of 1024.

I suspect the default is 1024 because many PC networking cards used to
have tiny buffers on them.
--
  Roland Latour  Slackware3.4 & PPP     http://home.cdsnet.net/~rolandl
      Change is inevitable, except from a vending machine.

 
 
 

Poor NFS performance from Solaris to Linux

Post by Erik de Castro Lop » Sun, 13 Dec 1998 04:00:00



> This is an old problem, but it doesn't seem to have been fixed yet.
> NFS data transfer speed between a server running Solaris 2.6 and a
> PC running Linux (Slackware 3.6 with kernel 2.1.130) is 5 to 8
> times slower than between two Solaris 2.6 machines.  This is true
> even using udp and NFS version 2 between the Solaris machines.

> Does anyone have any information on the cause of this?  I hoping
> to have some time to put into working on a fix in the next month
> or two, and it would be nice not to have to start from scratch.

I know a networking guru who noticed the same thing between IRIX
boxes. He dug a bit deeper and found that whenever an IRIX box
was talking to another IRIX box it used 8k packets instead of
1.5k packets.

Something similar may be going on between the Solaris boxes. DO
you have a network analyser to check this out?

Cheers,
Erik
--
+-------------------------------------------------+

+-------------------------------------------------+
Microsoft is not the answer.
Microsoft is the question. NO is the answer.

 
 
 

Poor NFS performance from Solaris to Linux

Post by Pete Zaitc » Sun, 13 Dec 1998 04:00:00



>This is an old problem, but it doesn't seem to have been fixed yet.
>NFS data transfer speed between a server running Solaris 2.6 and a
>PC running Linux (Slackware 3.6 with kernel 2.1.130) is 5 to 8
>times slower than between two Solaris 2.6 machines.  This is true
>even using udp and NFS version 2 between the Solaris machines.

Traditionally it is attributed to bad default transfer size.
Did you mount with rsize=8192,wsize=8192?

--Pete

 
 
 

Poor NFS performance from Solaris to Linux

Post by Steve Lint » Tue, 15 Dec 1998 04:00:00




>>This is an old problem, but it doesn't seem to have been fixed yet.
>>NFS data transfer speed between a server running Solaris 2.6 and a
>>PC running Linux (Slackware 3.6 with kernel 2.1.130) is 5 to 8
>>times slower than between two Solaris 2.6 machines.  This is true
>>even using udp and NFS version 2 between the Solaris machines.
>Traditionally it is attributed to bad default transfer size.
>Did you mount with rsize=8192,wsize=8192?

The full story is the following:

1. NFS specifiies that a write transaction cannot complete until the
data is physically on the disk

2. This is utterly inappropriate for modern network speeds

3. Sun get round this using a client-side daemon to hide the latency

4. Linux gets round this by ignoring the NFS spec on the server side

5  Hence Sun server and Linux client gets the worst of all possible
worlds.

The problem can be ameliorated by increasing the size of the NFS
transfers, as suggested. Try also 4K and 16K sizes.

On Solaris 2.6 it may be possible to set an option to the NFS server
to make it work asynchronously.

Runnign Linux on the Sun file server is another option.

        Steve

 
 
 

Poor NFS performance from Solaris to Linux

Post by James Hewit » Tue, 15 Dec 1998 04:00:00



> The full story is the following:

> 1. NFS specifiies that a write transaction cannot complete until the
> data is physically on the disk

> 2. This is utterly inappropriate for modern network speeds

It should be optional, at least.

Quote:> 3. Sun get round this using a client-side daemon to hide the latency

This I don't quite understand.  In the Solaris -> Solaris case, the
whole file appears to be available when the copy ends, and the
packets for the transfer stop as well.  I don't see how a client-side
daemon could help, unless the transfer protocol is broken, which
doesn't seem to be the case.

Quote:> 4. Linux gets round this by ignoring the NFS spec on the server side

> 5  Hence Sun server and Linux client gets the worst of all possible
> worlds.

> The problem can be ameliorated by increasing the size of the NFS
> transfers, as suggested. Try also 4K and 16K sizes.

I tried 4k and 8k sizes, as recommended in the man pages (thanks to
all of you who suggested it).  I'll try some larger sizes as well.

Quote:> On Solaris 2.6 it may be possible to set an option to the NFS server
> to make it work asynchronously.

I had heard about the sychronous/asynchronous issue before, but
the Solaris NFS server doesn't have an asynchronous option (which
figures since they wrote the standard :/ ).  I tried setting the
sync option on Linux, but that made transfers very, very slow.

Quote:> Runnign Linux on the Sun file server is another option.

:)  I have thought about this once or twice, but unfortunately
most of our servers are workstations as well, and the apps we
use are not available for Sparc Linux.

Thanks for the info.  If the client-side daemon does the trick on
Solaris, I will probably try to write one for Linux.

--
James Hewitt

My real email address is the part in parenthesis:

May the Source be with you

 
 
 

Poor NFS performance from Solaris to Linux

Post by Steve Lint » Thu, 17 Dec 1998 04:00:00



>> 3. Sun get round this using a client-side daemon to hide the latency
>This I don't quite understand.  In the Solaris -> Solaris case, the
>whole file appears to be available when the copy ends, and the
>packets for the transfer stop as well.  I don't see how a client-side
>daemon could help, unless the transfer protocol is broken, which
>doesn't seem to be the case.

The description from man nfs_mount on Solaris is:

 SunOS 4.X used the biod  maintenance  procedure  to  perform
     parallel  read-ahead and write-behind on NFS clients.  SunOS
     5.X obsoleted biod  with  multi-threaded  processing,  which
     transparently performs parallel read-ahead and write-behind.

I imagine the net result is to overlap the transactions more than
Linux does. Maybe the answer is to modify the Linux nfs client-side
code to do something similar. In their new spirit of open-ness to
Linux Sun might even help.

        Steve

 
 
 

Poor NFS performance from Solaris to Linux

Post by Peter Samuels » Wed, 06 Jan 1999 04:00:00


  [Steve Linton]

Quote:> > On Solaris 2.6 it may be possible to set an option to the NFS
> > server to make it work asynchronously.
[James Hewitt]
> I tried setting the sync option on Linux, but that made transfers
> very, very slow.

Nope, not what you want: `-o sync' basically disables write buffering
on the client.  Which would just exacerbate your latency problem.

Quote:> > Runnign Linux on the Sun file server is another option.
> :) I have thought about this once or twice, but unfortunately most of
> our servers are workstations as well, and the apps we use are not
> available for Sparc Linux.

Hold that smiley.  You might at least try it on a spare machine --
SPARC Linux *does* run Solaris binaries.  At least that's the claim.  I
don't know how well it works, since I have never run SPARC Linux.  (It
would help if I had a SPARC.)

--
Peter Samuelson
<sampo.creighton.edu!psamuels>

 
 
 

Poor NFS performance from Solaris to Linux

Post by Martin.Sp.. » Mon, 11 Jan 1999 04:00:00



> Hold that smiley.  You might at least try it on a spare machine --
> SPARC Linux *does* run Solaris binaries.  At least that's the claim.  I
> don't know how well it works, since I have never run SPARC Linux.  (It
> would help if I had a SPARC.)

They claim that SPARC Linux runs SunOS4 aka. Solaris1-binaries, but no
SunOS5 aka. Solaris2-binaries. Did I miss something ?

Martin.
--
 Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------

 
 
 

Poor NFS performance from Solaris to Linux

Post by Paul Voj » Mon, 11 Jan 1999 04:00:00




>They claim that SPARC Linux runs SunOS4 aka. Solaris1-binaries, but no
>SunOS5 aka. Solaris2-binaries. Did I miss something ?

Who is ``they''?

SunOS 4 emulation:  earlier sparc kernels supported it (provided you compiled
the kernel with support for a.out binaries); recent kernels have it disabled
with the following comment in linux/include/asm-sparc/namei.h:

/* Until we solve, why SunOS apps sometime crash, disable gnemul support
  for Sun OS */

SunOS 5 emulation:  you need to install the iBCS package.  I've never tried
it myself.


 
 
 

1. Poor NFS performance between Linux and Solaris

I am having performance problems with NFS between a Linux client (running
RedHat 6.0) and a Solaris server (E450 running Solaris 2.6).  The two machines
are connected together using 100BaseT ethernet.

As a simple test I am trying to copy a 100m file

        client% cd /mnt
        client% time cp 100m.dat copy.dat

Using two Solaris systems, this takes about 23 seconds (using NFS V3) and
1 minute 55 seconds (using NFS V2).  However, the same test on the Linux
system takes 4 minutes 30 seconds (with rsize and wsize set to 8192), and
even longer with the default parameters.  In both cases, the E450 is acting
as the NFS server.

Has anybody else seen a similar degredation in performance, and how can I
fix things?  Is NFSV3 available under Linux, and if so what performance
gains are there over NFSV2?

Thanks in advance,

Cheers,

Martin.

2. Linux Reliability

3. Poor NFS write performance from Linux clients to NFS Ver 3 servers.

4. trying to understand xmodmap... :(

5. Poor Solaris 2.6 NFS client performance

6. cron does not launch any job

7. Poor NFS performance between 2 Linux boxes

8. Vacuum, void, null... .NET

9. poor NFS client performance under linux?

10. Is NFS performance under Linux known to be poor?

11. poor nfs performance with SuSE Linux 6.2

12. linux and NFS performance (poor)

13. Poor linux NFS client write performance