BUG REPORT: kernel nfs between 2.4.19-pre2 (server) and 2.2.21-pre3 (client)

BUG REPORT: kernel nfs between 2.4.19-pre2 (server) and 2.2.21-pre3 (client)

Post by Stephan von Krawczynsk » Sun, 10 Mar 2002 21:30:06



Hello all,

I just upgraded a host from 2.2.19 to 2.2.21-pre3 and discovered a problem with kernel nfs. Setup is this:

knfs-server is 2.4.19-pre2
knfs-client is 2.2.21-pre3

First mount some fs (mountpoint /backup). Then go and mount some other fs from the same server (mountpoint /mnt), do some i/o on the latter and umount it again. Now try to access /backup. You see:
1) /backup (as a fs) vanished, you get a stale nfs handle.
2) umount /backup; mount /backup does not work. client tells "permission denied". server tells "rpc.mountd: getfh failed: Operation not permitted"

Only solution: restart nfs-server (no reboot required), then everything works again.
Same setup works with 2.2.19-client.

Any hints?

Regards,
Stephan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

BUG REPORT: kernel nfs between 2.4.19-pre2 (server) and 2.2.21-pre3 (client)

Post by Alan Co » Mon, 11 Mar 2002 04:00:10


Quote:> I just upgraded a host from 2.2.19 to 2.2.21-pre3 and discovered a problem with kernel nfs. Setup is this:

> knfs-server is 2.4.19-pre2
> knfs-client is 2.2.21-pre3

Do you see this with 2.2.20 (2.2.20 has NFS changes in the client, 2.2.21pre
does not) ?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

BUG REPORT: kernel nfs between 2.4.19-pre2 (server) and 2.2.21-pre3 (client)

Post by Stephan von Krawczynsk » Tue, 12 Mar 2002 03:00:09


On Sat, 9 Mar 2002 19:10:52 +0000 (GMT)


> > I just upgraded a host from 2.2.19 to 2.2.21-pre3 and discovered a problem with kernel nfs. Setup is this:

> > knfs-server is 2.4.19-pre2
> > knfs-client is 2.2.21-pre3

> Do you see this with 2.2.20 (2.2.20 has NFS changes in the client, 2.2.21pre
> does not) ?

Hello Alan,

sorry for delayed answer. The machine in question is in production, and I had to find some other test candidate first. So here we go:

1) The problem is reproducable on another hardware with 2.2.21-pre3 client.
2) The problem is reproducable with 2.2.20 either.

What to test next?

Regards,
Stephan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

BUG REPORT: kernel nfs between 2.4.19-pre2 (server) and 2.2.21-pre3 (client)

Post by Trond Myklebus » Tue, 12 Mar 2002 07:50:08


     > Hello all, I just upgraded a host from 2.2.19 to 2.2.21-pre3
     > and discovered a problem with kernel nfs. Setup is this:

     > knfs-server is 2.4.19-pre2 knfs-client is 2.2.21-pre3

     > First mount some fs (mountpoint /backup). Then go and mount
     > some other fs from the same server (mountpoint /mnt), do some
     > i/o on the latter and umount it again. Now try to access
     > /backup. You see:
     > 1) /backup (as a fs) vanished, you get a stale nfs handle.
     > 2) umount /backup; mount /backup does not work. client tells
     >    "permission denied". server tells "rpc.mountd: getfh failed:
     >    Operation not permitted"

By 'some fs' do you mean ext2?

Not all filesystems work well with knfsd when things start to drop out
of the (d|i)caches. In particular things like /backup == VFAT might
give the above behaviour, since VFAT does not know how to map the NFS
file handles into on-disk inodes.

Cheers,
  Trond
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

BUG REPORT: kernel nfs between 2.4.19-pre2 (server) and 2.2.21-pre3 (client)

Post by Stephan von Krawczynsk » Tue, 12 Mar 2002 09:30:08



>      > Hello all, I just upgraded a host from 2.2.19 to 2.2.21-pre3
>      > and discovered a problem with kernel nfs. Setup is this:    

>      > knfs-server is 2.4.19-pre2 knfs-client is 2.2.21-pre3        

>      > First mount some fs (mountpoint /backup). Then go and mount  
>      > some other fs from the same server (mountpoint /mnt), do some
>      > i/o on the latter and umount it again. Now try to access    
>      > /backup. You see:                                            
>      > 1) /backup (as a fs) vanished, you get a stale nfs handle.  
>      > 2) umount /backup; mount /backup does not work. client tells
>      >    "permission denied". server tells "rpc.mountd: getfh      

failed:                                                              
Quote:>      >    Operation not permitted"                                  

> By 'some fs' do you mean ext2?                                      

> Not all filesystems work well with knfsd when things start to drop  

out                                                                  
Quote:> of the (d|i)caches. In particular things like /backup == VFAT might
> give the above behaviour, since VFAT does not know how to map the  

NFS                                                                  

Quote:> file handles into on-disk inodes.                                  

Sorry Trond,                                                          

this is a weak try of an explanation. All involved fs types are      
reiserfs. The problem occurs reproducably only after (and including)  
2.2.20 and above and _not_ in 2.2.19. There must be some problem.    
Though I do not know whether the problem is on the client side, or    
simply produced by this client side and effectively located on 2.4.18
server, I really can't tell. But giving me something to try might    
clear the picture.                                                    

Any hints?                                                            

Stephan                                                              

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

BUG REPORT: kernel nfs between 2.4.19-pre2 (server) and 2.2.21-pre3 (client)

Post by Trond Myklebus » Tue, 12 Mar 2002 09:40:05


     > this is a weak try of an explanation. All involved fs types are
     > reiserfs. The problem occurs reproducably only after (and

Which ReiserFS format? Is it version 3.5?

   'cat /proc/fs/reiserfs/device/version'

     > including)
     > 2.2.20 and above and _not_ in 2.2.19. There must be some
     > problem.

The client code in 2.2.20 is supposed to be the same as in 2.4.x. The
only thing I can think might be missing is the fix to cope with broken
servers that reuse filehandles (this violates the RFCs). Reiserfs 3.5
+ knfsd is one such broken combination. Another broken server is
unfsd...

     > Though I do not know whether the problem is on the client side,
     > or simply produced by this client side and effectively located
     > on 2.4.18 server, I really can't tell. But giving me something
     > to try might clear the picture.

You might try keeping a file open on /backup while you play with /mnt...

Cheers,
   Trond
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

BUG REPORT: kernel nfs between 2.4.19-pre2 (server) and 2.2.21-pre3 (client)

Post by Oleg Droki » Tue, 12 Mar 2002 15:20:06


Hello!


>      > this is a weak try of an explanation. All involved fs types are
>      > reiserfs. The problem occurs reproducably only after (and
> Which ReiserFS format? Is it version 3.5?
>    'cat /proc/fs/reiserfs/device/version'

If this does not work because you have no such file, then look through your
kernel logs, if you use reiserfs v3.5 on 2.4 kernel, it will show itself
as such record in the log file: "reiserfs: using 3.5.x disk format"

Bye,
    Oleg
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

BUG REPORT: kernel nfs between 2.4.19-pre2 (server) and 2.2.21-pre3 (client)

Post by Stephan von Krawczynsk » Tue, 12 Mar 2002 20:00:07


On Mon, 11 Mar 2002 09:14:58 +0300


> Hello!


> >      > this is a weak try of an explanation. All involved fs types are
> >      > reiserfs. The problem occurs reproducably only after (and
> > Which ReiserFS format? Is it version 3.5?

> >    'cat /proc/fs/reiserfs/device/version'

> If this does not work because you have no such file, then look through your
> kernel logs, if you use reiserfs v3.5 on 2.4 kernel, it will show itself
> as such record in the log file: "reiserfs: using 3.5.x disk format"

Hello Oleg, hello Trond, hello Alan,

I have several reiserfs fs in use on this server. boot.msg looks like

<4>reiserfs: checking transaction log (device 08:03) ...
<4>Using r5 hash to sort names
<4>ReiserFS version 3.6.25
<4>VFS: Mounted root (reiserfs filesystem) readonly.
<4>Freeing unused kernel memory: 224k freed
<6>Adding Swap: 265032k swap-space (priority 42)
<4>reiserfs: checking transaction log (device 21:01) ...
<4>Using r5 hash to sort names
<4>ReiserFS version 3.6.25
<4>reiserfs: checking transaction log (device 22:01) ...
<4>Using tea hash to sort names
<4>reiserfs: using 3.5.x disk format
<4>ReiserFS version 3.6.25
<4>reiserfs: checking transaction log (device 08:04) ...
<4>Using r5 hash to sort names
<4>ReiserFS version 3.6.25

I tried to find out which device has which numbers and did a cat /proc/devices:

Block devices:
  2 fd
  8 sd
 11 sr
 22 ide1
 33 ide2
 34 ide3
 65 sd
 66 sd

Interestingly there is no #21. Shouldn't I see a block device 21 here?
More strange the only two existing ide-drives in this system are located on
ide2 and ide3 and should therefore have device numbers 33 and 34, or not?
There is no hd on ide1, only a CDROM (not used during the test). ide0 is
completely empty.
As told earlier this is kernel 2.4.19-pre2.

Enlighten me, please...

Stephan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

BUG REPORT: kernel nfs between 2.4.19-pre2 (server) and 2.2.21-pre3 (client)

Post by Oleg Droki » Tue, 12 Mar 2002 20:00:11


Hello!


> <4>reiserfs: checking transaction log (device 22:01) ...
> <4>Using tea hash to sort names
> <4>reiserfs: using 3.5.x disk format

This means you have reiserfs v3.5 format on /dev/hdc1
And this one won't behave very good with nfs.
Does this one contain your nfs exports?

Bye,
    Oleg
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

BUG REPORT: kernel nfs between 2.4.19-pre2 (server) and 2.2.21-pre3 (client)

Post by Stephan von Krawczynsk » Tue, 12 Mar 2002 20:10:05


On Mon, 11 Mar 2002 13:52:56 +0300


> Hello!


> > <4>reiserfs: checking transaction log (device 22:01) ...
> > <4>Using tea hash to sort names
> > <4>reiserfs: using 3.5.x disk format

> This means you have reiserfs v3.5 format on /dev/hdc1
> And this one won't behave very good with nfs.
> Does this one contain your nfs exports?

There is _no_ /dev/hdc1.

Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/sda3              6297280   6146232    151048  98% /
/dev/sda2                31111     24695      4810  84% /boot
/dev/hde1             60049096  30161576  29887520  51% /p2
/dev/hdg1             20043416  16419444   3623972  82% /p3
/dev/sda4             29245432  27525524   1719908  95% /p5
shmfs                  1035112         0   1035112   0% /dev/shm

Exported fs is on /dev/hde1.

/dev/hdc could only be a cdrom, but is not in use nor mounted, and for sure has
never been related to reiserfs.

Regards,
Stephan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

BUG REPORT: kernel nfs between 2.4.19-pre2 (server) and 2.2.21-pre3 (client)

Post by Oleg Droki » Tue, 12 Mar 2002 20:20:08


Hello!



> > > <4>reiserfs: checking transaction log (device 22:01) ...
> > > <4>Using tea hash to sort names
> > > <4>reiserfs: using 3.5.x disk format
> > This means you have reiserfs v3.5 format on /dev/hdc1
> > And this one won't behave very good with nfs.
> > Does this one contain your nfs exports?
> There is _no_ /dev/hdc1.

Stupid me! Numbers are in hex! ;)
So that's /dev/hdg1 that is reiserfs v3.5

Quote:> /dev/hdg1             20043416  16419444   3623972  82% /p3
> Exported fs is on /dev/hde1.

Hm. Strange. Are you sure you do not export /dev/hdg1?

Bye,
    Oleg
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

BUG REPORT: kernel nfs between 2.4.19-pre2 (server) and 2.2.21-pre3 (client)

Post by Stephan von Krawczynsk » Tue, 12 Mar 2002 20:30:21


On Mon, 11 Mar 2002 14:11:54 +0300


> Hello!
> [...]
> > There is _no_ /dev/hdc1.

> Stupid me! Numbers are in hex! ;)

aahh, shoot me, a lot of trees, but no forest in sight ... ;-)

Quote:> So that's /dev/hdg1 that is reiserfs v3.5

> > /dev/hdg1             20043416  16419444   3623972  82% /p3
> > Exported fs is on /dev/hde1.

> Hm. Strange. Are you sure you do not export /dev/hdg1?

Yes, you are right here, one of the exports comes from hdg1. I will reformat with 3.6 and re-check the problem. Anyways I find it interesting that the problem does not occur with 2.2.19 client side ...

Stay tuned, I'll be back.

Regards,
Stephan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

BUG REPORT: kernel nfs between 2.4.19-pre2 (server) and 2.2.21-pre3 (client)

Post by Stephan von Krawczynsk » Tue, 12 Mar 2002 21:50:09


On Mon, 11 Mar 2002 14:11:54 +0300


> So that's /dev/hdg1 that is reiserfs v3.5

> > /dev/hdg1             20043416  16419444   3623972  82% /p3
> > Exported fs is on /dev/hde1.

> Hm. Strange. Are you sure you do not export /dev/hdg1?

Ok, Oleg,

I re-checked the setup with all server fs as reiserfs 3.6 and the problem stays
the same.

Mar 11 13:05:07 admin kernel: reiserfs: checking transaction log (device 22:01)
...
Mar 11 13:05:09 admin kernel: Using r5 hash to sort names
Mar 11 13:05:09 admin kernel: ReiserFS version 3.6.25

What else can I try?

I checked the setup with another client kernel 2.4.18, and guess what: it has
the same problem. I have the impression that the problem is somewhere on the
nfs server side - possibly around the umount case. Trond, Ken?

Can anyone reproduce this? It should be fairly simple to check.

Regards,
Stephan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

BUG REPORT: kernel nfs between 2.4.19-pre2 (server) and 2.2.21-pre3 (client)

Post by Oleg Droki » Tue, 12 Mar 2002 22:10:06


Hello!


> What else can I try?
> I checked the setup with another client kernel 2.4.18, and guess what: it has
> the same problem. I have the impression that the problem is somewhere on the
> nfs server side - possibly around the umount case. Trond, Ken?

Just to be sure - have you tried 2.4.17 at the server?
2.4.18 have 2 patches included that were supposed to have another
stale filehandle problem resolved.
Our test have not shown any problems, but I am interested can you still
reproduce with these 2 patches reversed off the 2.4.18?
Also if you still can trigger, apply back only 1st hunk of G-... patch.

Bye,
    Oleg

  A-bigendian-lookup-fix.diff
< 1K Download

  G-nfs_stale_inode_access.diff
1K Download
 
 
 

BUG REPORT: kernel nfs between 2.4.19-pre2 (server) and 2.2.21-pre3 (client)

Post by Stephan von Krawczynsk » Tue, 12 Mar 2002 22:50:11


On Mon, 11 Mar 2002 15:59:37 +0300


> Hello!


> > What else can I try?
> > I checked the setup with another client kernel 2.4.18, and guess what: it has
> > the same problem. I have the impression that the problem is somewhere on the
> > nfs server side - possibly around the umount case. Trond, Ken?
> Just to be sure - have you tried 2.4.17 at the server?

Hello Oleg,

I just checked with 2.4.17 on the server side: the problem stays.
I guess it will not make any sense to try your patches (reversing).

Regards,
Stephan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/