consistent read() on file while rename() file.new -> file?

consistent read() on file while rename() file.new -> file?

Post by Michael Wa » Tue, 27 Feb 2001 13:31:09



In shell, I have

grep ... file (open() on file)

lock
sed ... file > file.new
mv file.new file (rename())
unlock

In reading the file, either new version or the old version is ok.
*without* placing the open() portion between lock and unlock,
can I be sure I will get either the old version or the new version
but no something messed up in between?

I think the question can be rephrased as can open() get in between
rename(): ren- open() -name(). I am on Solaris 2.x with multi processors.

PS: when both file and file.new exists,
"mv file.new file" does not use "unlink() file". Then how
the "file" get removed? Thanks.

 
 
 

consistent read() on file while rename() file.new -> file?

Post by Casper H.S. Dik - Network Security Engine » Tue, 27 Feb 2001 20:17:03


[[ PLEASE DON'T SEND ME EMAIL COPIES OF POSTINGS ]]


>In reading the file, either new version or the old version is ok.
>*without* placing the open() portion between lock and unlock,
>can I be sure I will get either the old version or the new version
>but no something messed up in between?

Yes, after open() the name of the file is irrelevant.  If you make
sure that the file name always refers to a "finalized" file, then you
will always open the good file.

(Rename() is atomic; after open(), any changes to the file's name
are irrelevant)

Casper
--
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

 
 
 

consistent read() on file while rename() file.new -> file?

Post by Michael Wa » Tue, 27 Feb 2001 22:32:42


Quote:>>In reading the file, either new version or the old version is ok.
>>*without* placing the open() portion between lock and unlock,
>>can I be sure I will get either the old version or the new version
>>but no something messed up in between?

>Yes, after open() the name of the file is irrelevant.  If you make
>sure that the file name always refers to a "finalized" file, then you
>will always open the good file.

>(Rename() is atomic; after open(), any changes to the file's name
>are irrelevant)

atomic means it has to succeed to fail as whole,
as I understand. Does it also mean other calls
can not get in between, i.e.,

re- read() -name()

Does the kernel has to execute one system calls serially?
i.e. either

rename()
read()

or

read()
rename()?

 
 
 

consistent read() on file while rename() file.new -> file?

Post by Casper H.S. Dik - Network Security Engine » Tue, 27 Feb 2001 23:08:32


[[ PLEASE DON'T SEND ME EMAIL COPIES OF POSTINGS ]]


>atomic means it has to succeed to fail as whole,
>as I understand. Does it also mean other calls
>can not get in between, i.e.,
>re- read() -name()

You cannot detect the interleaving of the two calls; renaming a file
does not affect the readers of the file.

(It's atomic wrt the object it acts on: the filesystem namespace)

Casper
--
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

 
 
 

consistent read() on file while rename() file.new -> file?

Post by Ed L Cashi » Tue, 27 Feb 2001 23:12:17



> >>In reading the file, either new version or the old version is ok.
> >>*without* placing the open() portion between lock and unlock,
> >>can I be sure I will get either the old version or the new version
> >>but no something messed up in between?

> >Yes, after open() the name of the file is irrelevant.  If you make
> >sure that the file name always refers to a "finalized" file, then you
> >will always open the good file.

> >(Rename() is atomic; after open(), any changes to the file's name
> >are irrelevant)

> atomic means it has to succeed to fail as whole,
> as I understand. Does it also mean other calls
> can not get in between, i.e.,

> re- read() -name()

The read call operates on the file descriptor.  The file has already
been opened, so whether some other process renames or deletes the file
or whatever, they're just operating on the directory the file lives
in.  The filename belongs more to the directory than to the file
itself.  

Since you have the file descriptor itself, you will continue to read
from the same data on disk (the same file), and the OS won't reclaim
the disk space until you close the file.

--
--Ed Cashin                     integrit file-verification system:

    Note: If you want me to send you email, don't munge your address.

 
 
 

consistent read() on file while rename() file.new -> file?

Post by Michael Wa » Wed, 28 Feb 2001 01:31:03


Quote:>>re- open() -name()

>You cannot detect the interleaving of the two calls; renaming a file
>does not affect the readers of the file.

>(It's atomic wrt the object it acts on: the filesystem namespace)

The interleaving of the two calls does or does not happen?

If it does, why I can not detect it?

Thanks.

 
 
 

consistent read() on file while rename() file.new -> file?

Post by Kenneth Ch » Wed, 28 Feb 2001 01:49:51




>>You cannot detect the interleaving of the two calls; renaming a file
>>does not affect the readers of the file.

>>(It's atomic wrt the object it acts on: the filesystem namespace)

>The interleaving of the two calls does or does not happen?

>If it does, why I can not detect it?

A file's identity is not its name.  Opening a file returns a file
descriptor which is bound much more tightly to a file than just the
mere name.  As an analogy, I can change my name, but that doesn't
change my identity.  Any contractual obligations I had before are still
in force.  If I was married before, I'm still married.

In fact, once a file is opened, even 'rm' doesn't affect it.  It can
still be read from and written to.

 
 
 

consistent read() on file while rename() file.new -> file?

Post by Casper H.S. Dik - Network Security Engine » Wed, 28 Feb 2001 01:49:09


[[ PLEASE DON'T SEND ME EMAIL COPIES OF POSTINGS ]]


>>>re- open() -name()

>>You cannot detect the interleaving of the two calls; renaming a file
>>does not affect the readers of the file.

>>(It's atomic wrt the object it acts on: the filesystem namespace)
>The interleaving of the two calls does or does not happen?

It can happen, but it has no consequence.

Quote:>If it does, why I can not detect it?

Because your read will work on the file you've opened; and not
on one of the two file names.

I.e., it makes absolutely no difference to the read() call if it
happens before, after or concurrent with the rename() call.

Casper
--
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

 
 
 

consistent read() on file while rename() file.new -> file?

Post by Michael Wa » Wed, 28 Feb 2001 02:04:05


Quote:>>>>re- open() -name()

>Because your read will work on the file you've opened; and not
>on one of the two file names.

>I.e., it makes absolutely no difference to the read() call if it
>happens before, after or concurrent with the rename() call.

It makes no difference if read() happens in between rename(),
but what happens if open() happens between the rename():

re- open() -name()

So open() will fail and subsquence read() will fail?

 
 
 

consistent read() on file while rename() file.new -> file?

Post by Michael Wa » Wed, 28 Feb 2001 02:12:31


Quote:>A file's identity is not its name.  Opening a file returns a file
>descriptor which is bound much more tightly to a file than just the
>mere name.  As an analogy, I can change my name, but that doesn't
>change my identity.  Any contractual obligations I had before are still
>in force.  If I was married before, I'm still married.

>In fact, once a file is opened, even 'rm' doesn't affect it.  It can
>still be read from and written to.

The question is, can we always get a consistent read() on the file while
while rename() file.new -> file?

_once a file is opened_, then rename() does not matter.

However, will open() will always successful? i.e. will the following
happen?

rename() starts
open()
rename() ends

To use your analogy, as the registration office moves, can you always
find the office to register your name or your marriage?

 
 
 

consistent read() on file while rename() file.new -> file?

Post by Kenneth Ch » Wed, 28 Feb 2001 03:07:57




>However, will open() will always successful? i.e. will the following
>happen?

>rename() starts
>open()
>rename() ends

No.  Rename() is atomic.  Assuming the new name exists before the
rename starts, it will always be visible and valid.

A quote from SUSv2

    . . . If the link named by the new argument exists, it is removed
    and old renamed to new. In this case, a link named new will remain
    visible to other processes throughout the renaming operation and will
    refer either to the file referred to by new or old before the operation
    began.

 
 
 

consistent read() on file while rename() file.new -> file?

Post by Barry Margoli » Wed, 28 Feb 2001 08:03:05




>atomic means it has to succeed to fail as whole,
>as I understand. Does it also mean other calls
>can not get in between, i.e.,

Yes.  Atomic also means that it appears to be a single operation, not
composed of any smaller parts that can be detected.

--

Genuity, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.