Very Computer

Two (or more) processes 'open' (not fopen) the same file with
O_APPEND.  Then both processes 'write', say, 80 byte blocks to the
file.

Do the two (or more) processes stomp on each other's blocks of data?
And by stomp, I mean will characters or blocks be interleaved or over-
written.  And what happens at block boundaries?

And what happens with nfs and afs in the equation?

The big question is how do we get a whole bunch of processes to open a
single file (a log file, in fact) and write to the end of it at the same
time.

We are thinking HP/UX here, but a general answer would be useful too.

Thanx,
--
     ____/     /     /     __  /    _  _/    ____/
    /         /     /     /   /      /     /          Chris Sherman
   /         ___   /        _/      /          /

The answer is, generally, the writes will be atomic and the data won't
be overwritten or interleaved.  Apparently there is a race condition in
4.2 BSD, so any systems derived from said may or may not have this
problem.

Quote:>          And what happens at block boundaries?

Quote:>And what happens with nfs and afs in the equation?

Quote:>The big question is how do we get a whole bunch of processes to open a
>single file (a log file, in fact) and write to the end of it at the same
>time.

Dylan.
--
This is the ordinary signature

--

Quote:>To mingle appends, suggest you consider fopen(filename, "a");

Quote:> And what happens with nfs and afs in the equation?

Quote:> The big question is how do we get a whole bunch of processes to open a
> single file (a log file, in fact) and write to the end of it at the same
> time.

Quote:> We are thinking HP/UX here, but a general answer would be useful too.

--
All opinions Copyright (c) 1991 by John Chambers.  Inquire for licensing at:
Home: 1-617-484-6393 ...!{bu.edu,harvard.edu,ima.com,eddie.mit.edu,ora.com}!minya!jc
Work: 1-508-486-5475 {sppip7.lkg.dec.com!jc,ub40::jc}

[context: O_APPEND and network filesystems]

|> > And what happens with nfs and afs in the equation?
|>
|> No idea.  I wouldn't trust them unless I could get a definitive answer
|> from someone who I personally believed was knowledgeable.

If you open an NFS file with O_APPEND, the client kernel checks the file
size before each write request and writes where it is told the end of the
file is.  There is a race condition if two clients are writing at the same
time, but appending will often work if the interval between calls to write()
is longer than the NFS server response time (i.e. for important data, don't
rely on this feature).

O_APPEND does not work right in AFS, because clients cache files and the
last client to store back to the server will replace the old contents.
(General advice: always assume that AFS does the wrong thing.  You will rarely
be disappointed and sometimes you will be pleasantly surprised when it works.)

--

The only case where a write is not atomic is on a pipe when the size
of the write is greater than the _maximum_ size of the pipe (I'm talking
about V7 pipe semantics here; not socket pipes, stream pipes or any other conceivable form of pipe).

So just who is guilty?  Or are you people confused?

``When the going gets wierd, the weird turn pro...''

Quote:> If you open an NFS file with O_APPEND, the client kernel checks the file
> size before each write request and writes where it is told the end of the
> file is.  There is a race condition if two clients are writing at the same
> time, but appending will often work if the interval between calls to write()
> is longer than the NFS server response time (i.e. for important data, don't
> rely on this feature).
> O_APPEND does not work right in AFS, because clients cache files and the
> last client to store back to the server will replace the old contents.
> (General advice: always assume that AFS does the wrong thing.  You will
> rarely be disappointed and sometimes you will be pleasantly surprised
> when it works.)

The AFS-3 file semantics are that all clients will see the state of the
file as of the last time it was close() - ed.  So, for processes A and B
on one client workstation, and process C on another, if process A is
writing the file, process B will see all the writes, but process C
won't.  The way to guarantee that process C sees the writes is to
write(),close(),open(), and continue, or write(),fsync(), continue, but
there is still a race condition there.  

AFS-4  (DCE DFS, available sometime from a vendor near you, nag one)
will support fine-grain read-write sharing, so any two processes writing
a file with O_APPEND will behave essentially identically regardless of
whether they are on the same host or different ones, albeit at a small
performance penalty (due to the added communications involved with
repeatedly notifying each client where the end of the file is, and what
the contents of the file are).

AFS-3 is designed for general usage, (which is read-mostly,
write-rarely, and most of the writes are complete rewrites of a file,
which is usually re-read immediately) and
not for append-always, read-almost-never applications.  NFS would be
suitable for the cited use.

Ob Tip:  If you know you're going to delete a file which you currently
have open (eg, a temp file), unlink() before the close() instead of
vice-versa: it's faster, especially with log-based filesystems, or AFS.  

Lyle            Transarc                707 Grant Street
412 338 4474    The Gulf Tower  Pittsburgh 15219
Ich bin ein Virus. Mach' mit und kopiere mich in Deine .signature.