Very Computer

This seems to be true even if some totally unrelated process has the
file opened for writing.
Sonsider the following sequence of events, in a multi processor
machine.
1.process A opened teh file O_APPEND
2. Process B also opens with O_APPEND
3. Process A write() s
4. Process B also write() s

Process B s data will be appended after that of process A.

If this is true ( somebody correct me if I m wrong ), I m sure there
are many people out there who shared this misconcept with me ( that
O_APPEND is just to open a file so that the file pointer will be at
the end of it after opening ).

--sony

Quote:> Till recently I was under the impression that the flag O_APPEND is
> used for opeing a file so that the file pointer will be at the end of
> the file.
> But it looks like it s actually a property of the the succeeding
> write() s, whereby whenever a write() is done of teh file descriptor,
> *atomically* the file pointer will be moved to the end of the file and
> the data appended. ( Not only is the seeking and writing atomic, but
> also will be will be done in an exclusively "locked" fashion, so that
> any other process writing to the same file, opened with O_APPEND, will
> not write over one another's data )

> This seems to be true even if some totally unrelated process has the
> file opened for writing.
> Sonsider the following sequence of events, in a multi processor
> machine.
> 1.process A opened teh file O_APPEND
> 2. Process B also opens with O_APPEND
> 3. Process A write() s
> 4. Process B also write() s

> Process B s data will be appended after that of process A.

> If this is true ( somebody correct me if I m wrong ), I m sure there
> are many people out there who shared this misconcept with me ( that
> O_APPEND is just to open a file so that the file pointer will be at
> the end of it after opening ).

> --sony

    O_APPEND
        If set, the file offset shall be set to the end of the
        file prior to each write.

This doesn't say anything about atomicity though.

For write(), SUSv3 says:

    If the O_APPEND flag of the file status flags is set, the
    file offset shall be set to the end of the file prior to
    each write and no intervening file modification operation
    shall occur between changing the file offset and the write
    operation.

This, however, sounds like atomicity is guaranteed (for moving
the file pointer and initiating writing at least).

--
Andreas K?h?ri
--------------------------------------------------------------
Stable, secure, portable, free:     www.netbsd.org

Quote:> Till recently I was under the impression that the flag O_APPEND is
> used for opeing a file so that the file pointer will be at the end of
> the file.
> But it looks like it s actually a property of the the succeeding
> write() s, whereby whenever a write() is done of teh file descriptor,
> *atomically* the file pointer will be moved to the end of the file and
> the data appended. ( Not only is the seeking and writing atomic, but
> also will be will be done in an exclusively "locked" fashion, so that
> any other process writing to the same file, opened with O_APPEND, will
> not write over one another's data )

> This seems to be true even if some totally unrelated process has the
> file opened for writing.
> Sonsider the following sequence of events, in a multi processor
> machine.
> 1.process A opened teh file O_APPEND
> 2. Process B also opens with O_APPEND
> 3. Process A write() s
> 4. Process B also write() s

> Process B s data will be appended after that of process A.

> If this is true ( somebody correct me if I m wrong ), I m sure there
> are many people out there who shared this misconcept with me ( that
> O_APPEND is just to open a file so that the file pointer will be at
> the end of it after opening ).

> --sony

--sony

     O_APPEND
           If set, the file offset is set to the end of the  file
           prior to each write.

Casper
--
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

Actually APUE discusses this specifically at 3.10. I had read this
book some 5 years back. It s scary that I couldn t even remember the
mention of it.

Hmm that also brings up some questions.
1. For a file that is opened with O_APPEND, what will be the behavior
if an lseek() is done so as to position the file offset at somewhere
in the middle, and then a write(). The moment you do an lseek() will
you lose the O_APPEND property of the file descriptor. ?

2. Also where is the attribute for O_APPEND stored, file table or in
teh inode/vnode table.

3. When 2 processes are opening the same file in O_APPEND, are they
sharing the vnode/inode.

4. To get teh atomic semantics of write() where does the kernel put
the lock.

--sony

--sony

Quote:

> Casper

Quote:>2. Also where is the attribute for O_APPEND stored, file table or in
>the inode/vnode table.

Quote:>3. When 2 processes are opening the same file in O_APPEND, are they
>sharing the vnode/inode.

A file can have multiple file pointers associated with it; but those
can be shared between fds and processes.

(i.e., multiple fds point to a single file pointer which keeps a file
offset, some flags such as FAPPEND [file opened w/ O_APPEND])

You can change the flag with fcntl().

Multiple file pointers can point to a single vnode/inode.

Quote:>4. To get teh atomic semantics of write() where does the kernel put
>the lock.

Casper
--
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

Quote:> Yes it was Solaris man page that prompted me to discover this. Till
> then I had skipped this part of the man page, since I thought I "knew"
> what O_APPEND does from my reading of Unix books.

> Actually APUE discusses this specifically at 3.10. I had read this
> book some 5 years back. It s scary that I couldn t even remember the
> mention of it.

> Hmm that also brings up some questions.
> 1. For a file that is opened with O_APPEND, what will be the behavior
> if an lseek() is done so as to position the file offset at somewhere
> in the middle, and then a write(). The moment you do an lseek() will
> you lose the O_APPEND property of the file descriptor. ?

--

Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.

| Yes it was Solaris man page that prompted me to discover this. Till
| then I had skipped this part of the man page, since I thought I "knew"
| what O_APPEND does from my reading of Unix books.

This is an inherint danger of either reading something that appears to
be right but isn't, or reading something poorly written that gives an
incorrect understanding, or just plain not understanding it but thinking
that you do.  No easy way around it until you encounter something that
contradicts it, then dealing with it.

| Actually APUE discusses this specifically at 3.10. I had read this
| book some 5 years back. It s scary that I couldn t even remember the
| mention of it.

I be there are lots of things that, being not so important at that
time, might not have sunk in, and thus won't be remembered.  So when
friends ask to borrow your APUE, say "no, go buy your own".

| 2. Also where is the attribute for O_APPEND stored, file table or in
| teh inode/vnode table.

In whatever holds the descriptor state (implementation dependent I am
sure).

| 3. When 2 processes are opening the same file in O_APPEND, are they
| sharing the vnode/inode.

This is probably also implementation dependent.  But there is a distinction
between 2 separate opens, and duplicating a descriptor.  So there has to be
the ability to reference something multiple times that describes each open
file, which in turn references the file that is actually open.

| 4. To get teh atomic semantics of write() where does the kernel put
| the lock.

I'm sure that is implementation dependent.  It could be achieved in a single
processor by making sure interrupts are masked and just updating the current
position and queueing the write buffer.  A more elaborate system, like we
have today on most machines, might have to briefly place a lock on a file,
or queue the various writes to be handled by a single thread.  An OS I started
to write many years ago actually had a thread internal to it for each open
file, and I/O was done by sending messages to it and getting messages back.
Every active device and filesystem also had a thread.

--
-----------------------------------------------------------------
| Phil Howard - KA9WGN |   Dallas   | http://linuxhomepage.com/ |

-----------------------------------------------------------------

Quote:> >in the middle, and then a write(). The moment you do an lseek() will
> >you lose the O_APPEND property of the file descriptor. ?

> The write will append at the end; you can lseek for reading.

Quote:

> >2. Also where is the attribute for O_APPEND stored, file table or in
> >the inode/vnode table.

> The open file table.

1. The table/array indexed by the file descriptor number I call "file
descriptor table".
2. A pointer from file descriptor table points to another table entry
is called "file table". this is where fileposition is stored.
3. A pointer from "file table" points to another table entry called
inode/vnode

So 3 tables

My understanding was that file descriptor table holds only some
special kind of flags like close_on_exec etc.

Quote:

> >3. When 2 processes are opening the same file in O_APPEND, are they
> >sharing the vnode/inode.

> Yes; a file only had a single inode/vnode; all processes using the file
> share it.

> A file can have multiple file pointers associated with it; but those
> can be shared between fds and processes.

Quote:

> (i.e., multiple fds point to a single file pointer which keeps a file
> offset, some flags such as FAPPEND [file opened w/ O_APPEND])

Actually looking at /usr/include/sys/vnode.h it shows that struct
vnodeops defined write() with some flags as arguments. So I guess
whenever a write is executed on a file with O_APPEND, vnodes wtite() (
a function pointer ) is called with the flag and with the vnode
locked. Later inside the kernel, this might be invoking eithe a
different routine - due to teh FAPPEND - or the same routine,
preceeded by something equivalent to an lseek() to teh end of the
file. Since the vnode is locked, everything is atomic and safe.

Now I feel better.

--sony

--sony

--
Andreas K?h?ri
--------------------------------------------------------------
Stable, secure, portable, free:     www.netbsd.org