OS-wars: Unix file system(s)

OS-wars: Unix file system(s)

Post by GORE.. » Sun, 02 Dec 1990 06:12:00



Item 1:  I am a VMS partisan wondering if there will ever be (or need to
be) a Unix in my future.  I don't know much about Unix, but high on my list
of why I'd like to avoid it is the file system, or lack thereof (sometimes
referred to as a _feature_ by Unix partisans.)

Item 2:  Recent Data General ads tout their Unix as having a "commercial
grade file system."  I don't know if that means RMS-type file structures,
or multiple file versions, or what, but it sounds intriguing.  It offers
a faint glimmer of hope -- maybe this would be my type of Unix.  (Although
after having read Tracy Kidder's book nearly 10 years ago, I'm amazed
that Data General computers run at all.)

Item 3:  Whenever a VMS OS-warrior complains about Unix's primitive (or
simple, if you prefer) file system, the comeback given is that the file
system is separate from Unix per se -- you could substitute any file
system you want, theoretically.

OK, it seems that Data General has done just that -- built a Unix with a
different file system.  But if I were to develop software for such a system,
wouldn't the main advantage (to me) of Unix -- relative portability of
applications -- have been tossed out the window?  If a company were to buy
into the Data General version of Unix, wouldn't it be locked into DG just as
hard as a VMSer is locked into DEC?

Does anyone on this list know the details DG's "commercial grade" file
system, or has anyone taken an interest in it?  Or is it of no interest
precisely because it is different?

John Gorentz

 
 
 

OS-wars: Unix file system(s)

Post by Scott Schwar » Sun, 02 Dec 1990 08:22:00


No flames, no OS wars, just an honest discussion of interesting
issues, ok?

Quote:John Gorentz writes:
>Item 1:  I am a VMS partisan wondering if there will ever be (or need to
>be) a Unix in my future.  I don't know much about Unix, but high on my list
>of why I'd like to avoid it is the file system, or lack thereof (sometimes
>referred to as a _feature_ by Unix partisans.)

Yes it is a feature.  In my judgement some of the nice things about it are
  - it is tree structured
  - there are few restriction on filenames
  - files have a simple, flexible structure
  - it provides a generalized namespace for files, devices, and ipc endpoints

Think of it as a RISC filesystem. :-)

Quote:>Item 2:  Recent Data General ads tout their Unix as having a "commercial
>grade file system."  I don't know if that means RMS-type file structures,
>or multiple file versions, or what, but it sounds intriguing.  It offers
>a faint glimmer of hope -- maybe this would be my type of Unix.

Is there any reason these things need to be in the filesystem, as
opposed to a system library?  I mean, if I wrote you a version of the
unix open() system call that did version numbering and put it in your
system library, would that be good enough?

Quote:>Item 3:  Whenever a VMS OS-warrior complains about Unix's primitive (or
>simple, if you prefer) file system, the comeback given is that the file
>system is separate from Unix per se -- you could substitute any file
>system you want, theoretically.

And in practice.  It has been done.

Quote:> If a company were to buy
>into the Data General version of Unix, wouldn't it be locked into DG just as
>hard as a VMSer is locked into DEC?

Sure, if you take advantage of features only DG supplies.  But if
someone like Berkeley, or the Free Software Foundation writes a new
filesystem you could get it easily enough.  

If your system has NFS you can easily build a filesystem yourself and
connect it to your vendor's system by using NFS.  (This gets done
because it is easier than putting new stuff in the kernel, which is
where the filesystem lives right now.)  If your system is MACH your
filesystem is outside the kernel anyway, and you can easily add a
different one.  In any case, the idea is to take your custom filesystem
with you when you move to a different vendor.

Again, though, I have to wonder if most of the stuff that you want
couldn't be supplied with compatability libraries.

--

Now back to our regularly scheduled programming....

 
 
 

OS-wars: Unix file system(s)

Post by Kenneth Adelm » Sun, 02 Dec 1990 07:28:00


Quote:> For what it's worth:       It IS possible to implement alternate file systems
> (on-disk structures) under VMS.  You "simply" have to write an ACP (Ancillary
> Control Process, much like the XQP, but it runs in a separate process) which
> implements it.  One such is supported by DEC and allows you to mount ODS-I
> (eg RSX-11M) disk volumes on VMS systems.  I imagine that it would be possible
> to write one that would let you mount a Unix file system as well.

    It is.  We've written an XQP to implement an NFS Client under VMS.
This allows us to remotely mount Unix filesystems on a VMS machine and
have them appear to VMS as if they were VMS filesystems.

    The great thing about the VMS filesystem is that you can mix-and-match
its components. You can add a new XQP portion but still use the old RMS.

    Witness:

$ nfsmount nfs0: drakes::"/"
%NFSMOUNT-I-MOUNTED, DRAKES::/ NFS mounted on _NFS0:
$ show device/mounted
Device                  Device           Error    Volume         Free  Trans Mnt
 Name                   Status           Count     Label        Blocks Count Cnt
KAOS$DUA0:              Mounted              0  VAXVMSRL5       195235   250   1
NFS0:                   Mounted              0  /                 5884     1   1
$ dire nfs0:[000000]

Directory NFS0:[000000]

.CSHRC;1                  2/2       [DEFAULT]             (RWED,RWED,RE,RE)
.LOGIN;1                  2/2       [DEFAULT]             (RWED,RWED,RE,RE)
000000.DIR;1              3/3       [DEFAULT]             (RWED,RWED,RE,RE)
DEV.DIR;1                11/11      [DEFAULT]             (RWED,RWED,RE,RE)
ETC.DIR;1                 6/6       [DEFAULT]             (RWED,RWED,RE,RE)
HOME.DIR;1                2/2       [DEFAULT]             (RWED,RWED,RE,RE)
LOST$5KFOUND.DIR;1       16/16      [DEFAULT]             (RWED,RWED,RE,RE)
...
(directory edited for brevity)

                                                Kenneth Adelman
                                                TGV, Incorporated

 
 
 

OS-wars: Unix file system(s)

Post by j.. » Sun, 02 Dec 1990 20:36:00


In article <1989Dec6.072218.27...@psuvax1.cs.psu.edu>,
 schwa...@shire.cs.psu.edu (Scott Schwartz) writes:

> No flames, no OS wars, just an honest discussion of interesting
> issues, ok?

Sounds good to me!

> John Gorentz writes:
>>[he's a VMS partisan, and wonders if DG Unix's "commercial grade file
>>system" has RMS-type file structures, or multiple file versions, etc.]

> Is there any reason these things need to be in the filesystem, as
> opposed to a system library?  I mean, if I wrote you a version of the
> unix open() system call that did version numbering and put it in your
> system library, would that be good enough?

First, we VMS types need to clarify something for you Unix fans.  

In VMS the "file system" refers to things like disk volumes, the directory
structure (a tree, much like Unix), file names (not quite as generalized as
Unix's), version numbers, UIC-based and ACL-based file protection, how blocks
on the disk are allocated to files, how we can go from a file name to the
blocks that it owns, and so on.  For disks this is implemented partly (a tiny
bit, really) in the disk device driver and (mostly) in something called the XQP
(eXtended Qio Processor), which is kernel-mode AST-driven code mapped into each
user's process and invoked by the disk driver when necessary (not often).

The file system on VMS is called, variously, Files-11 II, or On-Disk-Structure
(ODS) II.  Its precursor was ODS I, which used numbered (ie [1,4]) directories
and a flat directory structuree, and was used various versions of RSX-11x.  
(Unix fans will no doubt pronounce "ODS" as "odious"...)

And then there is RMS -- Record Management Services -- which deals with the
structure of records within files. That is, it is RMS that knows about file
organization (sequential, relative, or indexed), record format (fixed-length,
variable-length, streamLF, streamCR, stream, or VFC), record attributes (such
as carriage control attributes -- carriage return, none, fortran, or VFC), and
so on.

        By the way, before you blanch in horror at all of those options,
        I should point out that the goal of RMS is to make dealing with all
        types of records in all types of files on all types of devices as
        much the same as possible.  While the setup to $CREATE an indexed
        file on disk must be different from that to $CREATE a sequential
        file on tape (or disk, or mailboxes, or network links), the $GET and
        $PUT calls can be the same in both (and almost all other) cases.  
        Even $OPENs (accessing an existing file) can be the same.  The only
        cases where they're not is when the program is exploiting some
        feature of a particular file organization, like accessing records
        in an indexed file by key value, or doing read-with-timeout on a
        terminal.

RMS is implemented as a set of system services -- ie when you call an RMS
procedure you are calling a little bit of code that does a change mode
instruction, which causes a change mode exception, which changes to (in this
case) exec mode and vectors (via an entry in the system control block, the same
structure through which interrupts are dispatched) to a change mode dispatcher.
The operand to the change mode instruction (which is determined by which RMS
service you called) then is used to transfer control to the correct RMS
procedure.  The only difference between this and the invocation of
kernel-mode services like $QIO and $SETIMR is that RMS services run in
executive mode, which is at an "outer level" of protection from kernel mode
(ie kernel-mode code can access memory that's accessible from executive mode,
but not the other way around).  

Examples of RMS services include $OPEN, $CREATE, $CONNECT, $GET, $PUT,
$FIND (locate a record by key in an indexed or relative file), $CLOSE, and
so on.  

There is obviously some overlap in the functions offered by RMS and the file
system XQP. If you want you can create files, delete files, look up files in
directories, and even do I/O to files in 512-byte (or multiples thereof)
increments, all by doing $QIO calls to the disk driver.  All regular VMS file
protections still apply (since they're implemented by the XQP, not RMS).  
Problems:  You have complete freedom to create files with any data you like
in them... including files that are completely non-understandable to RMS (ie
to any program but yours)... and the program is locked into dealing with files
on disk.  

RMS makes it much easier to perform these functions (other than creating bogus
files :-).  For instance, when you $CREATE an indexed file, RMS puts
information about the index structure in the file header.  Upon a later $OPEN
to such a file, RMS notices this and sets things up appropriately.  This is why
you can do sequential $GETs, and even $PUTs, to an indexed file and never know
the difference.  (RMS knows where the key fields are in your records, so when
you do $PUTs it just updates the indexes accordingly.)

As far as I can tell, mostly from reading and from discussions like this one
(I've typed a few commands at Unix systems here and there, and that's about
it), Unix has no analog to RMS, other than a set of conventions that
cooperating programs (and a few I/O calls) "know about".  For instance, all
programs that handle text files "know" that the line feed character is a
record separator in such files.  

        (Or a line separator, if you absolutely insist that there are no
        "records" in Unix files; I say if it looks like a record separator,
        acts like a record separator, etc., it's a record separator.  But
        never mind.)

Now, to the question:  Why not offer ODS-2- or RMS-like functions under
Unix in a compatibility library?

Well... there are good reasons why we run code that's "close to the operating
system" out of exception handlers rather than as simple callable routines.  

The exception dispatch table (the System Control Block on a VAX) and the change
mode dispatchers (the CASE instructions which transfer to the intended system
service or RMS service, according to the operand of the change mode
instruction) are in pages that are only writable from kernel mode.  This
protects the integrity of the system.  In particular, it prevents the
non-privileged user from replacing these services with ones that are similar
but which break system security.  

For the file system, the reasons for running in kernel mode are clear.  If the
XQP lived in user mode, I could get into it and replace parts of its code and
data.  It's part of my process, remember?  That means that, for instance, all
file protection on the system goes out the window; we might as well set all
files to world:read, write,exec,delete.  And since overall system security is
determined by the file protection on SYSUAF.DAT...

What about RMS?  What if RMS was implemented as routines which were called and
executed in user mode?  The exploitation methods there are a little less
obvious, and can't be explained in a news posting that's way too long already.
Suffice it to say that RMS and a lot of its data structures live in system
address space, ie they're mapped into the address space that's visible to all
processes (but not necessarily accessable from user mode).  They have to be,
for RMS supports shared file access, complete with record locking.  Though I
don't know if you could crack overall system security by twiddling this stuff
in the general case, you could certainly get into SYSUAF.DAT *if* some other
process was accessing it at the time.  Just for example.

For what it's worth:  It IS possible to implement alternate file systems
(on-disk structures) under VMS.  You "simply" have to write an ACP (Ancillary
Control Process, much like the XQP, but it runs in a separate process) which
implements it.  One such is supported by DEC and allows you to mount ODS-I
(eg RSX-11M) disk volumes on VMS systems.  I imagine that it would be possible
to write one that would let you mount a Unix file system as well.  

Alternate record-structure-implementation packages (to exist alongside RMS,
implemented as user-written system services) are possible too.  But it'd be
difficult to make them available "transparently", ie to give them the same
names as RMS services.  

Now, Unix fans, can you answer one for me?  

Suppose I have an old Fortran program that emits print records with a leading
"carriage control" character (1 for top of form, etc.).  On VMS this works
because it opens its output files with "Carriage control = Fortran".  This
does not effect the data on the disk at all, but when such a file is
printed, or otherwise copied (eg TYPEd) on a "carriage control" device,
this attribute causes the leading carriage control byte to automatically
be interpreted by the printer or terminal's device driver.  

On Unix I could see this being handled by a filter (pipe the output through
a filter that interprets the cc byte, and send the result to the printer).  
But... here's a case where (I don't think) a filter will help:

In a text file on Unix, lines are separated by line feeds.  Now, most "human
readable" I/O devices don't want just line feeds.  When such text files are
sent to printers or terminals, somebody (the driver?) must precede or follow
each line feed with a carriage return.  

This works fine until you need to do something like download a bitmapped
font file to a laser printer.  How do you do this under Unix?  If you use
the same command that you would use to print any other file, I'd expect that
the printer driver would expand any LFs it happens to find in the font data
to CRLF (or LFCR, whatever).  This isn't what's wanted.  

Ditto if the printer driver (or spooler or whatever) is usually called upon
to expand tabs to spaces, simulate form feeds on a printer that doesn't
have a hardware form feed, etc.

On VMS I can just PRINT the font file, because I'll have created it with
"Carriage control = none".  This does not affect the data on disk at all, but
the attribute is automatically "passed along" when ...

read more »

 
 
 

OS-wars: Unix file system(s)

Post by David S. Masters » Sun, 02 Dec 1990 23:47:00


   Now, Unix fans, can you answer one for me?

I'll give it a shot -- I don't claim to be an expert.

   [munched example explaination concerning printing "funny" files]

   How are cases like this handled on Unix?  Is there a special print command,
   or something, that says "print this in raw mode"?  One nice answer might be
   to configure the printer to NOT need CRs, only LFs... but I can see that
   that isn't what's done, as we have a printer here that's on a print sharer
   btw VMS and Unix systems, and the printer is set up to need both CRs and
   LFs.

Unix seems to have been built on an object-oriented paradigm before
"object-orientation" was well-known.  Basically, one builds tools to do very
specific functions and then builds on those tools to create other tools that
do higher level functions.  This printer case is an example of that
capability.

The Unix printer driver only needs to understand the mechanism for sending an
eight-bit value over the communication (for instance, parallel) line to the
printer.  The expectation is that eight-bit values that deals with will be
understood by the printer.  This is an assumption, but when you build on these
assumptions, you can achieve a consistent whole.

On top of the printer driver is typically a program that controls the printer.
As the printer is a device like any other device, it has a file system
connection to it that can be protected from unauthorized use (if desired).
The controlling program (the daemon) is authorized to open this device and
send things to it.  This program is configured to know how to handle the
printer(s) via a configuration file.

Part of this daemon's responsibility is to make the file intelligible to the
printer.  To do this, it may call upon the services of filters prescribed in
the configuration file.  For instance, there may be a filter to convert
Postscript form into codes understood by the printer.  In theory, this
conversion could be done while the daemon goes off and does something else
(and be informed when ready later), but typically its a straight filter.

The command to "print" a file in Unix is just a program like any other.  Its
job is to register the file with the print queue with any command line options
specified.  Simple queueing mechanism with privileges for getting to certain
areas of the system.  These privileges can only be given by something having
the privileges to give (SUID), so a user cannot "easily" replace this
mechanism and subvert it.

The configuration file in this may actually have several configurations for a
single printer at one time.  For instance, a laser printer might be called
POSTSCRIPT when a user wants to send Postscript files to it or LA23 when a
user wants to send a straight ASCII file to it.  Usually, the user needs to
know what type of file it is so he can send it to the right printer name.
However, this is not a hard thing to do and some people have built simple
commands that guess the type and send it to the right printer.
--
===================================================================
David Masterson                                 Consilium, Inc.
uunet!cimshop!davidm                            Mt. View, CA  94043
===================================================================
"If someone thinks they know what I said, then I didn't say it!"