In article <1989Dec6.072218.27...@psuvax1.cs.psu.edu>,
schwa...@shire.cs.psu.edu (Scott Schwartz) writes:
> No flames, no OS wars, just an honest discussion of interesting
> issues, ok?
Sounds good to me!
> John Gorentz writes:
>>[he's a VMS partisan, and wonders if DG Unix's "commercial grade file
>>system" has RMS-type file structures, or multiple file versions, etc.]
> Is there any reason these things need to be in the filesystem, as
> opposed to a system library? I mean, if I wrote you a version of the
> unix open() system call that did version numbering and put it in your
> system library, would that be good enough?
First, we VMS types need to clarify something for you Unix fans.
In VMS the "file system" refers to things like disk volumes, the directory
structure (a tree, much like Unix), file names (not quite as generalized as
Unix's), version numbers, UIC-based and ACL-based file protection, how blocks
on the disk are allocated to files, how we can go from a file name to the
blocks that it owns, and so on. For disks this is implemented partly (a tiny
bit, really) in the disk device driver and (mostly) in something called the XQP
(eXtended Qio Processor), which is kernel-mode AST-driven code mapped into each
user's process and invoked by the disk driver when necessary (not often).
The file system on VMS is called, variously, Files-11 II, or On-Disk-Structure
(ODS) II. Its precursor was ODS I, which used numbered (ie [1,4]) directories
and a flat directory structuree, and was used various versions of RSX-11x.
(Unix fans will no doubt pronounce "ODS" as "odious"...)
And then there is RMS -- Record Management Services -- which deals with the
structure of records within files. That is, it is RMS that knows about file
organization (sequential, relative, or indexed), record format (fixed-length,
variable-length, streamLF, streamCR, stream, or VFC), record attributes (such
as carriage control attributes -- carriage return, none, fortran, or VFC), and
By the way, before you blanch in horror at all of those options,
I should point out that the goal of RMS is to make dealing with all
types of records in all types of files on all types of devices as
much the same as possible. While the setup to $CREATE an indexed
file on disk must be different from that to $CREATE a sequential
file on tape (or disk, or mailboxes, or network links), the $GET and
$PUT calls can be the same in both (and almost all other) cases.
Even $OPENs (accessing an existing file) can be the same. The only
cases where they're not is when the program is exploiting some
feature of a particular file organization, like accessing records
in an indexed file by key value, or doing read-with-timeout on a
RMS is implemented as a set of system services -- ie when you call an RMS
procedure you are calling a little bit of code that does a change mode
instruction, which causes a change mode exception, which changes to (in this
case) exec mode and vectors (via an entry in the system control block, the same
structure through which interrupts are dispatched) to a change mode dispatcher.
The operand to the change mode instruction (which is determined by which RMS
service you called) then is used to transfer control to the correct RMS
procedure. The only difference between this and the invocation of
kernel-mode services like $QIO and $SETIMR is that RMS services run in
executive mode, which is at an "outer level" of protection from kernel mode
(ie kernel-mode code can access memory that's accessible from executive mode,
but not the other way around).
Examples of RMS services include $OPEN, $CREATE, $CONNECT, $GET, $PUT,
$FIND (locate a record by key in an indexed or relative file), $CLOSE, and
There is obviously some overlap in the functions offered by RMS and the file
system XQP. If you want you can create files, delete files, look up files in
directories, and even do I/O to files in 512-byte (or multiples thereof)
increments, all by doing $QIO calls to the disk driver. All regular VMS file
protections still apply (since they're implemented by the XQP, not RMS).
Problems: You have complete freedom to create files with any data you like
in them... including files that are completely non-understandable to RMS (ie
to any program but yours)... and the program is locked into dealing with files
RMS makes it much easier to perform these functions (other than creating bogus
files :-). For instance, when you $CREATE an indexed file, RMS puts
information about the index structure in the file header. Upon a later $OPEN
to such a file, RMS notices this and sets things up appropriately. This is why
you can do sequential $GETs, and even $PUTs, to an indexed file and never know
the difference. (RMS knows where the key fields are in your records, so when
you do $PUTs it just updates the indexes accordingly.)
As far as I can tell, mostly from reading and from discussions like this one
(I've typed a few commands at Unix systems here and there, and that's about
it), Unix has no analog to RMS, other than a set of conventions that
cooperating programs (and a few I/O calls) "know about". For instance, all
programs that handle text files "know" that the line feed character is a
record separator in such files.
(Or a line separator, if you absolutely insist that there are no
"records" in Unix files; I say if it looks like a record separator,
acts like a record separator, etc., it's a record separator. But
Now, to the question: Why not offer ODS-2- or RMS-like functions under
Unix in a compatibility library?
Well... there are good reasons why we run code that's "close to the operating
system" out of exception handlers rather than as simple callable routines.
The exception dispatch table (the System Control Block on a VAX) and the change
mode dispatchers (the CASE instructions which transfer to the intended system
service or RMS service, according to the operand of the change mode
instruction) are in pages that are only writable from kernel mode. This
protects the integrity of the system. In particular, it prevents the
non-privileged user from replacing these services with ones that are similar
but which break system security.
For the file system, the reasons for running in kernel mode are clear. If the
XQP lived in user mode, I could get into it and replace parts of its code and
data. It's part of my process, remember? That means that, for instance, all
file protection on the system goes out the window; we might as well set all
files to world:read, write,exec,delete. And since overall system security is
determined by the file protection on SYSUAF.DAT...
What about RMS? What if RMS was implemented as routines which were called and
executed in user mode? The exploitation methods there are a little less
obvious, and can't be explained in a news posting that's way too long already.
Suffice it to say that RMS and a lot of its data structures live in system
address space, ie they're mapped into the address space that's visible to all
processes (but not necessarily accessable from user mode). They have to be,
for RMS supports shared file access, complete with record locking. Though I
don't know if you could crack overall system security by twiddling this stuff
in the general case, you could certainly get into SYSUAF.DAT *if* some other
process was accessing it at the time. Just for example.
For what it's worth: It IS possible to implement alternate file systems
(on-disk structures) under VMS. You "simply" have to write an ACP (Ancillary
Control Process, much like the XQP, but it runs in a separate process) which
implements it. One such is supported by DEC and allows you to mount ODS-I
(eg RSX-11M) disk volumes on VMS systems. I imagine that it would be possible
to write one that would let you mount a Unix file system as well.
Alternate record-structure-implementation packages (to exist alongside RMS,
implemented as user-written system services) are possible too. But it'd be
difficult to make them available "transparently", ie to give them the same
names as RMS services.
Now, Unix fans, can you answer one for me?
Suppose I have an old Fortran program that emits print records with a leading
"carriage control" character (1 for top of form, etc.). On VMS this works
because it opens its output files with "Carriage control = Fortran". This
does not effect the data on the disk at all, but when such a file is
printed, or otherwise copied (eg TYPEd) on a "carriage control" device,
this attribute causes the leading carriage control byte to automatically
be interpreted by the printer or terminal's device driver.
On Unix I could see this being handled by a filter (pipe the output through
a filter that interprets the cc byte, and send the result to the printer).
But... here's a case where (I don't think) a filter will help:
In a text file on Unix, lines are separated by line feeds. Now, most "human
readable" I/O devices don't want just line feeds. When such text files are
sent to printers or terminals, somebody (the driver?) must precede or follow
each line feed with a carriage return.
This works fine until you need to do something like download a bitmapped
font file to a laser printer. How do you do this under Unix? If you use
the same command that you would use to print any other file, I'd expect that
the printer driver would expand any LFs it happens to find in the font data
to CRLF (or LFCR, whatever). This isn't what's wanted.
Ditto if the printer driver (or spooler or whatever) is usually called upon
to expand tabs to spaces, simulate form feeds on a printer that doesn't
have a hardware form feed, etc.
On VMS I can just PRINT the font file, because I'll have created it with
"Carriage control = none". This does not affect the data on disk at all, but
the attribute is automatically "passed along" when
read more »