It would seem that a large class of UNIX security holes are based
on tricking some suid program into writing to files which it's
not supposed to.
Swapping euid and uid works, but is error - prone (out of 100
opens, you'll probably only secure 99).
I would suggest adding a special file type to make this impossible.
Suppose we have a special bit associated with each file (let's
call it the "system bit"), turned on for all security - related
files (/etc/passwd, /etc/hosts.equiv, /etc/inetd.conf, ...)
The system insures that they have to be opened with a special
flag for writing, say with
open("/etc/passwd", O_RDWR | O_SYSTEM);
Unless the O_SYSTEM flag is present, all processes (including, and
especially those with root privileges) get 'permission denied'.
This would mean that a cracker who has subverted some suid root
program into appending a new root id to /etc/passwd won't
succeed, because said suid root program won't open any random
file with that flag set.
Programs like 'useradd', 'userdel' etc. will open /etc/passwd with
this flag, but those won't be userid root; the most common editors
could be patched easily to detect the presence of this flag.
Bang, goes one class of security risks inherent in UNIX systems.
This would seem to be such a trivial hack to, let's say, Linux, that
I'd be surprised if nobody had thought of this before; yet, at least
on the UNIX systems I know, nobody does this. (I once heard something
about BSD 4.4 implementing something called 'immutable files', which
may be related, but I dont't know BSD 4.4).
The joy of engineering is to find a straight line on a double