Very Computer

I am new to Solaris system administration.  I have just been
given a SunFire v240 with 2 GB RAM and a pair of 36 GB SCSI
drives.  I want to have maximum flexibility and uptime (no
surprise!)  After reading a bunch of Solaris 9 manuals, I came up
with the following:

Disk layout:
slice 0: most of the disk--break into SVM soft partitions as needed
slice 1: swap (2 GB)
slice 3: dedicated crash dump (2 GB) partition

I would also use Solaris Volume Manger (SVM) to mirror slices 0
and 1 to the second disk.  Then, I could create soft partitions
on slice 1 for /, /var, /export, etc.  If needed, I could also
grow those soft partitions and filesystems as the need arises.

To do routine patches, I would use Live Upgrade to create one
or more alternate boot environments on additional soft
partitions.  Once I reboot the system into its new environment
after patching and check that everything works, I could remove
the soft partitions that contained the earlier system
filesystems.  For routine backups, I would use fssnap to make
online filesystem backups.

But, older, wiser Solaris system administrators do not agree with
me.  They would rather I put all the system filesystems (/, /var,
/usr, etc) on slices (no soft partitions) and deal with 7 slice
limit.  They also do think live upgrade is overkill for patches
(but almost every Sun patch I have seen wants you to go to single
user mode and reboot after it is done?)

I wonder why SVM and soft partitions are not integrated into the
Solaris OE install CD--when I tried it, the only option I was
given was to everything onto disk slices.

So, is their any consensus or "best practices" when it comes to
using SVM/soft partitions and Solaris OE system filesystems?

Thanks,
Doug

I think you *must* have at least one small slice for / otherwise the boot
loader seems to have problems finding the kernel, putting an abrupt end to
what would have been otherwise a successful boot. ;) At least, this was
true for Solaris 8. No idea if this is still a requirement for Solaris 9.

We recently went through this whole exercise with a 280R and soft
partitions with a similar setup as you proposed.

Quote:> I would also use Solaris Volume Manger (SVM) to mirror slices 0
> and 1 to the second disk.  Then, I could create soft partitions
> on slice 1 for /, /var, /export, etc.  If needed, I could also
> grow those soft partitions and filesystems as the need arises.

Sounds reasonable.

Quote:> To do routine patches, I would use Live Upgrade to create one
> or more alternate boot environments on additional soft
> partitions.  Once I reboot the system into its new environment
> after patching and check that everything works, I could remove
> the soft partitions that contained the earlier system
> filesystems.  For routine backups, I would use fssnap to make
> online filesystem backups.

Sounds good to me.

Quote:> But, older, wiser Solaris system administrators do not agree with

Well, there are some older, wiser Solaris system administrators that agrees
with you. Some will, some won't. Such is the nature and variety of human
experience. :)

The important part is that you must have well-grounded reasons for the
various procedures and configurations that you want, and make sure you
aren't busting some limit or rule... or creating a difficult to support
situation in the long term.

From what you've described so far, sounds good to me with exception of
missing / on its own slice.

Quote:> me.  They would rather I put all the system filesystems (/, /var,
> /usr, etc) on slices (no soft partitions) and deal with 7 slice
> limit.  They also do think live upgrade is overkill for patches
> (but almost every Sun patch I have seen wants you to go to single
> user mode and reboot after it is done?)

Well, it depends... some places have complex software setups that are
*really* sensitive to changes in OS or application behavior, like these
introduced by patches sometimes, and may have higher availability
requirements. For these setups, rolling back to a known good setup simply
by booting off the original drive then re-mirroring or re-doing patches
after determining the culprit, is a hard requirement at some sites.

At other sites, they can back out the patches and reboot if they had
installed patches that saved the previous version (for easy backout).

Sometimes some sites prefer to apply minor patches with a save option (for
easy backout if necessary), and to do the break mirror, patch one drive if
it's a major or significant patch.

It just depends on your needs and requirements, really.

Quote:> I wonder why SVM and soft partitions are not integrated into the
> Solaris OE install CD--when I tried it, the only option I was
> given was to everything onto disk slices.

That's a good question. Perhaps Sun wanted to preserve expected behavior
since there are a large base of existing Solaris system admins (including
certified ones) that "knows" slice x has y, or existing homemade (eg
Jumpstart) install scripts that makes these assumptions and would need to
be overhauled to use a SVM+soft partition environment properly?

Don't know the reason... can only speculate. Perhaps someone from Sun will
be able to give a more definitive answer.

Quote:> So, is their any consensus or "best practices" when it comes to
> using SVM/soft partitions and Solaris OE system filesystems?

In general, I see use of SVM and soft partitions as being a big win for
system administration in the long term and strongly encourage its adoption
and use whereever possible.

-Dan

If you're already using slice 1 for swap, how are you going to put other
items on it?  Perhaps a typo?

You cannot put root on a soft partition.  It needs to be real.

Depending on the purpose of the box and where files go, you may prefer
to simply merge most OS slices (/var, /export) into root and not deal
with soft partitions for any OS filesystem.

Quote:> To do routine patches, I would use Live Upgrade to create one
> or more alternate boot environments on additional soft
> partitions.  Once I reboot the system into its new environment
> after patching and check that everything works, I could remove
> the soft partitions that contained the earlier system
> filesystems.  For routine backups, I would use fssnap to make
> online filesystem backups.

I have no idea how you're going to boot into a soft partition.  The OBP
must find a filesystem on a real slice, and that filesystem must have
the kernel on it.

Quote:> But, older, wiser Solaris system administrators do not agree with
> me.  They would rather I put all the system filesystems (/, /var,
> /usr, etc) on slices (no soft partitions) and deal with 7 slice
> limit.  They also do think live upgrade is overkill for patches
> (but almost every Sun patch I have seen wants you to go to single
> user mode and reboot after it is done?)
> I wonder why SVM and soft partitions are not integrated into the
> Solaris OE install CD--when I tried it, the only option I was
> given was to everything onto disk slices.

Because SVM and soft partitions are not integrated into the OBP (which
looks at VTOC slices) and the UFS boot loader (which knows how to find a
kernel on a UFS filesystem on a VTOC slice).  Because of those
limitations, root (and probably /usr if separate) must be "normal".

Presumably the installer could allow some non-critical OS and other
filesystems to be integrated into soft partitions, but no one has
bothered to change the installer to support it.  SVM wasn't part of the
OS until 9 anyway.

Quote:> So, is their any consensus or "best practices" when it comes to
> using SVM/soft partitions and Solaris OE system filesystems?

I've yet to use soft partitions on any OS filesystem myself.  Then
again, I tend to have only root and swap on the OS disk anyway, so SPs
would have little use.

--

Senior Technical Consultant         TAOS            http://www.taos.com/
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >

Quote:

> Because SVM and soft partitions are not integrated into the OBP (which
> looks at VTOC slices) and the UFS boot loader (which knows how to find a
> kernel on a UFS filesystem on a VTOC slice).  Because of those
> limitations, root (and probably /usr if separate) must be "normal".

Incidentally, /usr doesn't need to be on a real slice -- the Sol 8 box that
we recently installed only has / on a real slice, and all the rest
(including swap) are on soft partitions. Seems to work ok. Haven't tested
it extensively with odd situations (fsck, major upgrades, etc) so don't
know what the downsides are, if any.

Quote:> I've yet to use soft partitions on any OS filesystem myself.  Then
> again, I tend to have only root and swap on the OS disk anyway, so SPs
> would have little use.

Not everybody runs with VxVM for non-boot drives so it's still got a place
if one is only planning to use SDS for their setup. We use VxVM where it
makes sense, and SDS where it's more practical.

-Dan

And /usr is a separate filesystem?  If so that's new.  Older versions of
SDS wouldn't let you do anything with / or /usr except mirror because
components of /usr were required to load before SDS could start doing
the magic.

Quote:>> I've yet to use soft partitions on any OS filesystem myself.  Then
>> again, I tend to have only root and swap on the OS disk anyway, so SPs
>> would have little use.
> Not everybody runs with VxVM for non-boot drives so it's still got a place
> if one is only planning to use SDS for their setup. We use VxVM where it
> makes sense, and SDS where it's more practical.

I'm not talking SDS vs. VxVM here.  I'm just saying that on the *OS*
drives, where I only have one filesystem, the soft partitions wouldn't
be a big help.

--

Senior Technical Consultant         TAOS            http://www.taos.com/
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >

Yup, /usr is a separate filesystem and double-checked it was a soft
partition. (metastat reports it as being one if that's the case.)

# df -k /usr
Filesystem            kbytes    used   avail capacity  Mounted on
/dev/md/dsk/d3       8268461  221838 7963939     3%    /usr

# metastat d3
d3: Soft Partition
    Component: d107
    State: Okay
    Size: 16793600 blocks
        Extent              Start Block              Block count
             0                 10485763                 16793600

(it's showing only one mirror half because the other half is currently
detached for some rearranging work)

Quote:> SDS wouldn't let you do anything with / or /usr except mirror because
> components of /usr were required to load before SDS could start doing
> the magic.

Don't know what to say except it seems to work for us -- the box's booted
and in full multiuser mode in the other window. :) Perhaps some sort of
magic worked into the late Sol8 code?

This is latest version of Sol8 + SDS code, with all patches applied and
current as of perhaps a week ago? Although due to a Sol8 LUN bug, looks
like we'll reinstall it with Sol9 since Sun has declined to backport the
LUN fix to Sol8, and I can't afford to lose 150 GB of usable data space
with a Sol8 workaround.

Quote:> I'm not talking SDS vs. VxVM here.  I'm just saying that on the *OS*
> drives, where I only have one filesystem, the soft partitions wouldn't
> be a big help.

I'd still see it as a win for either LVM manager on the boot drives for
purposes of growing filesystems on the fly as needs requires it, adding new
filesystems temporarily (eg to store a large ISO image for 10 mins without
doing an irreversible growth of /usr or whatever) without having to worry
about available slices and so forth.

-Dan

I can confirm that.  I just configured several soft partitions on a
disk slice.
I then used Live Upgrade to create a clone of my Solaris 9 system.  I
had LU put / on a plain disk slice and had it put /usr on one of the
soft partitions.  Then I activated the alternate boot environment,
rebooted and the system came up.  I even put the /usr partition on a
fragmented soft partition (I grew it after adding other soft
partitions, so its extents are non-contiguous.)  So, I guess the OBP
bootloader does not care about /usr to get the system going (SVM/soft
partition kernel drivers are probably loaded before /usr needs to be
mounted?)

So, thanks to both of you Darren and Dan for your quick, insightful
and very helpful comments.  I had not previously considered the role
of OBP bootloader with respect to booting to a root filesystem on a
soft partition.  I had read in a Sun adverti*t that later editions
of Solaris 9 come with LU 2.1 which supports SVM.  I had assumed that
meant you could put a bootable system on any SVM component.  But when
I tried it, LU complained "the only metadevice that supports the root
filesystem are a stripe with only a single disk or a mirror on a
single-disk stripe."

For practical purposes, I guess it is simplest just to dedicate a set
of disk drives for the system filesystems (probably just a single, big
disk slice) and use SVM for mirroring if desired.  The non-system
files and data can go on other drives where SVM or VxVM can be used as
needed.

Still it would be nice (especially on single disk x86 systems) to have
the flexiblity to boot directly to soft partitions.  Maybe OBP will
get added functionality or maybe a Linux-like multistage boot loader
could overcome this limitation.  Or, taking another idea from linux,
get by with a really small /boot slice containing just a kernel with
built-in SVM drivers or use an initrd style system to boot the other
filesystems.

Thanks again,
Doug