Solaris 10 N1 Containers vs AIX 5.3 Micro-Partitioning

Solaris 10 N1 Containers vs AIX 5.3 Micro-Partitioning

Post by Rodrick Brow » Sun, 31 Oct 2004 11:07:40



What's everyone's take on this Sun's Zones being heavily software
dependant vs IBM approach which uses more hardware for isolation and
seperation for their logical os partitioning.

I have a big project coming up 4-6 months and one of the key factors in
determining if I should deploy on AIX 5.3 or Solaris 10 will be the
virtualization features offered by both and ability to have granular
processor utilization the ability to break a physical processor into
smaller virtual processors.

Solaris 10
Allows up to 8196 zones
Single Kernel managing all zones

I've been using zones for the past month or so now and i'm very happy
with the flexability, speed and over all ease of use. The best possible
use of zones would probably be in the higher end Sun Fire servers ie
the midrange and high end stuff because of the added ras and
availabiltiy features.

AIX 5.3
LPAR Micro partition allows a maximum of 10 LPAR's per physical CPU or
a maximum of 254 LPARS (Logical Partitions)
Multiple kernel managing each independant LPAR

I'm not sure how quickly 5.3 will be accepted among the AIX community
or even the Micro technology features most AIX sysadmins i've meet seem
to be very dependent on IBM services and usually use the bare minmim
AIX feature set,  so bugs and user accepts is a bit of a concern to me.
I'm very Pro Solaris and coming from a Solaris background were most Sun
admins are very feature rich and tend to utilize every feature Solaris
OS offers I'm not worried about this with Solaris 10 at all :)

Just food for taught.

--
Unix Systems Engineer
The City of New York
Dept. of Information Technology
http://www.nyc.gov/doitt

http://www.rodrickbrown.com

 
 
 

Solaris 10 N1 Containers vs AIX 5.3 Micro-Partitioning

Post by Nathan Dietsc » Sun, 31 Oct 2004 15:24:05


Hello Rodrick,


> What's everyone's take on this Sun's Zones being heavily software
> dependant vs IBM approach which uses more hardware for isolation and
> seperation for their logical os partitioning.

The main problem I see with Zones can also be seen as its largest
benefit; all of the Zones run the same kernel level.

In a situation where you are running an ISP and doing virtual hosting,
this is great. However, in a situation where you want to run different
instances of an application on different patch-levels (Solaris) for
testing, Zones do not help.

 From a software maintenance perspective, Zones are a good thing. All
your system administrators have to take care of is one OS, this is good.

 From a patch-testing perspective, all your Zones must run the same
patch-level. This is not good.

So to answer your question, I think the way IBM does things in hardware
is good and also the way Sun handle it in software is good.
Like always: It depends on your requirements.

I would assume the software way of doing things would be more scalable
because you would not have the overhead of a VM. I am not sure how
granular the breakdown of the IBM machines would be and I am not an AIX
admin, so I may be wrong here.

I would be interested to see what the AIX admins have to say about it.

Kind Regards,

Nathan Dietsch

 
 
 

Solaris 10 N1 Containers vs AIX 5.3 Micro-Partitioning

Post by Adrian Bridge » Tue, 02 Nov 2004 03:38:27


[snip]

Quote:>  From a patch-testing perspective, all your Zones must run the same
> patch-level. This is not good.

Zones start to sound awfully like WLM - you look after one OS but have
some barriers between images.  TBH I don't much care about having
seperate adapters etc etc, since that is only going to add to the cost
of a machine (always a big problem with LPARs - although with 5.3 you
can share them if you are willing to use an IO server partition (which
due to the granularity of uLPARs looks like it's best with a whole CPU
to avoid latency issues))

WLM lets me control CPU usage, memory usage, disk bandwidth and that
is what I care about.  I'm alway suprised at how few people use WLM -
it seems the best way to get lower admin cost with higher box
utilisation.  If you want true isolation get seperate boxes -
otherwise you are still going to need outages to put that latest
firmware on.

Adrian

 
 
 

Solaris 10 N1 Containers vs AIX 5.3 Micro-Partitioning

Post by Gundam17 » Thu, 04 Nov 2004 03:13:19


Hi,

Sun Containers is actually AIX WLM.
It is the way sun market it.

Basically, multiple containers in one OS instance.
OS down all containers gone.

For pSeries, besides WLM, in p5, there is this
Partition Load Manager which allows automate managing of
resources across partitions which Sun and Hp do not have.

-Thanks.


> What's everyone's take on this Sun's Zones being heavily software
> dependant vs IBM approach which uses more hardware for isolation and
> seperation for their logical os partitioning.

> I have a big project coming up 4-6 months and one of the key factors in
> determining if I should deploy on AIX 5.3 or Solaris 10 will be the
> virtualization features offered by both and ability to have granular
> processor utilization the ability to break a physical processor into
> smaller virtual processors.

> Solaris 10
> Allows up to 8196 zones
> Single Kernel managing all zones

> I've been using zones for the past month or so now and i'm very happy
> with the flexability, speed and over all ease of use. The best possible
> use of zones would probably be in the higher end Sun Fire servers ie
> the midrange and high end stuff because of the added ras and
> availabiltiy features.

> AIX 5.3
> LPAR Micro partition allows a maximum of 10 LPAR's per physical CPU or
> a maximum of 254 LPARS (Logical Partitions)
> Multiple kernel managing each independant LPAR

> I'm not sure how quickly 5.3 will be accepted among the AIX community
> or even the Micro technology features most AIX sysadmins i've meet seem
> to be very dependent on IBM services and usually use the bare minmim
> AIX feature set,  so bugs and user accepts is a bit of a concern to me.
> I'm very Pro Solaris and coming from a Solaris background were most Sun
> admins are very feature rich and tend to utilize every feature Solaris
> OS offers I'm not worried about this with Solaris 10 at all :)

> Just food for taught.

> --
> Unix Systems Engineer
> The City of New York
> Dept. of Information Technology
> http://www.nyc.gov/doitt

> http://www.rodrickbrown.com

 
 
 

Solaris 10 N1 Containers vs AIX 5.3 Micro-Partitioning

Post by Nigel P. Longbotto » Sun, 07 Nov 2004 05:20:36




> What's everyone's take on this Sun's Zones being heavily software
> dependant vs IBM approach which uses more hardware for isolation and
> seperation for their logical os partitioning.

> I have a big project coming up 4-6 months and one of the key factors in
> determining if I should deploy on AIX 5.3 or Solaris 10 will be the
> virtualization features offered by both and ability to have granular
> processor utilization the ability to break a physical processor into
> smaller virtual processors.

> Solaris 10
> Allows up to 8196 zones
> Single Kernel managing all zones

> I've been using zones for the past month or so now and i'm very happy
> with the flexability, speed and over all ease of use. The best possible
> use of zones would probably be in the higher end Sun Fire servers ie
> the midrange and high end stuff because of the added ras and
> availabiltiy features.

> AIX 5.3
> LPAR Micro partition allows a maximum of 10 LPAR's per physical CPU or
> a maximum of 254 LPARS (Logical Partitions)
> Multiple kernel managing each independant LPAR

> I'm not sure how quickly 5.3 will be accepted among the AIX community
> or even the Micro technology features most AIX sysadmins i've meet seem
> to be very dependent on IBM services and usually use the bare minmim
> AIX feature set,  so bugs and user accepts is a bit of a concern to me.
> I'm very Pro Solaris and coming from a Solaris background were most Sun
> admins are very feature rich and tend to utilize every feature Solaris
> OS offers I'm not worried about this with Solaris 10 at all :)

> Just food for taught.

What IBM have done with the p-series is steal the ideas from their mainframe
products and incorporate them in AIX. The LPARS in AIX are very much
independent - each one contains a completely separate installation of the OS
and consists of completely separate hardware - although the hardware may
reside in the same physical box. A good example is that if you have a SCSI
card that has a CD-ROM and tape drive attached - you can assign that to an
LPAR. The other LPARS cannot see or touch this SCSI card or it's devices.
You can then de-allocate this SCSI card from one LPAR then re-allocate to
another LPAR. Not usually worth doing. Where AIX does come in is in large
CPU based servers. Lets say you have 24 CPU's in a server and 12 are
allocated to each LPAR - remember these are completely separate entities! If
box B is struggling, you can deallocate at a hardware level, a number of
these CPU's from box A and re-allocate these to box B.
 
 
 

Solaris 10 N1 Containers vs AIX 5.3 Micro-Partitioning

Post by Bernardo Cabra » Sun, 07 Nov 2004 08:47:07


Quote:

> While the OS is running?  How does it deal with hardware all of a sudden
> "going away"?  On Solaris, various data structures are created on boot
> and are dynamic.  So to change them, you have to reboot.

Can you elaborate more on your question/answer? IBM has been doing DLPAR
on AIX since 10/2002, both for memory and CPU's. And you can add to that
CuOD. And they have been doing CPU dynamic deallocation since at least
2000 for CPU faults. So, it's nothing too new... just a little bit
strange! And I'm just talking about CPU/memory.

BC

 
 
 

Solaris 10 N1 Containers vs AIX 5.3 Micro-Partitioning

Post by Nigel P. Longbotto » Sun, 07 Nov 2004 20:20:02


On 6/11/04 7:23 am, in article




>>> While the OS is running?  How does it deal with hardware all of a sudden
>>> "going away"?  On Solaris, various data structures are created on boot
>>> and are dynamic.  So to change them, you have to reboot.

>> Can you elaborate more on your question/answer? IBM has been doing DLPAR
>> on AIX since 10/2002, both for memory and CPU's. And you can add to that
>> CuOD. And they have been doing CPU dynamic deallocation since at least
>> 2000 for CPU faults. So, it's nothing too new... just a little bit
>> strange! And I'm just talking about CPU/memory.

>> BC

> I meant that Solaris creates static kernel datastructures on boot, not
> dynamic as I stated.   I have no experience with the newer systems
> beyond E4500 or E5500.  Those systems required Solaris be shutdown to
> replace faulty hardware or install new hardware (I/O boards, tape
> drives, anything that's electrically connected to the system) as none of
> it was hot-plugable.

> I understand you can deallocate memory and CPU modules from a running OS
> and allocate it to another instance now (I think that's Zones on Solaris
> 10), but to remove or change hardware, you have to take the system down.

> Is IBM hardware hotplugable and the system tolerant of changes in the
> hardware, so it can all stay up and running while the hardware is being
> changed out?  That'd be like a car allowing cylinders to be replaced or
> added while it's running.  How do they do that?

On the p-series you can de-allocate CPUs. IBM has a facility where you ask a
CPU to be deallocated. AIX then stops allocating processes to that processor
and drains the run queue. The processor is then marked as offline. You can
then re-allocate that processor another LPAR. You then tell that LPAR that
it has a new processor and it starts using it.

Same with adapter cards. You tell AIX you are placing an adapter card
offline. The os stops using it and dynamically dealloctes it. Remember that
AIX has a dynamic kernel whereas solaris has a static one (i.e. /etc/system
on Solaris - there is no equivalent on AIX as it allocates as much as is
needed but you can set limits in /etc/security/limits). Anyway, once the
card is marked offline and shows as 'defined' in AIX, you can use the
hot-plug manager to set that card for removal and then you unplug with the
system up and fully working. All others LPARS on that box are completely
unaffected. IBM have done some really clever stuff with the p-series and AIX
5.2+

 
 
 

Solaris 10 N1 Containers vs AIX 5.3 Micro-Partitioning

Post by Casper H.S. Di » Sun, 07 Nov 2004 20:38:19



>I meant that Solaris creates static kernel datastructures on boot, not
>dynamic as I stated.   I have no experience with the newer systems
>beyond E4500 or E5500.  Those systems required Solaris be shutdown to
>replace faulty hardware or install new hardware (I/O boards, tape
>drives, anything that's electrically connected to the system) as none of
>it was hot-plugable.

I'm fairly sure most of the higher end systems of that era did allow
for Dynamic Reconfiguration of CPU and memory boards.

See, e.g.,: http://sunsolve.sun.com/data/805/805-3530/pdf/805-3530-10.pdf

This is for Solaris 2.6 so we're talking about a 1997 timeframe.
Initially only DR for I/O boards was supported; later support
for memory and CPU boards was added.

And: http://sunsolve.sun.com/data/806/806-3984/pdf/806-3984-10.pdf

Solaris 8 with support for Memory & CPU boards (2000?)

This was long before AIX developed such features; even when it
could dynamically off-line "faulty" CPUs, the system had to be shut down
to replace them.  (And those CPUs hadn't failed, yet, of course,
they would have shown a number of correctable errors; systems cannot
continue to run when a CPU fails with an uncorrectable error)

Quote:>I understand you can deallocate memory and CPU modules from a running OS
>and allocate it to another instance now (I think that's Zones on Solaris
>10), but to remove or change hardware, you have to take the system down.

No; zones are not "OS instances"; they're virtual OS instances which
do not generally own hardware, thoguh you can allocate disks to them.

All CPU/memory resources are managed on a software level; so you could
allocate a zone "0.1" CPU or any other fractional number.

The original E10K (starfire) allowed for board level partitioning;
that was partly dynamic too, IIRC.  Those domains run their own
OS instances; this feature also exists in the followon 12K, 15K, E20K
and E25K and the hardware partitioning was moved down the line to
other system.

Quote:>Is IBM hardware hotplugable and the system tolerant of changes in the
>hardware, so it can all stay up and running while the hardware is being
>changed out?  That'd be like a car allowing cylinders to be replaced or
>added while it's running.  How do they do that?

So is the Sun hardware and has been for quite a while in the server
lines (midsize and up).  The hardware had the capabilities before the
software; the software is a lot harder to do.

For hardware, it generally requires the ability to route power to boards
individually and power them down; to add hardware to a system is fairly
easy.  Removing it requires the system to first stop using the
resources; easy enough with CPUs and individual devices, considerably harder
with memory as memory is used for everything the system does.

Zones add some form of software partitioning; but it is done at a very
high level in the kernel so the overhead is minimal to non-existant.
They're also extremely fine grained.
The hardware partitioning in the larger systems is coarse grained; but
it also has no overhead to speak of (but half of a system only has
half the memory bandwith ...)

Casper
--
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

 
 
 

Solaris 10 N1 Containers vs AIX 5.3 Micro-Partitioning

Post by Joerg Schilli » Sun, 07 Nov 2004 20:57:12




Quote:>While the OS is running?  How does it deal with hardware all of a sudden
>"going away"?  On Solaris, various data structures are created on boot
>and are dynamic.  So to change them, you have to reboot.

This works for Solaris since about 8 years.....

--



URL:  http://www.fokus.fraunhofer.de/usr/schilling ftp://ftp.berlios.de/pub/schily

 
 
 

Solaris 10 N1 Containers vs AIX 5.3 Micro-Partitioning

Post by Casper H.S. Di » Sun, 07 Nov 2004 21:38:02



Quote:>On the p-series you can de-allocate CPUs. IBM has a facility where you ask a
>CPU to be deallocated. AIX then stops allocating processes to that processor
>and drains the run queue. The processor is then marked as offline. You can
>then re-allocate that processor another LPAR. You then tell that LPAR that
>it has a new processor and it starts using it.

Same on Solaris.

Quote:>Same with adapter cards. You tell AIX you are placing an adapter card
>offline. The os stops using it and dynamically dealloctes it. Remember that
>AIX has a dynamic kernel whereas solaris has a static one (i.e. /etc/system
>on Solaris - there is no equivalent on AIX as it allocates as much as is
>needed but you can set limits in /etc/security/limits). Anyway, once the
>card is marked offline and shows as 'defined' in AIX, you can use the
>hot-plug manager to set that card for removal and then you unplug with the
>system up and fully working. All others LPARS on that box are completely
>unaffected. IBM have done some really clever stuff with the p-series and AIX
>5.2+

The Solaris kernel is not at all static; even though there is a limited
number of tunables set in /etc/system, much of the system is dynamic
and self-tuning.  Anything which is "fixed" is considered a bug.
Some of the variables set in "/etc/system" can be modified while
the system is runnng.

Of these bugs, more and more are fixed every release (Solaris 10
does away with the fixed System V IPC tunables and replaces them with
dynamic resource control.  I think these are the last set of tunables
for which the static nature was a large hurdle.

Adding/removing memory, CPUs and I/O boards has been possible for
many years.

Casper
--
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.