RAID on Hitachi 9900 w/Oracle?

RAID on Hitachi 9900 w/Oracle?

Post by <der.. » Mon, 19 Aug 2002 00:46:59



Can anyone please comment on the use of RAID in an Oracle OLTP database configuration,
specifically with storage on a Hitachi 9900 serious storage system?  The Hitachi and Sun
people that I have spoken with so far would have me believe that I can put my entire Oracle
database on a Hitachi 9900 using all RAID-5 -- because the RAID-5 on this box is so fast and
has so much cache, etc., etc...

Anyway, I'm not convinced that any of the people I've spoken with know a thing about Oracle,
and despite the fact that I've asked them over and over for some specific real-world examples
of Oracle configurations with the 9900 series...  So far, I've seen nothing.

Everything I've seen and heard tells me that RAID-5 should be avoided, for the most part --
and that an entire Oracle database should most definitely NOT be put all on RAID-5.  The
Hitachi 9900 also supports RAID1+0.  I desperately want to hear from anyone who has some
real-world experience with this hardware, with Oracle.

Please help!

--Jon

 
 
 

RAID on Hitachi 9900 w/Oracle?

Post by Rick Hunte » Mon, 19 Aug 2002 09:52:58



>Can anyone please comment on the use of RAID in an Oracle OLTP database configuration,
>specifically with storage on a Hitachi 9900 serious storage system?  The Hitachi and Sun
>people that I have spoken with so far would have me believe that I can put my entire Oracle
>database on a Hitachi 9900 using all RAID-5 -- because the RAID-5 on this box is so fast and
>has so much cache, etc., etc...

>Anyway, I'm not convinced that any of the people I've spoken with know a thing about Oracle,
>and despite the fact that I've asked them over and over for some specific real-world examples
>of Oracle configurations with the 9900 series...  So far, I've seen nothing.

>Everything I've seen and heard tells me that RAID-5 should be avoided, for the most part --
>and that an entire Oracle database should most definitely NOT be put all on RAID-5.  The
>Hitachi 9900 also supports RAID1+0.  I desperately want to hear from anyone who has some
>real-world experience with this hardware, with Oracle.

Your apprehension about RAID 5 is understandable; especially
considering its long-standing, and well-known performance limitations.
However cache centric arrays like the HDS 99xx don't behave or perform
like a traditional array.  You will be happily surprised by Oracle's
performance on these boxes running RAID 5.  Switching to RAID 1 or 1+0
will simply be a waste of disk capacity.  As for suggestions for best
performance on these arrays...

-Follow best practices for Oracle, especially the use of multiple
table spaces, ie. make your DB modular vs monolithic

-Keep your primary index small(ish) to give your self the option of
locking it into cache (ala FlashAccess) for solid state disk like
performance

-Keep or move "hot" table spaces to under utilized back-end array
controllers (ACP pairs).  HDS has utilities to help you identify and
move LUNs as needed.

-Think of, and treat, each ACP pair as if it were a separate disk
array

-Use VxVM DPM, or the like, to trunk multiple FC HBAs together to form
a fat pipe and stripe across multiple LUNs controlled by multiple ACP
pairs

Some of the old rules simply don't apply with these arrays because
they are not like other arrays.  My first experience with this disk
type in an open systems environment was with an HDS 7700E.  Like the
99xx series after it, this array has separate data and command paths,
which I think is a major factor in their surprising OLTP performance.
A muli-Kbyte data block doesn't have to compete with a multi-byte
command for transportation.  Personally I think this is a huge and
under recognized feature.

 
 
 

RAID on Hitachi 9900 w/Oracle?

Post by Rodrick Brow » Tue, 20 Aug 2002 08:26:02


Dude dont listen to there marketing team from my experiecne you want raid
0+1

We had issues with Sun T3 + raid 5 got almost a 50% perf gain when we
configured it for 0+1
have fun.

- Rodrick Brown

- Computer Systems Engineer


Quote:> Can anyone please comment on the use of RAID in an Oracle OLTP database
configuration,
> specifically with storage on a Hitachi 9900 serious storage system?  The
Hitachi and Sun
> people that I have spoken with so far would have me believe that I can put
my entire Oracle
> database on a Hitachi 9900 using all RAID-5 -- because the RAID-5 on this
box is so fast and
> has so much cache, etc., etc...

> Anyway, I'm not convinced that any of the people I've spoken with know a
thing about Oracle,
> and despite the fact that I've asked them over and over for some specific
real-world examples
> of Oracle configurations with the 9900 series...  So far, I've seen
nothing.

> Everything I've seen and heard tells me that RAID-5 should be avoided, for
the most part --
> and that an entire Oracle database should most definitely NOT be put all
on RAID-5.  The
> Hitachi 9900 also supports RAID1+0.  I desperately want to hear from
anyone who has some
> real-world experience with this hardware, with Oracle.

> Please help!

> --Jon

 
 
 

RAID on Hitachi 9900 w/Oracle?

Post by John » Tue, 20 Aug 2002 20:22:23




> Your apprehension about RAID 5 is understandable; especially
> considering its long-standing, and well-known performance limitations.
> However cache centric arrays like the HDS 99xx don't behave or perform
> like a traditional array.  You will be happily surprised by Oracle's
> performance on these boxes running RAID 5.  Switching to RAID 1 or 1+0
> will simply be a waste of disk capacity.  As for suggestions for best
> performance on these arrays...

> -Follow best practices for Oracle, especially the use of multiple
> table spaces, ie. make your DB modular vs monolithic

> -Keep your primary index small(ish) to give your self the option of
> locking it into cache (ala FlashAccess) for solid state disk like
> performance

> -Keep or move "hot" table spaces to under utilized back-end array
> controllers (ACP pairs).  HDS has utilities to help you identify and
> move LUNs as needed.

> -Think of, and treat, each ACP pair as if it were a separate disk
> array

> -Use VxVM DPM, or the like, to trunk multiple FC HBAs together to form
> a fat pipe and stripe across multiple LUNs controlled by multiple ACP
> pairs

> Some of the old rules simply don't apply with these arrays because
> they are not like other arrays.  My first experience with this disk
> type in an open systems environment was with an HDS 7700E.  Like the
> 99xx series after it, this array has separate data and command paths,
> which I think is a major factor in their surprising OLTP performance.
> A muli-Kbyte data block doesn't have to compete with a multi-byte
> command for transportation.  Personally I think this is a huge and
> under recognized feature.

Well said, Rick! I completely agree with you. Need to get people to
understand that these new arrays act and run completely different than
trying to run say software raid 5 on JBOD or even the small hardware
raid arrays. With the right hardware, a little tuning and paying a
little attention to the details there is no problem running raid 5....

just my $.02

john

 
 
 

RAID on Hitachi 9900 w/Oracle?

Post by Malcolm Wei » Wed, 21 Aug 2002 11:47:27



appear as if it was written:

[ Snip ]

Quote:>Your apprehension about RAID 5 is understandable; especially
>considering its long-standing, and well-known performance limitations.
>However cache centric arrays like the HDS 99xx don't behave or perform
>like a traditional array.  You will be happily surprised by Oracle's
>performance on these boxes running RAID 5.  Switching to RAID 1 or 1+0
>will simply be a waste of disk capacity.  As for suggestions for best
>performance on these arrays...

HOWEVER: for a given capacity and subsystem, a mirrored solution will ALWAYS
provide a greater theoretical maximum throughput.

Here's the algebraic analysis that prooves this.  Note that this concerns
the interaction between the disks and the subsystem; the subsystem / host
interaction can (and will) be buffered by cache.

Suppose your capacity requires N disks.

With RAID 5, you can do N+1 simultaneous random reads OR between N/2 and N
simultaneous random writes (depending on how close the writes are.  If the
writes can be ordered so that a single stripe is updated, then N user writes
can be satisfied with N+1 disk writes).

With RAID 1, you can do 2N random reads OR N random writes.

In the read-mostly situation, if N > 1, 2N > N + 1.

In the write-mostly case, N is greater than (something between N/2 and N).

As the (total) size of your caches approaches the size of your working set,
the I/O pattern at the disk tends towards all writes.  This explains why
RAID 5 looks better on a large cache array like the Symmetrix or the HDS:
the difference between mirroring and RAID 5 is less pronounced with writes
than with reads.

Malc.

 
 
 

RAID on Hitachi 9900 w/Oracle?

Post by Larr » Sat, 24 Aug 2002 12:41:54



> Dude dont listen to there marketing team from my experiecne you want raid
> 0+1

> We had issues with Sun T3 + raid 5 got almost a 50% perf gain when we
> configured it for 0+1
> have fun.

I dont think its the RAID issue you're experiencing here but the T3...
 
 
 

RAID on Hitachi 9900 w/Oracle?

Post by Rick Hunte » Sun, 25 Aug 2002 05:57:38


On Sun, 18 Aug 2002 23:26:02 GMT, "Rodrick Brown"


>Dude dont listen to there marketing team from my experiecne you want raid
>0+1

>We had issues with Sun T3 + raid 5 got almost a 50% perf gain when we
>configured it for 0+1
>have fun.

   The Sun T3 and HDS 99xx are completely different animals.  The
performance characteristics of one can not be attributed to the other.
 
 
 

RAID on Hitachi 9900 w/Oracle?

Post by Tarjei T. Jense » Tue, 27 Aug 2002 15:45:28




> > We had issues with Sun T3 + raid 5 got almost a 50% perf gain when we
> > configured it for 0+1
> > have fun.

> I dont think its the RAID issue you're experiencing here but the T3...

Raid 0+1 means you have 2 disks which can service your reads. That means
less waiting for reads. I would expect an performance increase regardless of
type of storage as long as you can allocate entire disks.

I don't think it is economically feasible to cache a database in a manner
which makes raid 5 perform as well as raid 0+1.

Greetings,

 
 
 

RAID on Hitachi 9900 w/Oracle?

Post by Nicholas Pieter Heesters J » Wed, 28 Aug 2002 22:20:09





>> Dude dont listen to there marketing team from my experiecne you want raid
>> 0+1

>> We had issues with Sun T3 + raid 5 got almost a 50% perf gain when we
>> configured it for 0+1
>> have fun.

>I dont think its the RAID issue you're experiencing here but the T3...

I have to agree with the argument against RAID 5.  We believed all of
the marketing hype and had our original EMC configuration at RAID-S.
We set up an IDENTICAL EMC Symmetrix and HP host at a DR site and ran
the SAME test suites against our Informix database.  The ONLY difference
was the DR EMC Symmetrix was set up as mirrored (not RAID-S).  We
experienced a 50% increase in disk throughput.  We will NEVER use
RAID 5 (or RAID-S) again.

Nick

 
 
 

RAID on Hitachi 9900 w/Oracle?

Post by Bill Tod » Thu, 29 Aug 2002 03:46:06




...

Quote:> I have to agree with the argument against RAID 5.  We believed all of
> the marketing hype and had our original EMC configuration at RAID-S.
> We set up an IDENTICAL EMC Symmetrix and HP host at a DR site and ran
> the SAME test suites against our Informix database.  The ONLY difference
> was the DR EMC Symmetrix was set up as mirrored (not RAID-S).  We
> experienced a 50% increase in disk throughput.  We will NEVER use
> RAID 5 (or RAID-S) again.

If that's a blanket condemnation rather than a comment about the need to use
RAID-5 selectively, that's your loss, I'm afraid:  for some applications it
works fine, and for others it doesn't.  In particular, for a sequential
write pattern which a stable write-back cache can aggregate into full-stripe
writes (which is the nature of writing a log, the question Rodrick raised in
a different post), RAID-5 works fully as well as a RAID-1 of equivalent
usable capacity, whereas for random database read access it's somewhat
slower than RAID-1 (since RAID-1 provides nearly twice as many spindles to
spread the reads over for a given usable capacity) and for random small
database update activity it's considerably slower than RAID-1.

IIRC, EMC's 'RAID-S' is a RAID-5 implementation that's so coarsely striped
that aggregating sequential writes into a full stripe in a write-back cache
becomes infeasible.  So in situations such as the logging case your
comparison fails on multiple counts (it's not normal RAID-5 and not
necessarily experiencing a sequential or other access pattern that a stable
write-back cache could optimize).  Now, if you were to confine your comments
to 'RAID-S' rather than including normal RAID-5, I'd have less to disagree
with:  it's not clear that RAID-S has *any* redeeming qualities when
compared with a normal RAID-5, though there are cases where coarsening the
per-disk stripe-unit size up to the 1 - 4 MB range can have virtues.

- bill

 
 
 

RAID on Hitachi 9900 w/Oracle?

Post by Nicholas Pieter Heesters J » Fri, 30 Aug 2002 03:37:05




>IIRC, EMC's 'RAID-S' is a RAID-5 implementation that's so coarsely striped
>that aggregating sequential writes into a full stripe in a write-back cache
>becomes infeasible.  So in situations such as the logging case your
>comparison fails on multiple counts (it's not normal RAID-5 and not
>necessarily experiencing a sequential or other access pattern that a stable
>write-back cache could optimize).  Now, if you were to confine your comments
>to 'RAID-S' rather than including normal RAID-5, I'd have less to disagree
>with:  it's not clear that RAID-S has *any* redeeming qualities when
>compared with a normal RAID-5, though there are cases where coarsening the
>per-disk stripe-unit size up to the 1 - 4 MB range can have virtues.

The RAID Advisory Board has certified RAID-S's conformance to RAID-5
specifications.  I don't know what is meant by "normal RAID-5", but I
accept the RAID Advisory Board's definition of what is RAID-5 and what
is not.

Nick

 
 
 

RAID on Hitachi 9900 w/Oracle?

Post by Malcolm Wei » Fri, 30 Aug 2002 05:07:57



Heesters Jr.) caused to appear as if it was written:



>>IIRC, EMC's 'RAID-S' is a RAID-5 implementation that's so coarsely striped
>>that aggregating sequential writes into a full stripe in a write-back cache
>>becomes infeasible.  So in situations such as the logging case your
>>comparison fails on multiple counts (it's not normal RAID-5 and not
>>necessarily experiencing a sequential or other access pattern that a stable
>>write-back cache could optimize).  Now, if you were to confine your comments
>>to 'RAID-S' rather than including normal RAID-5, I'd have less to disagree
>>with:  it's not clear that RAID-S has *any* redeeming qualities when
>>compared with a normal RAID-5, though there are cases where coarsening the
>>per-disk stripe-unit size up to the 1 - 4 MB range can have virtues.

>The RAID Advisory Board has certified RAID-S's conformance to RAID-5
>specifications.  I don't know what is meant by "normal RAID-5", but I
>accept the RAID Advisory Board's definition of what is RAID-5 and what
>is not.

All true, Nick...

However, the RAB doesn't discuss architectural optimizations.

In particular, the classic RAID-5 optimization of doing full-stripe writes
is not exactly an option with RAID-S, since a full-stripe with RAID-S is the
whole RAID-S set's usable storage.

So unless you have enough cache and stupidity to defer writing to a RAID-S
volume until you can write the *entire* volume, you cannot perform this
optimization.

Against which, RAID-S implementations do include the use of on-disk XOR
engines, which helps somewhat...

Quote:>Nick

Malc.
 
 
 

RAID on Hitachi 9900 w/Oracle?

Post by Rick Hunte » Fri, 30 Aug 2002 11:01:44


Quote:

>I have to agree with the argument against RAID 5.  We believed all of
>the marketing hype and had our original EMC configuration at RAID-S.
>We set up an IDENTICAL EMC Symmetrix and HP host at a DR site and ran
>the SAME test suites against our Informix database.  The ONLY difference
>was the DR EMC Symmetrix was set up as mirrored (not RAID-S).  We
>experienced a 50% increase in disk throughput.  We will NEVER use
>RAID 5 (or RAID-S) again.

  RAID-S is not RAID-5 no matter how much EMC paid RAB to say it is.
And no matter how much EMC tries to make an "S" look like a "5".  Do
you think EMC picked the letter S for any reason other than the fact
bit can be mistaken for a 5?  Puhlease!

   Comparing RAID-S with real RAID-5 is...  If you really believe...

   Oh forget it.  Some EMC victims are just too far gone to save.

   (Can you believe this Bill?  Even my persistence has it's limits.)

 
 
 

RAID on Hitachi 9900 w/Oracle?

Post by Nicholas Pieter Heesters J » Fri, 30 Aug 2002 22:07:35




Quote:

>>I have to agree with the argument against RAID 5.  We believed all of
>>the marketing hype and had our original EMC configuration at RAID-S.
>>We set up an IDENTICAL EMC Symmetrix and HP host at a DR site and ran
>>the SAME test suites against our Informix database.  The ONLY difference
>>was the DR EMC Symmetrix was set up as mirrored (not RAID-S).  We
>>experienced a 50% increase in disk throughput.  We will NEVER use
>>RAID 5 (or RAID-S) again.

>  RAID-S is not RAID-5 no matter how much EMC paid RAB to say it is.
>And no matter how much EMC tries to make an "S" look like a "5".  Do
>you think EMC picked the letter S for any reason other than the fact
>bit can be mistaken for a 5?  Puhlease!

>   Comparing RAID-S with real RAID-5 is...  If you really believe...

>   Oh forget it.  Some EMC victims are just too far gone to save.

>   (Can you believe this Bill?  Even my persistence has it's limits.)

I think I'll take my chances on the recognized industry standard orginization's
definition of what RAID-5 is over somebody's personal definition from a Usenet
group any day.

Nick

 
 
 

RAID on Hitachi 9900 w/Oracle?

Post by Bill Tod » Sat, 31 Aug 2002 02:00:11






> >>I have to agree with the argument against RAID 5.  We believed all of
> >>the marketing hype and had our original EMC configuration at RAID-S.
> >>We set up an IDENTICAL EMC Symmetrix and HP host at a DR site and ran
> >>the SAME test suites against our Informix database.  The ONLY difference
> >>was the DR EMC Symmetrix was set up as mirrored (not RAID-S).  We
> >>experienced a 50% increase in disk throughput.  We will NEVER use
> >>RAID 5 (or RAID-S) again.

> >  RAID-S is not RAID-5 no matter how much EMC paid RAB to say it is.
> >And no matter how much EMC tries to make an "S" look like a "5".  Do
> >you think EMC picked the letter S for any reason other than the fact
> >bit can be mistaken for a 5?  Puhlease!

> >   Comparing RAID-S with real RAID-5 is...  If you really believe...

> >   Oh forget it.  Some EMC victims are just too far gone to save.

> >   (Can you believe this Bill?  Even my persistence has it's limits.)

> I think I'll take my chances on the recognized industry standard
orginization's
> definition of what RAID-5 is over somebody's personal definition from a
Usenet
> group any day.

Better yet, however, would be to actually learn something about the subject.

- bill

 
 
 

1. Hitachi StorEdge 9900 Series with Oracle?

Where the heck was my head at?  In a previous post, I referred to this hardware as Sun
StorEdge 9900  -- I meant Hitachi, not Sun...

Is anyone out there using a Hitachi StorEdge 9900 series storage system with their Oracle
database(s)?

We are in the beginning stages of configuring our brand new Hitachi 9970 for use with
our Oracle database environment and we would very much like to have the benefit of
other Oracle shops' experience with the 9900 series.  If you're using that type of hardware
and would be willing to entertain some questions about your configuration, please respond.
We'd be very grateful!

--Jon

2. netlabs.org ???

3. Sun StorEdge 9900 Series with Oracle?

4. Comparision of Outliner

5. Data migration from EMC Symmetrix to StorageTek D178 and Hitachi 9900

6. ADMT v2 Computer Migration problems

7. Any problems with Hitachi 9900 series storage arrays and Sun E420Rs

8. AAA for Amiga from ESCOM?? I say _yes_

9. Mirror Oracle-Redologfiles on Oracle-Level even when on Raid-1 or EMC-Systems ?

10. Oracle Redo/Archive Logs w/T3 Raid 0+1 vs Raid 5

11. Can LUN size expansion (LUSE) in HDS 9900 be done on command line ?

12. TI 9900 family databook - anybody need it?

13. TMS 9900