RAID controler?

RAID controler?

Post by Greg Leblan » Tue, 20 Jul 1999 04:00:00



I need to build a rather "beefy" server for linux.  I'm looking about a
P-II 350 (possibly faster), 256 megs of ram, probably about 18gigs of
drive space.  I would prefer if the hard drive was a Hardware RAID 5
array.  Can somebody recomend which controllers are best supported under
linux?  Maybe the HP controlers, or the Adaptec AAA-131 SA?  Thanks,
     Greg
--
It's pronounced "*" not "scuzzy"!

Sent via Deja.com http://www.veryComputer.com/
Share what you know. Learn what you don't.

 
 
 

RAID controler?

Post by Marc Mut » Wed, 21 Jul 1999 04:00:00



> I need to build a rather "beefy" server for linux.  I'm looking about a
> P-II 350 (possibly faster), 256 megs of ram, probably about 18gigs of
> drive space.  I would prefer if the hard drive was a Hardware RAID 5
> array.  Can somebody recomend which controllers are best supported under
> linux?  Maybe the HP controlers, or the Adaptec AAA-131 SA?  Thanks,
>      Greg

Mylex DAC960 (number may be different)
(AMI?) MegaRaid
come to mind. Check the compile options in recent kernels.

Marc

--

University of Bielefeld, Dep. of Mathematics / Dep. of Physics

PGP-keyID's:   0xd46ce9ab (RSA), 0x7ae55b9e (DSS/DH)

 
 
 

RAID controler?

Post by Tim Moor » Wed, 21 Jul 1999 04:00:00


Quote:> I need to build a rather "beefy" server for linux.  I'm looking about a
> P-II 350 (possibly faster), 256 megs of ram, probably about 18gigs of
> drive space.  I would prefer if the hard drive was a Hardware RAID 5
> array.  Can somebody recomend which controllers are best supported under
> linux?  Maybe the HP controlers, or the Adaptec AAA-131 SA?  Thanks,
>      Greg

RAID 5 performance is not very good until you have at least 4 spindles,
so a total of 18GB doesn't make much sense.  If you use older  2 or 4GB
drives, performance will not be good [by today's standards] due to
slower access time.

For small spindle counts, RAID1 or RAID0+1 is a significantly better
performer than RAID5.

--
direct replies substitute timothymoore for user name

"Everything is permitted.  Nothing is forbidden."
                                   WS Burroughs.

 
 
 

RAID controler?

Post by Paul Koty » Thu, 22 Jul 1999 04:00:00


Tim,

Could you elaborate on the no of spindles/performance issue?

Regards, Paul
--
Paul Kotyk
Nornet System Integrators

Quote:>> snip

 RAID 5 performance is not very good until you have at least 4 spindles,
Quote:> so a total of 18GB doesn't make much sense.  If you use older  2 or 4GB
> drives, performance will not be good [by today's standards] due to
> slower access time.

> For small spindle counts, RAID1 or RAID0+1 is a significantly better
> performer than RAID5.

> --
> direct replies substitute timothymoore for user name

> "Everything is permitted.  Nothing is forbidden."
>                                    WS Burroughs.

 
 
 

RAID controler?

Post by Greg Leblan » Thu, 22 Jul 1999 04:00:00




Quote:> > I need to build a rather "beefy" server for linux.  I'm looking
about a
> > P-II 350 (possibly faster), 256 megs of ram, probably about 18gigs
of
> > drive space.  I would prefer if the hard drive was a Hardware RAID 5
> > array.  Can somebody recomend which controllers are best supported
under
> > linux?  Maybe the HP controlers, or the Adaptec AAA-131 SA?  Thanks,
> >      Greg

> RAID 5 performance is not very good until you have at least 4
spindles,
> so a total of 18GB doesn't make much sense.  If you use older  2 or
4GB
> drives, performance will not be good [by today's standards] due to
> slower access time.

> For small spindle counts, RAID1 or RAID0+1 is a significantly better
> performer than RAID5.

RAID 1 shouldn't have any performance benefits over a RAID 5 on a three
disk config, since it's just mirroring data, unless I'm wrong.  However,
the main thing that I need is data redundancy, and hot swap capability.
The LC3 only has three internal hot swap drive bays, and purchasing an
external chassis is very expensive.  I was really more interested in
finding out the best controlers to use with Linux, and any expierence
that people have had with them, and saving debates about which RAID
configuration is best for another thread.  :-)  Thanks,
        Greg

Quote:

> --
> direct replies substitute timothymoore for user name

> "Everything is permitted.  Nothing is forbidden."
>                                    WS Burroughs.

--
It's pronounced "*" not "scuzzy"!

Sent via Deja.com http://www.veryComputer.com/
Share what you know. Learn what you don't.

 
 
 

RAID controler?

Post by Tim Moor » Thu, 22 Jul 1999 04:00:00


[note: 'controller' means either hardware or software in this
conversation]

RAID0 divides up data writes in chunks across all drives in the raid
group.  For RAID0 pairs on separate controller channels this almost
doubles the write and read speed as there are multiple controllers
working the same data stream.  Fastest, but no parity and no drive loss
protection.

RAID1 is a simple binary mirror with no parity calculation.  The
controller writes the same data to two disks, so there is a small
incremental latency.  Data reads may be faster than writes depending on
the controllers ability to pull from either drive.  Most commercial
implementations use mirrored RAID0 sets (RAID0+1 or RAID10) for
performance and protection.  Multiple disk failure protection if they
aren't in the same mirror set.  Twice as many disks.  Expensive.

Network Appliance uses a modified RAID4 which buffers writes in
nonvolatile memory rather than hitting the data drives for each write.
When the buffer is full, parity is calculated and all the bits are
written hitting the disks once.  This avoids the RAID4 hot parity drive
effect and yields RAID0 performance with RAID5 data integrity.  Single
disk failure protection.  Cost may be a factor.

RAID5 calculates parity for each write then stripes parity and data
across all physical disks in the raid group.  This tends to be the worst
performer but most cost effective.  Single disk failure protection.

For RAID0, 4 and 5, performance increases as the number of physical
drives in the raid group increases.  Best performance seems to start at
7 and begin to drop after 28 drives/raid group (different controllers
and/or raid implementations will vary of course).

Hope this helps.

Quote:> Could you elaborate on the no of spindles/performance issue?

> >  RAID 5 performance is not very good until you have at least 4 spindles,
> > so a total of 18GB doesn't make much sense.  If you use older  2 or 4GB
> > drives, performance will not be good [by today's standards] due to
> > slower access time.

> > For small spindle counts, RAID1 or RAID0+1 is a significantly better
> > performer than RAID5.

--
timothymoore    "Everything is permitted.  Nothing is forbidden."
bigfoot                                            WS Burroughs.
com
 
 
 

RAID controler?

Post by Mike Simo » Fri, 23 Jul 1999 04:00:00



> RAID 1 shouldn't have any performance benefits over a RAID 5 on a three
> disk config, since it's just mirroring data, unless I'm wrong.  However,
> the main thing that I need is data redundancy, and hot swap capability.
> The LC3 only has three internal hot swap drive bays, and purchasing an
> external chassis is very expensive.  I was really more interested in
> finding out the best controlers to use with Linux, and any expierence
> that people have had with them, and saving debates about which RAID
> configuration is best for another thread.  :-)  Thanks,

A good place to start is

www.redhat.com/corp/support/hardware/intel/60/rh6.0-hcl-i.ld-6.html#s...

If you plan on using RedHat to make life easy I would suggest purchasing
one of the RAID controllers on the HCL. I've had good experiences with
AMI MEGARAID controllers. VA Linux uses Mylex RAID controllers, they
seem work well. I haven't used any Mylex controllers yet, so I can't
speak from experience. I would probably stay away from Compaq or IBM's
RAID controller, but thats my own personal bias. :) DPT (www.dpt.com)
also has a RAID controller for Linux, never tried it but from the looks
of their web site it should work okay. As for performance I think the
Mylex controllers are probably faster, or thats what the DAC960-HOWTO
leads you to believe. I like the AMI MEGARAID controllers because they
are pretty easy to configure and there is a ncurses utility that allows
you to configure your RAID array under Linux.

Mike

 
 
 

RAID controler?

Post by Joe CADdud » Sun, 25 Jul 1999 04:00:00


I always thought that raid5 always uses sets of 3 drives.  The benifit is
that it doesn't write the parity to any one drive.  The speed hit is only in
the write phase of operation.  Reading is just as fast as raid 0 as you
divide the data in half and read one block from one drive and one block from
the second.  You get increased performance with increased number of drives
when reading more than one secord'ed files.  As it will queue up all the
reads to the drives in sequence.

This works like this.  you have 4 drives say, named A,B.,C,D.  It will write
the first block of data  between A, and B with parity writen to C.  The next
sector will be written to D and A, with parity writen to B. etc.. when you
go to read, it will queue up the reads from A, and B, then call up the next
blocks from D, and by that time A will have responded, so it then goes back
to A to get more info.  then queues up C, and since D will have responded,
it gets queued for the next half block.  If you had more drives, you would
see it gets shorter as the number of drives goes up it can retreave the data
in contiguous blocks faster than the request for that data required.

Raid 5 with X number of drives has the exact same perfomance as a Raid 0
with X-1 drives.  Raid 0 requires a minimum of 2 drives, where raid 5 needs
at least 3.  In most cases the number of controlers handling the raid is not
the bottleneck in performance, rather the speed of the drive itself.  As
most modern scsi drives do come with a considerable cache on them, the data
comming down the scsi bus is cached as the controler figures out how to
split up the data, and little time is spend in actually sending the data
down the piece of wire that connects the drives to the controler.


Quote:> [note: 'controller' means either hardware or software in this
> conversation]

> RAID0 divides up data writes in chunks across all drives in the raid
> group.  For RAID0 pairs on separate controller channels this almost
> doubles the write and read speed as there are multiple controllers
> working the same data stream.  Fastest, but no parity and no drive loss
> protection.

> RAID1 is a simple binary mirror with no parity calculation.  The
> controller writes the same data to two disks, so there is a small
> incremental latency.  Data reads may be faster than writes depending on
> the controllers ability to pull from either drive.  Most commercial
> implementations use mirrored RAID0 sets (RAID0+1 or RAID10) for
> performance and protection.  Multiple disk failure protection if they
> aren't in the same mirror set.  Twice as many disks.  Expensive.

> Network Appliance uses a modified RAID4 which buffers writes in
> nonvolatile memory rather than hitting the data drives for each write.
> When the buffer is full, parity is calculated and all the bits are
> written hitting the disks once.  This avoids the RAID4 hot parity drive
> effect and yields RAID0 performance with RAID5 data integrity.  Single
> disk failure protection.  Cost may be a factor.

> RAID5 calculates parity for each write then stripes parity and data
> across all physical disks in the raid group.  This tends to be the worst
> performer but most cost effective.  Single disk failure protection.

> For RAID0, 4 and 5, performance increases as the number of physical
> drives in the raid group increases.  Best performance seems to start at
> 7 and begin to drop after 28 drives/raid group (different controllers
> and/or raid implementations will vary of course).

> Hope this helps.

> > Could you elaborate on the no of spindles/performance issue?

> > >  RAID 5 performance is not very good until you have at least 4
spindles,
> > > so a total of 18GB doesn't make much sense.  If you use older  2 or
4GB
> > > drives, performance will not be good [by today's standards] due to
> > > slower access time.

> > > For small spindle counts, RAID1 or RAID0+1 is a significantly better
> > > performer than RAID5.

> --
> timothymoore    "Everything is permitted.  Nothing is forbidden."
> bigfoot                                            WS Burroughs.
> com

 
 
 

RAID controler?

Post by Scott Marlow » Mon, 26 Jul 1999 04:00:00



> This works like this.  you have 4 drives say, named A,B.,C,D.  It will write
> the first block of data  between A, and B with parity writen to C.  The next
> sector will be written to D and A, with parity writen to B. etc..

This isn't how I've learned RAID 5 theory.  What I learned goes like this:

Each write, you create a data stripe for all but one drive, create a parity
stripe by XORing all the data stripes, and place the parity stripe and data
stripes on the drives.  The parity stripe round robins across all the drives,
so that it looks like this for each write:

A  B  C  D
 D  D  D  P
 P  D  D  D
 D  P  D  D
 D  D  P  D

And so on adinifinitum.

The rest is all pretty much right.

Some insight on performance limitations.  The RAID array will get faster as the
drive count climbs, until you either saturate the parity generation engine
(CPU, Hardware RAID, whatever generates parity) or you saturate the SCSI busses
the or whatever buss the drive is connected to.

Parity Generation is best left to small fast RISC engines really good at it
that can offload the overhead from the CPU.  Many of the <$1000 cards don't do
very well, like the Adaptec AHA 133 series.  ugh, abysmal performance on any
size array we tried.  The MegaRAID by AMI and the newer DAC 960s seem to be
good controllers, with high numbers and good hardware parity gen, and several
options of multiple cards / channels all being combined into one
RAID subsystem.

Another thing that can kill you on a large drive number raid array is small
writes.  If you have a database that does changes in the range of hundreds of
bytes at a time, and you've got say, 20 drives with a stripe size of 4k, you've
got to retreive the 20x4k of data, change 1 or 100 bytes, recalculate the
parity strip, and write out the 20x4k.  Reads are fast, but all writes require
this type of operation.  If you're building a FTP server where the average file
size is 100k and above, then the 20x4k is no big deal, and you would likely see
good performance there.

In hardware controllers you'd use mirrored strip sets (RAID 0+1) to get best
database performance.  In Linux, you can do almost as well with software if you
make a 3 or 4 drive mirror set.

Reads are spread across the multiple drives, so things like databases are
likely to have three or so heads available at a time to read data.  Writes are
gonna be slower than RAID 0, but not terribly slow since the algorhythm for
mirrors is :  make a bunch of copies and write them out.  I don't if other Oses
support >2 drives in a mirror, but it is a wonderfully fast setup under Linux
for small data access.

So, here's my short recap, in table format:

----------------------------
| Application   |   RAID type I use       |
|---------------------------|
| Database         |  RAID-1 x 4               |
| Web Server    |  RAID 1 x 2                |
| File & Print    |  RAID 5 x [6-10]      |
----------------------------

 
 
 

RAID controler?

Post by Helge Haftin » Tue, 27 Jul 1999 04:00:00


You missed an optimization when doing small writes.  With 20 drives you
don't have to re-read from every drive order to recalculate parity.

Just read the stripe you are about to overwrite, and the parity stripe.
Then xor the parity stripe (in memory) with the stripe you are going
to overwrite.  Now the old stuff is effectively removed by calculation,
and you
have the same parity stripe you would get if you xored the 18 other
drive stripes - except you didn't have to read them.  Now
xor in the new stripe, and write the new stripe and the parity stripe
back.

You get no more than two reads and two writes for a single small write -
no
matter how many drives there is in your RAID-5 array.  

The time loss may be even smaller, as the two reads is done in parallel,
same for the two writes.

[...]

Quote:> This isn't how I've learned RAID 5 theory.  What I learned goes like this:

> Each write, you create a data stripe for all but one drive, create a parity
> stripe by XORing all the data stripes, and place the parity stripe and data
> stripes on the drives.  The parity stripe round robins across all the drives,
> so that it looks like this for each write:

> A  B  C  D
>  D  D  D  P
>  P  D  D  D
>  D  P  D  D
>  D  D  P  D
[...]
> Another thing that can kill you on a large drive number raid array is small
> writes.  If you have a database that does changes in the range of hundreds of
> bytes at a time, and you've got say, 20 drives with a stripe size of 4k, you've
> got to retreive the 20x4k of data, change 1 or 100 bytes, recalculate the
> parity strip, and write out the 20x4k.  Reads are fast, but all writes require
> this type of operation.  If you're building a FTP server where the average file
> size is 100k and above, then the 20x4k is no big deal, and you would likely see
> good performance there.

 
 
 

RAID controler?

Post by Tim Moor » Wed, 28 Jul 1999 04:00:00


Quote:> I always thought that raid5 always uses sets of 3 drives.  ...

3 to <limit of controller>.  Most examples use 3 drives.

Quote:> Raid 5 with X number of drives has the exact same perfomance as a Raid 0
> with X-1 drives.  Raid 0 requires a minimum of 2 drives, where raid 5 needs

RAID5 performance varies with implementation and is always a
performance/data integrity versus cost tradeoff.  Translation:
performance sucks but it's hard to lose data.  For Sun's Netra NFS
project, RAID5 performance was 1/3 slower than RAID0+1 (mirrored RAID0).

For my 2 drive linux software RAID0 array:
        data
r/w     size    MB/s    test    type
---     -----   ----    ------  ------
r        32MB    9.7    hdparm  single drive
r        32MB   18.4    hdparm  RAID0 array
r       300MB    9.9    dd(1)   single drive
r       300MB   11.8    dd(1)   RAID0 array
w       300MB   13.7    dd(1)   RAID0 array

IBM 6.4GB Deskstars, 5400RPM, UDMA, separate channels,
Promise U/33 controller, linux 2.0.37 kernel, raidtools-0.50beta10-2.

The original question was general spindle count vs performance:
                more
                spindles        data    easy
type    speed   = faster  cost  protect manage
-----   -----   --------  ----  ------- -------
RAID0   3       y         1     n       1
RAID1   2       n         3     y       2
RAID5   1       y         2     y       3
--
timothymoore    "Everything is permitted.  Nothing is forbidden."
bigfoot                                            WS Burroughs.
com