does software RAID5 performance suck?

does software RAID5 performance suck?

Post by Robert Nag » Wed, 22 Jan 2003 22:21:01



Well, the title summarizes my question.

To run a webserver with two or three 60 gig IDE hard drives, I have
512MB RAM and 1.2 Athlon. I'll be installing gentoo on reiser fs
today.

I have to choose between RAID 1 or 5. I read somewhere that RAID 5
consumes about 25% of your CPU, which sounds awful. Does that sound
about right?

That raises the question of how much power does a low end web server
really need?  Assuming that my server contains at least three web
apps/databases and possibly a mail server with low traffic but a good
number of web apps (zope, apache, mysql, cocoon, etc) running.

Robert Nagle
www.idiotprogrammer.com

 
 
 

does software RAID5 performance suck?

Post by Ian Pilche » Wed, 22 Jan 2003 23:22:52



> I have to choose between RAID 1 or 5. I read somewhere that RAID 5
> consumes about 25% of your CPU, which sounds awful. Does that sound
> about right?

It depends a lot on your read/write mix.  When you make a write to a
RAID-5 array, the system has to recalculate the parity information for
every "chunk" that is modified.  If you're not writing an entire chunk,
the system will also have to read the rest of the chunk before it can
calculate the new parity information.

So writes to RAID-5 arrays definitely have the potential to slow you
down.  Most applications are heavily weighted towards reads, however,
and a good caching algorithm (plus lots of memory) can go a long way
towards ameliorating the problem.

Reading from a RAID-5 array should be almost as fast as reading from a
(!R)AID-0 array.

--
========================================================================

========================================================================

 
 
 

does software RAID5 performance suck?

Post by Robert Nag » Thu, 23 Jan 2003 04:08:36


Well, I have one application that I would need to check on (zope) for
its caching ability.

Ok, I can easily boost the memory from 512 to 1 gig. I guess my
question boils down to wondering what the typical load on a web server
with several applications (zope, apache, mod_php, mod_perl, mysql and
possibly one other db).  I'm also thinking of trying evms, the new
logical volume manager.

This won't be a high traffic web server (it will be fairly low--just
your typical interesting personal web site/application server, with a
few hosted sites). How much is CPU a limiting factor on performance v.
memory for example?

I'm doing raid 5 for one reason mainly: to have more mirrored hard
drive space. Going from 2 60gig drives to 3 gives me 30 gig more
mirrored space to work with (and I will use it, I know that).

What are some good ways for measuring performance/speed?

Then, I wonder, can you downgrade from software RAID 5 to RAID 1 if
performance sucks too much?

Robert Nagle
www.idiotprogrammer.com



> > I have to choose between RAID 1 or 5. I read somewhere that RAID 5
> > consumes about 25% of your CPU, which sounds awful. Does that sound
> > about right?

> It depends a lot on your read/write mix.  When you make a write to a
> RAID-5 array, the system has to recalculate the parity information for
> every "chunk" that is modified.  If you're not writing an entire chunk,
> the system will also have to read the rest of the chunk before it can
> calculate the new parity information.

> So writes to RAID-5 arrays definitely have the potential to slow you
> down.  Most applications are heavily weighted towards reads, however,
> and a good caching algorithm (plus lots of memory) can go a long way
> towards ameliorating the problem.

> Reading from a RAID-5 array should be almost as fast as reading from a
> (!R)AID-0 array.

 
 
 

does software RAID5 performance suck?

Post by John-Paul Stewar » Thu, 23 Jan 2003 04:59:34



> I'm doing raid 5 for one reason mainly: to have more mirrored hard
> drive space. Going from 2 60gig drives to 3 gives me 30 gig more
> mirrored space to work with (and I will use it, I know that).

I think you're mis-understanding how RAID 5 works.  With 3 x
60 GB hard drives, you'll have 120 GB of available space.
(N drives in a RAID 5 array give you N-1 drives-worth of
capacity.)  This is _not_ mirrored, however.  Your data,
plus additional parity bits, are spread across all drives in
the array, so that if any one drive fails, sufficient
data+parity is available to reconstruct the array.  This not
at all how mirroring works (duplicating the data from one
disk onto another).

You're gaining more capacity than you think you are.

 
 
 

does software RAID5 performance suck?

Post by Robert Nag » Thu, 23 Jan 2003 07:42:31


Quote:> You're gaining more capacity than you think you are.

Thanks. I think I read that fact somewhere, but I forgot about it
(dismissing it as funny math). rj

www.idiotprogrammer.com

 
 
 

does software RAID5 performance suck?

Post by Steve Wolf » Thu, 23 Jan 2003 08:10:27


Quote:> I have to choose between RAID 1 or 5. I read somewhere that RAID 5
> consumes about 25% of your CPU, which sounds awful. Does that sound
> about right?

  Take a look at http://www.codon.com/docs/software-RAID.html .  While the
bonnie++ program uses significant CPU cycles, the RAID5 daemon took only a
negligible amount.  Even the P3/650 can generate over 1.3 GB/second of
parity information, which is far more than you need - in fact, more than
you could even use.  Even with a 64-bit, 66 MHz PCI slot, you'd run out of
bandwidth on the PCI bus long before the processor was short of parity.

Quote:> That raises the question of how much power does a low end web server
> really need?  Assuming that my server contains at least three web
> apps/databases and possibly a mail server with low traffic but a good
> number of web apps (zope, apache, mysql, cocoon, etc) running.

   For a "low-end" server, not a lot.  The database and CGI will be far
more "expensive" than the actual web serving, but if you're talking about
a few thousand hits per day (or even a few tens of thousands of hits), you
should be fine.  If you're talking a few *million* hits per day, that's
entirely a different matter.

steve

 
 
 

does software RAID5 performance suck?

Post by Steve Wolf » Fri, 24 Jan 2003 08:21:24


Quote:> Then, I wonder, can you downgrade from software RAID 5 to RAID 1 if
> performance sucks too much?

  I finished up software RAID testing today (
http://www.codon.com/docs/software-RAID.html ), and found some very
interesting trends.

  "Conventional Wisdom" states that RAID 5 is terrible for writes.  Of the
three forms of RAID I tested that actually have redundancy (1+0, 0+1, 5),
RAID 5 turned in the highest reads, and just as good (bad) writes as the
others, while giving you more usable space.

  A 2-drive RAID 1 array wouldn't have turned in much better scores, even
if they were better at all.  Here's why:

- A single drive read at 32.6 MB/s, and wrote at 20.6 Mb/s.  That means
that in a two-drive RAID 1 setup, you won't do any better than that, and
may very well do slightly worse on writes.

- The RAID 5 array read at up to 91 MB/s (maxing out the PCI bus), and
wrote at up to 20 MB/s.  In comparison to the two-drive RAID 1 array,
writes are equal, reads are nearly three times faster.

steve

 
 
 

does software RAID5 performance suck?

Post by Divin Marqui » Fri, 14 Feb 2003 03:47:00



>   A 2-drive RAID 1 array wouldn't have turned in much better scores,
>   even
> if they were better at all.  Here's why:

> - A single drive read at 32.6 MB/s, and wrote at 20.6 Mb/s.  That means
> that in a two-drive RAID 1 setup, you won't do any better than that, and
> may very well do slightly worse on writes.

> - The RAID 5 array read at up to 91 MB/s (maxing out the PCI bus), and
> wrote at up to 20 MB/s.  In comparison to the two-drive RAID 1 array,
> writes are equal, reads are nearly three times faster.

A perfect implementation of RAID-1 has the potential to be twice as fast
as a single disk for concurrent reads -- the subsystem can choose to serve
two simultaneous requests, one per drive -- and twice as fast in some
large reads -- read one big block on one drive while reading ahead the
next one on the second disk. I'm not sure how this actually works.

For RAID-5 I guess this has to do with the size of the chunk and the use
of the cache; if you write a full chunk, or part of a chunk which is
already cached, it doesn't need to read it on every drive to calculate the
parity before writing it. It just computes it and write.

As far as CPU usage is concerned, I don't think thisis gonna be a big
hit, I haven't noticed anything special but my only production software
RAID-5 systems are not very loaded, the heavy stuff goes to Mylex cards so
far. Programmatically, the computation of the parity is one simple
mathematic operation operating serially on a big chunk of data --
basically that's the ideal case for current memory systems, which can
serve large blocks of data very fast (esp. with stuff like double banked
DDR), as opposed to picking lots of scattered addresses.

--
Vous voulez encore tre pays, vous, les gens vertueux! Vous voulez avoir un
salaire pour votre vertu, le Ciel en rcompense de la terre et l'ternit pour
votre aujourd'hui? Et maintenant vous tes en colre contre moi parce que j'
enseigne qu'il n'existe pas d'agent payeur-comptable des rcompenses!

 
 
 

1. Software RAID5 Performance Tuning

Good morning,

  Several months ago I decided I wanted LOTS of disk space.. mostly for
archiving digital photos, video, music and the like.  I have a 9Gb
10kRPM Scsi drive for playback and any software I use frequently, but I
wanted somewhere to store stuff.  I decided I would go with four 80Gb
Maxtor drives.  I had two ATA/66 controllers (four buses) but they're
different brands.  One's a Promise, and the other is a no-name from Best
Buy.  I added md support to the kernel, built a nice 240Gb RAID-5 array,
and put ReiserFS on it for a journaling filesystem (fscking a 240gig
drive gives me the willies).  I went with a 32kb chunk size on the
array, and Reiser is using 4k block sizes.  /proc/mdstat has this to say
about the array:

+---------------
Personalities : [linear] [raid0] [raid1] [raid5]
read_ahead 1024 sectors
md0 : active raid5 hdk1[3] hdi1[2] hdg1[1] hde1[0]
      240082944 blocks level 5, 32k chunk, algorithm 2 [4/4] [UUUU]

unused devices: <none>
+---------------

Now, my questions:

Is it normal for a set up like this to take over TWO DAYS for md to
build/rebuild the array?

Is it normal for my system to run at a constant load of 0.10 even when
it's not doing anything?

Is it normal for kapmd to be using 53% of the CPU?
PID USER     PRI  NI  SIZE  RSS SHARE STAT  LIB %CPU %MEM   TIME COMMAND
2   root      17   0     0    0     0 RW      0 53.0  0.0 16508m kapmd

Is it normal for me to only be achieving about 200kb/sec transfer rates
to the array? (My DSL modem can almost outpace this puppy, and the LAN
stomps it.)

The rest of the machine's stats:
PIII/500
128MB RAM
Two ethernet cards (home LAN router)
Linux 2.4.0-test9
Redhat 6.2 initially, with lots of changes

Could anyone give me some advice on performance tuning a software RAID
array under Linux?  I'm going to spend a few days trying to decrease IO
latency in the kernel (online opinions seem to abound on this topic),
but I suspect I'm missing something big here.  Should I be using much
larger chunk/block sizes because of the size of the array?  Does having
two different brands of controllers do me in?   Any suggestions on a
systematic way of collecting more performance data?

With the array taking two days to build, trial-and-error is pretty much
out of the question. ;)

Any help is appreciated!

 -- Greg

Sent via Deja.com
http://www.deja.com/

2. Trying to partition 13GB; fdisk & mke2fs don't agree

3. X-windows sucks..sucks...sucks!!!!

4. Help! Xstation-130 boot failure!

5. Backing-up RAID5 SSA volumes SUCKS!

6. I love KDE

7. RAID5 performance

8. Digi PCI classicboard

9. RAID5 Performance Dilemma between ibm rs/6000 Model 590 and H50

10. Transfer old Raid5 on new Raid5 ?

11. FibreChannel Raid5 Array Performance

12. Adaptec 2010S raid5 low performance

13. adaptec 2010S raid5 low performance