DPT 3334UW Slow Performance/Filesystem probs (RAID-5)

DPT 3334UW Slow Performance/Filesystem probs (RAID-5)

Post by Bryn Wm. Moslo » Tue, 16 Feb 1999 04:00:00



I recently installed a DPT 3334UW with 64MB cache in a mail server
running RAID-5 with a 32K stripe on an external case. The array is
comprised of 6 Seagate Ultra Wide 4.5GB SCA drives. The system is a P2
300 with 384MB and uses an additional Seagate UW drive for boot, /usr,
/var, swap, and staff home directories. It doesn't go into swap often
but if it does it only hits about 5 to 10 MB. The system is running
FreeBSD 2.2.8

My first problem was that I initially tried to do 16K per inode to speed
things up a little bit (also, I didn't need the millions of inodes that
came with a default newfs on 43GB =).) However when trying to mount,
fsck, find, umount, or otherwise manipulate the filesystem about 1 times
out of 5 the system would hang to the point that I had to use the reset
switch to get it back. If I format with the default settings to newfs,
everything works fine. The same hang also occurs if I try to do 16K
block size. I haven't tried anything bigger as I had an extremely tight
window in which to get the machine online.

Does FreeBSD have a problem with non-default fs settings? Has anyone
else tried this sort of thing on such a large filesystem?

I ended up having to use the default settings for newfs to get the
system to work, wasting millions of inodes and bringing me to my next
problem: Under load the filesystem is horribly slow. I expected some of
this with the RAID-5 overhead but it's actually slower than a CCD I just
moved from that was using 5 regular 2GB fast SCSI-2 drives, much slower.
When running ktrace on the processes (qpopper and mail.local mainly) and
watching top I can see that most of the processes are waiting for disk
access. I've tried enabling/disabling various DPT options in the kernel
but it's all about the same. I'd really like to stick with RAID-5 so
using 0 or 1 just isn't what I'm looking for.

The user directories for delivery are broken out into 1st letter, 1st
two letters, username (i.e.: /home/u/us/username) to speed up dir
lookups already. Any suggestions on how else to speed things up? This
wasn't a problem on my old CCD, however.

Lastly, I tried to find another RAID controller besides DPT that was
compatible with FreeBSD 2.2.x with no luck. Upgrading to 3.1 is not an
option at the moment, at least until things are more stable. Is anyone
using anything in a host-based adapter (PCI) that is non-DPT?

 
 
 

DPT 3334UW Slow Performance/Filesystem probs (RAID-5)

Post by Nicolas Blai » Tue, 16 Feb 1999 04:00:00


We had a similar problem with a 18Gb system at work.  Changing it to version
3.0 solved the problem for some reason (there is now a 3.1).

Nicolas Blais.


> I recently installed a DPT 3334UW with 64MB cache in a mail server
> running RAID-5 with a 32K stripe on an external case. The array is
> comprised of 6 Seagate Ultra Wide 4.5GB SCA drives. The system is a P2
> 300 with 384MB and uses an additional Seagate UW drive for boot, /usr,
> /var, swap, and staff home directories. It doesn't go into swap often
> but if it does it only hits about 5 to 10 MB. The system is running
> FreeBSD 2.2.8

> My first problem was that I initially tried to do 16K per inode to speed
> things up a little bit (also, I didn't need the millions of inodes that
> came with a default newfs on 43GB =).) However when trying to mount,
> fsck, find, umount, or otherwise manipulate the filesystem about 1 times
> out of 5 the system would hang to the point that I had to use the reset
> switch to get it back. If I format with the default settings to newfs,
> everything works fine. The same hang also occurs if I try to do 16K
> block size. I haven't tried anything bigger as I had an extremely tight
> window in which to get the machine online.

> Does FreeBSD have a problem with non-default fs settings? Has anyone
> else tried this sort of thing on such a large filesystem?

> I ended up having to use the default settings for newfs to get the
> system to work, wasting millions of inodes and bringing me to my next
> problem: Under load the filesystem is horribly slow. I expected some of
> this with the RAID-5 overhead but it's actually slower than a CCD I just
> moved from that was using 5 regular 2GB fast SCSI-2 drives, much slower.
> When running ktrace on the processes (qpopper and mail.local mainly) and
> watching top I can see that most of the processes are waiting for disk
> access. I've tried enabling/disabling various DPT options in the kernel
> but it's all about the same. I'd really like to stick with RAID-5 so
> using 0 or 1 just isn't what I'm looking for.

> The user directories for delivery are broken out into 1st letter, 1st
> two letters, username (i.e.: /home/u/us/username) to speed up dir
> lookups already. Any suggestions on how else to speed things up? This
> wasn't a problem on my old CCD, however.

> Lastly, I tried to find another RAID controller besides DPT that was
> compatible with FreeBSD 2.2.x with no luck. Upgrading to 3.1 is not an
> option at the moment, at least until things are more stable. Is anyone
> using anything in a host-based adapter (PCI) that is non-DPT?


 
 
 

DPT 3334UW Slow Performance/Filesystem probs (RAID-5)

Post by Ted Mittelstaed » Wed, 17 Feb 1999 04:00:00


Just a question:  Are all 6 disk drives on the same SCSI chain?  I'm not
familiar
with the DPT controller so I don't know if it has multiple SCSI busses on
it.

Also, is the filesystem mounted sync or async?

Ted



Quote:> I recently installed a DPT 3334UW with 64MB cache in a mail server
> problem: Under load the filesystem is horribly slow. I expected some of
> this with the RAID-5 overhead but it's actually slower than a CCD I just

 
 
 

DPT 3334UW Slow Performance/Filesystem probs (RAID-5)

Post by Bryn Wm. Moslo » Wed, 17 Feb 1999 04:00:00



> Just a question:  Are all 6 disk drives on the same SCSI chain?  I'm not
> familiar
> with the DPT controller so I don't know if it has multiple SCSI busses on
> it.

> Also, is the filesystem mounted sync or async?

> Ted



> > I recently installed a DPT 3334UW with 64MB cache in a mail server

> > problem: Under load the filesystem is horribly slow. I expected some of
> > this with the RAID-5 overhead but it's actually slower than a CCD I just

Yes, all 6 on the same bus and they are mounted sync. (Async is not an
option for us unfortunately or fortunately depending on your view.<g>)

Bryn