Backup speed to DAT-tape rotten, ideas?

Backup speed to DAT-tape rotten, ideas?

Post by tho.. » Tue, 18 Jan 2000 04:00:00



Hi all,

I have a HP C1533A SCSI DDS-3 tape connected to a P166 running
FreeBSD 3.3 release.

No matter what I do, I can't get the backup speed over 1-2 MB/min.
I've tried dump, pax, and
find . -xdev -depth -print | cpio -o | dd of=/dev/nrsa0 bs=10k etc

I've experimented with blocksizes between 512b and 64k.

I keep looking at iostat, and the transfer rate rises to about
150k/s, then it appears to overrun the tape, the tape clicks
and whirrs a bit, and restarts with a transfer rate of ~5k/s.

What bugs me is that the Backup program supplied with Windows 98
gets a throughput of ~350k/s with the same hardware, without the
clicking and whirring.

Any ideas ?

The SCSI host adapter is an Adaptec 2940, the PC's got 32MB RAM,
only ran X in addition to the backup. No huge servers running.
mt <dev> stat reports that HW compression is on, and that things
look mostly good.

C'mon guys, help me beat Microsoft..

  -Terje

Sent via Deja.com http://www.deja.com/
Before you buy.

 
 
 

Backup speed to DAT-tape rotten, ideas?

Post by Carl Masco » Tue, 18 Jan 2000 04:00:00



>I have a HP C1533A SCSI DDS-3 tape connected to a P166 running
>FreeBSD 3.3 release.

>No matter what I do, I can't get the backup speed over 1-2 MB/min.
>I've tried dump, pax, and
>find . -xdev -depth -print | cpio -o | dd of=/dev/nrsa0 bs=10k etc

>I've experimented with blocksizes between 512b and 64k.

>I keep looking at iostat, and the transfer rate rises to about
>150k/s, then it appears to overrun the tape, the tape clicks
>and whirrs a bit, and restarts with a transfer rate of ~5k/s.

>What bugs me is that the Backup program supplied with Windows 98
>gets a throughput of ~350k/s with the same hardware, without the
>clicking and whirring.

I can think of a couple of things that could cause a lot of
stops and starts (clicks and whirs) during a backup:

1. Data underruns (not overruns).

2. Block rewrites.

The first step is to determine which of the above you are
experiencing.

Try to rule out block rewrites first.  These are caused
by bad media.  Have you tried several different tapes
w/ FBSD?  Can you try a new data grade tape?  The best test
is to use the same tape to back up FBSD and then Win 98.
If the FBSD backup is slow and the Win 98 backup is fast,
block rewrites are not the problem.

Data underruns are hard to deal with.  You can attack this
problem from both ends:

1. File system read throughput.

2. Tape system data rate requirement.

The object is to get #1 to be greater than #2.
Ideally, it's better to achieve this by increasing
#1 rather than by reducing #2.

There isn't much you can do about file system read
throughput, but one trick is to set the noatime
mount flag while doing a backup.  Of course, access
times won't be updated, but you may not mind (I
don't).

If your file systems ever got full or nearly full,
but are not so now, you might try laying them down
fresh (newfs, then restore).  FFS can't always put
things where it wants to when the file system is
nearly full.  Of course, if the file systems are just
going to fill up again tomorrow, don't bother: you
need larger file systems in that case.

Another way to improve file system read throughput is
to avoid having a zillion tiny files in a zillion
directories, but we generally don't have any control
over that.

As a last resort, the tape system data rate requirement
can be reduced in a couple of ways:

1. Use a lower density tape format.

2. Turn off compression.

You'll have to experiment here.  Trying either of the
above might speed up your backups, but might also
slow them down.

--
Carl Mascott


 
 
 

Backup speed to DAT-tape rotten, ideas?

Post by tho.. » Wed, 19 Jan 2000 04:00:00





> >I have a HP C1533A SCSI DDS-3 tape connected to a P166 running
> >FreeBSD 3.3 release.

> >No matter what I do, I can't get the backup speed over 1-2 MB/min.
> >I've tried dump, pax, and
> >find . -xdev -depth -print | cpio -o | dd of=/dev/nrsa0 bs=10k etc

> I can think of a couple of things that could cause a lot of
> stops and starts (clicks and whirs) during a backup:

> 1. Data underruns (not overruns).
> 2. Block rewrites.

[ lots of very good suggestions snipped ]

> --
> Carl Mascott


I have to say this is almos eerie, if that is the word I want.
During my previous attempts I had the notion that a disk was a disk and
a tape a tape. From following your suggestions I've been able to track
down a combination of causes : both a suspect tape and disk-throughput
problems.

All tapes tried came fresh from the box, but they're a few years old. By
turning off compression om the drive, I was able to get throughput
enough not to starve the tape. The remaining restarts I then put down to
rewrites, and did the rest of the testing on a(nother) fresh tape, with
better results.

Then your suggestions on a large number of small files
(/usr/local/ports, of course) and the speed of my drives came in handy.
Turns out my da1 has better throughput than da0. (confirmed this, and
got around the file angle with "dd if=/dev/<disk> of=/dev/<tape>")

I still have some tweaking to do, but atleast now it's a slightly more
educated approach.

Thank you.

  -Terje

Sent via Deja.com http://www.deja.com/
Before you buy.

 
 
 

Backup speed to DAT-tape rotten, ideas?

Post by Christian Weisgerb » Thu, 20 Jan 2000 04:00:00



> No matter what I do, I can't get the backup speed over 1-2 MB/min.
> I've tried dump, pax, and
> find . -xdev -depth -print | cpio -o | dd of=/dev/nrsa0 bs=10k etc

> I keep looking at iostat, and the transfer rate rises to about
> 150k/s, then it appears to overrun the tape, the tape clicks
> and whirrs a bit, and restarts with a transfer rate of ~5k/s.

Tape underrun. The backup program doesn't deliver the data quickly
enough or not uniformly enough => the data stream breaks, the tape
drive has to stop, backspace, and start up anew => overall throughput
collapses and wear on the drive and the tape increases immensely.

Rather than using dd(1), which uses a single process that blocks
for both read() and write(), try a buffering program, such as
misc/buffer or misc/team in the ports collection. If the disk can
deliver the data fast enough averaged over time, those tools should
help you out. dump already includes internal buffering, piping it
into buffer/team won't improve performance.

If the disk can't keep up with the tape even on average--which I
have a hard time believing, given your quote of 350kB/s tape
speed--then you've got a problem. In that case, you'll need to use
a holding partition, i.e. dump the backup archive to a spare disk
partition and later on transfer it from there to the tape.

--