Very Computer

Dear Linux users,

A:

# cat files | cpio -ocav | dd bs=32k of=/dev/st0
24478883 blocks                  # cpio  (output)   512 byte blocks
0+3336377 records in           # dd  (output)
0+3336377 records out

results in a 12.5GB  (1G = 10^9) backup and uses 7h30, or 28 Meg per
minute. The DLT 7000 specs. state that the drive is capable of a
sustained data transfer of  600 MB per minute (with compression). During

backup the drive continuously spins up and down, maybe we are unable to
feed the drive with enough data.

We have tried to tune the parameters in /etc/stinit.def but didn't
notice any speed improvements.
  /etc/stinit.def
  # The common definitions that can usually be used
  {buffer-writes read-ahead async-writes }

  # Tandberg DLT 7000
  manufacturer=QUANTUM model = "DLT7000" {
    buffering can-bsr can-partitions auto-lock
    mode1 blocksize=0 density=0x85 # 85937 bpi, 52 quad tracks, serial
cart tape 70 Gb, compression
    mode2 blocksize=0 density=0x84 # 85937 bpi, 52 quad tracks, serial
cart tape 35 Gb, no compression
  }

Please is there someone who could you help us in determining the
problem, is it a Linux problem ? Is there a way to boost the backup
speed ? Does anyone have experience with DLT 7000 drives ?

Regards,

Henk

--- system configuration ---
- cat /proc/version

Fri May 8 16:05:57 EDT 1998

- mt -f /dev/nst0 status
SCSI 2 tape drive:
File number=0, block number=0, partition=0.
Tape block size 0 bytes. Density code 0x1b (unknown).
Soft error count since last status=0
General status bits on (41010000):
 BOT ONLINE IM_REP_EN

- cat /proc/scsi/scsi
Host: scsi0 Channel: 00 Id: 01 Lun: 00
  Vendor: QUANTUM  Model: DLT7000          Rev: 2151
  Type:   Sequential-Access                ANSI SCSI revision: 02

Henk Vergonet.
--
REDWOOD Software and services B.V.
phone:+31.30.6354573
fax  :+31.30.6354550
http://www.redwood.nl

Looks like it.

Quote:>Please is there someone who could you help us in determining the
>problem, is it a Linux problem ? Is there a way to boost the backup
>speed ? Does anyone have experience with DLT 7000 drives ?

I recommend using amanda as a backup program. It is network capable
and uses a holding disk to buffer processed output before writing it
to the tape in one run.

It is very reliable and convenient for me. More info is at
http://www.amanda.org

Johannes Niess

It was suggest that you try something like Amanda.  Here's what I think...

The same thing could very well happen, using any backup program.  Think about
it.  Rather than spooling data directlty from disk to tape, you are asking that
the user introduce a layer of software that reads from disk, spools to disk,
and then and then guess what.... reads again from disk to finally output to
tape!

I would simply experiment with the cpio buffer parameters (and take out the
pipe to "dd".  This can create un-necessary slowdowns.)

For starters, I recommend "cpio -aocvB".  The uppercase "B" says to block the
data at 5120 bytes, rather than the default of 512 bytes (a ten-fold increase).
Part of your slowdown is the overhead incurred by cpio only reading 512
bytes before passing the data along your pipeline to "dd".

Next, I would experiment with "cpio -aocvC <some_number>", where <some_number>
is for block sizes larger than the 5K offered by "B".  For example, try:
"cpio -ocvC 65536" (for a 64K block size) or "cpio -ocvC 131072" (for 128K)

Introducing additional pipes, filters, or backup agents (especially those that
do software compression) will certainly slow things down.

Ken

==================================
Ken Cormack
Cuyahoga Falls, Ohio

http://home.neo.rr.com/kcormack/

: Dear Linux users,
:
: A:
:
: # cat files | cpio -ocav | dd bs=32k of=/dev/st0
: 24478883 blocks                  # cpio  (output)   512 byte blocks
: 0+3336377 records in           # dd  (output)
: 0+3336377 records out
:
: results in a 12.5GB  (1G = 10^9) backup and uses 7h30, or 28 Meg per
: minute. The DLT 7000 specs. state that the drive is capable of a
: sustained data transfer of  600 MB per minute (with compression). During
:
: backup the drive continuously spins up and down, maybe we are unable to
: feed the drive with enough data.
:
: We have tried to tune the parameters in /etc/stinit.def but didn't
: notice any speed improvements.
:   /etc/stinit.def
:   # The common definitions that can usually be used
:   {buffer-writes read-ahead async-writes }
:
:   # Tandberg DLT 7000
:   manufacturer=QUANTUM model = "DLT7000" {
:     buffering can-bsr can-partitions auto-lock
:     mode1 blocksize=0 density=0x85 # 85937 bpi, 52 quad tracks, serial
: cart tape 70 Gb, compression
:     mode2 blocksize=0 density=0x84 # 85937 bpi, 52 quad tracks, serial
: cart tape 35 Gb, no compression
:   }
:
: Please is there someone who could you help us in determining the
: problem, is it a Linux problem ? Is there a way to boost the backup
: speed ? Does anyone have experience with DLT 7000 drives ?

Try using of=/dev/null in your command above, and note the time it takes to
run.  Is there any significant change in the run time?  If not, then the tape
drive was not your problem.

Is the tape drive on the same SCSI bus with the disk?  Can they be placed on
separate SCSI busses?

Does the DLT7000 have lights on the front panel indicating the density and
state of compression?

        Stu

Thanks Guys,

Yes,  it's really a buffering problem. I'm now using
   cpio -oacvC 32768 < files > /dev/st0
and getting a 20 fold speed increase !

Backup speed is now limited by the hard disk transfer rate !

Thanks,

Henk Vergonet.
--
REDWOOD Software and services B.V.
phone:+31.30.6354573
fax  :+31.30.6354550
http://www.redwood.nl

I used to work for a company (1) that made a RAID-3 DLT array using
the DLT4000's(2).  The system had 4 data DLT's and 1 parity DLT
running 5 internal SCSI channels.

The biggest performance problem we had was buffer underruns.  The host
would dump out data as quickly as possible, and the tape array would
write it faster than it was coming in.  Due to the high streaming
speed of the tape, the tape wouldn't be able to stop instantly.  It
would overshoot and have to rewind and relocate the end of the data so
that it could write the next set of blocks.

Really killed the performance.  I don't think I ever saw a unit fully
stream and obtain the spec'd figures in our brochures.  But we
couldn't really do much to speed up the host I guess...

On the plus side the Ultera controller is pretty sweet - it lets you
do all the RAID functionality transparently to the host - no drivers
other than standard DLT drivers needed.  The down side is that a lot
of software was written with tape capacity assumptions hard coded in
rather than derived from SCSI inquiries.  They didn't expect a DLT4000
to backup 80GB at 40GB an hour :)

If I recall either Ultera(3) or Andataco(4) did some benchmarking and
IIRC had to do memory to tape dumps on their HP to keep up, anything
else I guess had so much overhead that we had buffer underruns.

Dave

(1)     Land-5, www.land-5.com - I was their Linux guy.

(2)     According to the website they got DLT-7000 working - but they
        hadn't managed to do so before I left, so I've never seen it
        working.  The 7000 required external termination which the
        4000 has jumperable on the board.  We didn't have enough
        room in the shuttle and had a heck of a time getting the bus
        to terminate reliably due to the kludgy test environment

(3)     The OEM of the tape controller - http://www.ultera.com/

(4)     One of our OEM customers - we made their Rapid Tape DLT
        enclosure.

Dave