Which backup tool to backup huge filesystems to tape?

Which backup tool to backup huge filesystems to tape?

Post by Florian Kirste » Wed, 31 Jul 2002 14:38:51



Hi,

after reading this group through it looked for me like amanda would
be the favoured tool, I installed it and it actually looks quite
sympatic (not too overloaded, understandable...), BUT: from
everything I read and tried out it seems uncapable of handling
filesystems whose backup is larger than the available tape size.

I have two RAIDs with 0.5 and 1 TB of data, currently only the 0.5 TB are
full but this will change soon (currently I backup to the 1TB RAID, but
this is not satisfying, especialy because we had a near-bad crash due to a
failure in the air conditioning, so I'm searching for something more
static :). We've also got a Tandberg SuperDLT with a per-tape capacity
of about 110GB, so it should be capable of doing the job to currently
5 tapes, which is OK. The RAID is connected to the fileserver, the tape
to a seperate box (both running RedHat Linux, 7.0 in the moment)

So what I'm looking for is a reliable backup program (from this point I
liked amanda because of the usage of standard formats like tar, I don't
want to use dumpfs like tools as I need only parts being backupped and
want to be able to work on the fs during the backup process) capable of
using my tapes optimal. Any good/bad experiences? It should also be
capable of backing up over the network, and possibly also backup some
windows and macintosh workstations, but I could mount these using samba.

What I've tried so far:
- homegrown tar solution with split option. Works. Drawback: no sophisticated
tape management (I'd for example like to do incremental backups, appending
them to existing tapes, so the tapes are used about equally) which I don't
want to re-invent once again, no good way for software compression (using
the z option in tar compresses the whole archive as one stream which I
think could bring me into trouble when one tape fails. Without compression
I should be able to recover at least the undamaged date).

- amanda: almost useless for me. The proposed solutions for the "tape too
small" problem I read so far were:
 - buy larger drive (1 TB? haha)
 - do full backup outside amanda using tar and just the increments using
   amanda (not much better than doing everything with tar, besides this
   it could well be that over 110 GB change on one day)
 - instead of backing up /raid backup /raid/smallerdir1, /raid/smallerdir2
   and so on. Unmanageable for my needed capacities and wastes a lot of
   tape space because no tape is used full.

- arkeia: the best I found so far, but I read of bad experiences with
restores from damaged tapes, problems after the arekeia installation
got lost, random crashes and such things. Never happend to me, but I
also didn't have the impression of a very reliable program (almost no
real error checking on various config files). But my experiences are
over 1.5 years old, did it improve? At least it can split a backup
to several tapes :)

My next candidate is BRU, I'll download the trial version tomorrow.
Any opinions on it? Any hints for other tools?


 
 
 

Which backup tool to backup huge filesystems to tape?

Post by Michael Heimin » Wed, 31 Jul 2002 15:13:51



[..]

Quote:> My next candidate is BRU, I'll download the trial version
> tomorrow. Any opinions on it? Any hints for other tools?

If you don't mind spending some money and you are on a secure
network, you could try Legato networker, it works with most tape
charger and is available for many OS.

Michael Heiming
--
Remove the +SIGNS case mail bounces.

 
 
 

Which backup tool to backup huge filesystems to tape?

Post by Jean-David Beye » Wed, 31 Jul 2002 21:09:56


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

| Hi,
|
| after reading this group through it looked for me like amanda would
| be the favoured tool, I installed it and it actually looks quite
| sympatic (not too overloaded, understandable...), BUT: from
| everything I read and tried out it seems uncapable of handling
| filesystems whose backup is larger than the available tape size.
|
| I have two RAIDs with 0.5 and 1 TB of data, currently only the 0.5 TB are
| full but this will change soon (currently I backup to the 1TB RAID, but
| this is not satisfying, especialy because we had a near-bad crash due to a
| failure in the air conditioning, so I'm searching for something more
| static :). We've also got a Tandberg SuperDLT with a per-tape capacity
| of about 110GB, so it should be capable of doing the job to currently
| 5 tapes, which is OK. The RAID is connected to the fileserver, the tape
| to a seperate box (both running RedHat Linux, 7.0 in the moment)
|
| So what I'm looking for is a reliable backup program (from this point I
| liked amanda because of the usage of standard formats like tar, I don't
| want to use dumpfs like tools as I need only parts being backupped and
| want to be able to work on the fs during the backup process) capable of
| using my tapes optimal. Any good/bad experiences? It should also be
| capable of backing up over the network, and possibly also backup some
| windows and macintosh workstations, but I could mount these using samba.

I use BRU that, from the instruction book, will easily span tapes. I do
not have enough data to test this, but I imagine it works. (My two
9.1GByte drives, even if full, would easily fit on the smallest V6 tapes
for my Ecrix VXA-1 tape drive.)

I do not recall if cpio will do this or not. If not, you could try a
shell script with find-cpio where each find is setup to be sure what it
finds will fit a tape. Not ideal, perhaps, but it should work.

I would not, personally, use tar for backups. When I worked at Bell
Labs, we deprecated tar for this purpose and wrote cpio (I did not write
it) to overcome the deficiencies and inconveniences of tar.

|
| What I've tried so far:
| - homegrown tar solution with split option. Works. Drawback: no
sophisticated
| tape management (I'd for example like to do incremental backups, appending
| them to existing tapes, so the tapes are used about equally) which I don't
| want to re-invent once again, no good way for software compression (using
| the z option in tar compresses the whole archive as one stream which I
| think could bring me into trouble when one tape fails. Without compression
| I should be able to recover at least the undamaged date).
|
| - amanda: almost useless for me. The proposed solutions for the "tape too
| small" problem I read so far were:
|  - buy larger drive (1 TB? haha)
|  - do full backup outside amanda using tar and just the increments using
|    amanda (not much better than doing everything with tar, besides this
|    it could well be that over 110 GB change on one day)
|  - instead of backing up /raid backup /raid/smallerdir1, /raid/smallerdir2
|    and so on. Unmanageable for my needed capacities and wastes a lot of
|    tape space because no tape is used full.
|
| - arkeia: the best I found so far, but I read of bad experiences with
| restores from damaged tapes, problems after the arekeia installation
| got lost, random crashes and such things. Never happend to me, but I
| also didn't have the impression of a very reliable program (almost no
| real error checking on various config files). But my experiences are
| over 1.5 years old, did it improve? At least it can split a backup
| to several tapes :)
|
| My next candidate is BRU, I'll download the trial version tomorrow.
| Any opinions on it? Any hints for other tools?
|

I like BRU a lot (I paid for mine). I started with 15.1 (after using
find-cpio for years). They sent me a free upgrade to 16.0 rather than
fixing a bug in 15.1 (which was obsolete at the time, but working OK for
me). When I did my latest upgrade, I bought 17.0 that I am using right now.

BRU has a feature that I consider of major importance, though you could
probably figure out how to do it yourself:

There is a (free) add-on, called CRU (Crash Recovery Utility), that
writes two floppies and a tape. If everything crashes, you buy new hard
drives, stick in the first (boot) floppy and reboot your machine. Then
you put in the second (root) floppy, and the tape. It will repartition
and reformat your new drives (that must be at least as large as your old
ones), restore your /dev and all the rest of the file systems. It can
take a while, depending on the size of your disks, but at least you know
it will be done correctly. Far easier that restoring from a bunch of
floppies, cd-roms, downloading forgotten stuff from unremembered web
locations, etc., and guessing how to configure them.

- --
~  .~.  Jean-David Beyer           Registered Linux User 85642.
~  /V\                             Registered Machine    73926.
~ /( )\ Shrewsbury, New Jersey     http://counter.li.org
~ ^^-^^ 8:00am up 5 days, 18:36, 2 users, load average: 2.18, 2.17, 2.17
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQE9RoITPtu2XpovyZoRAofWAJ9u/UYfNAr7wddelcOhmhbie4GL5wCeIEaF
OLePHI9qCKP5PTgtj4MN7w0=
=ZrMZ
-----END PGP SIGNATURE-----

 
 
 

Which backup tool to backup huge filesystems to tape?

Post by Joshua Baker-LePai » Wed, 31 Jul 2002 23:38:30



> - amanda: almost useless for me. The proposed solutions for the "tape too
> small" problem I read so far were:
>  - buy larger drive (1 TB? haha)
>  - do full backup outside amanda using tar and just the increments using
>    amanda (not much better than doing everything with tar, besides this
>    it could well be that over 110 GB change on one day)
>  - instead of backing up /raid backup /raid/smallerdir1, /raid/smallerdir2
>    and so on. Unmanageable for my needed capacities and wastes a lot of
>    tape space because no tape is used full.

I do the last solution on our .5TB RAID, and will be doing so on our new
2TB RAID.  Currently I have an AIT1 drive (35GB uncompressed, and our
data doesn't compress), but I may be moving to AIT3 or a loader.

Yes, there is a bit of administrative overhead breaking up the filesystem.
But I'm a bit at odds with your "wasted tape space", since amanda will
do what you tell her to.  If you base your dumpcycle and runspercycle
numbers on your usage, you can get very near to using all of each tape.
Also, since you rotate the tapes, if a tape doesn't get used to capacity
one tapecycle, it will next time.

The reason I *like* amanda and none of the other commercial solutions
is that it is possible to get *all* your data off of amanda backup tapes
even if your backup server goes up in flames.  If you lose all your
tape management info, mt, dd, and restore/tar will get you all your data
back.  Try that with a commercial product.

--
Joshua Baker-LePain
Department of Biomedical Engineering
Duke University

 
 
 

Which backup tool to backup huge filesystems to tape?

Post by Nick » Thu, 01 Aug 2002 07:20:16


There are several commercial Backup products, all the Super Tar's, that will
allow you to restore your data if everything goes up in smoke.
LONE-TAR is one of them, www.LONE-TAR.com.  It does not use tar, however, it
is completely tar compatible and has much more intelligence built into it,
such as Error recovery when restoring from a damaged tape, and true Bit
Level verification of the backup media when it's created. Lone-Tar is also
compatible on every platform of UNIX and Linux.

Plus Lone-Tar includes crash recovery software that rebuilds the system if
it goes down. It will even restore to a new hard drive if needed.

Nick



|
| > - amanda: almost useless for me. The proposed solutions for the "tape
too
| > small" problem I read so far were:
| >  - buy larger drive (1 TB? haha)
| >  - do full backup outside amanda using tar and just the increments using
| >    amanda (not much better than doing everything with tar, besides this
| >    it could well be that over 110 GB change on one day)
| >  - instead of backing up /raid backup /raid/smallerdir1,
/raid/smallerdir2
| >    and so on. Unmanageable for my needed capacities and wastes a lot of
| >    tape space because no tape is used full.
|
| I do the last solution on our .5TB RAID, and will be doing so on our new
| 2TB RAID.  Currently I have an AIT1 drive (35GB uncompressed, and our
| data doesn't compress), but I may be moving to AIT3 or a loader.
|
| Yes, there is a bit of administrative overhead breaking up the filesystem.
| But I'm a bit at odds with your "wasted tape space", since amanda will
| do what you tell her to.  If you base your dumpcycle and runspercycle
| numbers on your usage, you can get very near to using all of each tape.
| Also, since you rotate the tapes, if a tape doesn't get used to capacity
| one tapecycle, it will next time.
|
| The reason I *like* amanda and none of the other commercial solutions
| is that it is possible to get *all* your data off of amanda backup tapes
| even if your backup server goes up in flames.  If you lose all your
| tape management info, mt, dd, and restore/tar will get you all your data
| back.  Try that with a commercial product.
|
| --
| Joshua Baker-LePain
| Department of Biomedical Engineering
| Duke University

 
 
 

1. how backup multiple filesystems onto 1 tape using backup by i-node ?

I quote from man/info :
"Use the backup command when you want to back up large and multiple file
systems"

How can I achieve this, backing up multiple filesystems onto one tape using
the
backup (by i-node) command... Infoexplorer and man don't give any examples
on backing
up multiple filesystems (using backup),  at least I couldn't find any.

Who can point me in the right direction?

TIA,
Bruno

ps: AIX version : 4.1.5 & 4.2--

--

Coca-Cola Enterprises Services, Brussels

2. NMAKE gurus

3. "live" filesystems backup on AIX using backup/mksysb/sysback/storix

4. cleaning up /tmp on solaris 2.6

5. Solstice Backup 5.1 / DMO 2.1/Oracle 8.1.5: Tape backup failed : Need of 5.5 Upgrade ?

6. thread

7. How to designate full backup to specific tape using Solstice Backup?

8. Making man pages

9. Q: Looking for backup tool(s) to handle >2Gb filesystems.

10. Need reliable/stable BACKUP tool/method for 24hr filesystems

11. Backup Q: How to dump multiple filesystems to one tape?

12. Backup multiple filesystems on one tape

13. Backup a huge file accross floppies