Solaris 8 ufsdump Level 0 backup

Solaris 8 ufsdump Level 0 backup

Post by Ed F. de Guzma » Wed, 10 Sep 2003 00:04:56



I have a 450 server that uses a DLT IV tape unit to backup files using
ufsdump (Level 0 for weekly).  The system is running Oracle.

The odd thing is: about 3 weeks ago, a full system backup (Oracle
shutdown, of course) runs for 3 1/2 hours max.  Now it is 6 hours
without the database size changing!

The only thing that changed was the addition of a file system to backup
- /opt (around 200 mega bytes).  Currently, there are 12 filesystems to
backup versus the original 11 filesystems.  Oddly, the throughput has
gone down by 2/3 (6000+ kbytes/sec originally to 2000+ kbytes/sec) for
the Oracle filesystems.  Again, there were no increases in the database
size.

Any idea on what is going on? Where to start troubleshooting?

Thanks.

 
 
 

Solaris 8 ufsdump Level 0 backup

Post by Ed F. de Guzma » Wed, 10 Sep 2003 03:59:26


The funny thing is that throughput dives only in certain filesystems.
The filesystems associated with Solaris has throughput in the 7000+ Kbps
while the ones in associated with Oracle drops to 2000+ kbps.

I will try your suggestion.

Thanks.




> > I have a 450 server that uses a DLT IV tape unit to backup files using
> > ufsdump (Level 0 for weekly).  The system is running Oracle.

> > The odd thing is: about 3 weeks ago, a full system backup (Oracle
> > shutdown, of course) runs for 3 1/2 hours max.  Now it is 6 hours
> > without the database size changing!

> > The only thing that changed was the addition of a file system to backup
> > - /opt (around 200 mega bytes).  Currently, there are 12 filesystems to
> > backup versus the original 11 filesystems.  Oddly, the throughput has
> > gone down by 2/3 (6000+ kbytes/sec originally to 2000+ kbytes/sec) for
> > the Oracle filesystems.  Again, there were no increases in the database
> > size.

> > Any idea on what is going on? Where to start troubleshooting?

> > Thanks.

> Clean the drive a couple times.  Run some more backups.  If throughput
> doesn't improve, replace the drive.

> --
> DeeDee, don't press that button!  DeeDee!  NO!  Dee...


 
 
 

Solaris 8 ufsdump Level 0 backup

Post by Joerg Schilli » Wed, 10 Sep 2003 05:50:30




Quote:>I have a 450 server that uses a DLT IV tape unit to backup files using
>ufsdump (Level 0 for weekly).  The system is running Oracle.

>The odd thing is: about 3 weeks ago, a full system backup (Oracle
>shutdown, of course) runs for 3 1/2 hours max.  Now it is 6 hours
>without the database size changing!

>The only thing that changed was the addition of a file system to backup
>- /opt (around 200 mega bytes).  Currently, there are 12 filesystems to
>backup versus the original 11 filesystems.  Oddly, the throughput has
>gone down by 2/3 (6000+ kbytes/sec originally to 2000+ kbytes/sec) for
>the Oracle filesystems.  Again, there were no increases in the database
>size.

This may have many reasons.....

A point to start from may be a read speed test. Run:

        star -cM -time -C /mount-point . > /dev/null

to check the read speed with a speed optimized backup program.

ftp://ftp.berlios.de/pub/star/alpha/

--



URL:  http://www.fokus.fraunhofer.de/usr/schilling ftp://ftp.berlios.de/pub/schily

 
 
 

Solaris 8 ufsdump Level 0 backup

Post by Anthony Mandi » Wed, 10 Sep 2003 07:04:03



Quote:>   I have a 450 server that uses a DLT IV tape unit to backup files using
>   ufsdump (Level 0 for weekly).  The system is running Oracle.

        Orace rulez!

Quote:

>   The odd thing is: about 3 weeks ago, a full system backup (Oracle
>   shutdown, of course) runs for 3 1/2 hours max.  Now it is 6 hours
>   without the database size changing!

>   Again, there were no increases in the database size.

        Then check the UFS filesystems.

        find / -type f -size +60000 -ls

        That'll show files about 30 Mb and larger. As a matter of course, delete any
        core files you find before backing up (or put on a separate tape.)

-am     ? 2003

 
 
 

Solaris 8 ufsdump Level 0 backup

Post by Logan Sha » Wed, 10 Sep 2003 08:29:52



Quote:> I have a 450 server that uses a DLT IV tape unit to backup files using
> ufsdump (Level 0 for weekly).  The system is running Oracle.

> The odd thing is: about 3 weeks ago, a full system backup (Oracle
> shutdown, of course) runs for 3 1/2 hours max.  Now it is 6 hours
> without the database size changing!

Do a "ufsdump" on each filesystem, but replace your tape device
with "/dev/null".  (Be very careful to leave "u" out of the options,
otherwise this will mess up your /etc/dumpdates!)

This will tell you whether there is trouble reading the filesystems
quickly enough.  (It will show you the average data speed in
ufsdump's output.)  If there isn't trouble reading the filesystems,
you can assume it's probably trouble writing to the tape.

Also, when you added /opt, I assume you changed around the
partitions a little.  Is it possible that /etc/fstab has an
error or that your backup script is still using the
old /dev/rdsk/c?t?d?s? entries?  If so, it could be the
case that you're actually backing up the wrong devices
and writing too much data for that reason.

   - Logan

 
 
 

Solaris 8 ufsdump Level 0 backup

Post by Przemek Ba » Wed, 10 Sep 2003 15:14:46



Quote:>  I have a 450 server that uses a DLT IV tape unit to backup files using
>  ufsdump (Level 0 for weekly).  The system is running Oracle.

>  The odd thing is: about 3 weeks ago, a full system backup (Oracle
>  shutdown, of course) runs for 3 1/2 hours max.  Now it is 6 hours
>  without the database size changing!

>  The only thing that changed was the addition of a file system to backup
>  - /opt (around 200 mega bytes).  Currently, there are 12 filesystems to
>  backup versus the original 11 filesystems.  Oddly, the throughput has
>  gone down by 2/3 (6000+ kbytes/sec originally to 2000+ kbytes/sec) for
>  the Oracle filesystems.  Again, there were no increases in the database
>  size.

>  Any idea on what is going on? Where to start troubleshooting?

Did you remount oracle filesystem with 'forcedirectio' option ?
Can you show output of 'mount' command ?

przemol

 
 
 

Solaris 8 ufsdump Level 0 backup

Post by Ed F. de Guzma » Wed, 10 Sep 2003 20:51:26


Here is part of the mount output:

I am getting largefile errors.  The disk are mirrored using DiskSuite.
Again, I have not seen this errors before:

/ on /dev/md/dsk/d150
read/write/setuid/intr/largefiles/onerror=panic/dev=1540096 on Sat Se
p  6 12:55:42 2003
/usr on /dev/md/dsk/d151
read/write/setuid/intr/largefiles/onerror=panic/dev=1540097 on Sat
 Sep  6 12:55:43 2003
/proc on /proc read/write/setuid/dev=37c0000 on Sat Sep  6 12:55:41 2003
/dev/fd on fd read/write/setuid/dev=3880000 on Sat Sep  6 12:55:44 2003
/etc/mnttab on mnttab read/write/setuid/dev=3980000 on Sat Sep  6
12:55:46 2003
/var on /dev/md/dsk/d154
read/write/setuid/intr/largefiles/onerror=panic/dev=154009a on Sat
 Sep  6 12:55:47 2003
/var/run on swap read/write/setuid/dev=1 on Sat Sep  6 12:55:47 2003
/tmp on swap read/write/setuid/dev=2 on Sat Sep  6 12:55:52 2003
/u00 on /dev/md/dsk/d350
read/write/setuid/intr/largefiles/onerror=panic/dev=154015e on Sat
 Sep  6 12:55:52 2003
/syncsort_tmp on /dev/md/dsk/d250
read/write/setuid/intr/largefiles/onerror=panic/dev=15400
fa on Sat Sep  6 12:55:52 2003
/u01 on /dev/md/dsk/d450
read/write/setuid/intr/largefiles/onerror=panic/dev=15401c2 on Sat
 Sep  6 12:55:52 2003
/ftphome on /dev/md/dsk/d253
read/write/setuid/intr/largefiles/onerror=panic/dev=15400fd on
 Sat Sep  6 12:55:52 2003
/u02 on /dev/md/dsk/d550
read/write/setuid/intr/largefiles/onerror=panic/dev=1540226 on Sat
 Sep  6 12:55:52 2003

Any ideas?

Thanks



> >  I have a 450 server that uses a DLT IV tape unit to backup files using
> >  ufsdump (Level 0 for weekly).  The system is running Oracle.

> >  The odd thing is: about 3 weeks ago, a full system backup (Oracle
> >  shutdown, of course) runs for 3 1/2 hours max.  Now it is 6 hours
> >  without the database size changing!

> >  The only thing that changed was the addition of a file system to backup
> >  - /opt (around 200 mega bytes).  Currently, there are 12 filesystems to
> >  backup versus the original 11 filesystems.  Oddly, the throughput has
> >  gone down by 2/3 (6000+ kbytes/sec originally to 2000+ kbytes/sec) for
> >  the Oracle filesystems.  Again, there were no increases in the database
> >  size.

> >  Any idea on what is going on? Where to start troubleshooting?

> Did you remount oracle filesystem with 'forcedirectio' option ?
> Can you show output of 'mount' command ?

> przemol

 
 
 

Solaris 8 ufsdump Level 0 backup

Post by Przemek Ba » Wed, 10 Sep 2003 21:29:48



Quote:>  Here is part of the mount output:

>  I am getting largefile errors.  [ ... ]

Where ?

Quote:>  / on /dev/md/dsk/d150
>  read/write/setuid/intr/largefiles/onerror=panic/dev=1540096 on Sat Se

The above doesn't mean you have errors. It shows how filesystem behaves
when an error occur.

przemol

 
 
 

Solaris 8 ufsdump Level 0 backup

Post by Ed F. de Guzma » Wed, 10 Sep 2003 21:33:28


My mistake.  I read it wrongly.

There is no problem with the filesystem.  I will look at the other
answers and figure out where the throughput slows down.

Thanks.



> >  Here is part of the mount output:

> >  I am getting largefile errors.  [ ... ]

> Where ?

> >  / on /dev/md/dsk/d150
> >  read/write/setuid/intr/largefiles/onerror=panic/dev=1540096 on Sat Se

> The above doesn't mean you have errors. It shows how filesystem behaves
> when an error occur.

> przemol

 
 
 

Solaris 8 ufsdump Level 0 backup

Post by Tony Walto » Wed, 10 Sep 2003 22:43:21



Quote:> Here is part of the mount output:

> I am getting largefile errors.  The disk are mirrored using DiskSuite.

> / on /dev/md/dsk/d150
> read/write/setuid/intr/largefiles/onerror=panic/dev=1540096 on Sat Se

No, you're not. "onerror=panic" is the default, and means that if the
kernel detects an inconsistency in the filesystem it'll panic. There are
other options - see the mount_ufs(1M) manpage

                 onerror = action
                       This option specifies the action  that  UFS
                       should  take  to  recover  from an internal
                       inconsistency on  a  file  system.  Specify
                       action  as  panic,  lock,  or umount. These
                       values cause a forced  system  shutdown,  a
                       file  system lock to be applied to the file
                       system, or the file system to  be  forcibly
                       unmounted,  respectively.  The  default  is
                       panic.

 > Again, I have not seen this errors before:

Odd, the output of mount has looked like that since Solaris 8 was released.

--
Tony

 
 
 

Solaris 8 ufsdump Level 0 backup

Post by Ed F. de Guzma » Wed, 10 Sep 2003 23:16:17


My mistake.  I read it wrongly.

There is no problem with the filesystem.  I will look at the other
answers and figure out where the throughput slows down.

Thanks.



> > Here is part of the mount output:

> > I am getting largefile errors.  The disk are mirrored using DiskSuite.

> > / on /dev/md/dsk/d150
> > read/write/setuid/intr/largefiles/onerror=panic/dev=1540096 on Sat Se

> No, you're not. "onerror=panic" is the default, and means that if the
> kernel detects an inconsistency in the filesystem it'll panic. There are
> other options - see the mount_ufs(1M) manpage

>                  onerror = action
>                        This option specifies the action  that  UFS
>                        should  take  to  recover  from an internal
>                        inconsistency on  a  file  system.  Specify
>                        action  as  panic,  lock,  or umount. These
>                        values cause a forced  system  shutdown,  a
>                        file  system lock to be applied to the file
>                        system, or the file system to  be  forcibly
>                        unmounted,  respectively.  The  default  is
>                        panic.

>  > Again, I have not seen this errors before:

> Odd, the output of mount has looked like that since Solaris 8 was released.

> --
> Tony

 
 
 

1. Solaris 8 ufsdump much slower than Solaris 7 ufsdump

I have just submitted the following service request to Sun.  Am I the
only one to see this problem or have all of you just been suffering
in silence?  We just upgraded to Solaris 8 two weeks ago.

ufsdump under solaris 8 is taking about 50% longer to run than it
did under solaris 7.  Running the Solaris 7 ufsdump under the
Solaris 8 operating system gives us the faster run time.

Example: dumping an 8 GB disk using the Solaris 8 ufsdump takes
2 hours 9 minutes to dump 7327.44 MB at 975 KB/sec.  Dumping the
same 8 GB disk using the Solaris 7 ufsdump under the Solaris 8
operating system takes 1 hour 22 minutes to dump 7327.69 MB at
1536 KB/sec.

The system was quiescent while doing the dumps.
The command line is:
ufsdump 0ufb /dev/rmt/0cn 126 /dev/rdsk/c0t2d0s6
The system is a Sparc station 10 with 96 MB memory (adding more
memory made no difference).  The tape drive is a Sony Advanced
Intelligent Tape on it's own SCSI buss.

I did notice that the size of the Solaris 8 ufsdump is 82484 bytes
while the size of the Solaris 7 ufsdump is 162988.  Were these
built with different optimizations?

Note that this slow down makes it impossible to get our dumps
done without spilling over into our normal working hours.
--
Tom Schulz

2. Linux and PCMCIA card DEPCM-BA

3. Ufsdump and Solaris Backup

4. New improved INN 1.4 for Linux package available

5. Typical problem with ufsdump backup on solaris 7

6. kernel sources?

7. Backup with Solaris ufsdump

8. NIS+ and JumpStart info reqrd.

9. Solstice Backup 5.1: can't do level or incremental backups

10. Solstice backup, pools & backup level

11. Solaris 2.6 - Zero Level Backup???

12. ufsdump levels

13. ufsdump level 0 missing files