Synchronizing files

Synchronizing files

Post by Ofer Razo » Wed, 24 Dec 2003 07:32:21



Hi all.

I have a directory with a 40Gigs of files which are Oracle`s datafiles of
my database.
I want to make a copy of the files in another directory which will be
updated daily.
Where`s the trick?
Here :
On the first day - my copy will be a normal cp command to the new directory.
From now and on, I want to move only the changed blocks (I`ll decide what is
the block size).
I`ll explain : Let`s say that on the first day I copied the files using the
cp command.
On the second day, only a small percentage on each file was changed.
I want only these blocks that changed to be updated in the files already
placed in my target dir.
Of course that the solution must handle cases when the sizes of the files
will change.

Any ideas?

Ofer.

 
 
 

Synchronizing files

Post by Darren Dunha » Wed, 24 Dec 2003 09:52:23



> Hi all.
> I have a directory with a 40Gigs of files which are Oracle`s datafiles of
> my database.
> I want to make a copy of the files in another directory which will be
> updated daily.
> Where`s the trick?
> Here :
> On the first day - my copy will be a normal cp command to the new directory.
> From now and on, I want to move only the changed blocks (I`ll decide what is
> the block size).
> Any ideas?

rsync will do that.

The Solaris filesystem does't keep track of modified blocks in any
useful manner, so you're going to have to compare blocks on both sides
to see if they've changed or not, or you're going to have to keep
another copy of all the data (usually too expensive to do so).

The only way to do that without more traffic than simply copying the
whole file is to send some sort of checksum for each of the blocks and
send a block only if the checksums don't match.

Don't forget to be in hotbackup mode or shutdown before copying them.

--

Unix System Administrator                    Taos - The SysAdmin Company
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >

 
 
 

Synchronizing files

Post by Darren Dunha » Wed, 24 Dec 2003 14:36:47



>> Don't forget to be in hotbackup mode or shutdown before copying them.
> Without using OMAN or whatever Oracle supplied utility to database
> backups, it seems to me that this is why there can be no incremental
> backups of Oracle databases.

(RMAN).  Not true.  As long as you either shut the database down or put
the tablespaces into hotbackup mode, you are guaranteed (by the
application), that any copy of those tablespaces plus relevant log files
will yield a consistent database.  The tablespaces may still be updated
in that case, but you don't care about that.

Quote:> Any change to any row in a tablespace changes that tablespace.

True, but rsync will attempt to only copy changed blocks, so you don't
have to turn an incremental copy into an effective full copy.

Quote:> Other databases I've worked with had
> replication software in place to distribute changed tables to remote
> locations.  Is there something similar for Oracle.

A common way to do that is to copy a "cold" backup (all tablespaces from
a shutdown database) to another machine and start it up with a read-only
mount.  Then you copy over archive logs as they are created and apply
them.  That copies over all changes and you don't need RMAN.  If you
need to, you just shut down the database, modify the contol file, then
start it up and you're live on the standby system.

If you ever have a replication problem though, you'll probably have to
punt and get another cold backup to start over.  The rsync method will
always make the database consistent if it runs to completion, regardless
of the initial state.

Quote:> If you're just using
> ufsdump or cp, how would you be able to do any sort of incremental
> backup?  Seems you'd have to copy the entire database's state or just do
> nightly a nightly export.

You couldn't use ufsdump or cp to perform an effective incremental
because neither of those tools will allow you to copy anything less than
an entire file.  rsync can be configured to copy only changed portions
of a file.

--

Unix System Administrator                    Taos - The SysAdmin Company
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >

 
 
 

Synchronizing files

Post by mg » Wed, 24 Dec 2003 16:18:02



Quote:> Hi all.

> I have a directory with a 40Gigs of files which are Oracle`s datafiles of
> my database.
> I want to make a copy of the files in another directory which will be
> updated daily.
> Where`s the trick?
> Here :
> On the first day - my copy will be a normal cp command to the new
directory.
> From now and on, I want to move only the changed blocks (I`ll decide what
is
> the block size).
> I`ll explain : Let`s say that on the first day I copied the files using
the
> cp command.
> On the second day, only a small percentage on each file was changed.
> I want only these blocks that changed to be updated in the files already
> placed in my target dir.
> Of course that the solution must handle cases when the sizes of the files
> will change.

> Any ideas?

> Ofer.

http://samba.org/rsync/
 
 
 

Synchronizing files

Post by Logan Sha » Thu, 25 Dec 2003 03:37:47



> I have a directory with a 40Gigs of files which are Oracle`s datafiles of
> my database.
> I want to make a copy of the files in another directory which will be
> updated daily.
> Where`s the trick?

I am no Oracle expert, but...

Can you really afford that much downtime for your database?
(I assume you don't copy them with the database running and
hope that they will be consistent, because I don't think there
is any guarantee that they will.)

If it is possible, I would suggest using Oracle's tools to
dump the data from the database, then store that data.

Alternatively, if the filesystem is ufs (and not vxfs or something),
you could shut down Oracle, make a snapshot volume with
fssnap, and then bring Oracle back up.  Then copy everything
over and delete the snapshot.  (You don't want to leave the
snapshot around for too long, because it will reduce filesystem
and thus database performance.)

   - Logan

 
 
 

Synchronizing files

Post by Dave Savill » Fri, 26 Dec 2003 00:35:23




>> I have a directory with a 40Gigs of files which are Oracle`s datafiles of
>> my database.
>> I want to make a copy of the files in another directory which will be
>> updated daily.
>> Where`s the trick?

>I am no Oracle expert, but...

>Can you really afford that much downtime for your database?
>(I assume you don't copy them with the database running and
>hope that they will be consistent, because I don't think there
>is any guarantee that they will.)

>If it is possible, I would suggest using Oracle's tools to
>dump the data from the database, then store that data.

Or do hot backups - then you just tell Oracle which table space you are
dumping and copy the files at the OS level, then tell oracle you have
finished.

Regards

Dave Saville

NB switch saville for nospam in address

 
 
 

1. Synchronize Files Between Two Linux Machines

Anyone know of software which will synchronize directories between two
Linux machines on a network or across the internet?  How about Windows
machines or Linux to Windows??

I need to have users make changes at either machine and have the changes
propogate to the other machine so that in time both machines will become
identical (at least the directories specified).

Thanks in advance.  Please email me your response.

Steve Marcus

2. Light Speed Bourne Shell! (was: Bourne shell tricks)

3. Synchronising file access permissions?

4. Help: "Swapoff" error

5. Synchronizing Files

6. COMMERCIAL: New Linux Bundle

7. synchronize files with downloaderrors

8. Basic question about USB support

9. How to synchronize files across standalone machines ?

10. Looking for a program to synchronize files

11. Synchronizing files on laptop

12. Need to synchronize password files on multiple machines

13. Synchronizing 2 file systems