1. Perfomance: tar vs ftp vs rsync vs cp vs ?
I'm looking for the fastest way to transfer files between 2 SUN
boxes sitting on a 100MB full-duplex ethernet. Data: ~100Gb of
binary files ~30Mb each.
Copying and moving disks around is not an option - don't have
physical access to the servers.
Files are not organized by filesystem and/or directories. I.e. I
have to transfer a list of files spread all over the places.
Things I'm considering:
1. cd /net/src; tar cf - list_of_files | (cd /net/dst; tar xf -)
2. for i in list_of_files; do rsync -options /net/src/$i /net/dst/$i; done
3. for i in list_of_files; do ftp each file; done
4. for i in list_of_files; do cp each file; done
For all 4 options I can break the list_of_files into sublists and
do them in parallel.
- Approximately 5% of files are the same on src and dst servers;
other files may have the same names, but are different and not
even similar (chunks of a database)
- Files compress with gzip 1:2 - I could reduce the total amount
of data travelling over the link by 2 via "z" option in tar or
rsync, but I'm afraid the overhead of gziping/ungzipping them
will consume more time, than I save on the network transers.
Server details: SRC server - E6500, 22 400MHz cpu, 48Gb RAM;
DST server - E10K, 4 333MHz cpu, 4Gb RAM.
Any thoughts on the above 4 methods and any other ideas on moving
things will be appreciated.
2. RH NEWBIE: RH Router Question
3. SunOS 5.6 vs Solaris 2.6 vs Solaris 7 vs Solaris 8
4. YDL 2.3 + PB3500 + Lucent wireless PCMCIA card: how to connect to AirPort base station?
5. rsync hangs on RedHat 2.4.2 or stock 2.4.4
6. SCO & UNIX: A Comedy of Errors ALWAYSON-NETWORK.com
7. RDIST, RSYNC or something else? replication
8. GoldStar 1505 resolution trouble...
9. Best tool to distribute files (rdist, rsync, cfengine)
10. rsync replacement for rdist
11. Stock Market Question: COREL vs RHAT vs VA Linux
12. rdist and rsync