To compress or not to compress, that is the question.

To compress or not to compress, that is the question.

Post by Kevin Smi » Sun, 25 Jun 2000 04:00:00




>When backing up to a DDS tape which is large enough to contain all of a
>site's files as is, uncompressed, is it measurably slower (or faster) to
>let the DDS drive exercise its internal compression routine?

>(Obviously I could test this myself, but I'm trying not to reinvent any
>wheels, and am interested in any explanations of why the answer is
>whatever it is.)

I sat here for quite a while and read all the responses which were
mostly off topic :)

Tape drives have a fixed transfer rate to the tape.  Say 5mb/sec.
If you are getting 2:1 compression, every 5mb to tape is 10mb of
your data.

Data is streaming through the controller at 10mb/sec and compressed
data is going onto the tape at it's steady old rate of 5mb/sec.

So, your backup runs twice as fast with 2:1 compression, subject to
other limiting factors like file reads and controller and bus
performance.

With a pci bus and decent controller they are not the bottleneck.
It will be a combination of read speed and compression ratios.

- Binary data (executables and the like) seem to compress about 2:1.
- Database and text data often compress much more.
- Compressed data doesn't compress further (.Z, .gz, many graphic
  formats).
- YMMV

I've never noticed uncompressable data to a tape drive in compressing
mode to run any slower than in non-compressing mode.

So, there is no reason beyond compatability ( with older non-compressing
or proprietary compression schemes ( which are no longer used as far
as I know)) to use non-compressing mode.  IMHO.
--
Do two rights make | Kevin Smith, ShadeTree Software, Philadelphia, PA, USA
a libertarian      | 001-215-487-3811  shady.com,kevin   bbs.cpcn.com,sysop
                   | dvtug.org,kevins--Delaware Valley Transit Users Group

 
 
 

1. COMPRESS (how much does it compress?)

I was wondering if someone could tell me what the average compression
factor is for the compress command?  We are compressing some of the
filesystems in our nightly backups which makes it difficult to compute
how much data is actually being written to tape.  If I knew the average
compression factor then I could make better estimates.

The Sun man page says 50-60% for English text or source-code.  What
about for an entire filesystem?  What about for an oracle database?
What about other things like say binaries?

Does anyone know?

--
--
Alan W. McKay           | (902)542-2201.158     | Wolfville, N.S. Canada

2. App Title Bar Creeped Out of Reach

3. Compressed filesystem or Compressed loop?

4. [v850] Remove some unneeded register saving on the v850

5. compressing on the fly: father process seems to end before popened compress

6. Term woes

7. Compressed 2.2 kernel about 35% larger than compressed 2.0.x?

8. lan drivers

9. PPP-COMPRESS -- what am I missing?

10. kernel not compressed

11. Help: Kernel Builds but will not Compress (Very Frustrating)

12. 0ss603a not a compressed file?????

13. Compile did not compress kernel