Very Computer

by **Jan Moral** » Thu, 24 Jun 1993 22:34:37

I am posting this for a friend. He is not on netnews.

---BEGIN------------------------------------

I am looking for a compression algorithm that can compress binary (i.e.,
randomly distributed) data by about 2:1 (or thereabouts). Does anyone have any
expertise in this area? I am trying to improve effective file I/O throughput
using compression.

I may be willing to pay $$$ for a recognized compression experts consultation
time.

Bob Saunders

301-517-1739
301-251-1990 FAX
"I only learnt to count up to one, so it was a natural for me to get into
computers." Anonymous.

---END--------------------------------------
--

Objective Interface Systems, Inc. UUCP: uunet!ois!jan
Reston, *ia, U.S.A.

by **Lawrence Kir** » Fri, 25 Jun 1993 01:16:49

>I am posting this for a friend. He is not on netnews.

>---BEGIN------------------------------------

>I am looking for a compression algorithm that can compress binary (i.e.,
>randomly distributed) data by about 2:1 (or thereabouts). Does anyone have any

>expertise in this area? I am trying to improve effective file I/O throughput
>using compression.

What do you mean by 'randomly distributed'. If you want lossless compression
(i.e. you want the decompressor always to reconstruct the original data
precisely) then it simply isn't possible to compress random data. Compression
works by recognising likely properties of the original data like repeated
strings or uneven character distribution. If the original data is random it
has no properties than a compressor can take advantage of.

Quote:>I may be willing to pay $$$ for a recognized compression experts consultation
>time.

In the hope that your data isn't really randon it might be an idea to post
more information about it. Then you can get various opinions on this
newsgroup as to whether it can be compressed or not.

-----------------------------------------

-----------------------------------------

by **Jim Baches** » Fri, 25 Jun 1993 05:24:57

|> I am looking for a compression algorithm that can compress binary (i.e.,
|> randomly distributed) data by about 2:1 (or thereabouts). Does anyone have any
|> expertise in this area? I am trying to improve effective file I/O throughput
|> using compression.

In my expeirence the ZIP "implode" algorithm gives about 2:1
compression. The source code is available from the net.
I can't remember which site I pulled it from. The code
is fairly well written and portable. This may work for you.

--
Jim Bachesta

Data I/O Corporation

206-867-6274

by **Barry Margol** » Sat, 26 Jun 1993 06:25:56

>>I am looking for a compression algorithm that can compress binary (i.e.,
>>randomly distributed) data by about 2:1 (or thereabouts).
>What do you mean by 'randomly distributed'. If you want lossless compression
>(i.e. you want the decompressor always to reconstruct the original data
>precisely) then it simply isn't possible to compress random data.

This is only true if the data is randomly distributed through the entire
set of possible values. But for any real world database, this is probably
unlikely. It's more likely that it's randomly distributed through some
subset of the possible values; for instance, if the data is in 8-bit bytes,
it could be randomly distributed from 0 to 30. Or if it's floating point
data, the structure of floating point representation makes some values more
likely in certain bytes than others, or makes some floating point values
themselves unlikely (e.g. if the database contains no infinities or NaN's).

All of these features can be exploited by a suitable (but not necessarily
the same) compression algorithm.
--
Barry Margolin
System Manager, Thinking Machines Corp.

by **Lawrence Kir** » Sat, 26 Jun 1993 08:56:11

>This is only true if the data is randomly distributed through the entire
>set of possible values. But for any real world database, this is probably
>unlikely. It's more likely that it's randomly distributed through some
>subset of the possible values; for instance, if the data is in 8-bit bytes,
>it could be randomly distributed from 0 to 30. Or if it's floating point
>data, the structure of floating point representation makes some values more
>likely in certain bytes than others, or makes some floating point values
>themselves unlikely (e.g. if the database contains no infinities or NaN's).

Quite correct - which is why I suggested that the original poster explain what
his data is in more detail.

-----------------------------------------

-----------------------------------------

Very Computer

Compression algorithm/compression consultant wanted (MD/USA)

Compression algorithm/compression consultant wanted (MD/USA)

Compression algorithm/compression consultant wanted (MD/USA)

Compression algorithm/compression consultant wanted (MD/USA)

Compression algorithm/compression consultant wanted (MD/USA)

Compression algorithm/compression consultant wanted (MD/USA)