Very Computer

by **jim_marshall2..** » Wed, 23 Aug 2000 04:00:00

I'm in the midst's of writing a program that can be updated via the
internet. The program will download gzip'd versions of shared libraries
it uses. I found 'zlib' and have been using the file operation function
(gzopen, gzread) to decompress the file.

However; there doesn't appear to be any API in zlib to determine the
original CRC of the '.gz' file. Additionally if the '.gz' file is
corrupted performing a gzopen, gzread will not cause any problems (at
least they return success values).

The question is, while using zlib how can I determine if the '.gz' file
is not corrupted?

Thanks
-Jim

Sent via Deja.com http://www.deja.com/
Before you buy.

by **jbrick..** » Wed, 23 Aug 2000 04:00:00

Zlib API is designed to be a compression API, not a full package....
However a CRC is not that difficult to implement yourself, and I don't
know about you, **I** feel safer when I know what CRC was used. I
generally use two different CRC methods when I am truly worried about
data corruption. The standard CRC-32 polynomial you will find source
code for on the internet, plus a checksum-sum (16-bit check sum, and
16-bit sum of the sum). If you want to go REALLY far, you can apply a
32-bit CRC to the compressed data form to check BEFORE you de-compress
it and 32-bit CRC to check after, as much as you truly want to check,
but Zlib doesn't do it for you, you must do it yourself.
Just me,
jeff

> I'm in the midst's of writing a program that can be updated via the
> internet. The program will download gzip'd versions of shared
libraries
> it uses. I found 'zlib' and have been using the file operation
function
> (gzopen, gzread) to decompress the file.

> However; there doesn't appear to be any API in zlib to determine the
> original CRC of the '.gz' file. Additionally if the '.gz' file is
> corrupted performing a gzopen, gzread will not cause any problems (at
> least they return success values).

> The question is, while using zlib how can I determine if the '.gz'
file
> is not corrupted?

> Thanks
> -Jim

> Sent via Deja.com http://www.deja.com/
> Before you buy.

Sent via Deja.com http://www.deja.com/
Before you buy.

by **jim_marshall2..** » Thu, 24 Aug 2000 04:00:00

Jeff,
Thanks for the reply. yes I know I could do the CRC myself, but I'm
tryng to avoid adding that level of complexity to both my application
and the server app. The main issue is changing how I get the updated
file from the server (it will need to pass a CRC to me - which isn't the
end of the world but I know getting the plp writing the server to add
this will be a fight...).

Besides - the '.gz' file has a CRC within it. Why wouldn't 'libz' give
you access to that existing CRC value? 'gunzip' uses it, or uses some
information to verify the files integrity.

-Jim

> Zlib API is designed to be a compression API, not a full package....
> However a CRC is not that difficult to implement yourself, and I don't
> know about you, **I** feel safer when I know what CRC was used. I
> generally use two different CRC methods when I am truly worried about
> data corruption. The standard CRC-32 polynomial you will find source
> code for on the internet, plus a checksum-sum (16-bit check sum, and
> 16-bit sum of the sum). If you want to go REALLY far, you can apply a
> 32-bit CRC to the compressed data form to check BEFORE you de-compress
> it and 32-bit CRC to check after, as much as you truly want to check,
> but Zlib doesn't do it for you, you must do it yourself.
> Just me,
> jeff

> > I'm in the midst's of writing a program that can be updated via the
> > internet. The program will download gzip'd versions of shared
> libraries
> > it uses. I found 'zlib' and have been using the file operation
> function
> > (gzopen, gzread) to decompress the file.

> > However; there doesn't appear to be any API in zlib to determine the
> > original CRC of the '.gz' file. Additionally if the '.gz' file is
> > corrupted performing a gzopen, gzread will not cause any problems
(at
> > least they return success values).

> > The question is, while using zlib how can I determine if the '.gz'
> file
> > is not corrupted?

> > Thanks
> > -Jim

> > Sent via Deja.com http://www.deja.com/
> > Before you buy.

> Sent via Deja.com http://www.deja.com/
> Before you buy.

Sent via Deja.com http://www.deja.com/
Before you buy.

by **nom..** » Sun, 27 Aug 2000 04:00:00

according to the zlib spec (RFC1950), zlib data streams are supposed to
contain
an adler32 32-bit checksum....

i'm not sure if/how this is actually implemented, though.... try corrupting
a file on purpose, then see if the decompression process fails or if it
works anyway.......

the decompress functions will _probably_ return with some kind of error
code......?

by **jim_marshall2..** » Tue, 29 Aug 2000 04:00:00

There are no errors returned from the zlib functions if the file is
corrupted. Specifically I am using the gzopen, gzread, gzclose
functions. zlib does provide a mechanism to calculate a CRC on data
that has been read in (gzread) but it doesn't have a way to know the
original CRC, so I can't see a way to check the computed CRC.....

-Jim

> according to the zlib spec (RFC1950), zlib data streams are supposed
to
> contain
> an adler32 32-bit checksum....

> i'm not sure if/how this is actually implemented, though.... try
corrupting
> a file on purpose, then see if the decompression process fails or if
it
> works anyway.......

> the decompress functions will _probably_ return with some kind of
error
> code......?

Sent via Deja.com http://www.deja.com/
Before you buy.

by **nrav** » Thu, 31 Aug 2000 04:00:00

so gzread and gzclose never return with the value -1?

if they do, you could use gzerror to get the type of error....
_____

.....have a look at
http://www.info-zip.org/pub/infozip/zlib/manual.html#Introduction

- "The 'zlib' compression library provides in-memory compression and
decompression functions, __including integrity checks of the uncompressed
data.__ "

- "__The decoder checks the consistency of the compressed data__, so the
library should never crash even in case of corrupted input."

> There are no errors returned from the zlib functions if the file is
> corrupted. Specifically I am using the gzopen, gzread, gzclose
> functions. zlib does provide a mechanism to calculate a CRC on data
> that has been read in (gzread) but it doesn't have a way to know the
> original CRC, so I can't see a way to check the computed CRC.....

> -Jim

> > according to the zlib spec (RFC1950), zlib data streams are supposed
> to
> > contain
> > an adler32 32-bit checksum....

> > i'm not sure if/how this is actually implemented, though.... try
> corrupting
> > a file on purpose, then see if the decompression process fails or if
> it
> > works anyway.......

> > the decompress functions will _probably_ return with some kind of
> error
> > code......?

> Sent via Deja.com http://www.deja.com/
> Before you buy.

by **jim_marshall2..** » Thu, 31 Aug 2000 04:00:00

Not to keep this thread going for ever, but...

I believe the quote you mention is referring to the compression methods
in the library, not the file access methods.

regardless - the gzread, gzopen & gzclose functions do not seem to
return an error condition if the file is invalid.

Specifically the test I did (and it was an unintentional test) was the
HTTP server failed to send the file, so it created a 0 length file. When
I did a gzopen, gzread, gzclose I got no error messages.

I also took a gz file and cut out a 1/4 of the file (in the middle),
again no errors came from these functions.

-Jim

> so gzread and gzclose never return with the value -1?

> if they do, you could use gzerror to get the type of error....
> _____

> .....have a look at
> http://www.info-zip.org/pub/infozip/zlib/manual.html#Introduction

> - "The 'zlib' compression library provides in-memory compression and
> decompression functions, __including integrity checks of the
uncompressed
> data.__ "

> - "__The decoder checks the consistency of the compressed data__, so
the
> library should never crash even in case of corrupted input."

> > There are no errors returned from the zlib functions if the file is
> > corrupted. Specifically I am using the gzopen, gzread, gzclose
> > functions. zlib does provide a mechanism to calculate a CRC on data
> > that has been read in (gzread) but it doesn't have a way to know the
> > original CRC, so I can't see a way to check the computed CRC.....

> > -Jim

> > > according to the zlib spec (RFC1950), zlib data streams are
supposed
> > to
> > > contain
> > > an adler32 32-bit checksum....

> > > i'm not sure if/how this is actually implemented, though.... try
> > corrupting
> > > a file on purpose, then see if the decompression process fails or
if
> > it
> > > works anyway.......

> > > the decompress functions will _probably_ return with some kind of
> > error
> > > code......?

> > Sent via Deja.com http://www.deja.com/
> > Before you buy.

Sent via Deja.com http://www.deja.com/
Before you buy.

by **jim_marshall2..** » Thu, 31 Aug 2000 04:00:00

In case anyone cares, I figured out the problem. I had a loop which did
the gzread, I was only checking if gzread returned a negative number.
However; in the case of a zero length file, gzread will return zero.
Once I changed the code to detect the case of zero bytes read all was
well.

> I'm in the midst's of writing a program that can be updated via the
> internet. The program will download gzip'd versions of shared
libraries
> it uses. I found 'zlib' and have been using the file operation
function
> (gzopen, gzread) to decompress the file.

> However; there doesn't appear to be any API in zlib to determine the
> original CRC of the '.gz' file. Additionally if the '.gz' file is
> corrupted performing a gzopen, gzread will not cause any problems (at
> least they return success values).

> The question is, while using zlib how can I determine if the '.gz'
file
> is not corrupted?

> Thanks
> -Jim

> Sent via Deja.com http://www.deja.com/
> Before you buy.

Sent via Deja.com http://www.deja.com/
Before you buy.

Very Computer

zlib and checking CRC - how?

zlib and checking CRC - how?

zlib and checking CRC - how?

zlib and checking CRC - how?

zlib and checking CRC - how?

zlib and checking CRC - how?

zlib and checking CRC - how?

zlib and checking CRC - how?

zlib and checking CRC - how?