Compressing a directory structure with gzip

Compressing a directory structure with gzip

Post by Adam Coute » Fri, 21 Sep 2001 02:05:43



Hello,

I've been trying to compress a large directory structure using gzip, and
have tried the following commands:

gzip -r directory_name > zipped_file.gz
gzip -rc directory_name > zipped_file.gz

Neither one of these seems to work. I believe the problem is that there are
symbolic links within the directory tree that I am trying to compress that
link to other directories in the same tree. The program seems to get stuck
with this. Is there a flag for gzip that makes it ignore symbolic links? If
not, how else would I zip a directory tree like this.

Thanks in advance for the help,
Adam

 
 
 

Compressing a directory structure with gzip

Post by John Gord » Fri, 21 Sep 2001 02:41:08



> Hello,
> I've been trying to compress a large directory structure using gzip, and
> have tried the following commands:
> gzip -r directory_name > zipped_file.gz
> gzip -rc directory_name > zipped_file.gz

why are you redirecting the output to zipped_file.gz?  that file is
created by gzip itself.

Quote:> Neither one of these seems to work. I believe the problem is that there are

when you say it doesn't work, what do you mean?  the zipfile doesn't
get created?  it gets created, but is unreadable?  the contents aren't
as you would like?  please be more specific.

---
"... What with you being his parents and all, I think that you could
be trusted not to shaft him."  -- Robert Chang, rec.games.board



 
 
 

Compressing a directory structure with gzip

Post by John Doher » Fri, 21 Sep 2001 03:41:04



> I've been trying to compress a large directory structure using gzip, and
> have tried the following commands:

> gzip -r directory_name > zipped_file.gz
> gzip -rc directory_name > zipped_file.gz

> Neither one of these seems to work. I believe the problem is that there are
> symbolic links within the directory tree that I am trying to compress that
> link to other directories in the same tree. The program seems to get stuck
> with this. Is there a flag for gzip that makes it ignore symbolic links? If
> not, how else would I zip a directory tree like this.

I'm not sure that you've identified your problem correctly, but FWIW,
the gzip man page is rather explicit:

   Gzip will only attempt to compress regular files. In particular,
   it will ignore symbolic links.

Also, the resursive option of gzip does not create a single zip file
that contains the entire directory tree. Rather, it compresses each
file in the directory tree, like so:

  $ ls -l foo
  total 135
  -rw-r--r--  1 bozo clowns  53660 Feb 10 23:24 bar
  -rw-r--r--  1 bozo clowns  83089 Feb 10 23:24 baz
  $ gzip -r foo
  $ ls -l foo
  total 121
  -rw-r--r--  1 bozo clowns  47272 Feb 10 23:24 bar.gz
  -rw-r--r--  1 bozo clowns  75610 Feb 10 23:24 baz.gz
  $

Is that what you want? Maybe what you really want is something
like this:

$ tar cf - foo | gzip > foo.tar.gz

tar collects the directory tree into a single tar-format stream that
is written to stdout and then passed through gzip to the file named
foo.tar.gz. Is that what you want?

--

 
 
 

Compressing a directory structure with gzip

Post by Ryan Henness » Fri, 21 Sep 2001 04:06:05



> Hello,

> I've been trying to compress a large directory structure using gzip, and
> have tried the following commands:

> gzip -r directory_name > zipped_file.gz
> gzip -rc directory_name > zipped_file.gz

> Neither one of these seems to work. I believe the problem is that there are
> symbolic links within the directory tree that I am trying to compress that
> link to other directories in the same tree. The program seems to get stuck
> with this. Is there a flag for gzip that makes it ignore symbolic links? If
> not, how else would I zip a directory tree like this.

> Thanks in advance for the help,
> Adam

Okay, first of all, gzip ignores symbolic links all by itself, so you have no
worries there.

Second, gzip does not write to stdout by default, so

gzip -r directory_name > zipped_file.gz

does absolutely nothing.  If you want to write to stdout, you want "gzip -c"

Third, gzip -r individually zips every file it acts on into its own .gz

$ mkdir /test
$ cd /test
$ touch a b c d
$ ls
 a b c d
$ gzip -r /test
$ ls
 a.gz b.gz c.gz d.gz

I don't know if this is the behavior you want...  If you want one big .gz file
with everything in it, either cat the files together first:

$ cat a b c d | gzip -c > bigfile.gz

Or, (this is way better), use tar to collect all your files and directory
structure, uncompressed, into a single .tar file.  Then you gzip that.  This
is what is known as a tarball, and is a typical method of distributing unix
software.

Here we go:

$ tar -cf - (directory to compress) | gzip -f  > (filename.tar.gz)

Oh, and btw, tar also ignores symlinks, so no worries there.

Hope this helps,
Ryan.

 
 
 

Compressing a directory structure with gzip

Post by those who know me have no need of my nam » Sat, 22 Sep 2001 00:33:15



Quote:>I've been trying to compress a large directory structure using gzip, and
>have tried the following commands:

>gzip -r directory_name > zipped_file.gz
>gzip -rc directory_name > zipped_file.gz

i'll take a different tack from the other responses.  it seems to me that
you might be confusing gzip with zip.  if this is not the case you may as
well skip my article.

zip combines archiving and compression.  gzip only compresses.  so, if you
want archiving and compression you can use gzip along with an archival
tool, e.g., cpio or tar.

the following are equivalent, each directory and file in `directory_name'
or below is copied into an archive, and the archive features compression so
that it is (probably) smaller than the sum of the file sizes.  however, the
archives are completely different, and cannot (generally) be interchanged,
but their purpose and abilities are the same.

  zip -r zipped_file.zip directory_name

  tar -cpvf - directory_name | gzip > gzipped_file.tar.gz

  find directory_name | cpio -ov | gzip > gzipped_file.cpio.gz

a quick example ...

| $ mkdir foo
| $ touch foo/a
| $ touch foo/b
| $ mkdir foo/c
| $ touch foo/c/d
| $ touch foo/c/e
| $ zip -r foo.zip foo
|   adding: foo/ (stored 0%)
|   adding: foo/a (stored 0%)
|   adding: foo/b (stored 0%)
|   adding: foo/c/ (stored 0%)
|   adding: foo/c/d (stored 0%)
|   adding: foo/c/e (stored 0%)
| $ tar -cpvf - foo | gzip >foo.tar.gz
| foo/
| foo/a
| foo/b
| foo/c/
| foo/c/d
| foo/c/e
| $ find foo | cpio -ov | gzip > foo.cpio.gz
| foo
| foo/a
| foo/b
| foo/c
| foo/c/d
| foo/c/e
| 1 block
| $ unzip -v foo.zip
| Archive:  foo.zip
|  Length   Method    Size  Ratio   Date   Time   CRC-32    Name
| --------  ------  ------- -----   ----   ----   ------    ----
|        0  Stored        0   0%  09-20-01 14:56  00000000  foo/
|        0  Stored        0   0%  09-20-01 14:56  00000000  foo/a
|        0  Stored        0   0%  09-20-01 14:56  00000000  foo/b
|        0  Stored        0   0%  09-20-01 14:56  00000000  foo/c/
|        0  Stored        0   0%  09-20-01 14:56  00000000  foo/c/d
|        0  Stored        0   0%  09-20-01 14:56  00000000  foo/c/e
| --------          -------  ---                            -------
|        0                0   0%                            6 files
| $ gunzip -c foo.tar.gz | tar -tvf -
| drwx------ twkmhnno/twkmhnno 0 2001-09-20 14:56:19 foo/
| -rw------- twkmhnno/twkmhnno 0 2001-09-20 14:56:15 foo/a
| -rw------- twkmhnno/twkmhnno 0 2001-09-20 14:56:17 foo/b
| drwx------ twkmhnno/twkmhnno 0 2001-09-20 14:56:27 foo/c/
| -rw------- twkmhnno/twkmhnno 0 2001-09-20 14:56:26 foo/c/d
| -rw------- twkmhnno/twkmhnno 0 2001-09-20 14:56:27 foo/c/e
| $ gunzip -c foo.cpio.gz | cpio -itv
| drwx------ twkmhnno/twkmhnno 0 2001-09-20 14:56:19 foo
| -rw------- twkmhnno/twkmhnno 0 2001-09-20 14:56:15 foo/a
| -rw------- twkmhnno/twkmhnno 0 2001-09-20 14:56:17 foo/b
| drwx------ twkmhnno/twkmhnno 0 2001-09-20 14:56:27 foo/c
| -rw------- twkmhnno/twkmhnno 0 2001-09-20 14:56:26 foo/c/d
| -rw------- twkmhnno/twkmhnno 0 2001-09-20 14:56:27 foo/c/e
| 1 block

another example showing compression (using the contents of my $TMPDIR) but
with verbosity turned down ...

| $ du -s tmp
| 2016    tmp
| $ find tmp | wc -l
|     219
| $ zip -qr foo.zip tmp
| $ tar -cpf - tmp | gzip > foo.tar.gz
| $ find tmp | cpio -o | gzip > foo.cpio.gz
| 3742 blocks
| $ ls -s foo.*
| 1137 foo.cpio.gz
| 1141 foo.tar.gz
| 1224 foo.zip

--
okay, have a sig then

 
 
 

Compressing a directory structure with gzip

Post by Adam Coute » Sun, 23 Sep 2001 02:13:22


I want to thank everyone for their help. With your help, I learned quite a
bit and was able to complete my task (tar-ing and the gzip-ing a large
directory tree for transport to another machine). Thanks again.

Adam


Quote:> Hello,

> I've been trying to compress a large directory structure using gzip, and
> have tried the following commands:

> gzip -r directory_name > zipped_file.gz
> gzip -rc directory_name > zipped_file.gz

> Neither one of these seems to work. I believe the problem is that there
are
> symbolic links within the directory tree that I am trying to compress that
> link to other directories in the same tree. The program seems to get stuck
> with this. Is there a flag for gzip that makes it ignore symbolic links?
If
> not, how else would I zip a directory tree like this.

> Thanks in advance for the help,
> Adam

 
 
 

Compressing a directory structure with gzip

Post by Adam Coute » Sun, 23 Sep 2001 02:26:42


Come to think of it, I do have one other question. After I tar-ed the
directory, including all of the files and directories beneath it, I
transported the tar-ed file to a Windows based machine for the purpose of
backing up the data. Then, I used WinZip to extract all of the data into its
original directory structure. However, some files seemed not to know what
directory they belonged in and were just placed in the base directory. Why
does this happen? ...only to some files? Is that a limitation of tar, that
sometimes with larges jobs it produces errors lik that? My largest directory
that I tar-ed was about 1GB. The command line I used to tar the files
together was:

tar -cvf dir_name.tar dir_name        (from within the directory dir_name)

Anyways, thanks again for the help.

Adam


Quote:> Hello,

> I've been trying to compress a large directory structure using gzip, and
> have tried the following commands:

> gzip -r directory_name > zipped_file.gz
> gzip -rc directory_name > zipped_file.gz

> Neither one of these seems to work. I believe the problem is that there
are
> symbolic links within the directory tree that I am trying to compress that
> link to other directories in the same tree. The program seems to get stuck
> with this. Is there a flag for gzip that makes it ignore symbolic links?
If
> not, how else would I zip a directory tree like this.

> Thanks in advance for the help,
> Adam

 
 
 

Compressing a directory structure with gzip

Post by those who know me have no need of my nam » Sun, 23 Sep 2001 09:33:21



Quote:>some files seemed not to know what
>directory they belonged in and were just placed in the base directory. Why
>does this happen? ...only to some files?

if there is no path given to tar then none will be stored in the archive,
e.g., ``tar -cpf /tmp/foo.far .'' will only record `./' as the directory so
wherever you are when you extract the tar is where those files will be
placed.

inspect the output of ``tar tvf dir_name.tar''.

Quote:>tar -cvf dir_name.tar dir_name        (from within the directory dir_name)

don't do that.  many tar's don't know how to avoid tar'ing themselves.  i
should have made that clear in my examples.

--
okay, have a sig then

 
 
 

1. copying a directory structure with a particular file in directories below

Hello,
I need some help/advise on how to copy a directory structure with a
particular file in directories below the start directory.

Let's say I have many text files [*.txt] with many data files and other
files [*.data, *.*]
The directory I want to copy across to another area with only the text
files but which also retains the directory structure.

i.e.,

[root_directory]
 |__[data]
       |__[directory_1]
       |__[direcotry_2]
       |    |__[directory_2.1]
       |
       |__[directory_3]
       |
       |__[directory_N]

I want to copy the directory [data] to another area with just the *.txt
files but retain the directory structure.

I tried to do this

cp -r `find /[root_directory]/[data] -name "*.txt"` .

NOTE: The above when I include the square brackets is meant to
represent the full name of the
directory and is not to be confused with anything else.

cp -r `find /root_directory/data/ -name "*.txt"` .

When doing this, all I get is the *.txt files without the directory
structure. If the *.txt files have the same name it will cause
problems. So I can copy the directory structure it would be great.

I hope you can understand the issue/problem statement from my
explanation.
If you have any questions please ask me.

Thanks,
ROuNIN

2. Setting up PPP Server Problem

3. gzip compresses its own binary when running from the ksh scripts

4. kde 3.02 and codeweaver wine

5. gzip/compress compression routine/library want

6. Lilo [Q] PLEASE!!!!

7. gzip vs. winzip/pkzip vs. compress

8. Fdisk on new harddisk

9. Q: Compressing backup using dump and gzip

10. gzip and compressed tar

11. gzip / zip / compress : 2 gig limit?

12. Auto-compressing all output when browser says "Accept-Encoding: gzip": Possible?

13. gzip, compress, and others???