writing to pipe

writing to pipe

Post by Manja Fieber » Sat, 13 Nov 1999 04:00:00



Hi,

I'd like to decompress 3 files, concatenate them and pipe the output in
another process without storing intermediate files. The 3 compressed
files are about 10 GB, the decompressed ones 30-40 GB. Does something
like

mknod /dev/hugo p
uncompress file1.Z > /dev/hugo
uncompress file2.Z > /dev/hugo
uncompress file3.Z > /dev/hugo

work when I start a reading process like
 cp /dev/hugo file &
in background?

The question is, if the pipe is open the whole time and what happens, if
the read from the pipe is very slow.
Does anyone have a suggestion?
Thanks,
Manja Fieberg

 
 
 

writing to pipe

Post by Martin Shiel » Sat, 13 Nov 1999 04:00:00



> Does something
> like

> mknod /dev/hugo p
> uncompress file1.Z > /dev/hugo
> uncompress file2.Z > /dev/hugo
> uncompress file3.Z > /dev/hugo

> work when I start a reading process like
>  cp /dev/hugo file &
> in background?

> The question is, if the pipe is open the whole time and what happens, if
> the read from the pipe is very slow.

Since each of "uncompress file1.Z > /dev/hugo" ends with EOF, the
background cp process will finish before the second uncompress.

Why not just do:

zcat file1.Z file2.Z file3.Z > file

???

Martin Shields.

 
 
 

1. Writing to Pipe

Assuming a pipe has a buffer of 4096 bytes,
what happens when using the write system call
on the pipe with more bytes? Does write simply
return the value 4096, being the maximum
number of bytes writable to the pipe?

Marc

2. linux client crash my NFS server ?

3. Write to pipe : Bad file number

4. probs with Snort and Acid

5. writing to pipe

6. evm high event activity

7. Problem on writing named pipes - SOLARIS/SUN OS System

8. CD backup tools with soft RAID

9. writing a pipe to tape

10. how to write a pipe program ?

11. atomic writes on pipes and sockets

12. select(), non blocking writes to pipes and EAGAIN

13. dump pipe gzip pipe ssh pipe dd... blocksize?