I am trying to scp and compress on the fly:
#!/bin/ksh
compress<temp.dmp>file.Z &
rcp host:directory/file . > temp.dmp
This doesn't do the trick.
by rich for » Fri, 23 Feb 2001 14:47:50
#!/bin/ksh
compress<temp.dmp>file.Z &
rcp host:directory/file . > temp.dmp
This doesn't do the trick.
by Logan Sh » Fri, 23 Feb 2001 16:10:49
This doesn't work because the compress sees end of file whenever itQuote:>I am trying to scp and compress on the fly:
>#!/bin/ksh
>compress<temp.dmp>file.Z &
>rcp host:directory/file . > temp.dmp
>This doesn't do the trick.
The solution is to use a pipe:
#! /bin/sh
ssh host cat directory/file | compress > file.Z
This uses ssh instead of scp, but that should be O.K.
in most cases. If you want to use scp, I'm not sure if
it has the ability to send its output to its standard
output, so you might have to trick it, like this:
#! /bin/sh
scp host:directory/file /dev/fd/1 | compress > file.Z
/dev/fd/1 is, for any program, file #1 that *that* program
has open, so /dev/fd/1 is usually standard output.
Note that I've used "/bin/sh" and not "/bin/ksh" because
nothing that either you or I did requires ksh-specific
features, so you might as well make it portable. (Assuming
you care, which you might not, but it is good practice.)
- Logan
--
my your his her our their *its*
I'm you're he's she's we're they're *it's*
by Philip Bro » Mon, 12 Mar 2001 07:15:51
it would be more efficient on the network to doQuote:>The solution is to use a pipe:
> #! /bin/sh
> ssh host cat directory/file | compress > file.Z
ssh host compress -c directory/file >file.Z
[note that the ">" gets interpreted by the LOCAL shell, unless you quote
it to send to the remote machine]
--
[Trim the no-bots from my address to reply to me by email!]
[ Do NOT email-CC me on posts. Pick one or the other.]
The word of the day is mispergitude
1. compressing on the fly: father process seems to end before popened compress
Hi all.
Got this weird problem. Program A starts a child process (no forking)
with popen to compress data written to disk on the fly. This seems to
work ok. The thing is that immediately after program A terminates, an
invoking script moves the new file to another location. It turns out
that the file gets truncated after it is moved. I get the sense that
compress didn't finish before the file was moved. I'm doing a pclose
on the pipe of course so this is really strange. I'm flushing every
write too, but to no effect.
The strangest thing is that this doesn't happen always. Sometimes
moved files are chopped sometimes they arent. Probably due to system
load. If I dont call mv in the script the whole process works just
fine, files are always created ok. Also if I introduce a "sleep 1"
before de mv in the script, no files get truncated. Of course I dont
like this workaround. Would rather have prog_a stop execution till
compress is completely finished. But I thought pclose would arrange
for that.
Any ideas of what can be happening? Any of you guys ever walked into a
similar behavior? Im working on Digital Unix if that matters for
anything.
This is an excerpt of my code:
*** script.sh ***
prog_a newfile.Z
mv newfile.Z /another_fs/another_dir/newfile.Z
*** progr_a ***
fout = popen("/usr/bin/compress - > newfile.Z", "w");
if (fout==NULL){ perror("popen didn't work"); exit(1); }
/* Write data and flush every time */
while(...){
fprintf(fout, ...);
fflush(fout);
if( pclose(fout)!= 0){
/* never gets in here so pclose works fine */
perror("oops!");
Thanks in advance for any help you can provide.
Fernando Hevia
Telecom Personal
Cordoba, Argentina
2. Bug in NIS/SENDMAIL/Domains
4. "host to network long long"
7. Recording radio and mpeg compressing on the fly?
8. poppassd
9. GNU tar compress on the fly?
10. Compressing logfiles on the fly?
11. trying to compress sybase dump on-the-fly
12. on-the-fly file-compressing?
13. compress and ftp on the fly