problem with named piped giving no performance increase

problem with named piped giving no performance increase

Post by dave » Fri, 03 Aug 2001 06:44:12



we have a shell script that does 2 processes, a cut of a file and then a
execution of loader program.

With out a named pipe process takes ~20 minutes, the cut runs first then the
loading program starts after oit finishes

we create a named pipe and then have the cut program feed the loader program
we note that the loader program starts up right away.  hoever the total time
is still ~20 are we doing anything worng?

rm -f sanjeev.dat
mknod sanjeev.dat p
cut -c1-10 ../josejn/F_*2.dat >  sanjeev.dat &
sqlldr mdwbig/mdwbig control=sanctl.ctl log=sanjeev.log
date  

 
 
 

problem with named piped giving no performance increase

Post by Christoph Hintermülle » Fri, 03 Aug 2001 18:00:07


Hi
Depends how the cut and the loader are realised.
If the cut does the whole cut before it produces  any output on the named pipe
the loader has to wait for the cut to finish to get some Input
If the loader starts to process the input after the input is complete than
you will not get much difference either as whole behaves like the first case.

So change the cuter so that it places any line not to be cut onto its output,
when it occures. And change the loader so that it is capable to start processing
on incomplete input data and stopping when EOF arrives.

If this changes are not possible than you have at least the advantage that
you do not need to errase any tempory file from harddisk.

cu
Christoph


> we have a shell script that does 2 processes, a cut of a file and then a
> execution of loader program.

> With out a named pipe process takes ~20 minutes, the cut runs first then the
> loading program starts after oit finishes

> we create a named pipe and then have the cut program feed the loader program
> we note that the loader program starts up right away.  hoever the total time
> is still ~20 are we doing anything worng?

> rm -f sanjeev.dat
> mknod sanjeev.dat p
> cut -c1-10 ../josejn/F_*2.dat >  sanjeev.dat &
> sqlldr mdwbig/mdwbig control=sanctl.ctl log=sanjeev.log
> date  

--
THESIS:     God is alive
PROOVE:     Who else would have scheduled the mankind and world first
             recommendation of research????
CONCLUSION: Scientists do what he wants, willing or not:)