I have a question about performance and implementation about File I/O vs.
Right now I have a program of A which produces output file A.output. I have
a program B reading A.output to do some postprocessing. So the flow looks like
A --> A.output --> B
Since A.output if very large ( ~100MB ), it takes a long time to write and
read the file. I am looking into the possibility to set up a pipe between A
and B to transfer the data. I guess I can use either socket or pipe to do
it. Anyone having the experience how much performance will I gain and which
way is easier to implement( socket or stream ) ?
Now suppose A is a lot faster than B, I think it will result in a lot of
data storing in an internal buffer. Will it cause some memory problem?
Thank you very much for your answer.