Very Computer

by **Neo Brav** » Mon, 03 Jan 2000 04:00:00

Hi all,

I have a bit of a dilemma I'm hoping someone might be able
to shed some light on.

I'm working on some software which uses fopen(), fread(),
fseek(), and fclose() to operate on data in a file. This
is working for files on disk but I would like to extend
this to work on stdin as well. At the moment I'm doing
this by by setting the FILE* pointer to stdin and it works
on one case but not on another.

Case 1:
myprog <file - works fine

Case 2:
cat file |myprog - fseek() fails

My guess is that the "<" file redirection actually causes
the shell to open the disk file and connects it to the
program while "|" pipe command creates a temporary buffer
which feeds stdin of my program. Obviously, the temporary
buffer won't support fseek().

So, my question is, what's the best way of getting my program
to work correctly with piped input? I have thought of using a
temporary file but I'm hoping for a neater solution.

Thanks in advance,
Neo

PS: I'm working on Linux with gcc but the behaviour is the
same on Sparc Solaris.

PPS: Please post the answer to the group rather than emailing me.

--
Neo Brave
Going where no-one has gone before!

Sent via Deja.com http://www.deja.com/
Before you buy.

by **Andrew Giert** » Mon, 03 Jan 2000 04:00:00

Neo> Case 1:
Neo> myprog <file - works fine

Neo> Case 2:
Neo> cat file |myprog - fseek() fails

Neo> My guess is that the "<" file redirection actually causes the
Neo> shell to open the disk file and connects it to the program

correct

Neo> while "|" pipe command creates a temporary buffer which feeds
Neo> stdin of my program.

to be precise, what it creates is a pipe.

Neo> Obviously, the temporary buffer won't support fseek().

correct

Neo> So, my question is, what's the best way of getting my program to
Neo> work correctly with piped input?

don't use fseek.

it's hard to be more specific without knowing more about what your
program is doing....

--
Andrew.

comp.unix.programmer FAQ: see <URL: http://www.erlenstar.demon.co.uk/unix/>
or <URL: http://www.whitefang.com/unix/>

by **Russ Allber** » Mon, 03 Jan 2000 04:00:00

>> it's hard to be more specific without knowing more about what your
>> program is doing....
> Its reading from a binary file. The fseek()s usually don't move much
> more than 1024 bytes from the current position. Is there some (portable)
> way of making the pipe fseek()able over a small range?

The easy way is to use a temporary file. A harder but nicer way would be
to buffer the data you read into a buffer large enough to do the seeks
that you want to be able to do by looking back in the buffer.

--

by **Neo Brav** » Tue, 04 Jan 2000 04:00:00

> Neo> Case 1:
> Neo> myprog <file - works fine

> Neo> Case 2:
> Neo> cat file |myprog - fseek() fails

> Neo> My guess is that the "<" file redirection actually causes the
> Neo> shell to open the disk file and connects it to the program

> correct

> Neo> while "|" pipe command creates a temporary buffer which feeds
> Neo> stdin of my program.

> to be precise, what it creates is a pipe.

> Neo> Obviously, the temporary buffer won't support fseek().

> correct

> Neo> So, my question is, what's the best way of getting my program to
> Neo> work correctly with piped input?

> don't use fseek.

Well thats the problem, I do need to use fseek() although not
necessarily over the full contents of the file.

Quote:> it's hard to be more specific without knowing more about what your
> program is doing....

Its reading from a binary file. The fseek()s usually don't move
much more than 1024 bytes from the current position. Is there some
(portable) way of making the pipe fseek()able over a small range?

Thanks,
Neo
--
Neo Brave
Going where no-one has gone before!

Sent via Deja.com http://www.deja.com/
Before you buy.

by **Neo Brav** » Tue, 04 Jan 2000 04:00:00

> Neo> Case 1:
> Neo> myprog <file - works fine

> Neo> Case 2:
> Neo> cat file |myprog - fseek() fails

> Neo> My guess is that the "<" file redirection actually causes the
> Neo> shell to open the disk file and connects it to the program

> correct

> Neo> while "|" pipe command creates a temporary buffer which feeds
> Neo> stdin of my program.

> to be precise, what it creates is a pipe.

> Neo> Obviously, the temporary buffer won't support fseek().

> correct

> Neo> So, my question is, what's the best way of getting my program to
> Neo> work correctly with piped input?

> don't use fseek.

Well thats the problem, I do need to use fseek() although not
necessarily over the full contents of the file.

Quote:> it's hard to be more specific without knowing more about what your
> program is doing....

Its reading from a binary file. The fseek()s usually don't move
much more than 1024 bytes from the current position. Is there some
(portable) way of making the pipe fseek()able over a small range?

Thanks,
Neo
--
Neo Brave
Going where no-one has gone before!

Sent via Deja.com http://www.deja.com/
Before you buy.

by **David Schwart** » Tue, 04 Jan 2000 04:00:00

> >> it's hard to be more specific without knowing more about what your
> >> program is doing....

> > Its reading from a binary file. The fseek()s usually don't move much
> > more than 1024 bytes from the current position. Is there some (portable)
> > way of making the pipe fseek()able over a small range?

> The easy way is to use a temporary file. A harder but nicer way would be
> to buffer the data you read into a buffer large enough to do the seeks
> that you want to be able to do by looking back in the buffer.

Note that this is exactly what the operating system or library would
have to do if it had some magic way to make any arbitrary stream
seekable. So, just do it.

DS

by **Andrew Gabri** » Tue, 04 Jan 2000 04:00:00

>>> it's hard to be more specific without knowing more about what your
>>> program is doing....

>> Its reading from a binary file. The fseek()s usually don't move much
>> more than 1024 bytes from the current position. Is there some (portable)
>> way of making the pipe fseek()able over a small range?

>The easy way is to use a temporary file. A harder but nicer way would be
>to buffer the data you read into a buffer large enough to do the seeks
>that you want to be able to do by looking back in the buffer.

If your program happens to only ever seek forwards, you can or course
simulate this without a buffer, just by keeping a note of the byte
offset into the datastream, and reading and skipping data up to the
offset you were intending to seek to.

One other thing to be aware of - if a program was originally intended
to read from a file, it might not correctly cope with short reads
which it might sometimes get when reading from a pipe.

--
Andrew Gabriel
Consultant Software Engineer

by **Neo Brav** » Wed, 05 Jan 2000 04:00:00

<snip>

Quote:> One other thing to be aware of - if a program was originally intended
> to read from a file, it might not correctly cope with short reads
> which it might sometimes get when reading from a pipe.

Correct me if I'm wrong, but shouldn't that be taken care of
by using the buffered fread() function rather than the unbuffered
read() function?

The fread() function should only return a short read if the end of
file has been reached or an error has occurred.

Neo
--
Neo Brave
Going where noone has gone before!

Sent via Deja.com http://www.deja.com/
Before you buy.

Very Computer

Q: Input piped via stdin and fseek() problem

Q: Input piped via stdin and fseek() problem

Q: Input piped via stdin and fseek() problem

Q: Input piped via stdin and fseek() problem

Q: Input piped via stdin and fseek() problem

Q: Input piped via stdin and fseek() problem

Q: Input piped via stdin and fseek() problem

Q: Input piped via stdin and fseek() problem

Q: Input piped via stdin and fseek() problem