select() and write() behavior with non-blocking named-pipe

select() and write() behavior with non-blocking named-pipe

Post by Roger Hosie » Wed, 04 Jul 2001 10:46:42



I've got a process that sets a named-pipe to be non-blocking. The first
time that the process
enters a situation where the write() would block it adds the fd into the
select() calls set
for writers. The expectation was that the select() would indicate that
the fd was ready for
writing and the write() would be accomplished. On an HP/UX box we have a
situation
where the process that is doing this logic starts to take a large
portion of the CPU -- just
as if it was polling the fd. I used the HP/UX tusc/truss command and it
shows that what is
happening is that the select() returns a 1 (with the indication that the
fd that is being tested
for writing is set), but the write() returns a 0 and nothing is written.
I have always been
under the impression that write() would return a non-zero value and that
select() would not
return the fd for writing if it wasn't really ready for writing.

What is the problem with the logic in this case?

Thanks for the help!

Roger

 
 
 

select() and write() behavior with non-blocking named-pipe

Post by Nithyanandha » Wed, 04 Jul 2001 11:50:35



> I've got a process that sets a named-pipe to be non-blocking. The first
> time that the process
> enters a situation where the write() would block it adds the fd into the
> select() calls set
> for writers. The expectation was that the select() would indicate that
> the fd was ready for
> writing and the write() would be accomplished. On an HP/UX box we have a
> situation
> where the process that is doing this logic starts to take a large
> portion of the CPU -- just
> as if it was polling the fd. I used the HP/UX tusc/truss command and it
> shows that what is
> happening is that the select() returns a 1 (with the indication that the
> fd that is being tested
> for writing is set), but the write() returns a 0 and nothing is written.
> I have always been
> under the impression that write() would return a non-zero value and that
> select() would not
> return the fd for writing if it wasn't really ready for writing.

> What is the problem with the logic in this case?

socket is readable even when

Read p154,"Unix Network Programming"-Vol 1, W.Richard Stevens

When write return 0 , it means the kernel buffers for the socket are full.

Use setsockopt() to change these socket options.

--

Nithyanand.
Siemens, Bangalore, India.
(Opinions expressed are my own and do not reflect the opinions of my
employer, Siemens)

 
 
 

select() and write() behavior with non-blocking named-pipe

Post by Nithyanandha » Wed, 04 Jul 2001 11:52:10



> > enters a situation where the write() would block it adds the fd into the
> > select() calls set
> > for writers. The expectation was that the select() would indicate that
> > the fd was ready for
> > writing and the write() would be accomplished. On an HP/UX box we have a
> > situation
> > where the process that is doing this logic starts to take a large
> > portion of the CPU -- just
> > as if it was polling the fd. I used the HP/UX tusc/truss command and it
> > shows that what is
> > happening is that the select() returns a 1 (with the indication that the
> > fd that is being tested
> > for writing is set), but the write() returns a 0 and nothing is written.
> > I have always been
> > under the impression that write() would return a non-zero value and that
> > select() would not
> > return the fd for writing if it wasn't really ready for writing.

> > What is the problem with the logic in this case?

> socket is readable

Oops...socket is ready for write()-ing

Quote:> even when
> Read p154,"Unix Network Programming"-Vol 1, W.Richard Stevens
> When write return 0 , it means the kernel buffers for the socket are
> full.Use setsockopt() to change these socket options.

Nithyanand.
Siemens, Bangalore, India.
(Opinions expressed are my own and do not reflect the opinions of my employer,
Siemens)
 
 
 

select() and write() behavior with non-blocking named-pipe

Post by Andrew Giert » Wed, 04 Jul 2001 18:07:28


 Roger> but the write() returns a 0 and nothing is written.

are you sure? (write should not return 0)

make sure you used O_NONBLOCK and not O_NDELAY to set nonblocking
mode. Note also that select will return that the pipe is writable if
_anything_ can be written to it; it doesn't mean that an arbitrarily
large write will succeed.

--
Andrew.

comp.unix.programmer FAQ: see <URL: http://www.erlenstar.demon.co.uk/unix/>
                           or <URL: http://www.whitefang.com/unix/>

 
 
 

select() and write() behavior with non-blocking named-pipe

Post by Nithyanandha » Wed, 04 Jul 2001 23:34:55



> > select() returns a 1 (with the indication that the
> > fd that is being tested
> > for writing is set), but the write() returns a 0 and nothing is written.
> > I have always been
> > under the impression that write() would return a non-zero value and that
> > select() would not
> > return the fd for writing if it wasn't really ready for writing.

> > What is the problem with the logic in this case?

> socket is write()-able even when

> Read p154,"Unix Network Programming"-Vol 1, W.Richard Stevens

> When write return 0 , it means the kernel buffers for the socket are full.

> Use setsockopt() to change these socket options.

This is really absurd, I agree.  I didn't notice that, he asked about
'named-pipe'. If it was for 'sockets', what i posted would suit. Sorry for
that.

--

Nithyanand.
Siemens, Bangalore, India.
(Opinions expressed are my own and do not reflect the opinions of my employer,
Siemens)

 
 
 

select() and write() behavior with non-blocking named-pipe

Post by Roger Hosie » Thu, 05 Jul 2001 01:34:25


Well I can't think of any time write() would return the 0 and so that is why I
decided to ask here. I use truss all of the time on SUN boxes and trust it to
give me the complete and correct execution, but I've never used tusc/truss on
the HP box before and don't know if I should trust it or not. The output from
the truss is clearly indicating that select() is returning with a 1 and showing

that the fd I'm interested in is the one that caused the return. The output
from
truss on the write() is what I don't really understand -- I have basically said

that if I can't get an answer from here that I will have to conclude that there

is either a problem in the implementation of write() in the HP library or the
tusc/truss command isn't capable of tracing the execution correctly.

We only use O_NONBLOCK for this code and if the write() returned some
non-zero positive value the process would deal with that correctly it appears
from review of the code. We determine the size of the write() based upon
the number of bytes to be written or the OS's supported maximum size for a
named-pipe (in this case it appears to be 4096) whichever is less. I would
understand if the write() was sending a partial set of data out and returned a
number smaller than the requested size, but am perplexed by what tusc/truss
is showing me.

Thanks for the comments!

-Roger



>  Roger> but the write() returns a 0 and nothing is written.

> are you sure? (write should not return 0)

> make sure you used O_NONBLOCK and not O_NDELAY to set nonblocking
> mode. Note also that select will return that the pipe is writable if
> _anything_ can be written to it; it doesn't mean that an arbitrarily
> large write will succeed.

> --
> Andrew.

> comp.unix.programmer FAQ: see <URL: http://www.erlenstar.demon.co.uk/unix/>
>                            or <URL: http://www.whitefang.com/unix/>

 
 
 

select() and write() behavior with non-blocking named-pipe

Post by Barry Margoli » Thu, 05 Jul 2001 01:50:42




Quote:>Well I can't think of any time write() would return the 0 and so that is why I
>decided to ask here. I use truss all of the time on SUN boxes and trust it to
>give me the complete and correct execution, but I've never used tusc/truss on
>the HP box before and don't know if I should trust it or not. The output from
>the truss is clearly indicating that select() is returning with a 1 and showing

>that the fd I'm interested in is the one that caused the return. The output
>from
>truss on the write() is what I don't really understand -- I have basically said

>that if I can't get an answer from here that I will have to conclude that there

>is either a problem in the implementation of write() in the HP library or the
>tusc/truss command isn't capable of tracing the execution correctly.

It's just a gut feeling, but I think it's more likely that select() is
doing the wrong thing for a named pipe than that truss is lying about the
return value of write(), since what truss does is much simpler.

But if you want to be sure, why don't you modify your program so that it
displays what it's doing when it calls select() and write(), instead of
using truss?

--

Genuity, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.

 
 
 

select() and write() behavior with non-blocking named-pipe

Post by Roger Hosie » Thu, 05 Jul 2001 02:01:03


Barry,

Yes we are going to do that, the delay in doing so has been due to the fact that
we can only duplicate this on our customers installation and so it's a little more
difficult to deal with. I've never used DDE (not an HP kinda guy I guess and we
don't have lots of customers on that platform), but we have also requested that
it be install so we can trace the execution in the de* -- so much simpler
in the SUN world it seems to me.

I can believe that the select() is having the problem and shouldn't be returning
with the fd set, but do that explain the write() returning a 0? I would expect to
see a -1 and an errno value telling me that the write() would block or couldn't
be completed for some other reason. I don't know when write() can actually
return a 0, not that I'm an expert on that issue which is why I decide to seek any
insights from this forum.

Thanks for the comments!

-Roger




> >Well I can't think of any time write() would return the 0 and so that is why I
> >decided to ask here. I use truss all of the time on SUN boxes and trust it to
> >give me the complete and correct execution, but I've never used tusc/truss on
> >the HP box before and don't know if I should trust it or not. The output from
> >the truss is clearly indicating that select() is returning with a 1 and showing

> >that the fd I'm interested in is the one that caused the return. The output
> >from
> >truss on the write() is what I don't really understand -- I have basically said

> >that if I can't get an answer from here that I will have to conclude that there

> >is either a problem in the implementation of write() in the HP library or the
> >tusc/truss command isn't capable of tracing the execution correctly.

> It's just a gut feeling, but I think it's more likely that select() is
> doing the wrong thing for a named pipe than that truss is lying about the
> return value of write(), since what truss does is much simpler.

> But if you want to be sure, why don't you modify your program so that it
> displays what it's doing when it calls select() and write(), instead of
> using truss?

> --

> Genuity, Burlington, MA
> *** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
> Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.

 
 
 

select() and write() behavior with non-blocking named-pipe

Post by Roger Hosie » Thu, 05 Jul 2001 03:00:02


Folks, I got an e-mail from someone asking another question and decided
that I should also point out that the write() does eventually return a
positive
non-zero value and data is written to the named-pipe. The data indicates
that the reader is busy and that there probably was a buffer full of data in

the pipe and that the write() would in a blocking mode would have been
blocked as you would expect. The code is 'working' with the exception
of the select() returning the fd when I don't expect it to and the write()
returning a 0 when I don't expect it to.

The whole point of this design was to allow the use of the non-block
write() on the named-pipe so that the process could continue to service
it's input stream and other functions that it performs. The expectations is
that the select() will block until interrupted by a signal, data from any of

the processes read fds arrives, or data can be written to the output fd for
the named-pipe. The problem is that select() is returning immediately
and the write() fails, we essentially are in a polling loop and this eats
the CPU to a much higher degree than desired -- in this application there
can be hundreds of these processes running on the same host.

Just wanted to make sure that I clarified everything that perhaps wasn't
stated in the original post.

Regards,

Roger Hosier


> I've got a process that sets a named-pipe to be non-blocking. The first
> time that the process
> enters a situation where the write() would block it adds the fd into the
> select() calls set
> for writers. The expectation was that the select() would indicate that
> the fd was ready for
> writing and the write() would be accomplished. On an HP/UX box we have a
> situation
> where the process that is doing this logic starts to take a large
> portion of the CPU -- just
> as if it was polling the fd. I used the HP/UX tusc/truss command and it
> shows that what is
> happening is that the select() returns a 1 (with the indication that the
> fd that is being tested
> for writing is set), but the write() returns a 0 and nothing is written.
> I have always been
> under the impression that write() would return a non-zero value and that
> select() would not
> return the fd for writing if it wasn't really ready for writing.

> What is the problem with the logic in this case?

> Thanks for the help!

> Roger

 
 
 

select() and write() behavior with non-blocking named-pipe

Post by phil-news-nos.. » Thu, 05 Jul 2001 22:38:08



| Folks, I got an e-mail from someone asking another question and decided
| that I should also point out that the write() does eventually return a
| positive
| non-zero value and data is written to the named-pipe. The data indicates
| that the reader is busy and that there probably was a buffer full of data in

There is one condition when write() can return 0 validly.  That is when the
length given to it is 0.  But I'm assuming you would have seen a 0 length
on the write() call in the truss output.  I'm inclined to assume an
implementation error.  It wouldn't be the first I've heard of for that
system.

--
-----------------------------------------------------------------
| Phil Howard - KA9WGN |   Dallas   | http://linuxhomepage.com/ |

-----------------------------------------------------------------

 
 
 

select() and write() behavior with non-blocking named-pipe

Post by Roger Hosie » Sat, 07 Jul 2001 02:00:57


Phil,

Yes the tusc/truss output shows that the write() is attempting to put out 4K of
data and it gets the return value of 0. We have been able to duplicate the issue
with the write() returning a 0 on a SUN platform, but the select0 works as we
expect it to on the SUN and doesn't return that the fd is writeable until it is in
fact reading for writing. At this point we believe that the problem is in the way
that HP has implemented the select() functionality -- I should note that it does
not seem to resolve to a poll() call the way that the SUN implementation does
and so we are considering running a test to see if the poll() function for the
HP works as expected or not (I assume that HP has poll() but haven't really
checked on that yet).

Thanks for the help!

-Roger



> | Folks, I got an e-mail from someone asking another question and decided
> | that I should also point out that the write() does eventually return a
> | positive
> | non-zero value and data is written to the named-pipe. The data indicates
> | that the reader is busy and that there probably was a buffer full of data in

> There is one condition when write() can return 0 validly.  That is when the
> length given to it is 0.  But I'm assuming you would have seen a 0 length
> on the write() call in the truss output.  I'm inclined to assume an
> implementation error.  It wouldn't be the first I've heard of for that
> system.

> --
> -----------------------------------------------------------------
> | Phil Howard - KA9WGN |   Dallas   | http://linuxhomepage.com/ |

> -----------------------------------------------------------------

 
 
 

select() and write() behavior with non-blocking named-pipe

Post by phil-news-nos.. » Sun, 08 Jul 2001 15:41:10



| Yes the tusc/truss output shows that the write() is attempting to put out 4K of
| data and it gets the return value of 0. We have been able to duplicate the issue
| with the write() returning a 0 on a SUN platform, but the select0 works as we
| expect it to on the SUN and doesn't return that the fd is writeable until it is in
| fact reading for writing. At this point we believe that the problem is in the way
| that HP has implemented the select() functionality -- I should note that it does
| not seem to resolve to a poll() call the way that the SUN implementation does
| and so we are considering running a test to see if the poll() function for the
| HP works as expected or not (I assume that HP has poll() but haven't really
| checked on that yet).

I would assume (but that is often dangerous) that HP/UX has poll() since
that's a SysV thing.  Let us know if poll() works properly there.  I have
pretty much switched to using poll() for all the code I write that needs
non-blocking I/O.

--
-----------------------------------------------------------------
| Phil Howard - KA9WGN |   Dallas   | http://linuxhomepage.com/ |

-----------------------------------------------------------------

 
 
 

select() and write() behavior with non-blocking named-pipe

Post by Roger Hosie » Sat, 14 Jul 2001 02:43:23


Phil,

Sorry for the delay in the response, been busy!!

We never did the switch to poll() to test this since we had some issues with the
HP/UX patch levels that we were pursuing. We've got HP involved in this as of
today and so we'll know more soon I hope and I'll post the results. Why have you
gone to poll(), is it just a portability issue or some technical issue you had?

Thanks!

Roger Hosier



> | Yes the tusc/truss output shows that the write() is attempting to put out 4K of
> | data and it gets the return value of 0. We have been able to duplicate the issue
> | with the write() returning a 0 on a SUN platform, but the select0 works as we
> | expect it to on the SUN and doesn't return that the fd is writeable until it is in
> | fact reading for writing. At this point we believe that the problem is in the way
> | that HP has implemented the select() functionality -- I should note that it does
> | not seem to resolve to a poll() call the way that the SUN implementation does
> | and so we are considering running a test to see if the poll() function for the
> | HP works as expected or not (I assume that HP has poll() but haven't really
> | checked on that yet).

> I would assume (but that is often dangerous) that HP/UX has poll() since
> that's a SysV thing.  Let us know if poll() works properly there.  I have
> pretty much switched to using poll() for all the code I write that needs
> non-blocking I/O.

> --
> -----------------------------------------------------------------
> | Phil Howard - KA9WGN |   Dallas   | http://linuxhomepage.com/ |

> -----------------------------------------------------------------

 
 
 

select() and write() behavior with non-blocking named-pipe

Post by phil-news-nos.. » Sun, 15 Jul 2001 21:35:27



| We never did the switch to poll() to test this since we had some issues with the
| HP/UX patch levels that we were pursuing. We've got HP involved in this as of
| today and so we'll know more soon I hope and I'll post the results. Why have you
| gone to poll(), is it just a portability issue or some technical issue you had?

Since I mostly use Linux, and Linux is going to poll() as the principle
syscall, I decided to try it out.  I found it to be easier to interface
with than select().

--
-----------------------------------------------------------------
| Phil Howard - KA9WGN |   Dallas   | http://linuxhomepage.com/ |

-----------------------------------------------------------------

 
 
 

1. select() behavior for Blocking and non-blocking sockets.

Hi,

This is a question on select() usage:

I've a scenerio wherein I expect connect() requests
from multiple clients. My server application starts
up different instances to service the client requests.
I'm using select() call with readfd set to the bind'ed
socket to accept the connections.
I notice a peculiar thing:
If the client sends a connect() to the server, the
connect() call succeeds, but, the server doesn't come
to know of the client connect request.

However, if I change the listening socket on the server
to NON-Blocking, (using ioctl with FIONBIO), the server
does accept connection.

I would like to know the concept behind this. Why doesn't
the server get the connect request incase of blocking socket ?

I sincerely appreciate your comments/suggestions.

Regards,
Vishal Murgai.

------------------------------------------------------------------------

2. need help downloading w/ 14.4 modem in kermit

3. Non-blocking writing on a Named pipe

4. Default UID numbers out of whack!

5. select(), non blocking writes to pipes and EAGAIN

6. Monitor fading

7. [2.5] Non-blocking write can block

8. prepending text to a file

9. pipe write() blocks even though select() indicates writable

10. select() is not blocking on (named) pipes

11. Non Blocking Named Pipe

12. Select in non-blocking sockets

13. select and non-blocking sockets