Question about TCP/IP and OutOfBand bytes (MSG_OOB)

Question about TCP/IP and OutOfBand bytes (MSG_OOB)

Post by Kenny McCorma » Wed, 08 Nov 2000 04:00:00



I've read the relevant chapter (21, I believe) in UNP, but it kinda dances
around the issue.

Is it possible to send more than one byte as OOB?  And if you do, what
happens to it?  The book talks in terms of sending one byte and retrieving
that single byte.

My tests indicate if you send, say, "ABC" as OOB, the end result is that "AB"
gets sends as normal data, and the "C" ends up as the OOB byte.  Is this
correct?

 
 
 

Question about TCP/IP and OutOfBand bytes (MSG_OOB)

Post by Barry Margoli » Wed, 08 Nov 2000 04:00:00




>I've read the relevant chapter (21, I believe) in UNP, but it kinda dances
>around the issue.

>Is it possible to send more than one byte as OOB?  And if you do, what
>happens to it?  The book talks in terms of sending one byte and retrieving
>that single byte.

>My tests indicate if you send, say, "ABC" as OOB, the end result is that "AB"
>gets sends as normal data, and the "C" ends up as the OOB byte.  Is this
>correct?

OOB data is implemented using the TCP Urgent pointer.  This is a pointer to
a specific byte in the data stream.  The TCP specification says that
everything up to that byte is considered "urgent", but doesn't specify a
starting point.  The sockets API simply designates that last byte as the
OOB data, since there's no way to know where else the OOB data should
start.

--

Genuity, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.

 
 
 

Question about TCP/IP and OutOfBand bytes (MSG_OOB)

Post by Kenny McCorma » Wed, 08 Nov 2000 04:00:00




...

Quote:>>My tests indicate if you send, say, "ABC" as OOB, the end result is that "AB"
>>gets sends as normal data, and the "C" ends up as the OOB byte.  Is this
>>correct?

>OOB data is implemented using the TCP Urgent pointer.  This is a pointer to
>a specific byte in the data stream.  The TCP specification says that
>everything up to that byte is considered "urgent", but doesn't specify a
>starting point.  The sockets API simply designates that last byte as the
>OOB data, since there's no way to know where else the OOB data should
>start.

So, you are saying that the net result is that the sockets API does take
that last byte and shuffles it off somewhere else (unless the OOBINLINE
option [whatever it is called - I don't have the book handy right now] is
set), but it leaves the others ("AB" in my example) inline as normal data.
Right?
 
 
 

Question about TCP/IP and OutOfBand bytes (MSG_OOB)

Post by Barry Margoli » Wed, 08 Nov 2000 04:00:00






>...
>>>My tests indicate if you send, say, "ABC" as OOB, the end result is that "AB"
>>>gets sends as normal data, and the "C" ends up as the OOB byte.  Is this
>>>correct?

>>OOB data is implemented using the TCP Urgent pointer.  This is a pointer to
>>a specific byte in the data stream.  The TCP specification says that
>>everything up to that byte is considered "urgent", but doesn't specify a
>>starting point.  The sockets API simply designates that last byte as the
>>OOB data, since there's no way to know where else the OOB data should
>>start.

>So, you are saying that the net result is that the sockets API does take
>that last byte and shuffles it off somewhere else (unless the OOBINLINE
>option [whatever it is called - I don't have the book handy right now] is
>set), but it leaves the others ("AB" in my example) inline as normal data.
>Right?

Correct.  Since TCP doesn't have a way of indicating the start of urgent
data, only the end, how is it supposed to know that "AB" should have been
shuffled away as well?  Remember, TCP is a byte-stream protocol, it doesn't
have any way of indicating message boundaries, so there's no way for a
receiver to know that "ABC" should be treated as a unit.

--

Genuity, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.

 
 
 

Question about TCP/IP and OutOfBand bytes (MSG_OOB)

Post by Andrew Giert » Wed, 08 Nov 2000 04:00:00


 Kenny> I've read the relevant chapter (21, I believe) in UNP, but it
 Kenny> kinda dances around the issue.

 Kenny> Is it possible to send more than one byte as OOB?  And if you
 Kenny> do, what happens to it?  The book talks in terms of sending
 Kenny> one byte and retrieving that single byte.

The problem with OOB and TCP is that the socket API is trying to
implement an abstraction which is alien to TCP (but which is found in
some other protocol suites, e.g. SNA and others). Remember that the
socket API is supposed to be protocol-independent, after all.

These other protocol families have the notion of 'expedited data',
mechanisms for sending priority info that may need to be delivered
before previously-sent data in the normal flow. TCP does not provide
a mechanism for this.

TCP does, on the other hand, have the 'urgent' flag and pointer, which
is intended to indicate to the peer, "it is urgent that you read this
far". Telnet, for example, uses this to flush input or output when an
interrupt signal is being sent: when the peer gets informed of urgent
data, it is expected to read (and usually ignore) any pending normal
data until it gets to the urgent part. This corresponds more closely
to the socket API's behaviour when SO_OOBINLINE is set.

Using TCP Urgent data to implement the out-of-band data abstraction is
arguably bad (and certainly fails to work in some cases, resulting in
supposedly out-of-band bytes ending up in the normal data stream).
Matters are further confused by a long-standing disagreement between
the RFCs and (virtually all) implementations as to where the TCP
urgent pointer is actually supposed to point to. (The RFCs say that
the urgent pointer points to the last byte of data sent in urgent
mode; the BSD code and probably every other major implementation sets
the urgent pointer to point to the _following_ byte.)

--
Andrew.

comp.unix.programmer FAQ: see <URL: http://www.erlenstar.demon.co.uk/unix/>
                           or <URL: http://www.whitefang.com/unix/>

 
 
 

Question about TCP/IP and OutOfBand bytes (MSG_OOB)

Post by Andrew Gabri » Thu, 09 Nov 2000 04:00:00




Quote:> Matters are further confused by a long-standing disagreement between
> the RFCs and (virtually all) implementations as to where the TCP
> urgent pointer is actually supposed to point to. (The RFCs say that
> the urgent pointer points to the last byte of data sent in urgent
> mode; the BSD code and probably every other major implementation sets
> the urgent pointer to point to the _following_ byte.)

Actually, the RFCs don't even agree on this point. RFC793 (TCP RFC)
says it points to the byte following urgent data, and some other RFC
which I can't remember at the moment claims this is wrong and it
points to the last byte of urgent data.

When I implemented TCP on a minicomputer operating system, I went
with the RFC793 definition (only because I didn't see the other RFC
until afterwards, but even in hindsight, I believe this was the more
correct way round). However, I also provided a programming API which
was much better suited to TCP than the unix socket API.

All-in-all, urgent data is a badly thought out concept in TCP, when
compared with expedited data of other protocol familes.

--
Andrew Gabriel

 
 
 

Question about TCP/IP and OutOfBand bytes (MSG_OOB)

Post by Andrew Giert » Thu, 09 Nov 2000 04:00:00


 >> Matters are further confused by a long-standing disagreement between
 >> the RFCs and (virtually all) implementations as to where the TCP
 >> urgent pointer is actually supposed to point to. (The RFCs say that
 >> the urgent pointer points to the last byte of data sent in urgent
 >> mode; the BSD code and probably every other major implementation sets
 >> the urgent pointer to point to the _following_ byte.)

 Andrew> Actually, the RFCs don't even agree on this point. RFC793
 Andrew> (TCP RFC) says it points to the byte following urgent data,

actually 793 contradicts itself, saying different things in different
places. This is the original cause of the problem.

This error is corrected in RFC 924, and again in RFC 1122 (the Host
Requirements standard, which contains corrections and updates for many
protocol issues, and is required reading when working with any earlier
RFC).

--
Andrew.

comp.unix.programmer FAQ: see <URL: http://www.erlenstar.demon.co.uk/unix/>
                           or <URL: http://www.whitefang.com/unix/>

 
 
 

Question about TCP/IP and OutOfBand bytes (MSG_OOB)

Post by Chris Tor » Thu, 09 Nov 2000 04:00:00


>>>>... if you send, say, "ABC" as OOB, the end result is that "AB"
>>>>gets sends as normal data, and the "C" ends up as the OOB byte. ...


>>>OOB data is implemented using the TCP Urgent pointer. ...


>>... the sockets API [snips out] that last byte and shuffles it off
>>somewhere else ... but it leaves the others ("AB" in my example) inline
>>as normal data.  Right?



Quote:>Correct.  Since TCP doesn't have a way of indicating the start of urgent
>data, only the end, how is it supposed to know that "AB" should have been
>shuffled away as well?  Remember, TCP is a byte-stream protocol, it doesn't
>have any way of indicating message boundaries, so there's no way for a
>receiver to know that "ABC" should be treated as a unit.

Indeed, there is a major disconnect between "urgent data" and "out
of band", and setting that "in line" option (snipped above) is
probably the best way to deal with the situation at all.  The
"urgent pointer" in TCP is associated with each tcp segment, but
does it mean that *all* the data in that segment is "urgent", or
does it mean only that *some* of the data in that segment is
"urgent"?  And in any case, this has no effect on type- or
quality-of-service at lower layers.  Presumably, if data-set-X is
"more urgent" than data-set-Y, you would want it sent with a higher
low-level priority.

You can achieve this by opening two separate sockets.  Send "out
of band" data on the second (high-priority low-usage) socket, and
it will truly be out of band.
--
In-Real-Life: Chris Torek, Berkeley Software Design Inc


 
 
 

Question about TCP/IP and OutOfBand bytes (MSG_OOB)

Post by Barry Margoli » Fri, 10 Nov 2000 09:05:42



>Indeed, there is a major disconnect between "urgent data" and "out
>of band", and setting that "in line" option (snipped above) is
>probably the best way to deal with the situation at all.  The
>"urgent pointer" in TCP is associated with each tcp segment, but
>does it mean that *all* the data in that segment is "urgent", or
>does it mean only that *some* of the data in that segment is
>"urgent"?  

It means that some of the data up to the urgent pointer, potentially
including data in *preceding* segments, is urgent.  Remember, TCP is a byte
stream, so segment boundaries are not significant.  When TCP is performing
retransmissions, it's allowed to merge adjacent segments that were
originally sent separately, and there's nothing that says that a segment
with URG set can't be merged with one that doesn't.

--

Genuity, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.

 
 
 

Question about TCP/IP and OutOfBand bytes (MSG_OOB)

Post by Chris Tor » Fri, 10 Nov 2000 04:00:00



>>Indeed, there is a major disconnect between "urgent data" and "out
>>of band", and setting that "in line" option (snipped above) is
>>probably the best way to deal with the situation at all.  The
>>"urgent pointer" in TCP is associated with each tcp segment, but
>>does it mean that *all* the data in that segment is "urgent", or
>>does it mean only that *some* of the data in that segment is
>>"urgent"?  



Quote:>It means that some of the data up to the urgent pointer, potentially
>including data in *preceding* segments, is urgent.  Remember, TCP is a byte
>stream, so segment boundaries are not significant.  When TCP is performing
>retransmissions, it's allowed to merge adjacent segments that were
>originally sent separately, and there's nothing that says that a segment
>with URG set can't be merged with one that doesn't.

Well, yes; my question was meant more rhetorically, to illustrate
that the "urgent pointer" is rather tough to use effectively.

Consider also that each received segment might have a *different*
urgent pointer.  The sender might know what this means, but if the
receiver gets two segments at about the same time, before the
application can do anything with them, well, then what?  If the
receiver got the TCP headers, it might be able to do something,
but otherwise it will see only one urgent pointer, rather than two.
A BSD implementation will only get one of the "two" out-of-band
bytes, too ("two" in quotes here because, as we established earlier,
there is really no "out of band" byte at all).
--
In-Real-Life: Chris Torek, Berkeley Software Design Inc