How to do TCP tuning, set slowstart and sendstall timeouts?

How to do TCP tuning, set slowstart and sendstall timeouts?

Post by Fredda » Fri, 04 Apr 2003 14:58:22



Hi,

I want to be able to tune TCP in different ways, how can I do this?

My problem is following, when sending data to a socket and the link
get down, the send function hangs (fills its send buffer) and waits for the
link to get up again. This "up link"-wait takes around 1 minute after the
ethernet interface infact has got connected again (checked with ping),
can I, in some way, tune my TCP socket so this time will become minimal?

To test this, I start my application, start sending data and disconnect the
ethernet cable, the send function hangs (as it should) and are waiting on
the link to get up again, when I put the cable back it will take around one
minute before the send will release the send-hang. What should I do to get
rid of that minute delay?

I have tried using the setsockopt-function to decrease the SNDBUF size
to MTU size (1500 bytes) to get the send to go faster.
Also disabled the Nagle algorithm in the both ends of the link.

Are there any ways to adjust the slowstart, sendstall, congestion, etc. for
tuning the TCP protocol?

I'm running red hat 7.3 and developing in c.

Thanks in advance.

Greetings Fredrik.

 
 
 

How to do TCP tuning, set slowstart and sendstall timeouts?

Post by Glen Cummi » Sat, 05 Apr 2003 21:09:03


Freddan,

I'll probably show my ignorance here, but here goes.

First the book 'Unix network programming' Vol 1 by Stevens - Prentice
Hall is a bit of a hard read but contains lots of useful information
on things such as this (p185 talks about these sorts of scenarios).

TCP does time-out sockets eventually if SO_KEEPALIVE is enabled on the
socket although the TCP_KEEPALIVE parameter is usually set so this
takes ages and I think in linux this is a kernel parameter and so
changing it will affect all sockets - although you can mess with it by
echoing new values into /proc/sys/net/ipv4/tcp_keepalive_time &
tcp_keepalive_probes (check linux documentation for more info).

Not sure SO_RCVTIMEO & SO_SNDTIMEO are implemented, otherwise you
could have a timeout on the send implemented at the socket level.

Other things you can consider are setting the socket to non-blocking
and then using select to determine if you can write to the socket -
then do the timeout yourself.

Alternatively you could encapsulte the socket functionality into
something a bit higher level and using the above mechanism in
conjunction with sending your own keep-alive messages in both
directions so if say three of these messages are not acked the socket
it closed.

Just some ideas, I'm sure others will have better ones.

Cheers,

Glen

 
 
 

How to do TCP tuning, set slowstart and sendstall timeouts?

Post by Randy Howar » Sat, 12 Apr 2003 15:05:08




Quote:> My problem is following, when sending data to a socket and the link
> get down, the send function hangs (fills its send buffer) and waits for the
> link to get up again. This "up link"-wait takes around 1 minute after the
> ethernet interface infact has got connected again (checked with ping),
> can I, in some way, tune my TCP socket so this time will become minimal?

This sure sounds like the results of a spanning tree "hiccup" when a
link gets reset.  If so, that's in your switch, not your code.  If it
is a managed switch and you can get access to it, see if it has a
"fastlink" option under spanning tree.  Also, but more dangerous, is
to disable it entirely.  If you have a loop though, you'll get all
sorts of problems later on.

Quote:> I have tried using the setsockopt-function to decrease the SNDBUF size
> to MTU size (1500 bytes) to get the send to go faster.

That's probably counter-productive.  Most of the time you'll get the
best numbers (if streaming data for performance) with SNDBUF and
RCVBUF set to somewhere between 24K and 64K.  I've done this testing
on a lot of NICs, and you can get your paycheck on it.  If however
you are doing a lot of tiny send/rcv transactions, it won't help.  
Think something like ftp or ttcp style usage for the above.

--
Randy Howard
remove the obvious bits from my address to reply.

 
 
 

How to do TCP tuning, set slowstart and sendstall timeouts?

Post by RafBenso » Sat, 19 Apr 2003 01:56:04






> > My problem is following, when sending data to a socket and the link
> > get down, the send function hangs (fills its send buffer) and waits for
the
> > link to get up again. This "up link"-wait takes around 1 minute after
the
> > ethernet interface infact has got connected again (checked with ping),
> > can I, in some way, tune my TCP socket so this time will become minimal?

> This sure sounds like the results of a spanning tree "hiccup" when a
> link gets reset.  If so, that's in your switch, not your code.  If it
> is a managed switch and you can get access to it, see if it has a
> "fastlink" option under spanning tree.  Also, but more dangerous, is
> to disable it entirely.  If you have a loop though, you'll get all
> sorts of problems later on.

> > I have tried using the setsockopt-function to decrease the SNDBUF size
> > to MTU size (1500 bytes) to get the send to go faster.

> That's probably counter-productive.  Most of the time you'll get the
> best numbers (if streaming data for performance) with SNDBUF and
> RCVBUF set to somewhere between 24K and 64K.  I've done this testing
> on a lot of NICs, and you can get your paycheck on it.  If however
> you are doing a lot of tiny send/rcv transactions, it won't help.
> Think something like ftp or ttcp style usage for the above.

> --
> Randy Howard
> remove the obvious bits from my address to reply.

For my work I've modified parameters into kernel source... (slowstart,
SSTRSH... and so on).It is not a clean approach: doing so you have only
"modified TCP" on your machine, but it is quick and it works. Also TCP's
source code is well commented and sometimes there are some referrence to
RFCs too.

Ciao Luca

 
 
 

How to do TCP tuning, set slowstart and sendstall timeouts?

Post by David Schwart » Fri, 25 Apr 2003 05:10:10



Quote:> To test this, I start my application, start sending data and disconnect
the
> ethernet cable, the send function hangs (as it should) and are waiting on
> the link to get up again, when I put the cable back it will take around
one
> minute before the send will release the send-hang. What should I do to get
> rid of that minute delay?

    Make 100% sure that the delay isn't in your network. Do the following
test:

    1) Disconnect the ethernet cable *while* a ping is in progress between
the two machines.

    2) Confirm that pings stop getting through.

    3) Immediately return the ethernet cable to its connection.

    4) Time how long it takes before the first ping gets through.

Quote:> I have tried using the setsockopt-function to decrease the SNDBUF size
> to MTU size (1500 bytes) to get the send to go faster.
> Also disabled the Nagle algorithm in the both ends of the link.

    Wow, that should really make things worse. You thought that would help?

    DS

 
 
 

1. How to do TCP tuning, set slowstart and sendstall timeouts?

Hi,

I want to be able to tune TCP in different ways, how can I do this?

My problem is following, when sending data to a socket and the link
get down, the send function hangs (fills its send buffer) and waits for the
link to get up again. This "up link"-wait takes around 1 minute after the
ethernet interface infact has got connected again (checked with ping),
can I in some way tune my TCP socket so this time will bevome minimal?

To test this, I start my application, start sending data and disconnect the
ethernet cable, the send function hangs (as it should) and are waiting on
the link to get up again, when I put the cable back it will take around one
minute before the send will release the send-hang. What should I do?

I have tried using the setsockopt-function to decrease the SNDBUF size
to MTU size (1500 bytes) to get the send to go faster.
Also disabled the Nagle algorithm in the both ends of the link.

Are there any ways to adjust the slowstart, sendstall, congestion, etc. for
tuning the TCP protocol?

Thanks in advance.

Greetings Fredrik.

2. [v850] Whitespace and comment cleanups for v850 entry.S

3. Tuning timeout in tcp protocol state

4. squashed video resolution

5. Having problem with setting TCP Timeout?

6. What's the worst thing you have ever done as a Unix sysadmin?

7. TCP default timeouts badly set

8. *** CLASSIC MISSIONARY PROBLEM ***

9. TCP connect timeout setting in Solaris

10. How to set timeout on TCP/IP based client-server connection?

11. How to set timeout on my TCP/IP connection

12. application timeout vs. TCP timeout for solaris

13. Set timeout but why not timeout?