Gigabit Ethernet problems under Linux

Gigabit Ethernet problems under Linux

Post by Dave Turne » Sun, 08 Feb 1998 04:00:00



Gigabit Ethernet problems under Linux

   In measuring the bandwidth across gigabit ethernet between 2 Alphas
running Redhat 4.2, we've noticed some quirks for certain message sizes.
When the message size is roughly half of the TCP buffer size, the
performance drops by a considerable amount, only to rebound at around
75% of the buffer size (see the measurements below).  Can anyone
explain to me what Linux could be doing to cause this problem?  This is
a serious shortcoming that we've observed on both Alphas and Pentium
Pros
running Linux, while FreeBSD shows no similar problems on the same
hardware.  In applications, I've seen intermittent delays of upto 200 ms
for messages in this range, while breaking up the same messages into
multiple packets avoids these problems.

   I'ld also like to increase the maximum TCP buffer size beyond the
current maximum of 65535 bytes.  How easy is this, and how do I go
about doing it?

              Dave Turner

      NetPIPE results for a 20 kByte buffer size

  6:      7168 bytes  783 times -->  159.16 Mbps in 0.000344 sec
  7:      8189 bytes  726 times -->  160.72 Mbps in 0.000389 sec
  8:      8192 bytes  642 times -->  160.81 Mbps in 0.000389 sec
  9:      8195 bytes  642 times -->  160.87 Mbps in 0.000389 sec
 10:      9213 bytes  642 times -->  165.33 Mbps in 0.000425 sec
 11:      9216 bytes  587 times -->   59.48 Mbps in 0.001182 sec
 12:      9219 bytes  211 times -->   60.19 Mbps in 0.001169 sec
 13:     10237 bytes  213 times -->   71.13 Mbps in 0.001098 sec
 14:     10240 bytes  227 times -->   69.98 Mbps in 0.001116 sec
 15:     10243 bytes  223 times -->   68.64 Mbps in 0.001139 sec
 16:     11261 bytes  219 times -->   75.85 Mbps in 0.001133 sec
 17:     11264 bytes  220 times -->   75.77 Mbps in 0.001134 sec
 18:     11267 bytes  220 times -->   76.69 Mbps in 0.001121 sec
 19:     12285 bytes  222 times -->   96.63 Mbps in 0.000970 sec
 20:     12288 bytes  257 times -->   97.71 Mbps in 0.000959 sec
 21:     12291 bytes  259 times -->   95.47 Mbps in 0.000982 sec
 22:     13309 bytes  253 times -->   69.96 Mbps in 0.001451 sec
 23:     13312 bytes  171 times -->   70.71 Mbps in 0.001436 sec
 24:     13315 bytes  173 times -->   70.85 Mbps in 0.001434 sec
 25:     14333 bytes  173 times -->   87.66 Mbps in 0.001248 sec
 26:     14336 bytes  199 times -->   60.08 Mbps in 0.001821 sec
 27:     14339 bytes  136 times -->   69.57 Mbps in 0.001573 sec
 28:     15357 bytes  158 times -->   93.38 Mbps in 0.001255 sec
 29:     15360 bytes  198 times -->   99.41 Mbps in 0.001179 sec
 30:     15363 bytes  211 times -->  100.50 Mbps in 0.001166 sec
 31:     16381 bytes  212 times -->  151.15 Mbps in 0.000827 sec
 32:     16384 bytes  300 times -->  151.78 Mbps in 0.000824 sec
 33:     16387 bytes  301 times -->  151.41 Mbps in 0.000826 sec
 34:     17405 bytes  300 times -->  153.07 Mbps in 0.000868 sec
 35:     17408 bytes  286 times -->  154.35 Mbps in 0.000860 sec
 36:     17411 bytes  288 times -->  153.93 Mbps in 0.000863 sec

--

*******************************************************************
*   Dr Dave Turner               Asst Scientist - Ames Lab - ISU  *

*   Iowa State University        Phone:(515)-294-8872             *
*   Ames, IA  50011              Fax:  (515)-294-0689             *
*******************************************************************