Very Computer

If I reboot the box (shutdown -r now), the problem seems to stops manifesting
until I twiddle with the ethernet link status again.  (The first time this
happened I thought it was a hardware problem, and it did go away when I
plugged in an rtl8139 card and started using that instead, but now it's
happened on another system and it really does look like some kind of a device
driver problem...)

Here's an example run of ping when the problem is manifesting.  Notice the
sequence numbers and delay timestamps.  (We've tried swapping in three
different switches from two different manufacturers in between, so that's not
the problem...)  I think ping's getting confused by receiving 15 second-old
packets...

Help?

Rob

--------------------------------------------------

root@lithium:~# ping 216.143.22.140
PING 216.143.22.140 (216.143.22.140): 56 octets data
64 octets from 216.143.22.140: icmp_seq=0 ttl=254 time=0.7 ms
64 octets from 216.143.22.140: icmp_seq=1 ttl=254 time=0.5 ms
64 octets from 216.143.22.140: icmp_seq=2 ttl=254 time=0.4 ms
64 octets from 216.143.22.140: icmp_seq=3 ttl=254 time=0.5 ms
64 octets from 216.143.22.140: icmp_seq=4 ttl=254 time=0.5 ms
64 octets from 216.143.22.140: icmp_seq=11 ttl=254 time=0.4 ms
64 octets from 216.143.22.140: icmp_seq=12 ttl=254 time=0.4 ms
64 octets from 216.143.22.140: icmp_seq=13 ttl=254 time=0.4 ms
64 octets from 216.143.22.140: icmp_seq=14 ttl=254 time=0.4 ms
64 octets from 216.143.22.140: icmp_seq=15 ttl=254 time=0.7 ms
64 octets from 216.143.22.140: icmp_seq=5 ttl=254 time=10000.8 ms
wrong data byte #0 should be 0xf9 but was 0xefef 7c d2 3c 77 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=6 ttl=254 time=9000.9 ms
wrong data byte #0 should be 0xf9 but was 0xf0f0 7c d2 3c 74 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=7 ttl=254 time=8001.0 ms
wrong data byte #0 should be 0xf9 but was 0xf1f1 7c d2 3c 78 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=8 ttl=254 time=7001.2 ms
wrong data byte #0 should be 0xf9 but was 0xf2f2 7c d2 3c 72 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=9 ttl=254 time=6001.3 ms
wrong data byte #0 should be 0xf9 but was 0xf3f3 7c d2 3c 71 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=10 ttl=254 time=5001.4 ms
wrong data byte #0 should be 0xf9 but was 0xf4f4 7c d2 3c 88 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=26 ttl=254 time=0.4 ms
64 octets from 216.143.22.140: icmp_seq=27 ttl=254 time=0.5 ms
64 octets from 216.143.22.140: icmp_seq=28 ttl=254 time=0.4 ms
64 octets from 216.143.22.140: icmp_seq=29 ttl=254 time=0.4 ms
64 octets from 216.143.22.140: icmp_seq=30 ttl=254 time=0.4 ms
64 octets from 216.143.22.140: icmp_seq=31 ttl=254 time=0.8 ms
64 octets from 216.143.22.140: icmp_seq=16 ttl=254 time=15000.9 ms
wrong data byte #0 should be 0x9 but was 0xfafa 7c d2 3c 7c 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=17 ttl=254 time=14001.0 ms
wrong data byte #0 should be 0x9 but was 0xfbfb 7c d2 3c 76 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=18 ttl=254 time=13001.2 ms
wrong data byte #0 should be 0x9 but was 0xfcfc 7c d2 3c 71 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=19 ttl=254 time=12001.3 ms
wrong data byte #0 should be 0x9 but was 0xfdfd 7c d2 3c 71 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=20 ttl=254 time=11001.4 ms
wrong data byte #0 should be 0x9 but was 0xfefe 7c d2 3c 71 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=21 ttl=254 time=10001.5 ms
wrong data byte #0 should be 0x9 but was 0xffff 7c d2 3c 73 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=22 ttl=254 time=9001.7 ms
wrong data byte #0 should be 0x9 but was 0x00 7d d2 3c 72 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=23 ttl=254 time=8001.8 ms
wrong data byte #0 should be 0x9 but was 0x11 7d d2 3c 71 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=24 ttl=254 time=7001.9 ms
wrong data byte #0 should be 0x9 but was 0x22 7d d2 3c 71 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=25 ttl=254 time=6002.0 ms
wrong data byte #0 should be 0x9 but was 0x33 7d d2 3c 71 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=42 ttl=254 time=0.4 ms
64 octets from 216.143.22.140: icmp_seq=43 ttl=254 time=0.4 ms
64 octets from 216.143.22.140: icmp_seq=44 ttl=254 time=0.4 ms
64 octets from 216.143.22.140: icmp_seq=45 ttl=254 time=0.4 ms
64 octets from 216.143.22.140: icmp_seq=46 ttl=254 time=0.8 ms
64 octets from 216.143.22.140: icmp_seq=32 ttl=254 time=14000.9 ms
wrong data byte #0 should be 0x18 but was 0xaa 7d d2 3c 7d 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=33 ttl=254 time=13001.1 ms
wrong data byte #0 should be 0x18 but was 0xbb 7d d2 3c 73 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=34 ttl=254 time=12001.2 ms
wrong data byte #0 should be 0x18 but was 0xcc 7d d2 3c 71 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=35 ttl=254 time=11001.3 ms
wrong data byte #0 should be 0x18 but was 0xdd 7d d2 3c 70 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=36 ttl=254 time=10001.4 ms
wrong data byte #0 should be 0x18 but was 0xee 7d d2 3c 72 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=37 ttl=254 time=9001.5 ms
wrong data byte #0 should be 0x18 but was 0xff 7d d2 3c 77 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=38 ttl=254 time=8001.7 ms
wrong data byte #0 should be 0x18 but was 0x1010 7d d2 3c 72 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=39 ttl=254 time=7001.8 ms
wrong data byte #0 should be 0x18 but was 0x1111 7d d2 3c 71 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=40 ttl=254 time=6001.9 ms
wrong data byte #0 should be 0x18 but was 0x1212 7d d2 3c 70 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=41 ttl=254 time=5002.0 ms
wrong data byte #0 should be 0x18 but was 0x1313 7d d2 3c 72 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=57 ttl=254 time=0.4 ms
64 octets from 216.143.22.140: icmp_seq=58 ttl=254 time=0.4 ms
64 octets from 216.143.22.140: icmp_seq=59 ttl=254 time=0.4 ms
64 octets from 216.143.22.140: icmp_seq=60 ttl=254 time=0.7 ms
64 octets from 216.143.22.140: icmp_seq=47 ttl=254 time=13000.8 ms
wrong data byte #0 should be 0x26 but was 0x1919 7d d2 3c 81 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21
22 23 24 25 26 27
        28 29 2a 2b 2c 2d 2e 2f
64 octets from 216.143.22.140: icmp_seq=48 ttl=254 time=12001.0 ms
wrong data byte #0 should be 0x26 but was 0x1a1a 7d d2 3c 77 6e 4 0
        8 9 a b c d e f 10 11 12 13 14 15
...

read more »

On Fri, 3 May 2002 08:12:20 -0400

I hope this helps somehow...

Bye

Quote:> Hi,
> I have a very similar problem... when I ping the box with the sis900
> ethernet adapter everything wents fine except ~3 % of the pings. This pings
> need 10 times longer to get back to me than the other (eg. 72 ms <-> 1004
> ms) There is no other network activity on the network
> And sometimes I get corrupted echo replies when there is traffic over the
> sis900...

> I hope this helps somehow...

> Bye

I'm 99% certain it's not a hardware problem, because:

A) It occurs on multiple motherboards (built in sis900).

B) A soft reboot makes the problem go away.

C) Unplugging the cat 5 cable and plugging it back in makes the problem start
manifesting, until the next reboot.  (On the systems I've seen it on anyway,
I haven't tested otherwise healthy boxes to see if I can make them go nuts by
temporarily unplugging their ethernet cables, but if anybody thinks it would
help...)

D) The problem itself is packets getting delayed ~10 seconds.  (I think the
ping corruption warning is just ping getting confused by the delay and
profound out-of-sequenceness, but in any case 10 second old packets aren't
interesting so if they went bad in storage it's a side issue...)

That smells like a driver problem to me, but I can't honestly say I have too
much of a clue in this area.  I'm working with Advanced Clue Substitute at
best...

Rob
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/