Linux 2.6.39-rc7 - Wrong Maximum Segment Size being advertised

Linux 2.6.39-rc7 - Wrong Maximum Segment Size being advertised

Post by Mark Hoble » Sun, 22 May 2011 23:58:32



I am upgraded my Debian system to experimental kernel 2.6.39-rc7. This has
caused an old network problem to reoccur, which I believe is related to the
network Maximum Segment Size that my ISP can convey.

On startup of my computer, I issue a command to clamp the size of all
externally bound traffic (destined for routing via the default gateway at
10.0.0.1) as follows:

ip route change default via 10.0.0.1 mtu lock 1412

Where 1412 is the maximum transmission unit for my ISP (Three mobile).
I now attempt to post a usenet article, and the newsreader hangs.

I have used tcpdump to examine the communications, and here is the output:

18: Flags [S.], seq 3385954279, ack 3902988975, win 5792, options [mss 1380,sackOK,TS val 140346418 ecr 47079743,nop,wscale 7], length 0
03:40:13.539947 IP venus.markhobley.yi.org.51218 > news.eternal-september.org.nntp: Flags [.], ack 1, win 457, options [nop,nop,TS val 47079880 ecr 140346418], length 0
03:40:14.079843 IP news.eternal-september.org.nntp > venus.markhobley.yi.org.51218: Flags [P.], seq 1:107, ack 1, win 46, options [nop,nop,TS val 140346550 ecr 47079880], length 106
03:40:14.079914 IP venus.markhobley.yi.org.51218 > news.eternal-september.org.nntp: Flags [.], ack 107, win 457, options [nop,nop,TS val 47080015 ecr 140346550], length 0
03:40:14.080345 IP venus.markhobley.yi.org.51218 > news.eternal-september.org.nntp: Flags [P.], seq 1:23, ack 107, win 457, options [nop,nop,TS val 47080016 ecr 140346550], length 22
03:40:14.829571 IP news.eternal-september.org.nntp > venus.markhobley.yi.org.51218: Flags [.], ack 23, win 46, options [nop,nop,TS val 140346745 ecr 47080016], length 0
03:40:14.869553 IP news.eternal-september.org.nntp > venus.markhobley.yi.org.51218: Flags [P.], seq 107:127, ack 23, win 46, options [nop,nop,TS val 140346745 ecr 47080016], length 20
03:40:14.870604 IP venus.markhobley.yi.org.51218 > news.eternal-september.org.nntp: Flags [P.], seq 23:47, ack 127, win 457, options [nop,nop,TS val 47080213 ecr 140346745], length 24
03:40:15.509493 IP news.eternal-september.org.nntp > venus.markhobley.yi.org.51218: Flags [.], ack 47, win 46, options [nop,nop,TS val 140346913 ecr 47080213], length 0
03:40:15.759474 IP news.eternal-september.org.nntp > venus.markhobley.yi.org.51218: Flags [P.], seq 127:157, ack 47, win 46, options [nop,nop,TS val 140346967 ecr 47080213], length 30
03:40:15.759754 IP venus.markhobley.yi.org.51218 > news.eternal-september.org.nntp: Flags [P.], seq 47:60, ack 157, win 457, options [nop,nop,TS val 47080435 ecr 140346967], length 13
03:40:16.300519 IP news.eternal-september.org.nntp > venus.markhobley.yi.org.51218: Flags [.], ack 60, win 46, options [nop,nop,TS val 140347111 ecr 47080435], length 0
03:40:16.339579 IP news.eternal-september.org.nntp > venus.markhobley.yi.org.51218: Flags [P.], seq 157:263, ack 60, win 46, options [nop,nop,TS val 140347111 ecr 47080435], length 106
03:40:16.340742 IP venus.markhobley.yi.org.51218 > news.eternal-september.org.nntp: Flags [P.], seq 60:66, ack 263, win 457, options [nop,nop,TS val 47080581 ecr 140347111], length 6
03:40:16.941302 IP news.eternal-september.org.nntp > venus.markhobley.yi.org.51218: Flags [P.], seq 263:324, ack 66, win 46, options [nop,nop,TS val 140347268 ecr 47080581], length 61
03:40:16.941635 IP venus.markhobley.yi.org.51218 > news.eternal-september.org.nntp: Flags [.], seq 66:1434, ack 324, win 457, options [nop,nop,TS val 47080731 ecr 140347268], length 1368
03:40:16.941686 IP venus.markhobley.yi.org.51218 > news.eternal-september.org.nntp: Flags [P.], seq 1434:2113, ack 324, win 457, options [nop,nop,TS val 47080731 ecr 140347268], length 679
03:40:16.941731 IP venus.markhobley.yi.org.51218 > news.eternal-september.org.nntp: Flags [.], seq 2113:3481, ack 324, win 457, options [nop,nop,TS val 47080731 ecr 140347268], length 1368
03:40:16.941995 IP venus.markhobley.yi.org.51218 > news.eternal-september.org.nntp: Flags [.], seq 3481:4849, ack 324, win 457, options [nop,nop,TS val 47080731 ecr 140347268], length 1368
03:40:18.168057 IP news.eternal-september.org.nntp > venus.markhobley.yi.org.51218: Flags [P.], seq 263:324, ack 66, win 46, options [nop,nop,TS val 140347601 ecr 47080581], length 61
03:40:18.168130 IP venus.markhobley.yi.org.51218 > news.eternal-september.org.nntp: Flags [.], ack 324, win 457, options [nop,nop,TS val 47081037 ecr 140347601,nop,nop,sack 1 {263:324}], length 0
03:40:18.388264 IP news.eternal-september.org.nntp > venus.markhobley.yi.org.51218: Flags [.], ack 66, win 46, options [nop,nop,TS val 140347656 ecr 47080581,nop,nop,sack 1 {108444702:108445381}], length 0
03:40:18.544109 IP venus.markhobley.yi.org.51218 > news.eternal-september.org.nntp: Flags [.], seq 66:1434, ack 324, win 457, options [nop,nop,TS val 47081132 ecr 140347656], length 1368
03:40:21.752110 IP venus.markhobley.yi.org.51218 > news.eternal-september.org.nntp: Flags [.], seq 66:1434, ack 324, win 457, options [nop,nop,TS val 47081934 ecr 140347656], length 1368
03:40:28.160112 IP venus.markhobley.yi.org.51218 > news.eternal-september.org.nntp: Flags [.], seq 66:1434, ack 324, win 457, options [nop,nop,TS val 47083536 ecr 140347656], length 1368
03:40:40.992116 IP venus.markhobley.yi.org.51218 > news.eternal-september.org.nntp: Flags [.], seq 66:1434, ack 324, win 457, options [nop,nop,TS val 47086744 ecr 140347656], length 1368
03:40:46.942740 IP venus.markhobley.yi.org.51218 > news.eternal-september.org.nntp: Flags [FP.], seq 4849:5371, ack 324, win 457, options [nop,nop,TS val 47088231 ecr 140347656], length 522
03:40:48.745614 IP venus.markhobley.yi.org.52196 > news.eternal-september.org.nntp: Flags [S], seq 185692436, win 14600, options [mss 1460,sackOK,TS val 47088682 ecr 0,nop,wscale 5], length 0
03:40:48.745939 IP news.eternal-september.org.nntp > venus.markhobley.yi.org.51218: Flags [.], ack 66, win 46, options [nop,nop,TS val 140355243 ecr 47080581,nop,nop,sack 2 {108448117:108448640}{108444702:108445381}], length 0
03:40:48.863069 IP news.eternal-september.org.nntp > venus.markhobley.yi.org.52196: Flags [S.], seq 3936843225, ack 185692437, win 5792, options [mss 1380,sackOK,TS val 140355273 ecr 47088682,nop,wscale 7], length 0
03:40:48.863153 IP venus.markhobley.yi.org.52196 > news.eternal-september.org.nntp: Flags [.], ack 1, win 457, options [nop,nop,TS val 47088711 ecr 140355273], length 0
03:40:48.973427 IP news.eternal-september.org.nntp > venus.markhobley.yi.org.52196: Flags [P.], seq 1:107, ack 1, win 46, options [nop,nop,TS val 140355304 ecr 47088711], length 106
03:40:48.973497 IP venus.markhobley.yi.org.52196 > news.eternal-september.org.nntp: Flags [.], ack 107, win 457, options [nop,nop,TS val 47088739 ecr 140355304], length 0
03:40:48.973831 IP venus.markhobley.yi.org.52196 > news.eternal-september.org.nntp: Flags [P.], seq 1:23, ack 107, win 457, options [nop,nop,TS val 47088739 ecr 140355304], length 22
03:40:49.103319 IP news.eternal-september.org.nntp > venus.markhobley.yi.org.52196: Flags [.], ack 23, win 46, options [nop,nop,TS val 140355332 ecr 47088739], length 0
03:40:49.104431 IP news.eternal-september.org.nntp > venus.markhobley.yi.org.52196: Flags [P.], seq 107:127, ack 23, win 46, options [nop,nop,TS val 140355332 ecr 47088739], length 20
03:40:49.104729 IP venus.markhobley.yi.org.52196 > news.eternal-september.org.nntp: Flags [P.], seq 23:47, ack 127, win 457, options [nop,nop,TS val 47088772 ecr 140355332], length 24
03:40:49.263060 IP news.eternal-september.org.nntp > venus.markhobley.yi.org.52196: Flags [.], ack 47, win 46, options [nop,nop,TS val 140355373 ecr 47088772], length 0
03:40:49.443101 IP news.eternal-september.org.nntp > venus.markhobley.yi.org.52196: Flags [P.], seq 127:157, ack 47, win 46, options [nop,nop,TS val 140355420 ecr 47088772], length 30
03:40:49.443438 IP venus.markhobley.yi.org.52196 > news.eternal-september.org.nntp: Flags [P.], seq 47:60, ack 157, win 457, options [nop,nop,TS val 47088856 ecr 140355420], length 13
03:40:49.553169 IP news.eternal-september.org.nntp > venus.markhobley.yi.org.52196: Flags [.], ack 60, win 46, options [nop,nop,TS val 140355447 ecr 47088856], length 0
03:40:49.553943 IP news.eternal-september.org.nntp > venus.markhobley.yi.org.52196: Flags [P.], seq 157:263, ack 60, win 46, options [nop,nop,TS val 140355447 ecr 47088856], length 106
03:40:49.554864 IP venus.markhobley.yi.org.52196 > news.eternal-september.org.nntp: Flags [P.], seq 60:66, ack 263, win 457, options [nop,nop,TS val 47088884 ecr 140355447], length 6
03:40:49.663214 IP news.eternal-september.org.nntp > venus.markhobley.yi.org.52196: Flags [P.], seq 263:324, ack 66, win 46, options [nop,nop,TS val 140355475 ecr 47088884], length 61
03:40:49.663616 IP venus.markhobley.yi.org.52196 > news.eternal-september.org.nntp: Flags [.], seq 66:1434, ack 324, win 457, options [nop,nop,TS val 47088911 ecr 140355475], length 1368
03:40:49.663751 IP venus.markhobley.yi.org.52196 > news.eternal-september.org.nntp: Flags [P.], seq 1434:2113, ack 324, win 457, options [nop,nop,TS val 47088911 ecr 140355475], length 679
03:40:49.663820 IP venus.markhobley.yi.org.52196 > news.eternal-september.org.nntp: Flags [.], seq 2113:3481, ack 324, win 457, options [nop,nop,TS val 47088911 ecr 140355475], length 1368
03:40:49.663883 IP venus.markhobley.yi.org.52196 > news.eternal-september.org.nntp: Flags [.], seq 3481:4849, ack 324, win 457, options [nop,nop,TS val 47088911 ecr 140355475], length 1368
03:40:49.833205 IP news.eternal-september.org.nntp > venus.markhobley.yi.org.52196: Flags [.], ack 66, win 46, options [nop,nop,TS val 140355515 ecr 47088884,nop,nop,sack 1 {1136656415:1136657094}], length 0
03:40:51.120106 IP venus.markhobley.yi.org.52196 > news.eternal-september.org.nntp: Flags [.], seq 66:1434, ack 324, win 457, options [nop,nop,TS val 47089276 ecr 140355515], length 1368
03:40:54.040109 IP venus.markhobley.yi.org.52196 > news.eternal-september.org.nntp: Flags [.], seq 66:1434, ack 324, win 457, options [nop,nop,TS val 47090006 ecr ...

read more »

 
 
 

Linux 2.6.39-rc7 - Wrong Maximum Segment Size being advertised

Post by David W. Hodgin » Mon, 23 May 2011 04:16:02



> However, examining the binary version of the above file for strings, I can
> see that fragments of the usenet data are being retransmitted.

Most likely the isp, or an isp in between your system and e-s is giving
nntp traffic a low priority, causing packets to timeout.

I had the same problem when my isp's upstream (bell.ca) did the same thing.

To avoid the problems caused by the traffic shaping, use an ssl connection.
I use leafnode, which doesn't suppport ssl, so I'm using stunnel.

Install stunnel.  On my Mandriva/Mageia system, I have ...

# grep stunnel /etc/rc.d/rc.local
/usr/bin/stunnel /etc/ssl/stunnel/stunnel.conf & # Setup ssl for fetchnews

# cat /etc/ssl/stunnel/stunnel.conf
debug=info
foreground=no
syslog=no
compression=rle
[nntps]
client=yes
connect=news.eternal-september.org:563
accept=564

Then change the newsreader to use localhost, port 564.

If your newsreader supports ssl, use port 563 directly to e-s.

Regards, Dave Hodgins

--
Change nomail.afraid.org to ody.ca to reply by email.
(nomail.afraid.org has been set up specifically for
use in usenet. Feel free to use it yourself.)

 
 
 

Linux 2.6.39-rc7 - Wrong Maximum Segment Size being advertised

Post by Mark Hoble » Mon, 23 May 2011 05:20:20



> 18: Flags [S.], seq 3385954279, ack 3902988975, win 5792, options [mss
> 1380,sackOK,TS val 140346418 ecr 47079743,nop,wscale 7], length 0

I'm not sure what this first line is, but win 5792 is bigger than any
datagram that my ISP can carry.

Quote:> 03:40:48.745614 IP venus.markhobley.yi.org.52196 >
> news.eternal-september.org.nntp: Flags [S], seq 185692436, win 14600,
> options [mss 1460,sackOK,TS val 47088682 ecr 0,nop,wscale 5], length 0

The value of win 14600 here is presumably the number of bytes of data that
venus is prepared to receive in a single instance. Is that right? Presumably
that is just data, and excludes headers. Is that right? It is bigger than
any datagram that my ISP can carry. Does it have any effect on datagram
sizes?

Quote:> news.eternal-september.org.nntp > venus.markhobley.yi.org.52196: Flags
> [S.], seq 3936843225, ack 185692437, win 5792, options [mss
> 1380,sackOK,TS val 140355273 ecr 47088682,nop,wscale 7], length 0

Here I receive a window size. Presumably, that is the size that the remote
host at eternal-september is prepared to accept. Again, I would never be
able to deliver that many bytes in a single datagram.

Quote:> 03:41:21.453043 IP news.eternal-september.org.nntp >
> venus.markhobley.yi.org.52197: Flags [S.], seq 156904658, ack 693658576,
> win 5792, options [mss 1380,sackOK,TS val 140363421 ecr
> 47096829,nop,wscale 7], length 0 03:41:21.453127 IP

Here is another big window. I don't know whether that means anything.

Quote:> 03:41:54.123370 IP venus.markhobley.yi.org.52198 >
> news.eternal-september.org.nntp: Flags [S], seq 1215827121, win 14600,
> options [mss 1460,sackOK,TS val 47105026 ecr 0,nop,wscale 5], length 0

Another big window.

Quote:> news.eternal-september.org.nntp: Flags [S], seq 1725633662, win 14600,
> options [mss 1460,sackOK,TS val 47113211 ecr 0,nop,wscale 5], length 0

and another

Quote:> 03:42:26.982276 IP news.eternal-september.org.nntp >
> venus.markhobley.yi.org.52199: Flags [S.], seq 1178329563, ack
> 1725633663, win 5792, options [mss 1380,sackOK,TS val 140379803 ecr
> 47113211,nop,wscale 7], length 0 03:42:26.982362 IP
> 03:42:50.439130 IP news.eternal-september.org.nntp >
> venus.markhobley.yi.org.51218: Flags [R], seq 4178125634, win 0, length
> 0

Hmmm, a window of zero here. Presumably, the remote is busy.

Quote:> 03:42:59.663041 IP venus.markhobley.yi.org.52200 >
> news.eternal-september.org.nntp: Flags [S], seq 2233139354, win 14600,
> options [mss 1460,sackOK,TS val 47121411 ecr 0,nop,wscale 5], length 0

A big window.

Quote:> 03:42:59.770427 IP news.eternal-september.org.nntp >
> venus.markhobley.yi.org.52200: Flags [S.], seq 1693600159, ack
> 2233139355, win 5792, options [mss 1380,sackOK,TS val 140388003 ecr
> 47121411,nop,wscale 7], length 0 03:42:59.770510 IP

And another

03:43:04.320740 IP venus.markhobley.yi.org.52201 >

Quote:> news.eternal-september.org.nntp: Flags [S], seq 2304145392, win 14600,
> options [mss 1460,sackOK,TS val 47122576 ecr 0,nop,wscale 5], length 0

A big window again. Hey look at the mss value! That looks big to me.
My ISP cannot carry a datagram above 1412 bytes.

Quote:> 03:43:04.500693 IP news.eternal-september.org.nntp >
> venus.markhobley.yi.org.52201: Flags [S.], seq 1766167236, ack
> 2304145393, win 5792, options [mss 1380,sackOK,TS val 140389170 ecr
> 47122576,nop,wscale 7], length 0

A big window.

Quote:> 03:43:11.752242 IP news.eternal-september.org.nntp >
> venus.markhobley.yi.org.52202: Flags [S.], seq 1875583976, ack
> 2424765078, win 5792, options [mss 1380,sackOK,TS val 140390973 ecr
> 47124291,nop,wscale 7], length 0

Another big window

Quote:> 03:43:14.812402 IP venus.markhobley.yi.org.52203 >
> news.eternal-september.org.nntp: Flags [S], seq 2475930012, win 14600,
> options [mss 1460,sackOK,TS val 47125199 ecr 0,nop,wscale 5], length 0

There is that big maximum segment size again!

Mark.

--
Mark Hobley
Linux User: #370818  http://markhobley.yi.org/

 
 
 

Linux 2.6.39-rc7 - Wrong Maximum Segment Size being advertised

Post by Richard Kettlewel » Mon, 23 May 2011 06:47:22




>> 18: Flags [S.], seq 3385954279, ack 3902988975, win 5792, options [mss
>> 1380,sackOK,TS val 140346418 ecr 47079743,nop,wscale 7], length 0

> I'm not sure what this first line is, but win 5792 is bigger than any
> datagram that my ISP can carry.

>> 03:40:48.745614 IP venus.markhobley.yi.org.52196 >
>> news.eternal-september.org.nntp: Flags [S], seq 185692436, win 14600,
>> options [mss 1460,sackOK,TS val 47088682 ecr 0,nop,wscale 5], length 0

> The value of win 14600 here is presumably the number of bytes of data
> that venus is prepared to receive in a single instance. Is that right?
> Presumably that is just data, and excludes headers. Is that right? It
> is bigger than any datagram that my ISP can carry. Does it have any
> effect on datagram sizes?

Yes, it's the number of bytes the sender is able to accept.  It's not
connected to packet sizes.

--
http://www.greenend.org.uk/rjk/

 
 
 

Linux 2.6.39-rc7 - Wrong Maximum Segment Size being advertised

Post by Jorgen Grah » Mon, 23 May 2011 07:02:29




>> However, examining the binary version of the above file for strings, I can
>> see that fragments of the usenet data are being retransmitted.

> Most likely the isp, or an isp in between your system and e-s is giving
> nntp traffic a low priority, causing packets to timeout.

Why should that affect /all/ his large segments, and /only/ the large
segments? It seems very unlikely to me.

/Jorgen

--

\X/     snipabacken.se>   O  o   .

 
 
 

Linux 2.6.39-rc7 - Wrong Maximum Segment Size being advertised

Post by Jorgen Grah » Mon, 23 May 2011 07:28:56


["Followup-To:" header set to comp.protocols.tcp-ip. You should have
mentioned that you started crossposting.]



...
>> 03:40:48.745614 IP venus.markhobley.yi.org.52196 >
>> news.eternal-september.org.nntp: Flags [S], seq 185692436, win 14600,
>> options [mss 1460,sackOK,TS val 47088682 ecr 0,nop,wscale 5], length 0

> The value of win 14600 here is presumably the number of bytes of data that
> venus is prepared to receive in a single instance. Is that right? Presumably
> that is just data, and excludes headers. Is that right? It is bigger than
> any datagram that my ISP can carry. Does it have any effect on datagram
> sizes?

No.

...

Quote:> 03:43:04.320740 IP venus.markhobley.yi.org.52201 >
>> news.eternal-september.org.nntp: Flags [S], seq 2304145392, win 14600,
>> options [mss 1460,sackOK,TS val 47122576 ecr 0,nop,wscale 5], length 0

> A big window again. Hey look at the mss value! That looks big to me.
> My ISP cannot carry a datagram above 1412 bytes.

1460 looks perfectly normal to me; it's the mss for a normal Ethernet.
n.e-s.o cannot know that your ISP is weird.  As far as I can tell
you're not using Path MTU discovery, so you're relying on the routers
on the way handling IP fragmentation correctly. Which they must.

Perhaps you should have captured ICMP messages too, in case there are
any which are relevant.

...

I've looked some at your tcpdump.  It's a bit confusing because it
contains multiple sessions -- n.e-s.org times out and forces your
client to reconnect a few times.

Anyway, I don't see anything really strange, apart from the fact that
large segments from you (at least 1368 and upwards) don't get through.
You send a bunch of them and resend them, but the server never ACKs
any of them.  For example in this part (abbreviated for clarity):

1:55.072935 IP C > S: Flags [.], seq 66:1434, ack 324, win 457, length 1368
1:55.072986 IP C > S: Flags [P.], seq 1434:2113, ack 324, win 457, length 679
1:55.073028 IP C > S: Flags [.], seq 2113:3481, ack 324, win 457, length 1368
1:55.073074 IP C > S: Flags [.], seq 3481:4849, ack 324, win 457, length 1368
1:55.242762 IP S > C: Flags [.], ack 66, win 46, length 0
1:56.524109 IP C > S: Flags [.], seq 66:1434, ack 324, win 457, length 1368
1:59.424110 IP C > S: Flags [.], seq 66:1434, ack 324, win 457, length 1368
2:05.232112 IP C > S: Flags [.], seq 66:1434, ack 324, win 457, length 1368
2:16.832117 IP C > S: Flags [.], seq 66:1434, ack 324, win 457, length 1368

/Jorgen

--

\X/     snipabacken.se>   O  o   .

 
 
 

Linux 2.6.39-rc7 - Wrong Maximum Segment Size being advertised

Post by Mark Hoble » Mon, 23 May 2011 10:15:07



> 1460 looks perfectly normal to me; it's the mss for a normal Ethernet.

Yeah, but I am not connected via ethernet (well, I am, but only as far as
the external gateway), the router uses a wireless broadband and it cannot
route ethernet frames, because they are too large.

Quote:> n.e-s.o cannot know that your ISP is weird.

I don't know what that means. I tell the routing table that my maximum
transmission unit is 1412. Shouldn't the value provided in the mss field
be less than the routable mtu?

ip route change default via 10.0.0.1 mtu lock 1412

Quote:>  As far as I can tell you're not using Path MTU discovery

I have PMTUD enabled in the kernel:

cat /proc/sys/net/ipv4/ip_no_pmtu_disc
0

However, there is a problem with routers downsteam, so I have to lock the
maximum transmission unit value to 1412, which is the highest value that
downstream can handle.

Quote:> Perhaps you should have captured ICMP messages too, in case there are
> any which are relevant.

I don't think that I get any ICMP messages when the system fails, but I will
try and do some retesting to see if I can capture any.

Quote:> I've looked some at your tcpdump.  It's a bit confusing because it
> contains multiple sessions -- n.e-s.org times out and forces your client
> to reconnect a few times.

Yeah, I don't know what is happening there. This is the pan newsreader. I
have configured it to have a connection limit of 1 connection, to try and
cut down the confusion from multiple sessions, but hey, it is still
confusing. I guess that as the connection fails, pan retries whilst the
datagrams are still going to and fro causing that chaos.

Quote:> Anyway, I don't see anything really strange, apart from the fact that
> large segments from you (at least 1368 and upwards) don't get through.

Yeah, I cannot route the large segments over mobile broadband. A smaller
segment size is needed for outbound traffic going via the default gateway
at 10.0.0.1

Quote:> You send a bunch of them and resend them, but the server never ACKs any
> of them.  For example in this part (abbreviated for clarity):

Hmmm, maybe I need to trim the MTU value further.

There is something weird going on.

Back in September:

echo 0 > /proc/sys/net/ipv4/ip_no_pmtu_disc

ping -s 1384 www.yahoo.com
PING any-fp.wa1.b.yahoo.com (67.195.160.76) 1384(1412) bytes of data.
1392 bytes from ir1.fp.vip.ac4.yahoo.com (67.195.160.76): icmp_seq=2 ttl=45 time=305 ms

1384 was the maximum value that gave a reply.

Today:

ping -s 1384 news.eternal-september.org  
PING news.eternal-september.org (88.198.244.100) 1384(1412) bytes of data.
1392 bytes from news.eternal-september.org (88.198.244.100): icmp_req=1 ttl=50 time=1132 ms
1392 bytes from news.eternal-september.org (88.198.244.100): icmp_req=2 ttl=50 time=236 ms

ping -s 1386 news.eternal-september.org
(No reply)

However:

ping -s 1386 www.yahoo.com              
PING any-fp.wa1.b.yahoo.com (67.195.160.76) 1386(1414) bytes of data.
1394 bytes from ir1.fp.vip.ac4.yahoo.com (67.195.160.76): icmp_req=1 ttl=47 time=1158 ms
1394 bytes from ir1.fp.vip.ac4.yahoo.com (67.195.160.76): icmp_req=2 ttl=47 time=300 ms

Huh! I get a reply. That never used to happen.

I can go right up to 1472:

ping -s 1472 www.yahoo.com
PING any-fp.wa1.b.yahoo.com (67.195.160.76) 1472(1500) bytes of data.
1480 bytes from ir1.fp.vip.ac4.yahoo.com (67.195.160.76): icmp_req=1 ttl=47 time=462 ms
1480 bytes from ir1.fp.vip.ac4.yahoo.com (67.195.160.76): icmp_req=2 ttl=47 time=453 ms

Now that is suspicious. Either a downstream ISP has done some upgrades
since September, or firmware updates to my router have enabled me to route
larger packets.

I think its something that ISPs are doing, because the maximum for my
news provider is still 1384.

I will mention my router, just in case it matters. It is an Edimax 3g-6200n
and I am using the latest firmware (version 2.24g). The firmware is Linux
based and open source, but it does not provide shell access. On the back of
the router is a Three Mobile Broadband dongle made by Huawei, which makes
the connection to the outside world.

Mark.

--
Mark Hobley
Linux User: #370818  http://markhobley.yi.org/

 
 
 

Linux 2.6.39-rc7 - Wrong Maximum Segment Size being advertised

Post by Barry Margoli » Mon, 23 May 2011 13:44:01





> > 18: Flags [S.], seq 3385954279, ack 3902988975, win 5792, options [mss
> > 1380,sackOK,TS val 140346418 ecr 47079743,nop,wscale 7], length 0

> I'm not sure what this first line is, but win 5792 is bigger than any
> datagram that my ISP can carry.

Large window sizes are normal.  It's basically how many data bytes can
be sent before waiting for acknowledgement.

The "wscale" option scales this up, so the actual window size is
5792*2^7, i.e. 724KB.

http://en.wikipedia.org/wiki/TCP_window_scale_option

--

Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***

 
 
 

Linux 2.6.39-rc7 - Wrong Maximum Segment Size being advertised

Post by Jorgen Grah » Mon, 23 May 2011 15:01:14




>> 1460 looks perfectly normal to me; it's the mss for a normal Ethernet.

> Yeah, but I am not connected via ethernet (well, I am, but only as far as
> the external gateway), the router uses a wireless broadband and it cannot
> route ethernet frames, because they are too large.

>> n.e-s.o cannot know that your ISP is weird.
> I don't know what that means. I tell the routing table that my maximum
> transmission unit is 1412. Shouldn't the value provided in the mss field
> be less than the routable mtu?

> ip route change default via 10.0.0.1 mtu lock 1412

Ah, sorry! I assumed you were talking about the /server's/ advertised
mss. Please ignore everything I wrote above.

That points to another bizarre fact: the SYNs from n.e-s.org
advertise a low mss (1380).  But when I connect to it, it tells me:

  win 5792, options [mss 1460,sackOK,TS val ... ecr ...,nop,wscale 7]

This makes me think your ISP (or some equipment of yours) does
something tricky to the packets you're seeing. Like "MSS clamping".

[lots more]

...

Quote:> I will mention my router, just in case it matters. It is an Edimax 3g-6200n
> and I am using the latest firmware (version 2.24g). The firmware is Linux
> based and open source, but it does not provide shell access. On the back of
> the router is a Three Mobile Broadband dongle made by Huawei, which makes
> the connection to the outside world.

And then you have GTP tunneling between the routers (SGSN, GGSN) in
the 3G network. These frequently pull stupid stunts, too.

I don't know ... you also seem to say that IP fragmentation doesn't work
for you.  That's so abnormal that perhaps you should debug that first.
It would be easier though if you had tcpdump access to a machine on
the other side -- since something inbetween is obviously messing with
your packets.

/Jorgen

--

\X/     snipabacken.se>   O  o   .

 
 
 

Linux 2.6.39-rc7 - Wrong Maximum Segment Size being advertised

Post by Mark Hoble » Mon, 23 May 2011 16:28:10



> And then you have GTP tunneling between the routers (SGSN, GGSN) in the
> 3G network. These frequently pull stupid stunts, too.

> I don't know ... you also seem to say that IP fragmentation doesn't work
> for you.  That's so abnormal that perhaps you should debug that first.

I'm trying :)
What is also weird is that my system has been working with previous kernel
versions. It has only just become broken again.

I'll try dropping another 12 bytes off the ip route command and retest:

ip route change default via 10.0.0.1 mtu lock 1400

Am I right in that the below datagram is wrong, because the mss is 1460,
which is above the value of the mtu in the routing table? Shouldn't the
value of the mtu have been factored in here, and a new mss calculated based
on the mtu?

03:43:04.320740 IP venus.markhobley.yi.org.52201 >

Quote:> news.eternal-september.org.nntp: Flags [S], seq 2304145392, win 14600,
> options [mss 1460,sackOK,TS val 47122576 ecr 0,nop,wscale 5], length 0

--
Mark Hobley
Linux User: #370818  http://markhobley.yi.org/
 
 
 

Linux 2.6.39-rc7 - Wrong Maximum Segment Size being advertised

Post by Andy Furnis » Mon, 23 May 2011 19:40:16




>> And then you have GTP tunneling between the routers (SGSN, GGSN) in the
>> 3G network. These frequently pull stupid stunts, too.

>> I don't know ... you also seem to say that IP fragmentation doesn't work
>> for you.  That's so abnormal that perhaps you should debug that first.

> I'm trying :)
> What is also weird is that my system has been working with previous kernel
> versions. It has only just become broken again.

> I'll try dropping another 12 bytes off the ip route command and retest:

> ip route change default via 10.0.0.1 mtu lock 1400

> Am I right in that the below datagram is wrong, because the mss is 1460,
> which is above the value of the mtu in the routing table? Shouldn't the
> value of the mtu have been factored in here, and a new mss calculated based
> on the mtu?

> 03:43:04.320740 IP venus.markhobley.yi.org.52201>
>> news.eternal-september.org.nntp: Flags [S], seq 2304145392, win 14600,
>> options [mss 1460,sackOK,TS val 47122576 ecr 0,nop,wscale 5], length 0

I see the same, but don't know what the behavior should be.
If you set mtu on say eth0 then advertised tcp mss will be affected
(which in a way is setting mtu plus mru for tcp) - maybe the route
setting really just does mtu alone, which in a way is what you asked for
- don't send >1400.

If you want to fix up your wan use iptables mss clamping on the gateway
if you can.

Using ping to try and work out isp behavior can be misleading - you may
just be seeing the settings of the server/network you are pinging and
the pings may get fragged/re-assembled without you knowing.

 
 
 

Linux 2.6.39-rc7 - Wrong Maximum Segment Size being advertised

Post by Jorgen Grah » Mon, 23 May 2011 20:31:25


...

Quote:> I'll try dropping another 12 bytes off the ip route command and retest:

> ip route change default via 10.0.0.1 mtu lock 1400

> Am I right in that the below datagram is wrong, because the mss is 1460,
> which is above the value of the mtu in the routing table? Shouldn't the
> value of the mtu have been factored in here, and a new mss calculated based
> on the mtu?

> 03:43:04.320740 IP venus.markhobley.yi.org.52201 >
> news.eternal-september.org.nntp: Flags [S], seq 2304145392, win 14600,
> options [mss 1460,sackOK,TS val 47122576 ecr 0,nop,wscale 5], length 0

Yes, that seems wrong too. I have only messed with that once, as an
experiment, but I am pretty sure my TCP stack understood it and set
mss in the SYN accordingly. (I used 'route ... mss N' to do it, not
ip(8), but that shouldn't matter.)

/Jorgen

--

\X/     snipabacken.se>   O  o   .

 
 
 

Linux 2.6.39-rc7 - Wrong Maximum Segment Size being advertised

Post by Mark Hoble » Mon, 23 May 2011 20:50:04




>> I'll try dropping another 12 bytes off the ip route command and retest:

>> ip route change default via 10.0.0.1 mtu lock 1400

>> Am I right in that the below datagram is wrong, because the mss is
>> 1460, which is above the value of the mtu in the routing table?
>> Shouldn't the value of the mtu have been factored in here, and a new
>> mss calculated based on the mtu?

>> 03:43:04.320740 IP venus.markhobley.yi.org.52201 >
>> news.eternal-september.org.nntp: Flags [S], seq 2304145392, win 14600,
>> options [mss 1460,sackOK,TS val 47122576 ecr 0,nop,wscale 5], length 0

> Yes, that seems wrong too. I have only messed with that once, as an
> experiment, but I am pretty sure my TCP stack understood it and set mss
> in the SYN accordingly. (I used 'route ... mss N' to do it, not ip(8),
> but that shouldn't matter.)

Hmmm. So if the datagram is wrong, then we have a bug. Presumably this is
a kernel bug, because it has been working for a couple of months before I
upgraded the kernel. Or could it be a library bug? I might have upgraded
some libraries, for the purpose of being able to use the new kernel.

Mark.

--
Mark Hobley
Linux User: #370818  http://markhobley.yi.org/

 
 
 

Linux 2.6.39-rc7 - Wrong Maximum Segment Size being advertised

Post by Mark Hoble » Mon, 23 May 2011 21:25:59



> I see the same, but don't know what the behavior should be. If you set
> mtu on say eth0 then advertised tcp mss will be affected (which in a way
> is setting mtu plus mru for tcp) - maybe the route setting really just
> does mtu alone, which in a way is what you asked for

That would not be very good at all, because it breaks PMTUD. I would
obviously want the MTU to match the route.

Quote:> If you want to fix up your wan use iptables mss clamping on the gateway
> if you can.

I had a quick look through the router configuration settings and I could not
see such an option, so I guess that this facility is not provided.

Quote:> Using ping to try and work out isp behavior can be misleading - you may
> just be seeing the settings of the server/network you are pinging and
> the pings may get fragged/re-assembled without you knowing.

With PMTUD enabled, the Don't Fragment flag is set though, isn't it? So this
would mean that we have a single datagram end to end.

Mark.

--
Mark Hobley
Linux User: #370818  http://markhobley.yi.org/

 
 
 

Linux 2.6.39-rc7 - Wrong Maximum Segment Size being advertised

Post by Jorgen Grah » Mon, 23 May 2011 21:52:59





>>> I'll try dropping another 12 bytes off the ip route command and retest:

>>> ip route change default via 10.0.0.1 mtu lock 1400

>>> Am I right in that the below datagram is wrong, because the mss is
>>> 1460, which is above the value of the mtu in the routing table?
>>> Shouldn't the value of the mtu have been factored in here, and a new
>>> mss calculated based on the mtu?

>>> 03:43:04.320740 IP venus.markhobley.yi.org.52201 >
>>> news.eternal-september.org.nntp: Flags [S], seq 2304145392, win 14600,
>>> options [mss 1460,sackOK,TS val 47122576 ecr 0,nop,wscale 5], length 0

>> Yes, that seems wrong too. I have only messed with that once, as an
>> experiment, but I am pretty sure my TCP stack understood it and set mss
>> in the SYN accordingly. (I used 'route ... mss N' to do it, not ip(8),
>> but that shouldn't matter.)

> Hmmm. So if the datagram is wrong, then we have a bug. Presumably this is
> a kernel bug, because it has been working for a couple of months before I
> upgraded the kernel. Or could it be a library bug? I might have upgraded
> some libraries, for the purpose of being able to use the new kernel.

It seems to me it's more likely that you're doing something wrong --
people would immediately notice if these parts of the Linux kernel
broke! Is it the right route? Are there others?  What does route -ne say?

I also can't help wondering if you have caused all these problems
somehow.  If the ISP breaks things like this, it's commercial suicide
-- unless they somehow manage to keep Windows unaffected ...

/Jorgen

--

\X/     snipabacken.se>   O  o   .