Very Computer

I have a question about routing.

We have a network layout as in the diagram below. We have a linux server
connected to a Cabletron FN10 switch via a 100Mbs ethernet link(eth0). The
server will soon be connected to the outside world via a 10mbs ethernet
link (eth1) to a CISCO router. As you can see the Linux server has two
ethernet cards, one for the the 10 Mbs link to the outside and one for the
local 100Mbs link to the switch.

I have been reading all I can find about routing with Linux and I have
worked out how to talk to the intenet via the 10Mbs card but I am having
problems getting the internal networks (203.28.240.0, 203.28.241.0,
203.28.242.0) to talk to each other properly.

                                  ---------> University ---> Internet
                                 |
                                 |        |------->
                                 |        |------->
                    --------> Cisco Router|------->  Other nearby sites
                   |                      |------->
                   |                      |------->
                   |10 Mbs Ethernet eth1
  _________________|________________________________
 |                 |                                |
 v              -------   Linux Server/Router       v
Our            |       |  Firewall                 Our
Site            -------   203.28.241.1             Site
                   |
                   | 100 Mbs Ethernet eth0
                   |
               ----------
              |          | Cabletron FN10 24 port ethernet switch
               ----------
               ||||||||||
        ------- |||||||| ------------------
       |  ------ |||||| ----------------   |
       | |  ----- |||| ----------       |  |
       | | |  ---- || -------   |       |  |
       | | | |  ---  -----   |  |       |  |  
       | | | | |          |  |  |       |  |
       | | | | |          |  |  |       |  |
      [][][][][]         []  [] []      [] []  <- hubs with local computers
        Admin             Student1     Student2  * off them
     203.28.240.0      203.28.241.0   203.28.242.0

At present the Macintosh machines can quite happily use Appletalk to
communicate to each other throughout the entire campus. I believe this is
because they use the MAC address of each machine and the switch will
happily pass their packets. The problem occurs with IP communications. As
expected at present computers on each network can all talk to each other
happily, even if it means going via the switch , but they cannot talk to
computers on other networks (ie a computer on 203.28.241.0 cannot reach a
computer on 203.28.242.0). I have set up the linux box with the commands:

route add -net 203.28.240.0 netmask 255.255.255.0 dev eth0
route add -net 203.28.241.0 netmask 255.255.255.0 dev eth0
route add -net 203.28.242.0 netmask 255.255.255.0 dev eth0

This seems to allow a few computers to communicate across network
boundaries and the server is able to access all the networks. The problem
is that not all computers can communicate across the network boundaries.

I know the normal configuration is to have a separate ethernet card for
each network. Setting things up for a card for each network seems to be
well documented and obvious. That is what will happen for the connection
to the outside world, but we need to get working internal routing between
three networks via one ethernet card and a switch.

Can anyone help? Have I missed something, do I need to use a dummy device
or something, do I need to assign more than one IP number to the etho
ethernet card?

Richard

--
Richard Wraith
Department of Mechanical and Manufacturing Engineering
University of Melbourne
Parkville     3052
Victoria      Australia
Tel: (+61 3)/(03) 9344 7249
Fax: (+61 3)/(03) 9347 8784

WWW Home Page http://www.veryComputer.com/

I do however pose what I consider to be a more awkward question
towards the end of my answer.

Quote:> v              -------   Linux Server/Router       v
>Our            |       |  Firewall                 Our
>Site            -------   203.28.241.1             Site
>                   |
>                   | 100 Mbs Ethernet eth0
>                   |
>               ----------
>              |          | Cabletron FN10 24 port ethernet switch
>               ----------
>               ||||||||||
>        ------- |||||||| ------------------
>       |  ------ |||||| ----------------   |
>       | |  ----- |||| ----------       |  |
>       | | |  ---- || -------   |       |  |
>       | | | |  ---  -----   |  |       |  |  
>       | | | | |          |  |  |       |  |
>       | | | | |          |  |  |       |  |
>      [][][][][]         []  [] []      [] []  <- hubs with local computers
>        Admin             Student1     Student2  * off them
>     203.28.240.0      203.28.241.0   203.28.242.0

>At present the Macintosh machines can quite happily use Appletalk to
>communicate to each other throughout the entire campus. I believe this is
>because they use the MAC address of each machine and the switch will
>happily pass their packets. The problem occurs with IP communications. As
>expected at present computers on each network can all talk to each other
>happily, even if it means going via the switch , but they cannot talk to
>computers on other networks (ie a computer on 203.28.241.0 cannot reach a
>computer on 203.28.242.0). I have set up the linux box with the commands:

>route add -net 203.28.240.0 netmask 255.255.255.0 dev eth0
>route add -net 203.28.241.0 netmask 255.255.255.0 dev eth0
>route add -net 203.28.242.0 netmask 255.255.255.0 dev eth0

>This seems to allow a few computers to communicate across network
>boundaries and the server is able to access all the networks. The problem
>is that not all computers can communicate across the network boundaries.

Try giving eth0 multiple addresses, one in each subnet:

ifconfig eth0 203.28.241.1 netmask 255.255.255.0
ifconfig eth0:1 203.28.240.1 netmask 255.255.255.0
ifconfig eth0:2 203.28.242.1 netmask 255.255.255.0
route add -net 203.28.240.0 dev eth0
route add -net 203.28.241.0
route add -net 203.28.242.0 dev eth0

Note: You don't should usually need to give the device and mask
explicitly on the "route" command.  I have just taken a look at
ip_forward.c I suspect that the kernel won't generate ICMP_REDIRECTS
if you have your routing table referring to eth0:1 and eth0:2.  Maybe
I'm wrong about this.  Maybe it's a bug.  Maybe there's a good reason
for doing it this way.  Does anyone care to comment?

Note: If you don't have 2.0.x kernel with IP aliasing you will have to
play about with dummy devices and proxy ARP.

Next make sure that machines on 203.28.240.0 think that their router
is 203.28.240.1 not 203.28.241.1.

--

 .  _\\__[oo       from       | Phones: +44 121 471 3789 (home)

.  l___\\    /~~) /~~[  /   [ | PGP-fp: D7 03 2A 4B D8 3A 05 37...
 # ll  l\\  ~~~~ ~   ~ ~    ~ | http://www.veryComputer.com/~bam/

Quote:>I have just taken a look at
>ip_forward.c I suspect that the kernel won't generate ICMP_REDIRECTS
>if you have your routing table referring to eth0:1 and eth0:2.  Maybe
>I'm wrong about this.  Maybe it's a bug.  Maybe there's a good reason
>for doing it this way.  Does anyone care to comment?

If you configure the way I said with the routing table on the Linux
box using eth0 for all outgoing packets then the Linux box will
generate ICMP_REDIRECT messages so only the *first* packet from each
machine will go via the router.  After that all other packets will go
direct (assuming the clients are configured/capable of respecting
ICMP_REDIRECT from their router).  This achieves the same as the other
person suggested but allows for machines that do not support
non-byte-aligned netmasks.

If on the other hand you don't explicitly set the routes via eth0 and
let them default to eth0:1 and so on then my reading of the kernel
sources is that while the kernel will still route the packets back out
the same physical interface it will not realise that it is doing so
thus there will be no ICMP_REDIRECTs generated.  This means all
packets between subnets will go via the router and thus can
potentially be filtered and/or accounted.

I've not had it confirmed that I have correctly understood
ip_forward.c, however if I have correctly understood ip_forward.c and
the reason why it behaves the way it does I think a comment should be
inserted into the source code to explain why it does not consider net
aliases as candidates for ICMP_REDIRECTs so that some well-meaning
person doesn't "fix" it in future.

Note: I wouldn't trust this mechanism for security unless you are
really sure people can't alter their subnet masks.

--

 .  _\\__[oo       from       | Phones: +44 121 471 3789 (home)

.  l___\\    /~~) /~~[  /   [ | PGP-fp: D7 03 2A 4B D8 3A 05 37...
 # ll  l\\  ~~~~ ~   ~ ~    ~ | http://www.veryComputer.com/~bam/