Last night we did some upgrades on a system in our datacenter. Among other things moving a few services from physical computers to virtual ones. One of these new machines needed contact with three different physical networks, and even more subnets.
If you don’t want to read about my whole example network, skip to the “fun part”.

In this blog entry I will use some bogus internal network addresses. We had the following:

eth0 directly connected to 10.0.100.0/24
eth1 directly connected to 192.168.0.0/24
eth2 directly connected to 192.168.10.0/24


Our new (virtual) server was configured using 192.168.0.1 as default gateway, via eth1. But we also needed to reach the following networks via eth2:

• 192.168.20.0/24
• 192.168.30.0/24
• 192.168.55.0/24
• 10.50.0.0/16

Configuring this “on-the-fly” is easy. All we have to do is run the following commands as root:

route add -net 192.168.20.0/24 gw 192.168.10.5
route add -net 192.168.30.0/24 gw 192.168.10.5
route add -net 192.168.55.0/24 gw 192.168.10.5
route add -net 10.50.0.0/16 gw 192.168.10.5


As you have guessed, 192.168.10.5 is the gateway being connected to eth2. Now the following is taking place:

Traffic for 10.0.100.0/24 is directly pushed out eth0, no routing needed.
Traffic for 192.168.0.0/24 is directly pushed out eth1, no routing needed.
Traffic for 192.168.10.0/24 is directly pushed out eth2, no routing needed.
Traffic for 192.168.20.0/24, 192.168.30.0/24, 192.168.55.0/24 and 10.50.0.0/16 is pushed to gateway 192.168.10.5 via eth2.
Everything else is directed to gateway 192.168.0.1 via eth1.

Fun Part

To make this routing permanent, meaning it will return upon reboot, we need to store this information somewhere. In this case we’re using CentOS 4, so the file we need to edit is /etc/sysconfig/static-routes. Per default this file doesn’t exists, at least it didn’t on my machine, so I created one and entered the following:

any net 192.168.20/24 gw 192.168.10.5
any net 192.168.30/24 gw 192.168.10.5
any net 192.168.55.0/24 gw 192.168.10.5
any net 10.50.0.0/16 gw 192.168.10.5


Also, check the files /etc/sysconfig/network-scripts/ifcfg-ethx, replace x. Only eth1, in my example, should have a line which says “GATEWAY=192.168.0.1”. If anyone of the other files also has a line which starts with “GATEWAY”, something will most likely go wrong.

I’m not sure how interesting this is for anyone. But at least I hope someone will benefit from it. I might start some more “in-depth” articles about network configuration in the future.

Please leave a comment if you found this useful, or ask questions if there is something I can improve.