static and default routes on unixoide OS question

Question

I have a question concerning static routes and default gateways. I am trying to rebuild an exercise similiar to an older CCNA question (https://www.omnisecu.com/cisco-certified-network-associate-ccna/how-to-configure-static-routes-and-default-routes.php) with OpenBSD. So no CISCO involved. I am already stuck adding the second router and setting default routes ...

I am not involving any firewall rules. Really only at the routing layer. And yes, ip forwarding is activated on the boxes. My setup is currently with OpenBSD 6.6, but the logic certainly applies to other BSD and Linux as well. In a simple example I have a standard setup of two (IPv4) networks connected by a router:

|-----|   |---------|
| pc1 |---| router1 |---
|-----|   |em0   em1|  |
          |---------|  |
                       |
                    |-----|
                    | pc2 |
                    |-----|

I can set the default route of pc1 to the IP addresses of router1's em0, and on pc2 to the IP address of router1's em1 network card. Then I can ping each other.

I extend my setup with another router (pc3 could also be another router...). I set the default routes (goal is to have default traffic going from pc1 and pc2 to pc3), and cannot ping between pc1 and pc2 without adding a static route:

|-----|   |---------|     |---------|   |-----|
| pc1 |---| router1 |-----| router2 |---| pc3 |
|-----|   |em0   em1|  |  |em0   em1|   |-----|
          |---------|  |  |---------|
                       |
                    |-----|
                    | pc2 |
                    |-----|

Setup of default routes:
  pc1     -> IP address of em0 on router1
  router1 -> IP address of em0 on router2
  pc2     -> IP address of em0 on router2
  router2 -> IP address of pc3

I think I get this. I can observe with tcpdump, that ping echo request leaves pc1, goes through router1 and reaches the network card of pc2. As the default route of pc2 is pointing to router2, the ping echo reply is sent to router2, who doesn't know what to do with the IP address of pc1, and ignores the package. So far ok. So I can set a static route on the pc2, like this

route add -inet {network of pc1} {IP address of em1 on router1}

and then pinging back and forth works. But setting static routes on all pcs in the same network as pc2 doesn't seem obvious to me.

So I thought I could set the same static route on router2. My understanding was, that when router2 receives the ping echo reply from pc2 (with a destination network address of pc1), router2 would forward the packages to router1. But obviously it doesn't. I can't see with tcpdump any ping echo replies arriving at router1's em1 interface. As with pc2 before, when on the router2 console, pinging pc1 works. Shouldn't a static route on router2 allow to forward {pc1 network packeges} to router1, independant from it's origin? I'm sure I am missing something obvious here. What is my missing piece?

Answer 1

Many routers do not route a packet back out of the interface it was received. They may require explicit configuration to do that.

Routing "backwards" is inefficient since those packets need to cross a subnet multiple times. Your scenario isn't very practical since most often, you'd have a single router taking care of all routing into and out of a subnet. Accordingly, the subnets from PC1, PC2, and PC3 would all be connected to Router1, making the default gateway an obvious choice. Connecting Router2 behind Router1 would be a more practical example.

If multiple routers do sit next to each other, a host in the subnet should be able to make a routing choice on its own. If you don't want to set up static routes manually or perhaps by policy, you could use DHCP to configure them (depending on the host capabilities).