0

I have multiple servers which share 2 networks. That is the LAN Network at 192.168.1.0/24, aswell as the Backbone Network at 192.168.5.0/24.

                       ┌────────────────┐
                       │     Server 1   │
                       │LAN     Backbone│
                       │ |         |    │
                       └─┬─────────┬────┘
                         │         │       (Backbone)
                         │         │   192.168.5.0/24
┌────────────────┐       │         │      │
│Notebook/Client ├───────┤         │◄─────┘
└────────────────┘       │         │
                   ┌────►│         │
                   │     │         │
                   │   ┌─┴─────────┴────┐
                   │   │ |         |    │
                   │   │LAN     Backbone│
       192.168.1.0/24  │     Server 2   │
       (LAN)           └────────────────┘

Now, my Server 1 should route the packets between the subnets. It is a PfSense Firewall.

And well, it does that. My Notebook/Clients a can connect to the server 2 on its Backbone interface and get responses.

Now, this has been written online multiple times, the connection for me drops after one minute.

As I have read somewhere else, this happens due to the fact that the server 2 sends the packet not back over the Server 1 (Firewall).

Meaning, while the packet to the server goes through the Server 1 and then over the Backbone to Server 2, all responses of Server 2 can go directly over LAN to the client, without passing through the firewall again.

Essentially, the Server 1 is never receiving a Acknowledgement or anything. Since its a stateful firewall (right?), it just drops the connection since it never knew the connection was actually established.

Now, the LAN/Backbone configuration has to stay this way. I need both interfaces operating at the same time.

EDIT:
TLDR: I want to avoid using asymetric routing. According to @Zac67's comment, it is not possible to use stateful firewalls and asymetric routing. Can I tell Linux that it should use the original interface for sending back the requests?

3
  • Asymmetric routing doesn't work with firewalls using stateful inspection. You need to make up your mind and configure routing and firewalling accordingly. Most likely, you can adjust the (default or specific) routing on the host. Note that host configurations and issues are explicitly off topic here.
    – Zac67
    Dec 23, 2022 at 16:11
  • Please edit the question to include a good network description or diagram, the network device models, and the network device configurations. It is difficult to see why the hosts will not send directly to the server if they are on the same network. Also, remember that host/server configurations are off-topic here.
    – Ron Maupin
    Dec 23, 2022 at 17:38
  • I have edited the question and added a Ascii-Diagram of the network structure. Dec 23, 2022 at 17:56

2 Answers 2

3

This is all working "as designed". The client doesn't know anything about Server 2's configuration, so to reach it (and any other 5.x) the default route to Server 1 applies. When that traffic arrives at Server 2 (src:1.client dst:2.server2), it doesn't know that traffic arrived via Server 1.[*] Since Server 2 has a direct connection to 1.x, it will send any reply directly into that network. Server 1, being a firewall, will be unhappy with this asymmetry -- not seeing ACK's.

The best way to "fix" this is tell the client it can reach 5.server2 via 1.server2; thus bypassing the firewall. [/32 route is more specific than the /24 "connected" LAN, or /0 default route] You don't want to tell Server 2 to send all of 1.client back to 5.server1, because that would break 1.client to 1.server2 communications. (trading one asymmetry for another.)

Of course, the more common answer would be "NAT". If the firewall rewrites the source of 1.client to 5.server1, Server 2 will send the traffic back to Server 1. This is "source NAT". (I've done that a few times to address some "backdoor" asymmetries -- the terminal server needs to be accessible on any interface, from any network... traffic must return by the same path.)

[*] Layer-3 doesn't care about layer-2, so the fact that it came from Server 1's MAC is irrelevant.

1
  • The idea here is to access the servers webdashboards/configurations only over the management/backbone interfaces. I have disabled the LAN-Interface for the Hypervisors, so the server wont route the traffic the wrong way back. I actually assumed that PfSense was doing SNAT while routing traffic between interfaces, therefore i was confused. Thanks! Dec 24, 2022 at 11:33
0

You need to set up a static route on Server2 for 192.168.1.0/24 via the PfSense with a better metric (=lower value) than its direct connection to LAN.

Since LAN clients seem to be supposed to use the firewall, that direct connection circumvents that. Since it's also messing with stateful inspection, you might want to remove that connection altogether.

5
  • The idea behind the "parallel" servers are my proxmox hosts, which can have their virtual machines transferred between them for e.g. loadbalncing/failover etc. Therefore i cannot remove that connection. I will try to edit the metric on the nodes. Dec 23, 2022 at 18:06
  • Well, you could use a different IP subnet (or VLAN altogether) for that, then it wouldn't mess with your routing. You should really consider a dedicated VLAN and subnet for VM migration.
    – Zac67
    Dec 23, 2022 at 19:37
  • You cannot override "directly connected" with metrics.
    – Ricky
    Dec 23, 2022 at 21:51
  • @Ricky ... but you should be able to add an additional route with lower metrics. But that's only a workaround, a better design with dedicated VLANs doesn't require that.
    – Zac67
    Dec 23, 2022 at 22:30
  • 1
    You can add a more specific route (/32, pair of /25's), but you cannot override the automatic /24 for the connecting interface.
    – Ricky
    Dec 24, 2022 at 1:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.