I currently have a set of four virtual hosts on a private network, with a fifth host running pfSense to act as the gateway.

When communicating with hosts outside of the private network, all traffic is routed via the pfSense gateway ( However, when two hosts on the same subnet communicate, they do not do so via the gateway. I have gathered that these hosts realise they are on the same subnet and switch and opt to communicate via MAC address w/ARP rather than using the IP gateway I have set up.

I would like these hosts to communicate via the gateway so that I can monitor the traffic between the hosts on the private network. I have attempted to set static routes, for example: ip route add via dev eth0

The first traceroute subsequent to this will correctly route via the gateway:

root@file-server ~ # traceroute
traceroute to (, 30 hops max, 60 byte packets
 1 ( 0.241ms 0.287ms 0.274ms
 2 ( 0.559ms 0.567ms 0.574ms

If I then perform the traceroute again however, the route stops using the gateway.

root@file-server ~ # traceroute
 1 ( 0.345ms 0.376ms 0.362ms

My guess would be that perhaps my host has populated its ARP table with the MAC address of the traceroute target after the initial traceroute, and opts to use that for communication in future.

If that is the case, I am not sure how to counter it. I believed that my static route would take precedent over any other decisions taken by the system. I would like to research further, but I am not actually sure where to look to solve this issue as I am not sure why my static route is not adhered to.

Can someone please provide insight as to why my static route is only adhered to with my initial traceroute, and how I would go about forcing a static route to the gateway for same-subnet communications?

**Edit: ** I should add that this set-up is for experimentation, so I am not restricted to solely things that would be a good idea in the real world.

2 Answers 2


You would have to capture packets to be sure, but I suspect what is going on is an ICMP Redirect.

If a Router receives a packet for which a better next-hop exists, it can send an ICMP Redirect to the original sender to inform them to use the better next-hop when trying to speak to that same destination.

The initial sender will update its routing table temporarily with the new information. I believe for 10 minutes, but this is probably different with each OS/vendor/implementation.

There is generally a way to disable ICMP redirects at the Router. And there is more than likely a way to disable accepting ICMP redirects on the Host. If not specifically, then indirectly by blocking it with an ACL or iptables entry.

You can read more about ICMP Redirects here: http://www.cymru.com/gillsr/documents/icmp-redirects-are-bad.htm

The are now-a-days considered a security vulnerability, but when they were created they were based on good intentions. As the Internet got bigger, and less trust-able, however, their use has been significantly reduced -- and discouraged, for that matter.

  • 1
    That was exactly the problem, thanks! I hadn't heard of ICMP redirects before. I'll read into them a bit.
    – Joe
    Commented Feb 20, 2016 at 21:08
  • 1
    The problem with relying on disabling ICMP redirects is that your hosts are the ones who make the decision whether to use the gateway or not (with routes). I know you are in a lab so it does not matter, but if you are doing that for security your goal is not achieved. :) See my answer for a way to force your hosts to use the gateway. Commented Feb 23, 2016 at 8:22

You want to look into two features called private VLAN and ARP proxy.

When your host's VLAN is configured as a private VLAN, the switch will forbid your host (let's assume it has IP from communicating with other hosts in the same VLAN, as for instance When I say forbid, I mean that it will not forward ethernet frames incoming from a host in the VLAN to another host in the same VLAN. That means no ARP, no ICMP, nothing.

That's a bit against everything we learned in networking class but that's how it works. Bear with me because that is not all there is to know with private VLAN.

The VLAN where our two hosts (host .3 and .4) are configured is known as the secondary VLAN. You can configure a port (typically the gateway's) to be a little special : this port will be in another VLAN, the primary VLAN. Both VLAN are linked somehow (configuration varies with vendors) in the switch's config. A host in the primary VLANs will be able to communicate (i.e. the switch will allow frames to be forwarded) with any hosts, whether they are in the primary or the secondary VLAN. And for the return frames : hosts in the secondary VLAN -while still unable to talk to each other- are allowed to send frames to hosts in the primary VLAN.

From an IP point of view, all hosts of both VLANs should be in the same subnet.

So there we are. Our hosts ( and .4) cannot communicate between them but can communicate with the gateway (, and, per extension, the rest of the internet.

That can be very useful for hosting providers that do not want different clients to communicate and do not want to waste IP addresses in a lot of /30 networks. But that is not what you want. Yet.

The twin feature of private VLAN is ARP proxy. That is just a config on the gateway to instruct it to respond on ARP requests originated from host .3, and asking for's MAC, with its own MAC address. The gateway then initiate its own ARP request to This time host .4 sees it (the gateway is in the primary VLAN, remember?) and responds.

This way, the gateway is performing some sort of legitimate man-in-the-middle between your two host, and can control the traffic. It may perform an ICMP redirect, you might want to disable that.

That's it. You can now control traffic between hosts in the same subnet. Well, you need a switch with the correct features, of course ! ;)

(or you can use two different VLAN, as Ron outlined.)

EDIT : some more information about how private VLANs work.

The primary (say VLAN ID 10) and secondary VLAN (ID 20) have different 802.1q numbers.

A frame from the gateway will travel your switched network tagged with ID 10. Switches will then forward your frame to trunk ports, and access ports in VLAN 10 and 20, because of the relation between primary and secondary VLANs that is configured on the device.

A frame from an isolated host will travel your network tagged as VLAN 20. It will be forwarded on trunk ports, and access ports in VLAN 10 but not access ports in VLAN 20.

There is no communication involved between switches because none is required : it is only a matter of local treatment. But it is advised to have the same primaries/secondaries config on all switches, or expect headaches.

If you send frames to your hosts tagged, you will have to play with allowed vlan and some VLAN mapping magic. Refer to your vendor's doc.

Also, I only talked about isolated secondary VLANs. There are also community secondary VLANs, but that is for another time/another edit.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.