No route to host for IPs in same subnet being routed by bgp

Question

I have set up BGP to do some custom routing for a subset of addresses in the range 192.168.1.0/24.

I have verified that the BGP is working by testing ping/curl on a different host in a different subnet (in the 192.168.2.0/24 range), so I think that the routing configuration is working correctly. I have also verified that the IP is accessible from pfsense.

However, pings and curls fail with a no route to host message when attempting to hit an IP from a host in the same subnet.

curl -v http://192.168.1.11
* Rebuilt URL to: http://192.168.1.11/
*   Trying 192.168.1.11...
* connect to 192.168.1.11 port 80 failed: No route to host
* Failed to connect to 192.168.1.11 port 80: No route to host
* Closing connection 0
curl: (7) Failed to connect to 192.168.1.11 port 80: No route to host

Arp is showing (incomplete) for the IP being routed to, which I though was strange but maybe is normal?

arp -n
Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.1.1              ether   40:62:31:00:52:40   C                     eth0
...
192.168.1.11                     (incomplete)                              eth0

Here's the routes for a host in the subnet.

ip route
default via 192.168.1.1 dev eth0
172.17.0.0/16 dev docker0  proto kernel  scope link  src 172.17.0.1 linkdown
172.18.0.0/16 dev br-f1b94ce4ec9c  proto kernel  scope link  src 172.18.0.1 linkdown
192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.154

I stumbled across a post that I thought was describing my situation, where IP redirect was causing the hosts not to use the route, but after updating the net.inet.ip.redirect and net.inet6.ip6.redirect values (and rebooting pfsense) I am still left with the same problem. Is there some way to force IPs in the same subnet to use the correct route?

Edit: Updated with network topology

Answer 1

The problem is because the destination is in the same subnet the client arps for the destination IP, not the IP of your pfsense box. By default your pfsense box does not respond to arp requests for that IP, so the traffic never reaches the pfsense box.

Normal practice is to use IPs in seperate subnets for different networks, sometimes however you are stuck with IPs you can't change.

I'm not a pfsense user myself, but I believe the fix if you really need to keep your current addressing is to add the IP to the interface on the pfsense box as a "proxy arp" IP. This will bring the traffic into the pfsense box where the /32 routes can route it.

http://pfsensesetup.com/tag/proxy-arp/

P.S. note that "no route to host" can appear in at least two different scenarios. One is where there is no route in the routing table. The other is where there is no arp response for the "next hop" IP that was determined from the routing table.

Answer 2

The router and BGP have nothing to do with a problem where the source and destination are on the same network. Destinations in the same network as the source do not use a router, they are delivered directly from host-to-host by the layer-2 protocol. The ICMP message and lack of an entry in the ARP table means that the destination host did not respond to an ARP request to tell the source host what the destination MAC address is, and the ARP request timed out so network stack in the source host generated and ICMP error back to the application.

You have not properly described the network or provided a diagram or other information to be able to help further.

---

Edit for your diagram:

The problem is that you do not understand the network layers and what routers do as opposed to switches and how hosts address layer-2 frames. Routers route between networks, not from a network back to the same network, so each router interface must be in a different network. Router strip off the layer-2 frames to get to the layer-3 packet, and they look in their routing tables to determine where to send the packets. Router do not forward broadcasts, which is what ARP uses.

A host will mask the destination address to see if the destination is on the same network. If it is, it tries to use ARP to find the MAC address and sends the layer-2 frame directly to the destination. Only if the destination is on a different network will the host address the frame to the configured gateway (router). The problem you have is that the layer-2 frame never gets sent because the destination host doesn't respond because it is on the other side of the router.

---

Also, routing protocols do not route packets. Routing protocols simply exchange route information with neighbor routers. Routers use their routing tables to route packets. Routing protocols are one way to populate a routing table. Directly connected networks are another way, and that will take precedence over equivalent routes learned by a routing protocol. A router will automatically route traffic between its directly connected networks, even if it doesn't run a routing protocol.