3

My situation is that I have 10 Linux servers announcing the same VIP (10.0.0.10 for example) to the router with bird, so the router is doing kind of load balancing with ECMP.

                             +---------------------------+
                             | BGP router (192.168.0.10) |
                             +---------------------------+
                                           ^
                                           |   BGP session
               +---------------------------+---------------------------------+
               |                           |                                 |
VIP: 10.0.0.10 v            VIP: 10.0.0.10 v                  VIP: 10.0.0.10 v
   +----------------------+      +-----------------------+              +----------+
   | Node 1 (192.168.0.20)|      | Node 2 (192.168.0.21) |      ...     | Node 10  |
   +----------------------+      +-----------------------+              +----------+  
      (Linux with Bird)

It works well, but I want to understand how does BGP router know the MAC addresses of clients, as the BGP rfc does not specify how RIB is translated to FIB or Forwarding table.

Does it also depends on ARP protocol?

2
  • Unfortunately, questions about hosts/servers are off-topic here, and your router is off-topic because the manufacturer does not offer optional, paid support. We will answer questions about BGP, but not your specific router implementation or configuration.
    – Ron Maupin
    Mar 29, 2022 at 14:55
  • I'm not asking about router specfic question, just how BGP works :)
    – fwsousjt
    Mar 30, 2022 at 8:32

3 Answers 3

5

Frames, including any layer-2 addressing (remember that MAC addressing is not the only layer-2 addressing), are stripped off the packet at the interface. The routing only sees the packet.

BGP is an application that uses TCP, so the IP packet and TCP segment are stripped off before the data are passed to BGP. BGP never sees the frame, packet, or segment, only the data contained in the segments.

I want to understand how does BGP router know the MAC addresses of clients, as the BGP rfc does not specify how RIB is translated to FIB or Forwarding table.

It happens the same way as any other application. Also, remember that routing protocols, like BGP, do not route packets. Routing protocols are one of three ways a router uses to build its routing table (directly connected networks, statically configured routes, or dynamic routing protocol). Routers route packets based on their routing tables and the packet layer-3 address (IPv4 or IPv6). The layer-2 addresses (MAC-48, MAC-64, DLCI, VPI/VCI, etc., or no layer-2 addressing for things like PPP) are stripped off or added at the interface as an incoming frame is stripped or a frame is added for an outgoing packet.

3
  • Thanks for your answer. But I'm still not clear how traffic is routed to backend nodes. Suppose the router (ROUTER_MAC) determined to forward PKT1 to Node 2 (192.168.0.21) with MAC2, how does the forwarding looks like? Looks like Ether(src=ROUTER_MAC, dst=MAC2)/IP(src=ORIGINAL_CLIENT_IP, dst=10.0.0.10)/PAYLOAD or other way?
    – fwsousjt
    Mar 29, 2022 at 16:56
  • It is the same way as any host sending works. If the routers use an FHRP, then ARP returns the MAC address of the VIP, and that is what is used to build the frames at the outgoing interface.
    – Ron Maupin
    Mar 29, 2022 at 16:58
  • 2
    @fwsousjt The router uses ARP to determine the MAC address corresponding to the next hop address, which is 192.168.0.21. BGP has nothing to do with this - it works the same with a static route.
    – user253751
    Mar 29, 2022 at 23:19
6

You state that you're using ECMP for load balancing. The routing table (FIB) fed by BGP likely looks something like this:

10.0.0.0/24 gateway 192.168.0.20 metric x
10.0.0.0/24 gateway 192.168.0.21 metric x
10.0.0.0/24 gateway 192.168.0.22 metric x
...

Accordingly, the top-level router does not use the VIP 10.0.0.10 for forwarding over layer 2. Instead, it chooses a gateway from its FIB (by round robin, load, ...) and uses that gateway's MAC address for frame encapsulation delivery.

Instead of BGP you could use any other routing protocol, or even static routes with the caveat that a failed/offline server/gateway would stay in the routing table.

You should note that this kind of stateless load balancing - without 'stickyness' - is only suitable for web servers that don't maintain a client state - e.g. simple content servers or when the entire client state is maintained on the client and tracked by a cookie.

2
  • Thanks. Does the forwared packet look like this Ether(src=ROUTER_MAC, dst=MAC2)/IP(src=ORIGINAL_CLIENT_IP, dst=10.0.0.10)/PAYLOAD as I commented for previous answer by @Ron Maupin? If that's the case, is ARP is used to retrieve the MAC address of Node2 (192.168.0.21)?
    – fwsousjt
    Mar 29, 2022 at 17:00
  • 1
    The packet is addressed to IP 10.0.0.10 on L3 and to the MAC address of either Linux box on L2 (for the gateway chosen by ECMP). ARP has been used to resolve the gateway's address.
    – Zac67
    Mar 29, 2022 at 17:10
0

In your case, I would assume that the Linux servers are advertising routes with a destination of 10.0.0.10/32 and a NEXT_HOP attribute reflecing thier IP address on the LAN.

The last two paragraphs of section 5.1.3 of rfc4271 specify how the BGP NEXT_HOP is translated to a local interface and immediate next hop.

In your case, the recursive route lookup finds the implicit route created when the IP address and subnet mask were assigned to the interface of the router. This implicit route does not specify a next hop, so the BGP NEXT_HOP is used as the immediate next hop.

Assuming an Ethernet like interface, the "immediate next hop" is looked up in the ARP table to determine the MAC address. That isn't really a BGP thing though, it's an IP over Ethernet thing.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.