I maintain two cross-connected switches, R1 and R2, which both have a link to an external switch, R3. R1 and R2 both use eBGP to receive routes to various subnets behind R3. Additionally, R1 and R2 use iBGP to exchange routes between each other. The cross-connect between R1 and R2 is considerably larger than the links to R3, and there is enough traffic to completely utilize both links to R3 if traffic is routed appropriately.
I want to use ECMP to divide R1->R3 traffic so that a share of it follows R1->R3 directly and a share of it follows R1->R2->R3. I want R2->R3 traffic to be similarly divided.
R1 and R2 are Cisco NX-OS switches. R1 sees both potential paths to 192.168.240.0/24 (behind R3) and correctly determines that one was received with eBGP and one with iBGP.
R1# show ip bgp vrf vrf-100 BGP routing table information for VRF vrf-100, address family IPv4 Unicast BGP table version is 49, Local Router ID is 10.10.100.2 Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - best2 Network Next Hop Metric LocPrf Weight Path * i10.10.100.0/24 10.10.100.3 100 0 i *>l 0.0.0.0 100 32768 i * i192.168.240.0/24 10.10.100.3 100 0 YYYYY i *>e 169.254.159.2 0 YYYYY i
I tried setting maximum-paths to 2, but Cisco NX-OS switches (and as I understand it, most/all switches) don't consider paths like this equal. eBGP paths are always preferred over iBGP paths and ECMP won't install both in the routing table.
R1# show run bgp router bgp XXXXX vrf vrf-100 address-family ipv4 unicast network 10.10.100.0/24 maximum-paths 2 neighbor 10.10.100.3 remote-as XXXXX password 3 b6d56a77617af8e3 address-family ipv4 unicast neighbor 169.254.159.2 remote-as YYYYY password 3 b6d56a77617af8e3 address-family ipv4 unicast
I tried using "maximum-paths mixed" because my understanding was that this would allow eBGP and iBGP paths to be considered equal.
R1# show run bgp router bgp XXXXX vrf vrf-100 address-family ipv4 unicast network 10.10.100.0/24 maximum-paths mixed 2 neighbor 10.10.100.3 remote-as XXXXX password 3 b6d56a77617af8e3 address-family ipv4 unicast neighbor 169.254.159.2 remote-as YYYYY password 3 b6d56a77617af8e3 address-family ipv4 unicast
In both cases, only the eBGP route is installed in the route table.
R1# show ip route vrf vrf-100 IP Route Table for VRF "vrf-100" '*' denotes best ucast next-hop '**' denotes best mcast next-hop '[x/y]' denotes [preference/metric] '%<string>' in via output denotes VRF <string> 10.10.100.0/24, ubest/mbest: 1/0, attached *via 10.10.100.2, Vlan100, [0/0], 6w2d, direct 10.10.100.1/32, ubest/mbest: 1/0, attached *via 10.10.100.1, Vlan100, [0/0], 6w2d, vrrp_engine 10.10.100.2/32, ubest/mbest: 1/0, attached *via 10.10.100.2, Vlan100, [0/0], 6w2d, local 169.254.159.0/30, ubest/mbest: 1/0, attached *via 169.254.159.1, Po2.100, [0/0], 1w5d, direct 169.254.159.1/32, ubest/mbest: 1/0, attached *via 169.254.159.1, Po2.100, [0/0], 1w5d, local 192.168.240.0/24, ubest/mbest: 1/0, all-best (0x0) *via 169.254.159.2, [20/0], 00:31:48, bgp-XXXXXX, external, tag YYYYYY
How do I either:
- Convince R1 to multipath over the combination of an eBGP and iBGP route, or
- Change what I'm doing to allow for multipathing by some other mechanism?