Routing traffic out different links from the same BGP AS

Question

I have two sites, A and B, in BGP AS 65000, and a third site, C, in AS 65001. All three sites have connectivity via carrier MPLS and there is internal connectivity between sites A and B. I am trying to influence BGP so that traffic from site A to site C will be routed via site A's MPLS link, and traffic from site B to site C will route via site B's MPLS link. The topology is similar to what is described in this example.

The edge routers at sites A and B will both see their own MPLS link as the best path, because EBGP routes are preferred over IBGP routes. However, routers further inside AS 65000 will all prefer either one link or the other. My goal is to force all the routers at either site to prefer the closest link. (Unfortunately, I'm not able to split the two sites into separate ASes at this time.)

Is there a sane way to accomplish this while still allowing failover connectivity to site C between the site A and B links?

Edit: I should have noted that there is no IGP in use here. In fact, the networks at each site exist within a VRF as part of a much, much larger network. As such, any solution needs to be rely entirely on BGP.

Answer 1

Is there a clear definition between site A and site B?

If so then I would look to define a policy on the edge routers to inject a community when receiving routes from the carrier MPLS.

Once this community has been put on the prefixes (say 100:1 for site A and 100:2 for site B) you could then add a policy to each of site A's routers to increase the LP for any routes with community 100:1 and likewise for site B with community 100:2.

This solution would match the requirement of only using BGP and also would be flexible enough to still allow B to use A's uplink if it lost its own uplink to the carrier.

Answer 2

When I did something similar I did not use eBGP between the routers. I had the BGP speaking router only send the default route down to the Site A and Site B routers via OSPF and then redistribute the OSPF routes into BGP. On the link between the two sites I applied an OSPF cost.

This allows Site A to have routes to its networks, Site B networks, and a default route to get out of Site A (the OSPF cost keeps the default route from Site B as a secondary option should you lose the Site A link). Site A will talk directly to site B, not using MPLS unless there is a link failure between the sites.

As well, the OSPF costs translate into the BGP MED during redistribution which causes the carrier MPLS network to prefer sending Site A traffic directly to Site A but also use Site B to get to Site A if need be.

Answer 3

I feel some information is missing. Why are edge routers not preferring best path already?

Do you have full-mesh iBGP in 65000? Or are you using route-reflection?

If you have full-mesh iBGP, then each edge router learns route from both [AB] and will resort to comparing IGP cost to next-hop, which should translate to closest edge box.

If there is route-reflection in place, it'll only reflect best route from its own POV, which may remove best-path-forwarding. This is fixable also by adding another RR which will choose the other route as best, then edge boxes once again can choose the best route. If fixing RR is not possible like this. You could add same IPV4 address in both border routes loopbacks, and when border routers advertise prefix to RR, they'd set next-hop to this anycast address. Then even after reflection, you'll follow IGP to closest border.

Answer 4

If you have control over the IGP:

What we've done is to only have eBGP routers in the iBGP mesh. The rest of our internal routers are OSPF routers. We redistribute from BGP to OSPF within each AS. Our setup is a little different from yours, but this should allow OSPF's cost metric to influence which path traffic takes by directing it to the nearest eBGP router.

If you do not have control over the IGP:

Perhaps you can apply an import policy to the internal routers to make them statically prefer the route announcements of the desired exit eBGP router. So for example, if iBGP contains two announcements for a given prefix (one from the site A router and one from the site B router, but only one is installed into the RIB) then you could local-pref up the desired announcement as it comes in. I would have to lab it up, but I don't see why it wouldn't work.