I would like to get some opinions regarding ways in which I can improve a BGP dual-provider, dual-router design. Each provider supplies a /24 public subnet. I will refer to the routers, circuits, subnets, HSRP groups and providers as A and B, respectively. The bandwidth on each circuit is adequate for the entire load.
Current Design
The current design attempts to achieve per-provider symmetry. In a steady state, the intended routing logic is that traffic to/from subnet A transits only circuit A and traffic to/from subnet B transits only circuit B. The circuits would back each other up in a failed state.
The providers only advertise the default route. Outbound routing entails a mix of PBR and HSRP. The routers have no routing between them: No iBGP, no OSPF, no static routing. Instead, there are two HSRP groups tracking the default route. Router A is primary for HSRP group A and router B is primary for HSRP group B. Downstream devices have a default route pointing to HSRP group A and PBR which directs traffic sourced from subnet B to HSRP group B. Inbound routing is influenced with prepending and communities. Subnet A is prepended and communitied on circuit B and subnet B is prepended and communitied on circuit A.
I see much room for improvement in this design. The lack of Internet topology awareness combined with circuit affinity completely eliminates best path selection. There are concerns about the tier designation of the providers and the design has been rationalized as providing ‘acceptable performance’ and simpler to troubleshoot. Indeed, the design couldn’t possibly get any simpler. I have demonstrated that transiting an extra AS adds 6 hops and 63ms (+421%) to the RTT. I would prefer not to settle for acceptable.
Better Design
The better design provides the routers with as much Internet topology awareness as possible. The best path algorithm is left to determine the inbound and outbound routing logic. The circuits would back each other up in a failed state.
The providers advertise the full view. The routers run iBGP and OSPF. HSRP is eliminated. The outbound routing would be purely destination-based best path and the inbound routing would be left to the best path algorithm and transit provider whims.
Now that I type it out, it does seem simpler. At the very least, it took fewer words to explain. There are concerns about asymmetry, but I have seen plenty of asymmetry in the current design. I would think they are probably equally prone to asymmetry and it really doesn’t worry me. We have never seen problems as a result. Presently it is relegated to the realm of ifs, “What ‘if’ we had to troubleshoot something?”
Am I way off base here or did I hit the nail on the head? How have others solved this problem? What would Google do?