ISP HA using GLBP vs BGP

Question

I have a tech startup environment where I'm replacing consumer-grade routers with enterprise-grade.

Now, they are asking if we can have dual ISPs to minimize downtime and any disruption.

I have:

• VLAN 70: 10.70.0.0/16 with 10.70.0.1 default gw
• VLAN 5: 10.2.10.0/24 with 10.2.10.1 default gw
• Mgmt VLAN is 70 where router has 10.70.0.2, layer 3 switch has 10.70.0.3, and so forth.
• Virtual-IP GW of 10.70.0.1 or iBGP?

Usually, I use GLBP with two Cisco routers for CEF load-balancing, but we are not sticking with Cisco this time. Shall I use VRRP between vendors or use BGP with our ISP gateway being iBGP neighbors with each other?

Requirements:

1. Later on, we want hybrid cloud with AWS and it uses BGP that I'm eventually going to set a default route in OSPF for.
2. Minimize CPU on routers
3. We are very ARP Layer 2 loop prone due to multiple adapters on servers acting unexpectedly.

In the long run, which would provide simple yet effective HA and load balancing?

Answer 1

If you have a single layer-3 switch to which the LAN connects, you don't need an FHRP for the LAN. The question then becomes how to load balance from the layer-3 switch to the Internet.

If you need the layer-3 switch to know about the specific routes advertised from each ISP in order to make intelligent decisions about which direction to route particular packets, then BGP would certainly be the right choice.

Any FHRP will need to have a layer-2 connection between the routers. That would make the uplinks from the layer-3 switch be layer-2 links, but that doesn't seem to be how the layer-3 switch should be used.

I really don't see the need for two separate routers on each ISP link. That could be handled by a single router.

Another way to go is to have the links from the layer-3 switches toward the Internet be routed links, and just use the default route on each one. The default routes could be equal cost, and, depending on your layer-3 switch, could naturally load balance.

You should think carefully about load balancing. You can end up with asymmetric traffic flows which may cause problems. You can also slow down because of the out of order packet delivery problem. There is no guarantee that to and from a particular place on the Internet is equidistant from your site on each of the ISPs, or even that to a place and from a place will use the same ISP (asymmetric).