3

I have two EdgeRouter 4s. I have two /28 blocks of public IP addresses. The EdgeRouters each have an uplink to my ISP, and can ping each other.

I'm trying to setup VRRP, and used this resource as a guide: https://help.ui.com/hc/en-us/articles/204962174-EdgeRouter-Virtual-Router-Redundancy-Protocol-VRRP-

I have removed the two middle octets, but below is my relevant config for both routers. You'll notice that I've assigned a static out of the same /28 to both routers, and then configured the rest of the addresses as virtual.

Both routers can ping each other. 209.x.x.82 can ping 209.x.x.83, and vice-versa.

Primary Router's Config:

set interfaces ethernet eth0 address 209.x.x.82/28
set interfaces ethernet eth0 vrrp vrrp-group 3 advertise-interval 1
set interfaces ethernet eth0 vrrp vrrp-group 3 preempt true
set interfaces ethernet eth0 vrrp vrrp-group 3 priority 200
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.84/28
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.85/28
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.86/28
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.87/28
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.88/28
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.89/28
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.90/28
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.91/28
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.92/28
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.93/28
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.94/28

Backup Router's Config:

set interfaces ethernet eth0 address 209.x.x.83/28
set interfaces ethernet eth0 vrrp vrrp-group 3 advertise-interval 1
set interfaces ethernet eth0 vrrp vrrp-group 3 preempt true
set interfaces ethernet eth0 vrrp vrrp-group 3 priority 100
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.84/28
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.85/28
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.86/28
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.87/28
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.88/28
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.89/28
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.90/28
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.91/28
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.92/28
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.93/28
set interfaces ethernet eth0 vrrp vrrp-group 3 virtual-address 209.x.x.94/28

Here is the primary router's vrrp summary for eth0

$ show vrrp summary
                VRRP    Addr                    Interface       VRRP            
Interface       Group   Type    Address         State           State           
---------       -----   ----    -------         -----           -----           
eth0            3       vip     209.x.x.84/28up              master          
                        vip     209.x.x.85/28
                        vip     209.x.x.86/28
                        vip     209.x.x.87/28
                        vip     209.x.x.88/28
                        vip     209.x.x.89/28
                        vip     209.x.x.90/28
                        vip     209.x.x.91/28
                        vip     209.x.x.92/28
                        vip     209.x.x.93/28
                        vip     209.x.x.94/28

The problem Every time I put the Backup router's config into place, the backup router makes itself the master - when at the same time, the Primary continues to show it is master. So BOTH routers think they are master.

I have tried modifying the backup router to NOT preempt (set interfaces ethernet eth0 vrrp vrrp-group 3 preempt false). Additionally, as soon as I've committed the changes to the backup router, I've tried to force clear its vrrp state, and force it into backup mode, with preempt set to false:

So on the Backup Router, after applying the above config, I run:

$ clear vrrp master interface eth0 group 3
Forcing vyatta-eth0-3 to BACKUP...

And then I run show vrrp summary and I see that Group 3 is in a VRRP State of Backup. But then I wait a few seconds, and run the same show vrrp summary command on the Backup router, and it is in a state of Master again.

How can I troubleshoot why my Backup router is continuing to try to make itself Master, even though each router can ping each other, and even when I've tried to set preempt to false?

For context, it should be noted my upstream ISP uses Cisco arp caching, and if a MAC address gets into their arp cache, I have to wait 3 hours for it to clear before the IP will start working again. That's quite annoying for my own troubleshooting purposes.

Are there additional firewall flows I have to setup on either of the routers in order to allow vrrp communication to work (i.e. do I need anything besides ICMP ping)? Is it possible that my ISP is doing some sort of filtering on their switch? Any other suggestions?

Improve this question
7
  • VRRP is to fool the LAN hosts, not to use on the WAN. The LAN interfaces of the routers must be on the same layer-2 LAN so that they can communicate on the same LAN.
    – Ron Maupin ♦
    Jul 24, 2021 at 15:21
  • I do understand that VRRP was originally intended for LAN interfaces. That said, I do have my WAN interfaces on the same layer 2 subnet. There's a number of folks who have gotten it to work, e.g. community.ui.com/questions/VRRP-on-the-WAN-side-any-caveats/…. I suppose I could ask my ISP to validate that their config have both my router's WAN interfaces on the same vlan, but they are also already aware that I'm attempting to setup vrrp on my WAN interfaces.
    – David W
    Jul 24, 2021 at 15:26
  • "I do have my WAN interfaces on the same layer 2 subnet." No, because layer-2 does not have subnets. It must be the same layer-2 broadcast domain. For example, you need an ethernet switch and connect both interfaces to the switch on the same VLAN (broadcast domain) so that the interfaces communicate on the layer-2 broadcast domain. VRRP is for your LAN to fool the LAN hosts.
    – Ron Maupin ♦
    Jul 24, 2021 at 15:31
  • Interesting. Thank you. So it sounds like my only option for router high availability is to use something like BGP, if I can get my ISP to let me do that.
    – David W
    Jul 24, 2021 at 15:56
  • 1
    If you have two ISP connections, those will be totally independent links -- most likely routed, not switched. VRRP requires a bridged network, not two routed interfaces. BGP is how one gets redundancy in this situation, but I understand how much of an ask that is. (ISPs don't like running BGP with "dumb" customers.) Can you add a picture of your physical setup?
    – Ricky
    Jul 25, 2021 at 4:29

3 Answers 3

Reset to default
1

The EdgeRouters each have an uplink to my ISP

With separate uplinks, there's likely no transparent L2 connectivity between the routers, as required for VRRP.

Just being able to ping between them is not enough. VRRP datagrams use the multicast destination address 224.0.0.18 with IP protocol 112. If those cannot pass, each router assumes the other one is offline and takes mastership.

To make sure, run a packet capture on a WAN port to see if the partner's VRRP packets can be received.

Improve this answer
2

VRRP PDUs use IP protocol 112 as described in RFC 5798 §5.1.1.4.

Are there additional firewall flows I have to setup on either of the routers in order to allow vrrp communication to work (i.e. do I need anything besides ICMP ping)?

Yes, you must allow the protocol 112 packets. If they're being blocked, it would cause behavior like what you've described.

Improve this answer
0
1

A VRRP topology without a direct link between both routers can cause a "Split brain" situation, that I believe you are having in your scenario. As mentioned before, if you don't allow the multicast datagrams to flow between the hosts, the election doesn't work properly and you have issues like this.

On a side note, if you have a router in your local network connected to these edge routers, why not use dynamic routing instead of VRRP? You can set a BGP peer between your local and edge routers and set different weights to each link.

Improve this answer
1
  • Excellent point - more likely OSPF or IS-IS should be available within the DC which could provide more granular routing and include some load balancing.
    – Zac67 ♦
    Jul 28, 2021 at 15:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.