I drew a diagram for the network to make it easy to understand my network issue.
- Site B,C and D communicate through MPLS and in case of failover they communicate through VPN tunnels.
- Site X communicate with Site B,C and D through MPLS only.
- Site A is a new site which has a firewall and P2P -EPL to SITE B.
I configured Site A so that traffic to Sites B, C, D, and X go through P2P and in case of failover it goes to Firewall.
Case 1: (Traffic goes through P2P)
OSPF is enabled (L3 switch) in (Site A), so all networks of (site A) will be advertised and see all other sites through MPLS. >>>> No issues and works like a charm!
Case 2: (Failover to Firewall)
In case of P2P is down, OSPF will not take an effect and traffic from Site A to Sites B, C, and D will go through Firewall, Site X will not be visible to Site A.
Solution: Since OSPF is enabled in Sites B, C and D with redistribute subnets, I added a static route for site A in each of the three sites and point it to the firewall for each site. Now site X can see site A. Awesome!!!
Case3: (Fall back to P2P)
Here is my issue, when P2P goes up OSPF in site A will take an effect again, and all static routes that I added for site A in all of the other sites will take an effect and point traffic to FW.
Solution 1: Remove static routes manually (which doesn’t make sense)
Solution 2: Keep the static routes but add AD to each route higher than OSPF AD, let’s say 200.
In this case if P2P is up OSPF (110) < 200, OSPF will win
If P2P is down Static 1<200 , Static will win and traffic will go through firewalls.
For some reason solution 2 worked only in Site B and didn’t work on Site C or D!!!!
I checked the configurations in Sites B, C, and D, and I found that Sites C and D has this extra command which was added by the previous network engineer
redistribute static route-map STATIC>OSPF
Why do you think solution 2 didn’t work on Sites C or D?
Your help and suggestions are much appreciated.