Connecting two OSPF sites both directly and via MPLS

Question

Here at Stack Exchange, we recently added an MPLS link between data centers to supplement the VPN tunnels that we had been using up until now. The network looks like this:

   +-----------+                                                                  
   |    NY     |                                                                  
   |10.7.0.0/16|                                                                  
   ++----+----++                                                                  
    |    |    |                                                                   
  VPN 1  |    |                                                                   
OSPF P2P |  MPLS Peer--+                                                          
    |    |             |                                                          
    |    |             |                                                          
    |  VPN 2   MPLS Provider Net                                                  
    |OSPF P2P          |                                                          
    |    |             |                                                          
    |    |  MPLS Peer--+                                                          
    |    |    |                                                                   
    |    |    |                                                                   
   ++----+----++                                                                  
   |    CO     |                                                                  
   |10.8.0.0/16|                                                                  
   +-----------+    

The two sites are connected directly using VPN links (OSPF is running on these in point to point mode) and via the MPLS (we're OSPF peering with the MPLS provider's routers on each end). Everything is in area 0; nothing other than these two devices and the MPLS provider's routers are participating in this OSPF network.

The idea was to be able to adjust OSPF costs to have traffic flow over the (faster) MPLS connection - but what's happening instead, is that when only the MPLS is up the traffic takes the MPLS, but when the VPNs come up, the MPLS is no longer considered a valid route.

With just the MPLS up, there are type 5 LSAs for the remote site's /16 present in the database (with the advertising-router set to the MPLS provider's router); when the VPNs are brought up and the "original" type 5 LSA (with the advertising router set to the site router) populates across, the MPLS provider's routers stop sending their LSAs for the routes, so there's no way to use the MPLS route when the VPN is up.

• Is this strange withdrawl of the MPLS provider's type 5 LSA being caused by having the MPLS network set as area 0, since it's a disjointed area 0 which then gets joined together via the VPN when that's lit up? Since joining them together gives the MPLS neighbors knowledge of the 'direct' type 5 LSA that flooded via the VPN, would that be why they stopped advertising their own LSAs for those networks? (Could this be fixed by putting the MPLS peers in a different (NSSA?) area instead of area 0?)

• Because the MPLS network isn't using end-to-end OSPF, and instead distributes the routes to their endpoint routers in some other way then re-advertises new type 5 LSAs, will we be able to get to our desired end state of 'select route based on OSPF cost between VPN and MPLS', or will the fact that the LSAs that are being chosen between have differing advertising routers make that plan impossible?

Answer 1

I think this is the exact situation addressed by the OSPF "Sham Link" feature. It's hard to know if that's the exact issue, without knowing what your MPLS provider is doing with the routes you exchange with them. It's possible that even though the MPLS link is up, it's considered a Inter-area link, while your "backdoor" VPN tunnels are Intra-area, and thus, always preferred over Inter-area links.

I don't know what routing platform you use, but Juniper's docs contain a good explanation of the issue: http://www.juniper.net/techpubs/en_US/junos12.3/topics/topic-map/ospfv2-sham-links.html

Answer 2

FWIW, another solution is to just run a different routing protocol on your backup links. You could do RIP, EIGRP, or even just floating statics. Then just alter the AD of that protocol (if necessary) to ensure the MPLS routing protocol is preferred to the backup.

IMO, Sham-link are a bit of a hack to get around a design problem and are unnecessarily complex.