4

We trying to implement an scenario to provide L2 transport services from POP A to POP B, connected with three different carriers transport links. In the future, we may want to implement POP C and have the ability to provide L2 services between any of the POPs.

The goals we are trying to achieve are:

  1. Scalability. More POPs will be added
  2. Sub second convergence
  3. Traffic engineering capabilities in order to establish conditional business logic based on the state of the transport links.

We have tried a VXLAN approach based on BGP underlay using route-maps to establish local preference and to control what carrier link will transport each VNI, by manipulating the reachability of the peer VTEPS, using a loopback interface for each VLAN/VNI

The problem with below configuration comes when defining the source interface for the nve interface, as it has to be only one. If we define loopback1 as source, only vlan 101 is carried. If we define loopback 2 as source, only vlan 102 is carried, and so on. Disregarding the fact that every peer VTEP is reachable when sourcing traffic from any of the loopbacks (1 to 3).

Below configuration is for POP A red siwtch, but you can figure the other 3. Eht1/6 and Eth1/7 are 2 of the transport links. Blue switches have Eth1/5 for the third link. Eth1/53 is for POP switches interlink.

Is there any chance this configuration may work? Does anyone knows another approach to achieve above goals?

I also have tried underlay OSPF + BGP EVP approach. The convergence is awesome (below 200ms) and all the vlans are carried. The problem is we cant figure a way to selectively route vlans/vnis through the different transport links. Is it possible? Any clue?

Thanks in advance,

Miguel

nv overlay evpn
feature ospf
feature bgp
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature bfd
feature nv overlay

!
vlan 1,101-103
vlan 101
  vn-segment 10101

vlan 102
  vn-segment 10102

vlan 103
  vn-segment 10103
!
spanning-tree port type edge bpduguard default
spanning-tree vlan 101-103 priority 0
!
ip prefix-list REMOTE_TEP_101 seq 5 permit 31.31.31.31/32 
ip prefix-list REMOTE_TEP_101 seq 10 permit 41.41.41.31/32 
ip prefix-list REMOTE_TEP_102 seq 5 permit 32.32.32.32/32 
ip prefix-list REMOTE_TEP_102 seq 10 permit 42.42.42.42/32 
ip prefix-list REMOTE_TEP_103 seq 5 permit 33.33.33.33/32 
ip prefix-list REMOTE_TEP_103 seq 10 permit 43.43.43.43/32 

route-map FROM_LINK1 permit 10
  match ip address prefix-list REMOTE_TEP_101 
  set local-preference 300

route-map FROM_LINK1 permit 20
  match ip address prefix-list REMOTE_TEP_102 
  set local-preference 200

route-map FROM_LINK1 permit 30
  match ip address prefix-list REMOTE_TEP_103 
  set local-preference 100

route-map FROM_LINK1 permit 100

route-map FROM_LINK2 permit 10
  match ip address prefix-list REMOTE_TEP_101 
  set local-preference 100

route-map FROM_LINK2 permit 20
  match ip address prefix-list REMOTE_TEP_102 
  set local-preference 300

route-map FROM_LINK2 permit 30
  match ip address prefix-list REMOTE_TEP_103 
  set local-preference 200

route-map FROM_LINK2 permit 100

route-map FROM_LINK3 permit 10
  match ip address prefix-list REMOTE_TEP_101 
  set local-preference 200

route-map FROM_LINK3 permit 30
  match ip address prefix-list REMOTE_TEP_102 REMOTE_TEP_103 
  set local-preference 300

route-map FROM_LINK3 permit 100
!

vrf context management
!
interface Vlan1
  no shutdown
!
interface Vlan101
  no shutdown
  ip address 91.91.91.1/24
  mtu 9216
!
interface Vlan102
  no shutdown
  ip address 92.92.92.1/24
  mtu 9216
!
interface Vlan103
  no shutdown
  ip address 93.93.93.1/24
  mtu 9216
!
interface nve1
  no shutdown
  source-interface loopback1
  member vni 10101
    ingress-replication protocol static
      peer-ip 31.31.31.31
  member vni 10102
    ingress-replication protocol static
      peer-ip 32.32.32.32
  member vni 10103
    ingress-replication protocol static
      peer-ip 33.33.33.33
!
interface Ethernet1/53
  no switchport
  mtu 9216
  bfd interval 50 min_rx 50 multiplier 3
  bfd ipv4 interval 50 min_rx 50 multiplier 3
  no ip redirects
  ip address 201.201.201.1/24
  no ipv6 redirects
  no shutdown
!
interface Ethernet1/6
  no switchport
  mtu 9216
  bfd interval 50 min_rx 50 multiplier 3
  bfd ipv4 interval 50 min_rx 50 multiplier 3
  no ip redirects
  ip address 102.102.102.1/24
  no ipv6 redirects
  no shutdown
!
interface Ethernet1/7
  no switchport
  mtu 9216
  bfd interval 50 min_rx 50 multiplier 3
  bfd ipv4 interval 50 min_rx 50 multiplier 3
  no ip redirects
  ip address 101.101.101.1/24
  no ipv6 redirects
  no shutdown
!
interface Ethernet1/48
  switchport
  switchport access vlan 101
  spanning-tree port type edge
  mtu 9216
  no shutdown
!
interface loopback0
  description ### VXLAN - ROUTING PURPOSES ###
  ip address 1.1.1.1/32
!
interface loopback1
  description ### VXLAN - TEP-NVE - VLAN 101 ###
  ip address 11.11.11.11/32
  ip address 10.101.0.1/32 secondary
!
interface loopback2
  description ### VXLAN - TEP-NVE - VLAN 102 ###
  ip address 12.12.12.12/32
  ip address 10.102.0.1/32 secondary
!
interface loopback3
  description ### VXLAN - TEP-NVE - VLAN 103 ###
  ip address 13.13.13.13/32
  ip address 10.103.0.1/32 secondary
!
router bgp 100
  router-id 1.1.1.1
  timers bgp 5 15
  address-family ipv4 unicast
    network 11.11.11.11/32
    network 12.12.12.12/32
    network 13.13.13.13/32
    network 10.101.0.1/32
    network 10.102.0.1/32
    network 10.103.0.1/32
  neighbor 101.101.101.2
    remote-as 200
        bfd
    address-family ipv4 unicast
      send-community
      send-community extended
      route-map FROM_LINK1 in
  neighbor 102.102.102.2
    remote-as 200
    bfd
    address-family ipv4 unicast
      send-community
      send-community extended
      route-map FROM_LINK2 in
  neighbor 201.201.201.2
    remote-as 100
    bfd
    address-family ipv4 unicast
      send-community
      send-community extended
      next-hop-self
!
end
!
Improve this question
1
  • Did any answer help you? If so, you should accept the answer so that the question doesn't keep popping up forever, looking for an answer. Alternatively, you can provide and accept your own answer.
    – Ron Maupin ♦
    Commented Dec 25, 2018 at 9:57

1 Answer 1

Reset to default
3

I don't think that what you're trying to do fits within the design of VXLAN or VXLAN EVPN. The whole point is that the fabric is templated, scalable and has the same multiple paths everywhere. As soon as you are doing per-VNI per-VTEP manipulation the scalability and uniform forwarding model is impacted.

Furthermore, by having a single fabric split between multiple sites, you're creating a shared fault domain.

Instead of manipulating the underlay, I'd approach this differently, and look at implementing some kind of DCI technology, and then segmenting at the handoff, perhaps on a per-tenant (VRF) basis. e.g. run 3 sets of OTV or DCI VXLAN (e.g. Cisco multi-site EVPN) segments, and then route them differently. It could be as 'simple' as dot1q handoff to 3x separate DCI VXLANs then you could route each set differently (though that's at least 12 extra switches for redundancy across 2x DCs...). Ultimately on Cisco at least there is no way of splitting the NVE/VTEPs.

I'd also note that a single fabric approach will give you headaches when it comes to routing, regardless of whether you are running VXLAN with traditional handoff to border/central routing, or upgrade to distributed anycast GW with EVPN. This is especially problematic with symmetry (think firewalls at the edge of the fabrics...).

Improve this answer
1
  • Thanks for your comments. They have been really helpful.
    – Miguel Bérniz
    Commented Jan 9, 2019 at 11:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.