We trying to implement an scenario to provide L2 transport services from POP A to POP B, connected with three different carriers transport links. In the future, we may want to implement POP C and have the ability to provide L2 services between any of the POPs.
The goals we are trying to achieve are:
- Scalability. More POPs will be added
- Sub second convergence
- Traffic engineering capabilities in order to establish conditional business logic based on the state of the transport links.
We have tried a VXLAN approach based on BGP underlay using route-maps to establish local preference and to control what carrier link will transport each VNI, by manipulating the reachability of the peer VTEPS, using a loopback interface for each VLAN/VNI
The problem with below configuration comes when defining the source interface for the nve interface, as it has to be only one. If we define loopback1 as source, only vlan 101 is carried. If we define loopback 2 as source, only vlan 102 is carried, and so on. Disregarding the fact that every peer VTEP is reachable when sourcing traffic from any of the loopbacks (1 to 3).
Below configuration is for POP A red siwtch, but you can figure the other 3. Eht1/6 and Eth1/7 are 2 of the transport links. Blue switches have Eth1/5 for the third link. Eth1/53 is for POP switches interlink.
Is there any chance this configuration may work? Does anyone knows another approach to achieve above goals?
I also have tried underlay OSPF + BGP EVP approach. The convergence is awesome (below 200ms) and all the vlans are carried. The problem is we cant figure a way to selectively route vlans/vnis through the different transport links. Is it possible? Any clue?
Thanks in advance,
nv overlay evpn feature ospf feature bgp feature interface-vlan feature vn-segment-vlan-based feature lacp feature bfd feature nv overlay ! vlan 1,101-103 vlan 101 vn-segment 10101 vlan 102 vn-segment 10102 vlan 103 vn-segment 10103 ! spanning-tree port type edge bpduguard default spanning-tree vlan 101-103 priority 0 ! ip prefix-list REMOTE_TEP_101 seq 5 permit 22.214.171.124/32 ip prefix-list REMOTE_TEP_101 seq 10 permit 126.96.36.199/32 ip prefix-list REMOTE_TEP_102 seq 5 permit 188.8.131.52/32 ip prefix-list REMOTE_TEP_102 seq 10 permit 184.108.40.206/32 ip prefix-list REMOTE_TEP_103 seq 5 permit 220.127.116.11/32 ip prefix-list REMOTE_TEP_103 seq 10 permit 18.104.22.168/32 route-map FROM_LINK1 permit 10 match ip address prefix-list REMOTE_TEP_101 set local-preference 300 route-map FROM_LINK1 permit 20 match ip address prefix-list REMOTE_TEP_102 set local-preference 200 route-map FROM_LINK1 permit 30 match ip address prefix-list REMOTE_TEP_103 set local-preference 100 route-map FROM_LINK1 permit 100 route-map FROM_LINK2 permit 10 match ip address prefix-list REMOTE_TEP_101 set local-preference 100 route-map FROM_LINK2 permit 20 match ip address prefix-list REMOTE_TEP_102 set local-preference 300 route-map FROM_LINK2 permit 30 match ip address prefix-list REMOTE_TEP_103 set local-preference 200 route-map FROM_LINK2 permit 100 route-map FROM_LINK3 permit 10 match ip address prefix-list REMOTE_TEP_101 set local-preference 200 route-map FROM_LINK3 permit 30 match ip address prefix-list REMOTE_TEP_102 REMOTE_TEP_103 set local-preference 300 route-map FROM_LINK3 permit 100 ! vrf context management ! interface Vlan1 no shutdown ! interface Vlan101 no shutdown ip address 22.214.171.124/24 mtu 9216 ! interface Vlan102 no shutdown ip address 126.96.36.199/24 mtu 9216 ! interface Vlan103 no shutdown ip address 188.8.131.52/24 mtu 9216 ! interface nve1 no shutdown source-interface loopback1 member vni 10101 ingress-replication protocol static peer-ip 184.108.40.206 member vni 10102 ingress-replication protocol static peer-ip 220.127.116.11 member vni 10103 ingress-replication protocol static peer-ip 18.104.22.168 ! interface Ethernet1/53 no switchport mtu 9216 bfd interval 50 min_rx 50 multiplier 3 bfd ipv4 interval 50 min_rx 50 multiplier 3 no ip redirects ip address 22.214.171.124/24 no ipv6 redirects no shutdown ! interface Ethernet1/6 no switchport mtu 9216 bfd interval 50 min_rx 50 multiplier 3 bfd ipv4 interval 50 min_rx 50 multiplier 3 no ip redirects ip address 126.96.36.199/24 no ipv6 redirects no shutdown ! interface Ethernet1/7 no switchport mtu 9216 bfd interval 50 min_rx 50 multiplier 3 bfd ipv4 interval 50 min_rx 50 multiplier 3 no ip redirects ip address 188.8.131.52/24 no ipv6 redirects no shutdown ! interface Ethernet1/48 switchport switchport access vlan 101 spanning-tree port type edge mtu 9216 no shutdown ! interface loopback0 description ### VXLAN - ROUTING PURPOSES ### ip address 184.108.40.206/32 ! interface loopback1 description ### VXLAN - TEP-NVE - VLAN 101 ### ip address 220.127.116.11/32 ip address 10.101.0.1/32 secondary ! interface loopback2 description ### VXLAN - TEP-NVE - VLAN 102 ### ip address 18.104.22.168/32 ip address 10.102.0.1/32 secondary ! interface loopback3 description ### VXLAN - TEP-NVE - VLAN 103 ### ip address 22.214.171.124/32 ip address 10.103.0.1/32 secondary ! router bgp 100 router-id 126.96.36.199 timers bgp 5 15 address-family ipv4 unicast network 188.8.131.52/32 network 184.108.40.206/32 network 220.127.116.11/32 network 10.101.0.1/32 network 10.102.0.1/32 network 10.103.0.1/32 neighbor 18.104.22.168 remote-as 200 bfd address-family ipv4 unicast send-community send-community extended route-map FROM_LINK1 in neighbor 22.214.171.124 remote-as 200 bfd address-family ipv4 unicast send-community send-community extended route-map FROM_LINK2 in neighbor 126.96.36.199 remote-as 100 bfd address-family ipv4 unicast send-community send-community extended next-hop-self ! end !