I'm an absolute beginner with NX-OS and VXLAN. I'm trying to set up a very simple configuration for VXLAN/EVPN/BGP. I have one NX-OSv spine and two leaf switches. No VPC, no anycast SVI. The underlay IGP is OSPF, and NVE host reachability via BGP is configured. The NVE source interface is the BGP update source on the iBGP spine-link session.
What happens is as follows:
- Pings work
- Despite traffic going through the network, there are no NVE peer relationships.
- I see no EVPN routes in BGP.
- If I shut down the BGP session between the spine and a leaf, pings still succeed.
- If I shut down an NVE interface, pings fail.
On Wireshark I see headers as expected: IPv4 (source: NVE, dest: VNI MCAST group), UDP 4789, VXLAN, Ethernet, IPv4, ICMP.
nv overlay evpn
feature ospf
feature bgp
feature pim
feature vn-segment-vlan-based
feature nv overlay
ip pim rp-address 10.0.0.255 group-list 224.0.0.0/4
ip pim ssm range 232.0.0.0/8
ip pim anycast-rp 10.0.0.255 10.0.255.1
ip pim anycast-rp 10.0.0.255 10.0.255.2
vlan 1,192
vlan 192
vn-segment 100192
interface nve1
no shutdown
source-interface loopback1
host-reachability protocol bgp
member vni 100192
mcast-group 224.1.1.192
interface Ethernet2/1
no switchport
mac-address 0cd4.553c.ca01
medium p2p
ip unnumbered loopback0
ip router ospf 255 area 0.0.0.0
ip pim sparse-mode
no shutdown
interface Ethernet2/9
switchport
switchport access vlan 192
no shutdown
interface loopback0
description Underlay
ip address 10.0.255.12/32
ip router ospf 255 area 0.0.0.0
ip pim sparse-mode
interface loopback1
description Overlay
ip address 10.1.255.12/32
ip router ospf 255 area 0.0.0.0
ip pim sparse-mode
router ospf 255
router-id 10.0.255.12
router bgp 255
router-id 10.1.255.12
neighbor 10.1.255.1
remote-as 255
update-source loopback1
address-family l2vpn evpn
send-community both
evpn
vni 100192 l2
rd auto
route-target import auto
route-target export auto
I've omitted the spine configuration for brevity, but the basic facts are:
- The spine is the RP
- The spine has the same BGP AS as the leaf
- The BGP sessions only have AF L2VPN EVPN
- The spines are RRs for the leaf switches.
The VNI looks OK:
leaf2# sh nve vni 100192
Codes: CP - Control Plane DP - Data Plane
UC - Unconfigured SA - Suppress ARP
SU - Suppress Unknown Unicast
Interface VNI Multicast-group State Mode Type [BD/VRF] Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1 100192 224.1.1.192 Up CP L2 [192]
And the switch sees the MAC:
leaf2# sh sys int l2fw mac
Stl Static BD MAC-Address FTAG.Sid/L2_Intf GM Type Age
---|------|-----|------------------|----------------|---|-----|---------|
0 0 192 00:50:79:66:68:01 Eth2/9 0 0 03:59:23*
BGP sessions are up:
leaf2# sh bgp l2 e sum
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 10.1.255.12, local AS number 255
BGP table version is 3, L2VPN EVPN config peers 2, capable peers 1
0 network entries and 0 paths using 0 bytes of memory
BGP attribute entries [0/0], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [0/0]
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
10.1.255.1 4 255 290 291 3 0 0 04:45:11 0
But no NVE peer:
leaf2# sh nve peer
leaf2#
And no EVPN routes in BGP:
leaf2# sh bgp l2 e
leaf2#
So what's going on? It seems like the routes are being learned from the IGP. What else do I need to get them into BGP?