1

I'm an absolute beginner with NX-OS and VXLAN. I'm trying to set up a very simple configuration for VXLAN/EVPN/BGP. I have one NX-OSv spine and two leaf switches. No VPC, no anycast SVI. The underlay IGP is OSPF, and NVE host reachability via BGP is configured. The NVE source interface is the BGP update source on the iBGP spine-link session.

What happens is as follows:

  1. Pings work
  2. Despite traffic going through the network, there are no NVE peer relationships.
  3. I see no EVPN routes in BGP.
  4. If I shut down the BGP session between the spine and a leaf, pings still succeed.
  5. If I shut down an NVE interface, pings fail.

On Wireshark I see headers as expected: IPv4 (source: NVE, dest: VNI MCAST group), UDP 4789, VXLAN, Ethernet, IPv4, ICMP.

nv overlay evpn
feature ospf
feature bgp
feature pim
feature vn-segment-vlan-based
feature nv overlay

ip pim rp-address 10.0.0.255 group-list 224.0.0.0/4
ip pim ssm range 232.0.0.0/8
ip pim anycast-rp 10.0.0.255 10.0.255.1
ip pim anycast-rp 10.0.0.255 10.0.255.2
vlan 1,192
vlan 192
  vn-segment 100192

interface nve1
  no shutdown
  source-interface loopback1
  host-reachability protocol bgp
  member vni 100192
    mcast-group 224.1.1.192

interface Ethernet2/1
  no switchport
  mac-address 0cd4.553c.ca01
  medium p2p
  ip unnumbered loopback0
  ip router ospf 255 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet2/9
  switchport
  switchport access vlan 192
  no shutdown

interface loopback0
  description Underlay
  ip address 10.0.255.12/32
  ip router ospf 255 area 0.0.0.0
  ip pim sparse-mode

interface loopback1
  description Overlay
  ip address 10.1.255.12/32
  ip router ospf 255 area 0.0.0.0
  ip pim sparse-mode

router ospf 255
  router-id 10.0.255.12
router bgp 255
  router-id 10.1.255.12
  neighbor 10.1.255.1
    remote-as 255
    update-source loopback1
    address-family l2vpn evpn
      send-community both
evpn
  vni 100192 l2
    rd auto
    route-target import auto
    route-target export auto

I've omitted the spine configuration for brevity, but the basic facts are:

  1. The spine is the RP
  2. The spine has the same BGP AS as the leaf
  3. The BGP sessions only have AF L2VPN EVPN
  4. The spines are RRs for the leaf switches.

The VNI looks OK:

leaf2# sh nve vni 100192
Codes: CP - Control Plane        DP - Data Plane
       UC - Unconfigured         SA - Suppress ARP
       SU - Suppress Unknown Unicast

Interface VNI      Multicast-group   State Mode Type [BD/VRF]      Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1      100192   224.1.1.192       Up    CP   L2 [192]

And the switch sees the MAC:

leaf2# sh sys int l2fw mac

Stl Static   BD       MAC-Address   FTAG.Sid/L2_Intf  GM  Type      Age
---|------|-----|------------------|----------------|---|-----|---------|

  0      0   192  00:50:79:66:68:01           Eth2/9   0     0  03:59:23*

BGP sessions are up:

leaf2# sh bgp l2 e sum
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 10.1.255.12, local AS number 255
BGP table version is 3, L2VPN EVPN config peers 2, capable peers 1
0 network entries and 0 paths using 0 bytes of memory
BGP attribute entries [0/0], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.1.255.1      4   255     290     291        3    0    0 04:45:11 0

But no NVE peer:

leaf2# sh nve peer
leaf2#

And no EVPN routes in BGP:

leaf2# sh bgp l2 e
leaf2#

So what's going on? It seems like the routes are being learned from the IGP. What else do I need to get them into BGP?

3
  • Is this the complete leaf configuration? You'd usually have vrf's set up - and, indeed, if Lo1 is to be a member of the overlay then it needs to be part of a vrf in that overlay.
    – rnxrx
    Feb 25 at 3:01
  • This is a partial config, but I didn't configure any VRFs. Don't the VNIs do that work for a purely L2 EVPN setup like this one? There are lots of models in Cisco documentation with BGP and no VRFs, but they mostly have anycast gateways and/or vPC to clutter the config. Feb 25 at 15:00
  • So I tweaked this to get the NVE update source out of OSPF and into BGP. Now when I take down the BGP session the pings fail, as expected, but still don't see any EVPN routes or NVE peers. Feb 26 at 16:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.