Category: Nexus 9k

Ethanalyzer

Here’s some useful ‘built-in’ Wireshark capability, for troubleshooting on the supervisor. The feature only works with process switched traffic.

nexus#ethanalyzer local interface inband capture-filter  “udp port 161”

nexus#ethanalyzer local interface inband capture-filter  “udp port 161” detail

nexus#ethanalyzer local interface inband capture-filter “udp port 161” write bootflash:snmp.pcap

An ACL log is necessary to capture data plane traffic.  Steps for this can be found here: Ethanalyzer Link

http://www.cisco.com/c/en/us/support/docs/switches/nexus-7000-series-switches/116136-trouble-ethanalyzer-nexus7000-00.html

 

ethanalyzer

MP-BGP EVPN VXLAN Configuration

Logical Construct of a Multi Tenant VxLAN EVPN with a Single Tenant in a VRF on a Nexux 9k

One VLAN maps to one Layer-2 VNI Layer-2 VNI per Layer-2 segment
A Tenant can have multiple VLANs, therefore multiple Layer-2 VNIs
Traffic within one Layer-2 VNI is bridged
Traffic between Layer-2 VINs is routed

1 Layer-3 VNI per Tenant (VRF) for routing
VNI X’ is used for routed packets

Initial Config – Per Switch

Enable VXLAN and MP-BGP EVPN Control Plane

feature nv overlay
[enables VXLAN]
feature vn-segment-vlan-based
[enables VLAN-based VXLAN]
feature bgp
[enables BGP]
nv overlay evpn
[enables EVPN control plane for VXLAN]

Other features that may need to be enabled

feature eigrp                          
[enables EIGRP if that is the choosen underlay IGP]
feature pim  
[enables IP PIM multicast routing in the underlay network]
feature interface-vlan  
[enables VLAN SVI interfaces if the VTEP needs to be IP gateway and route for the VXLAN VLAN IP subnet]

EVPN Tenant VRF (create VXLAN tenant VRF)

vrf context evpn-tenant-1
[create a VXLAN Tenant VRF]
    vni 39000
[Specify the L3 VNI for VXLAN rotuing within the VRF]
    rd auto
[define VRF RD (route distinguisher]
       address-family ipv4 unicast
       route-target import 39000:39000
       route-target export 39000:39000
       route-target both auto evpn
[define VRF Route Target and import/export policies in address-family ipv4 unicast]

Example of a 2nd tenant in a VRF

vrf context evpn-tenant-2
   vni 39010
   rd auto
   address-family ipv4 unicast
       route-target import 39010:39010
       route-target export 39010:39010
       route-target both auto evpn

Layer-3 VNI Per Tenant for EVPN Routing – How to Configure Layer-3 VNI per EVPN Tenant VRF Routing Instance

vlan 3900
name l3-vni-vlan-for-tenant-1
   vn-segment 39000
[Create the VLAN for the Layer-3 VNI. One Layer-3 VNI per tenant VRF routing instance]
interface Vlan3900
description l3-vni-for-tenant-1-routing
no shutdown
   vrf member evpn-tenant-1
[Create the SVI interface for the Layer-3 VNI Put this SVI interface into the tenant VRF context]
vrf context evpn-tenant-1
   vni 39000
rd auto
address-family ipv4 unicast
route-target import 39000:39000
route-target export 39000:39000
route-target both auto evpn
[Associate the Layer-3 VNI with the tenant VRF routing instance]

EVPN Layer-3 VNI Per Tenant for Routing Instance – Create SVI interface for Layer-2 VNIs for VXLAN routing

interface Vlan200
no shutdown
[Create SVI interface for a Layer-2 VNI. Associate it with the tenant VRF]
   vrf member evpn-tenant-1
   ip address 20.1.1.1/8
[All VTEPs for this VLAN/VNI should have the same SVI interface IP address as the distributed IP gateway]
   fabric forwarding mode anycast-gateway
[Enable distributed anycast gateway for this VLAN/VNI]

EVPN Distributed Gateway

fabric forwarding anycast-gateway-mac 0002.0002.0002
[Configure distributed gateway virtual MAC address One virtual MAC per VTEP All VTEPs should have the same virtual MAC address]
interface Vlan210
no shutdown
vrf member evpn-tenant-2
   ip address 21.1.1.1/8
[Configure virtual IP address All VTEPs for this VLAN should have the same virtual IP address]
   fabric forwarding mode anycast-gateway
[Enable distributed gateway for this VLAN]

VXLAN Tunnel Interface Configuration – Configure VXLAN tunnel interface nve1

interface nve1
no shutdown
   source-interface loopback0
[specify loopback0 as the source interface]
   host-reachability protocol bgp
[Define BGP as the mechanism for host reachability advertisement]
   member vni 20000
[Associate tenant VNIs to the tunnel interface nve1
Define the mcast group on a per-VNI basis
Enable arp suppression on a per-VNI basis]
      suppress-arp
      mcast-group 239.1.1.1
   member vni 21000
      suppress-arp
      mcast-group 239.1.1.2
member vni 39000 associate-vrf
member vni 39010 associate-vrf
[Add Layer-3 VNIs, one per tenant VRF]

interface loopback 0
ip address 10.1.1.11/32

MP-BGP Configuration on VTEP

router bgp 100
router-id 10.1.1.11
log-neighbor-changes
   address-family ipv4 unicast
[Address-family ipv4 unicast for prefix-based routing]
   address-family l2vpn evpn
[Address-family l2vpn evpn for evpn host routes]
   neighbor 10.1.1.1 remote-as 100
[Define MP-BGP neighbors. Under each neighbor define address-family ipv4 unicast and l2vpn evpn]
update-source loopback0
address-family ipv4 unicast
      address-family l2vpn evpn
[Send extended community in l2vpn evpn address-family to distribute EVPN route attributes]
         send-community extended
   neighbor 10.1.1.2 remote-as 100
update-source loopback0
address-family ipv4 unicast
      address-family l2vpn evpn
         send-community extended

vrf evpn-tenant-1
[Under address-family ipv4 unicast of each tenant VRF instance, enable advertising EVPN routes]
   address-family ipv4 unicast
      advertise l2vpn evpn
vrf evpn-tenant-2
   address-family ipv4 unicast
      advertise l2vpn evpn

MP-BGP Configuration on iBGP Route Reflector

router bgp 100
router-id 10.1.1.1
log-neighbor-changes
   address-family ipv4 unicast
[Address-family ipv4 unicast for prefix-based routing]
   address-family l2vpn evpn
      retain route-target all
[Address-family l2vpn evpn for EVPN vxlan host routes Retain route-targets attributes]
template peer vtep-peer
[iBGP RR client peer template]
remote-as 100
update-source loopback0
      address-family ipv4 unicast

[Send both standard and extended community in address-family ipv4 unicast]
   send-community both
      route-reflector-client
   address-family l2vpn evpn
[Send both standard and extended community in address-family l2vpn evpn]
      send-community both
      route-reflector-client
neighbor 10.1.1.11
inherit peer vtep-peer
neighbor 10.1.1.12
inherit peer vtep-peer
neighbor 10.1.1.13
inherit peer vtep-peer
neighbor 10.1.1.14
inherit peer vtep-peer

Nexus 9396 duplicate ICMP echo-reply (DUP!)

Inconsistent echo-reply from devices connected via VPC to Nexus 9300 while pinging from the Nexus exec prompt.

With some of the testing I receive normal response when pinging from one Nexus,  but no response when pinging from the other.  In another test I receive a normal response to one Nexus, and duplicate replys to the other

HNINXP12# ping 172.22.36.1 source 172.22.82.3 vrf corp
PING 172.22.36.1 (172.22.36.1) from 172.22.82.3: 56 data bytes
64 bytes from 172.22.36.1: icmp_seq=0 ttl=251 time=3.249 ms
64 bytes from 172.22.36.1: icmp_seq=0 ttl=251 time=3.643 ms (DUP!)
64 bytes from 172.22.36.1: icmp_seq=1 ttl=251 time=3.213 ms
64 bytes from 172.22.36.1: icmp_seq=1 ttl=251 time=3.563 ms (DUP!)

“When you enable this feature (peer-gateway), Cisco NX-OS automatically disables IP redirects on all interface VLANs mapped over a vPC VLAN to avoid generation of IP redirect messages for packets switched through the peer gateway router.”

You will need to manually add “no ip redirects” on each VPC vlan interface to prevent duplicate pings.