0

We are setting up 2 nx 9k switches with port channel (LACP) enabled so we can have multiple links for redundancy. The two links in questions is eth1/3 on both switches going to UCS MLOM ports.

We are unable to get to the CIMC web gui without taking eth1/3 out of the port channel 13.

We think we are missing something basic but are unsure.

Edit: Including configs.

Switch 1 Brief

--------------------------------------------------------------------------------
Port   VRF          Status IP Address                              Speed    MTU
--------------------------------------------------------------------------------
mgmt0  --           up     192.168.111.4                           1000    1500 
   
--------------------------------------------------------------------------------
Ethernet      VLAN    Type Mode   Status  Reason                   Speed     Port
Interface                                                                    Ch #
--------------------------------------------------------------------------------
Eth1/1        17      eth  trunk  up      none                        10G(D) 11
Eth1/2        17      eth  trunk  up      none                        10G(D) 12
Eth1/3        17      eth  trunk  down    suspended(no LACP PDUs)    auto(D) 13
Eth1/4        17      eth  trunk  down    suspended(no LACP PDUs)    auto(D) 14
Eth1/5        17      eth  trunk  up      none                        10G(D) --
Eth1/6        1       eth  access up      none                        10G(D) --
Eth1/7        1       eth  access up      none                        10G(D) --
Eth1/8        1       eth  access up      none                        10G(D) --
Eth1/9        16      eth  access up      none                       1000(D) --
Eth1/10       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/11       1       eth  access down    XCVR not inserted          auto(D) --
[7m--More--[m
Eth1/12       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/13       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/14       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/15       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/16       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/17       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/18       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/19       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/20       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/21       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/22       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/23       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/24       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/25       17      eth  trunk  up      none                        10G(D) 10
Eth1/26       17      eth  trunk  up      none                        10G(D) 10
Eth1/27       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/28       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/29       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/30       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/31       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/32       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/33       16      eth  access up      none                       1000(D) --
Eth1/34       16      eth  access up      none                       1000(D) --
Eth1/35       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/36       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/37       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/38       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/39       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/40       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/41       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/42       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/43       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/44       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/45       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/46       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/47       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/48       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/49       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/50       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/51       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/52       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/53       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/54       1       eth  access down    XCVR not inserted          auto(D) --

------------------------------------------------------------------------------------------
Port-channel VLAN    Type Mode   Status  Reason                              Speed   Protocol
Interface                                                                       
     
------------------------------------------------------------------------------------------
Po10         17      eth  trunk  up      none                                 a-10G(D)  lacp
Po11         17      eth  trunk  up      none                                 a-10G(D)  lacp
Po12         17      eth  trunk  up      none                                 a-10G(D)  lacp
Po13         17      eth  trunk  down    No operational members                auto(D)  lacp
Po14         17      eth  trunk  down    No operational members                auto(D)  lacp

-------------------------------------------------------------------------------
Interface Secondary VLAN(Type)                    Status Reason                 
-------------------------------------------------------------------------------
Vlan1     --                                      down   VLAN/BD is down        
Vlan11    --                                      up     --        
Vlan16    --                                      up     --        

--------------------------------------------------------------------------------
Ethernet       VLAN    Type Mode   Status  Reason                  Speed     Port
Interface                                                                    Ch #
--------------------------------------------------------------------------------

SWITCH A ETH1/3 specific

Ethernet1/3 is down (suspended(no LACP PDUs))
admin state is up, Dedicated Interface
  Belongs to Po13
  Hardware: 100/1000/10000/25000 Ethernet, address: f8a7.3a4e.556a (bia f8a7.3a4e.556a)
  MTU 9216 bytes, BW 10000000 Kbit, DLY 10 usec
  reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, medium is broadcast
  Port mode is trunk
  auto-duplex, auto-speed, media type is 10G
  Beacon is turned off
  Auto-Negotiation is turned on  FEC mode is Auto
  Input flow-control is off, output flow-control is off
  Auto-mdix is turned off
  Rate mode is dedicated
  Switchport monitor is off 
  EtherType is 0x8100 
  EEE (efficient-ethernet) : n/a
    admin fec state is auto, oper fec state is off
  Last link flapped 00:03:25
  Last clearing of "show interface" counters 2w5d
  24 interface resets
  Load-Interval #1: 30 seconds
    30 seconds input rate 120 bits/sec, 0 packets/sec
    30 seconds output rate 176 bits/sec, 0 packets/sec
    input rate 120 bps, 0 pps; output rate 176 bps, 0 pps
  Load-Interval #2: 5 minute (300 seconds)
    300 seconds input rate 96 bits/sec, 0 packets/sec
    300 seconds output rate 568 bits/sec, 1 packets/sec
    input rate 96 bps, 0 pps; output rate 568 bps, 1 pps
  RX
    1997 unicast packets  49886 multicast packets  1784 broadcast packets
    53667 input packets  14871527 bytes
    1663 jumbo packets  0 storm suppression bytes
    0 runts  0 giants  0 CRC  0 no buffer
    0 input error  0 short frame  0 overrun   0 underrun  0 ignored
    0 watchdog  0 bad etype drop  0 bad proto drop  0 if down drop
    0 input with dribble  0 input discard
    0 Rx pause
  TX
    436 unicast packets  136858 multicast packets  145099 broadcast packets
    282393 output packets  30776179 bytes
    1 jumbo packets
    0 output error  0 collision  0 deferred  0 late collision
    0 lost carrier  0 no carrier  0 babble  0 output discard
    0 Tx pause

Switch B Brief

--------------------------------------------------------------------------------
Port   VRF          Status IP Address                              Speed    MTU
--------------------------------------------------------------------------------
mgmt0  --           up     192.168.111.5                           1000    1500 
   
--------------------------------------------------------------------------------
Ethernet      VLAN    Type Mode   Status  Reason                   Speed     Port
Interface                                                                    Ch #
--------------------------------------------------------------------------------
Eth1/1        17      eth  trunk  up      none                        10G(D) 11
Eth1/2        17      eth  trunk  up      none                        10G(D) 12
Eth1/3        17      eth  trunk  down    suspended(no LACP PDUs)    auto(D) 13
Eth1/4        17      eth  trunk  down    suspended(no LACP PDUs)    auto(D) 14
Eth1/5        1       eth  access up      none                        10G(D) --
Eth1/6        17      eth  trunk  up      none                        10G(D) --
Eth1/7        17      eth  trunk  up      none                        10G(D) --
Eth1/8        1       eth  access up      none                        10G(D) --
Eth1/9        1       eth  access down    XCVR not inserted          auto(D) --
Eth1/10       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/11       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/12       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/13       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/14       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/15       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/16       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/17       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/18       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/19       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/20       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/21       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/22       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/23       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/24       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/25       17      eth  trunk  up      none                        10G(D) 10
Eth1/26       17      eth  trunk  up      none                        10G(D) 10
Eth1/27       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/28       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/29       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/30       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/31       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/32       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/33       16      eth  access up      none                       1000(D) --
Eth1/34       16      eth  access up      none                       1000(D) --
Eth1/35       1       eth  access up      none                       1000(D) --
Eth1/36       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/37       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/38       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/39       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/40       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/41       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/42       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/43       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/44       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/45       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/46       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/47       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/48       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/49       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/50       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/51       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/52       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/53       1       eth  access down    XCVR not inserted          auto(D) --
Eth1/54       1       eth  access down    XCVR not inserted          auto(D) --

------------------------------------------------------------------------------------------
Port-channel VLAN    Type Mode   Status  Reason                              Speed   Protocol
Interface                                                                       
     
------------------------------------------------------------------------------------------
Po10         17      eth  trunk  up      none                                 a-10G(D)  lacp
Po11         17      eth  trunk  up      none                                 a-10G(D)  lacp
Po12         17      eth  trunk  up      none                                 a-10G(D)  lacp
Po13         17      eth  trunk  down    No operational members                auto(D)  lacp
Po14         17      eth  trunk  down    No operational members                auto(D)  lacp

-------------------------------------------------------------------------------
Interface Secondary VLAN(Type)                    Status Reason                 
-------------------------------------------------------------------------------
Vlan1     --                                      down   VLAN/BD is down        
Vlan16    --                                      up     --        

--------------------------------------------------------------------------------
Ethernet       VLAN    Type Mode   Status  Reason                  Speed     Port
Interface                                                                    Ch #
--------------------------------------------------------------------------------

Eth1/3 specific

Ethernet1/3 is down (suspended(no LACP PDUs))
admin state is up, Dedicated Interface
  Belongs to Po13
  Hardware: 100/1000/10000/25000 Ethernet, address: b08b.d081.4b2a (bia b08b.d081.4b2a)
  MTU 9216 bytes, BW 10000000 Kbit, DLY 10 usec
  reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, medium is broadcast
  Port mode is trunk
  auto-duplex, auto-speed, media type is 10G
  Beacon is turned off
  Auto-Negotiation is turned on  FEC mode is Auto
  Input flow-control is off, output flow-control is off
  Auto-mdix is turned off
  Rate mode is dedicated
  Switchport monitor is off 
  EtherType is 0x8100 
  EEE (efficient-ethernet) : n/a
    admin fec state is auto, oper fec state is off
  Last link flapped 00:00:12
  Last clearing of "show interface" counters 2w5d
  24 interface resets
  Load-Interval #1: 30 seconds
    30 seconds input rate 1184 bits/sec, 0 packets/sec
    30 seconds output rate 1168 bits/sec, 1 packets/sec
    input rate 1.18 Kbps, 0 pps; output rate 1.17 Kbps, 1 pps
  Load-Interval #2: 5 minute (300 seconds)
    300 seconds input rate 96 bits/sec, 0 packets/sec
    300 seconds output rate 424 bits/sec, 1 packets/sec
    input rate 96 bps, 0 pps; output rate 424 bps, 1 pps
  RX
    423117 unicast packets  49629 multicast packets  2545 broadcast packets
    475291 input packets  587954546 bytes
    359891 jumbo packets  0 storm suppression bytes
    0 runts  0 giants  0 CRC  0 no buffer
    0 input error  0 short frame  0 overrun   0 underrun  0 ignored
    0 watchdog  0 bad etype drop  0 bad proto drop  0 if down drop
    0 input with dribble  0 input discard
    0 Rx pause
  TX
    72282 unicast packets  136860 multicast packets  145284 broadcast packets
    354426 output packets  42593773 bytes
    75 jumbo packets
    0 output error  0 collision  0 deferred  0 late collision
    0 lost carrier  0 no carrier  0 babble  0 output discard
    0 Tx pause

Running Configuration

Lab-Switch-01(config-if)# show running conf

config-profile config-template

[J Lab-Switch-01(config-if)# show running config-[J[J[J[J[J[J[J[J-con[42D[J Lab-Switch-01(config-if)# show running-con[Jfig [J

!Command: show running-config
!Running configuration last done at: Thu Jun  9 10:20:27 2022
!Time: Thu Jun  9 10:23:22 2022

version 9.3(2) Bios:version 05.42 
switchname Lab-Switch-01
vdc Lab-Switch-01 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4096
  limit-resource port-channel minimum 0 maximum 511
  limit-resource u4route-mem minimum 248 maximum 248
  limit-resource u6route-mem minimum 96 maximum 96
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

cfs eth distribute
feature interface-vlan
feature lacp
feature vpc

username admin password 5 $5$NP5WMF6k$RPJoJxr8gJTVuDMQ0aUfu/LqWDksdTjZKJr4FW0PYc
9  role network-admin
[7m--More--[m
ip domain-lookup
system default switchport
copp profile strict
snmp-server user admin network-admin auth md5 0xd7a009c9655a2e76c81085ad9a3f4eac
 priv 0xd7a009c9655a2e76c81085ad9a3f4eac localizedkey
rmon event 1 description FATAL(1) owner PMON@FATAL
rmon event 2 description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 description ERROR(3) owner PMON@ERROR
rmon event 4 description WARNING(4) owner PMON@WARNING
rmon event 5 description INFORMATION(5) owner PMON@INFO
ntp master 3

ip route 0.0.0.0/0 192.168.111.1
ip route 192.168.111.0/24 192.168.111.5
vlan 1,11-17
vlan 11
  name NFS-VLAN
vlan 12
  name iSCSI-A-VLAN
vlan 13
  name iSCSI-B-VLAN
vlan 14
  name vMotion-VLAN
[7m--More--[m
vlan 15[K
  name VM-Traffic-VLAN
vlan 16
  name MGMT-VLAN
vlan 17
  name NATIVE-VLAN

spanning-tree port type edge bpduguard default
spanning-tree port type edge bpdufilter default
spanning-tree port type network default
vrf context management
  ip route 0.0.0.0/0 192.168.111.1
  ip route 192.168.111.0/24 192.168.111.1
port-channel load-balance src-dst ip-vlan 
vpc domain 1
  peer-switch
  role priority 10
  system-priority 3000
  peer-keepalive destination 192.168.111.5 source 192.168.111.4
  delay restore 150
  peer-gateway
  auto-recovery
  ip arp synchronize
[7m--More--[m

[K

interface Vlan1
  no shutdown
  no ip redirects
  no ipv6 redirects

interface Vlan11
  no shutdown
  no ip redirects
  no ipv6 redirects

interface Vlan16
  no shutdown
  management
  ip address 192.168.111.4/24
  no ipv6 redirects

interface port-channel10
  description vPC peer-link
  switchport mode trunk
  switchport trunk native vlan 17
  switchport trunk allowed vlan 11-16
[7m--More--[m
  spanning-tree port type network
  vpc peer-link

interface port-channel11
  description vPC to Controller-A
  switchport mode trunk
  switchport trunk native vlan 17
  switchport trunk allowed vlan 11-13,16
  spanning-tree port type edge trunk
  mtu 9216
  vpc 11

interface port-channel12
  description vPC to Controller-B
  switchport mode trunk
  switchport trunk native vlan 17
  switchport trunk allowed vlan 11-13,16
  spanning-tree port type edge trunk
  mtu 9216
  vpc 12

interface port-channel13
  description vPC to Server-A
[7m--More--[m
  switchport mode trunk
  switchport trunk native vlan 17
  switchport trunk allowed vlan 11,14-16
  spanning-tree port type edge trunk
  mtu 9216
  vpc 13

interface port-channel14
  description vPC to Server-B
  switchport mode trunk
  switchport trunk native vlan 17
  switchport trunk allowed vlan 11-16
  spanning-tree port type edge trunk
  mtu 9216
  vpc 14

interface Ethernet1/1
  switchport mode trunk
  switchport trunk native vlan 17
  switchport trunk allowed vlan 11-13,16
  mtu 9216
  channel-group 11 mode active

[7m--More--[m
interface Ethernet1/2
  switchport mode trunk
  switchport trunk native vlan 17
  switchport trunk allowed vlan 11-13,16
  mtu 9216
  channel-group 12 mode active

interface Ethernet1/3
  switchport mode trunk
  switchport trunk native vlan 17
  switchport trunk allowed vlan 11,14-16
  spanning-tree port type edge
  mtu 9216
  channel-group 13 mode active

interface Ethernet1/4
  switchport mode trunk
  switchport trunk native vlan 17
  switchport trunk allowed vlan 11-16
  spanning-tree port type edge trunk
  mtu 9216
  channel-group 14 mode active

[7m--More--[m
interface Ethernet1/5
  switchport mode trunk
  switchport trunk native vlan 17
  switchport trunk allowed vlan 11-12,14-16
  spanning-tree port type edge trunk
  mtu 9216

interface Ethernet1/6

interface Ethernet1/7

interface Ethernet1/8

interface Ethernet1/9
  switchport access vlan 16
  spanning-tree port type normal

interface Ethernet1/10

interface Ethernet1/11

interface Ethernet1/12

[7m--More--[m
interface Ethernet1/13

interface Ethernet1/14

interface Ethernet1/15

interface Ethernet1/16

interface Ethernet1/17

interface Ethernet1/18

interface Ethernet1/19

interface Ethernet1/20

interface Ethernet1/21

interface Ethernet1/22

interface Ethernet1/23

interface Ethernet1/24
[7m--More--[m

[K
interface Ethernet1/25
  switchport mode trunk
  switchport trunk native vlan 17
  switchport trunk allowed vlan 11-16
  channel-group 10 mode active

interface Ethernet1/26
  switchport mode trunk
  switchport trunk native vlan 17
  switchport trunk allowed vlan 11-16
  channel-group 10 mode active

interface Ethernet1/27

interface Ethernet1/28

interface Ethernet1/29

interface Ethernet1/30

interface Ethernet1/31

[7m--More--[m
interface Ethernet1/32

interface Ethernet1/33
  switchport access vlan 16
  spanning-tree port type normal
  speed 1000

interface Ethernet1/34
  switchport access vlan 16
  spanning-tree port type edge

interface Ethernet1/35

interface Ethernet1/36

interface Ethernet1/37

interface Ethernet1/38

interface Ethernet1/39

interface Ethernet1/40

[7m--More--[m
interface Ethernet1/41

interface Ethernet1/42

interface Ethernet1/43

interface Ethernet1/44

interface Ethernet1/45

interface Ethernet1/46

interface Ethernet1/47

interface Ethernet1/48

interface Ethernet1/49

interface Ethernet1/50

interface Ethernet1/51

interface Ethernet1/52
[7m--More--[m

[K
interface Ethernet1/53

interface Ethernet1/54

interface mgmt0
  vrf member management
  ip address 192.168.111.4/24
line console
line vty
boot nxos bootflash:/nxos.9.3.2.bin 
no system default switchport shutdown
5
  • Please edit the question to include the configurations. We cannot simply guess where it may have gone wrong.
    – Ron Maupin
    Jun 9, 2022 at 1:59
  • We need to see the switch configuration -- post the output of show running-configuration.
    – Ron Trunk
    Jun 9, 2022 at 12:27
  • It appears the UCS port is not configured for LACP, because your switch is not receiving any LACP PDUs. That's why the port channel is down.
    – Ron Trunk
    Jun 9, 2022 at 12:40
  • There does not appear to be any lacp settings on the UCS, there is one setting for enabling port channeling on the VNICs in the CIMC but it always deletes our vNICs and all the configs with it. Jun 9, 2022 at 12:49
  • Did you change this port in CIMC recently? If you change VIC configuration in CIMC, then Server restart is required. <br />Port must be in Trunk mode. <br /> Can you post output of show lacp interface e1/3 and your NX-OS version?
    – lbor
    Jul 2, 2022 at 21:35

1 Answer 1

1

Turns our it was not having a route properly set between the vlans. Our management van had an incorrect IP as well. So check the vrf context management.

1
  • 1
    You should accept your answer so that the question does not keep popping up forever, looking for an answer.
    – Ron Maupin
    Jul 16, 2022 at 15:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.