NX-OS: Importing already mirrored config into a switch-profile

Question

I'm running NX-OS version 5.2(1)N1(5) on a pair of Nexus 5020s that are mirrors of each other for all intents and purposes. After OS upgrades some of the configuration that was inside the switch-profile managed to end up outside of it. Apart form being not-synchronized, this causes a number of other problems like not being able to modify bits of the configuration (switchport trunk allowed vlan ...).

Example configuration for both switches in the VPC pair.

! Relevant pieces from: show running-config include-switch-profile

interface port-channel301
  speed 10000

interface Ethernet1/17
  channel-group 301 mode active


switch-profile PROFILE
  sync-peers destination 192.168.0.2 ! This is .2 on switch-2

  interface port-channel301
    description SERVER-1
    switchport mode trunk
    switchport trunk allowed vlan 10, 20, 30-50
    spanning-tree port type edge trunk
    vpc 301

  interface Ethernet1/17
    description PO301 - SERVER-1
    switchport mode trunk
    switchport trunk allowed vlan 10, 20, 30-50
    spanning-tree port type edge trunk

While trying to bring that configuration back into the switch-profile, simply running import interface port-channel301 will cause a mutual exclusion failure on commit since the configuration exists on both sides.

The best solution I can extract from the NX-OS troubleshooting guide is:

1. Remove the sync-peers destination stanza from both switch-1, and switch-2.
2. Import the desired configuration on both switches and commit.
3. Add the sync-peers destination back.

It seems like this is prone to human error. I'm specifically concerned about adding the sync-peers destination stanza back. There's not very much information on what happens if the profiles aren't the exact same and what to do in that case.

Is this really the best way? Are there any best-practices people have when doing this kind of import? What happens if the switch-profiles do end up different? There's some institutional knowledge about this, but not much in the official NX-OS documentation.

Answer 1

The bug-id you are hitting is probably CSCuf56076, which is fixed in 6.0(2)N2(1). Unfortunately this is the best solution for this problem. Luckily it doesn't impact the traffic on the switches.

For debugging you can see which part of the config is in the global database:

show system internal csm info global-db seq-tbl

and which part is in the synced database:

show system internal csm info switch-profile <switch-profile-name> cfgd-db seq-tbl

When the sync-peer are removed you should make sure that both switches have the same config, and that the same parts of the config are in the switch-profile. Then re-add the sync-peer.

Be aware that, depending on what exactly goes wrong, this bug might re-appear after every reload of the switch. When we hit this bug we first performed the fix in the middle of the night to make sure there was no downtime. After getting more confident that traffic was not impacted we dared to fix these problems at slightly more convenient hours of the day (but still early in the morning) until Cisco released a version that fixed this.