1

I'm having some trouble with STP on our network. I have created a 3 tier network made up of Dell and Cisco switches. The topology is as follows:

The core consists of two Dell N4032 10Gbps switches in stacked mode and three Dell Power Connect 5548 switches which are also in stacked mode. The two stacks connect to each other using two SFP+ cables in a LAG using LACP. The distribution layer switches are split into three physical rooms in the building. Each room has two Dell N3024 switches in stacked mode. The switches have a 10Gbps RJ45 module on the rear which have two ports each resulting in four 10Gbps ports per stack. I have four CAT 6A cables running from the core 10Gbps to the 10Gbps ports on the distribution switches. I have set these connections as a LAG again using LACP. The access layer is made up of Cisco SG300 switches. These switches use two connections in a LAG again using LACP to the distribution switches.

I'm having problems with some end devices being able to communicate with other end devices. From the log files on switches the problem seems to relate to STP. I haven't made any changes to STP on any of the switches (which are all using RSTP) so everything is default which from research I have discovered is not optimal. I'm struggling to work out what the best settings to use for STP on our LAGs. I considered disabling STP on the LAGs since our network isn't a loop network but I would like to use STP. But I may be wrong and disabling STP could be fine since the network isn't a loop network. Any input would be great, if needed I can paste configs and log outputs.

Thanks!

Network diagram: enter image description here

Core Switch config:

interface Te1/0/13
channel-group 3 mode active
switchport mode trunk
switchport access vlan 3
exit              
!                 
interface Te1/0/14
channel-group 3 mode active
switchport mode trunk
switchport access vlan 3
exit              
!                 
interface Te1/0/15
channel-group 4 mode active
switchport mode trunk
switchport access vlan 3
exit              
!                 
interface Te1/0/16
channel-group 4 mode active
switchport mode trunk
switchport access vlan 3
exit              
!                 
interface Te1/0/17
channel-group 5 mode active
exit              
!                 
interface Te1/0/18
channel-group 5 mode active
exit              
!                 
interface Te1/0/23
channel-group 2 mode active
exit              
!                 
interface Te1/0/24
channel-group 1 mode active
exit              
!                 
interface Te1/1/5 
channel-group 6 mode active
switchport mode trunk
switchport access vlan 3
exit              
!                 
interface Te1/1/6 
channel-group 6 mode active
switchport mode trunk
switchport access vlan 3
exit              
!                 
interface Te1/1/8 
channel-group 7 mode active
exit              
!                 
interface Te2/0/13
channel-group 3 mode active
switchport mode trunk
switchport access vlan 3
exit              
!                 
interface Te2/0/14
channel-group 3 mode active
switchport mode trunk
switchport access vlan 3
exit              
!                 
interface Te2/0/15
channel-group 4 mode active
switchport mode trunk
switchport access vlan 3
exit              
!                 
interface Te2/0/16
channel-group 4 mode active
switchport mode trunk
switchport access vlan 3
exit              
!                 
interface Te2/0/17
channel-group 5 mode active
exit              
!                 
interface Te2/0/18
channel-group 5 mode active
exit              
!                 
interface Te2/0/23
channel-group 1 mode active
exit              
!                 
interface Te2/0/24
channel-group 2 mode active
exit              
!                 
interface Te2/1/1 
channel-group 6 mode active
switchport mode trunk
switchport access vlan 3
exit              
!                 
interface Te2/1/2 
channel-group 6 mode active
switchport mode trunk
switchport access vlan 3
exit              
!                 
interface Te2/1/4 
channel-group 7 mode active
exit              
!                 
interface port-channel 3
description "SER Distribution"
switchport mode trunk
switchport access vlan 3
exit              
!                 
interface port-channel 4
switchport mode trunk
switchport access vlan 3
exit              
!                 
interface port-channel 5
switchport mode trunk
switchport access vlan 3
exit              
!                 
interface port-channel 6
switchport mode trunk
switchport access vlan 3
exit

Distribution Switch Config:

configure
vlan 2-4,99
exit
vlan 2
name "Exhibition"
exit
vlan 3
name "PublicLAN"
exit
vlan 4
name "SPC"
exit
vlan 99
name "Management"
exit
snmp-server location "Workshop"
hostname "Workshop Distribution"
slot 1/0 2    ! Dell Networking N3024F
slot 1/1 11   ! Dell 10GBase-T Card
slot 2/0 2    ! Dell Networking N3024F
slot 2/1 11   ! Dell 10GBase-T Card
stack
member 1 2    ! N3024F
member 2 2    ! N3024F
exit
interface vlan 1
ip address 10.0.6.46 255.255.0.0
exit

interface Gi1/0/1
channel-group 1 mode active
exit
!
interface Gi1/0/2
speed auto
channel-group 2 mode active
exit
!
interface Gi1/0/3
speed auto
channel-group 3 mode active
exit              
!
interface Gi1/0/4
speed auto
channel-group 5 mode active
exit
!
interface Gi1/0/5
speed auto
channel-group 6 mode active
exit
!
interface Gi1/0/6
speed auto
channel-group 7 mode active
exit
!
interface Gi1/0/7
channel-group 8 mode active
exit
!
interface Gi1/0/8
channel-group 9 mode active
exit              
!
interface Gi1/0/9
speed auto
channel-group 10 mode active
exit
!
interface Te1/1/1
channel-group 4 mode active
exit
!
interface Te1/1/2
channel-group 4 mode active
exit
!
interface Gi2/0/1
speed auto
channel-group 1 mode active
exit
!
interface Gi2/0/2
speed auto
channel-group 2 mode active
exit              
!
interface Gi2/0/3
speed auto
channel-group 3 mode active
exit
!
interface Gi2/0/4
speed auto
channel-group 5 mode active
exit
!
interface Gi2/0/5
speed auto
channel-group 6 mode active
exit
!
interface Gi2/0/6
speed auto
channel-group 7 mode active
exit
!
interface Gi2/0/7
speed auto        
channel-group 8 mode active
exit
!
interface Gi2/0/8
speed auto
channel-group 9 mode active
exit
!
interface Gi2/0/9
speed auto 1000
channel-group 10 mode active
exit
!
interface Gi2/0/11
speed auto 1000
exit
!
interface Gi2/0/21
speed auto
exit
!
interface Te2/1/1
channel-group 4 mode active
exit
!
interface Te2/1/2
channel-group 4 mode active
exit
!
interface port-channel 1
switchport mode trunk
switchport trunk allowed vlan 1-99
exit
!
interface port-channel 2
switchport mode trunk
switchport trunk allowed vlan 1-99
exit
!
interface port-channel 3
switchport mode trunk
switchport trunk allowed vlan 1-99
exit
!
interface port-channel 4
switchport mode trunk
switchport trunk allowed vlan 1-99
exit
!
interface port-channel 5
switchport mode trunk
switchport trunk allowed vlan 1-99
exit
!
interface port-channel 6
switchport mode trunk
switchport trunk allowed vlan 1-99
exit
!
interface port-channel 7
switchport mode trunk
switchport trunk allowed vlan 1-99
exit
!
interface port-channel 8
switchport mode trunk
switchport trunk allowed vlan 1-99
exit
!                 
interface port-channel 9
switchport mode trunk
switchport trunk allowed vlan 1-99
exit
!
interface port-channel 10
switchport mode trunk
switchport trunk allowed vlan 1-99
exit
!
interface port-channel 11
switchport mode trunk
switchport trunk allowed vlan 1-99
exit
!
interface port-channel 12
switchport mode trunk
switchport trunk allowed vlan 1-99
exit
!
interface port-channel 13
switchport mode trunk
switchport trunk allowed vlan 1-99
exit
!
interface port-channel 14
switchport mode trunk
switchport trunk allowed vlan 1-99
exit

All the distribution switches have the same config on them.

Example core config:

interface vlan 1                                      
 ip address 10.0.6.55 255.255.0.0                     
 no ip address dhcp                                   
!                                                     
interface vlan 2                                      
 name Exhibition                                      
!                                                     
interface vlan 3                                      
 name "Public Lan"                                    
!                                                     
interface vlan 5                                      
 name Office                                          
!                                                     
  interface gigabitethernet8
 switchport trunk allowed vlan add 2-3
!
interface gigabitethernet9
 channel-group 1 mode auto
!
interface gigabitethernet10
 channel-group 1 mode auto
!
interface Port-channel1
 description Uplink
 spanning-tree disable
 spanning-tree link-type point-to-point

Example log output:

<189> Jul  5 15:32:34 10.0.6.40-1 TRAPMGR[994142700]: traputil.c(740) 76551 %% Spanning Tree Topology Change Received: MSTID: 0 Po3
<189> Jul  5 15:32:32 10.0.6.40-1 TRAPMGR[994142700]: traputil.c(740) 76550 %% Spanning Tree Topology Change Received: MSTID: 0 Po3
<189> Jul  5 15:32:31 10.0.6.40-1 TRAPMGR[994142700]: traputil.c(740) 76549 %% Spanning Tree Topology Change Received: MSTID: 0 Po3
<189> Jul  5 15:32:30 10.0.6.40-1 TRAPMGR[994142700]: traputil.c(740) 76548 %% Spanning Tree Topology Change: 0, Unit: 1
<189> Jul  5 15:32:30 10.0.6.40-1 TRAPMGR[994142700]: traputil.c(740) 76547 %% Spanning Tree Topology Change Received: MSTID: 0 Po3
<189> Jul  5 15:32:28 10.0.6.40-1 TRAPMGR[994142700]: traputil.c(740) 76546 %% Spanning Tree Topology Change Received: MSTID: 0 Po3
<189> Jul  5 15:32:27 10.0.6.40-1 TRAPMGR[994142700]: traputil.c(740) 76545 %% Spanning Tree Topology Change Received: MSTID: 0 Po3
<189> Jul  5 15:32:26 10.0.6.40-1 TRAPMGR[994142700]: traputil.c(740) 76544 %% Spanning Tree Topology Change: 0, Unit: 1
<189> Jul  5 15:32:26 10.0.6.40-1 TRAPMGR[994142700]: traputil.c(740) 76543 %% Spanning Tree Topology Change Received: MSTID: 0 Po3
<189> Jul  5 15:32:24 10.0.6.40-1 TRAPMGR[994142700]: traputil.c(740) 76542 %% Spanning Tree Topology Change Received: MSTID: 0 Po3
<189> Jul  5 15:32:23 10.0.6.40-1 TRAPMGR[994142700]: traputil.c(740) 76541 %% Spanning Tree Topology Change Received: MSTID: 0 Po3
Improve this question
7
  • We need more info to help you. Please edit your question to include a simple diagram, and the switch configurations. You should also include the log messages, so we don't have to guess what they are. As a general rule, you should leave STP on. It will help prevent really bad things from happening on your network.
    – Ron Trunk
    Jul 5, 2016 at 15:29
  • Thanks for your comment Ron. I'll add a diagram, configs and log errors to the main question.
    – user23673
    Jul 5, 2016 at 15:33
  • What exactly is the issue though? Also, I'd recommend using the security features of RSTP+. And, keep in mind that STPs consider your aggregated links as one, so there's really no need to turn off STP on them.
    – Izy-
    Jul 5, 2016 at 17:36
  • We are having issues when devices are trying to talk to another internal device on another access switch so for example we have an internal web server which devices connect to for content. Some devices work fine others don't. If I was to disconnect one of the LAG cables all of the devices work fine. So I've come to the conclusion that the problem is with the LAGs and STP. I say STP due to the amount of errors in the logs.
    – user23673
    Jul 5, 2016 at 18:11
  • I've just read 'A major complication of existing link aggregation or bonding technologies is that all members interfaces of such a team/group need to terminate on one single logical switch.' My LAG teams don't all terminate to a single switch for example between a access switch and dist switch stack, one cable from dist sw a goes to one port on the access later sw and the other goes to sw b at the dist layer. This seems contradictory to what I've just read. Have I set my LAGs up incorrectly in this way?
    – user23673
    Jul 5, 2016 at 22:41

3 Answers 3

Reset to default
1

Stack mode is not recommended for a pair of core switches as it is a shared control plane / single point of failure and software upgrade will result in core outage. This is not acceptable design. The core should aim for better than 99.999% uptime (5m 15.6s / year).

Use Dell's VLT instead.

Improve this answer
1
  • Thanks for your answer Ron. I will look more into VLT in the morning.
    – user23673
    Jul 5, 2016 at 22:18
0

There isn't enough info to properly answer but here are a couple things.

One thing I'd change is having access commands on a trunk port. Remove the switchport access vlan 3 command if you want them to be trunks. (Note... the encapsulation command doesn't work on all cisco switches. If it fails that's ok.)

NO Switchport access vlan 3 
Switchport trunk encapsulation dot1q
Switchport mode trunk

Also, you've chosen to allow certain vlans. Make them match on both ends. Make sure vlan 1 can get from switch to switch for things like bpdu's for spanning tree. You have allowed vlan 1-99 on one switch and vlan 2-3 on others. I can't tell if those are linked together but vlan 1 isn't allowed and they are 'pruned' diffrently at each end.

Without the full configurations it's tough to tell but make sure vlan 1 isn't untagged on your up links attached to cisco switches. I've seen that issue several times. To explain: Cisco trunk ports tag all allowed vlans unless a native vlan (untagged) is identified. Some vendors untag vlan 1 or untag by having a pvid 1 or people tend to program vlan 1 untagged Without thinking about it. Easy mistake.

Improve this answer
6
  • "Make sure vlan 1 can get from switch to switch for things like bpdu's for spanning tree." That is unnecessary. You can prune VLAN 1, or any native VLAN, from all links, and link protocols, like BPDUs, will still work just fine. Those types of frames cannot even be sent to another interface, even if you have a native VLAN.
    – Ron Maupin ♦
    Dec 27, 2016 at 0:32
  • Regular STP derives from vlan1 is sent untagged. If you are using pvst+ (per vlan spanning tree) I would agree. But, I am speaking from a Cisco point of view
    – Fixitrod
    Dec 27, 2016 at 0:48
  • We are a Cisco company, and we disallow VLAN 1, and any other native VLANs. The link-local protocols (STP, CDP, etc.) still work just fine. They are sent untagged. These are link-only protocols, and they do not require a VLAN.
    – Ron Maupin ♦
    Dec 27, 2016 at 0:51
  • That's interesting. Maybe an ios diffrence I've run into. Since standard STP is untagged I thought you had to have an untagged (native) vlan. I've actually ran into issues before (been years ago now). PVST will tag so I know that works. I'll have to look into this. Thanks
    – Fixitrod
    Dec 27, 2016 at 0:59
  • There is a Cisco press book about layer-2 LAN security, and one of the primary recommendations is to shut down VLAN 1 and do not allow native VLANs. We did that years ago, and everything works well. Link-local LAN protocols still are sent untagged, and they still work. What you don't want is for that VLAN to be across all your switches. The link-local protocols are not allowed to be sent to another interface, anyway. BPDUs use a multicast OUI, reserved by the IEEE, that means the frames are not allowed off the link. Cisco STP has its own addresses for that, but the operation is the same.
    – Ron Maupin ♦
    Dec 27, 2016 at 1:04
0

Thank you all for your helpful input. According to Dell support the 'errors' I was seeing in the logs were not due to any loops but they related to end devices going up or down due to normal day to day activity. Many comments and answers above pointed to the use of the default vlan 1 Causing potential problems. This has now been rectified.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.