I have a quite big network with spanning tree diameter 26 (if I count it correctly). It is build on Cisco swithes (mainly Catalyst 2960 and Catalyst 1000, two SG350), running rapid-pvst. It is now running with default spanning tree timers, that are counted for diameter 7. I think that this makes troubles when topology change occurs. What timers should I set to make it working properly? There are five loops.
I know, that it is not good to have such a big diameter, but I cannot change it. It is connected via fiber optics and distance of switches is in kilometers.
-
3Seriously, you should rebuild your network into a tree with at most three hierarchy levels. Alternatively, consider MSTP with multiple regions.– Zac67 ♦Jan 13, 2021 at 16:14
-
2I certainly hope that whoever set that up is gone. That is really an unsupportable pile of crap. That network needs to be remediated because it is a giant problem waiting to happen, and it will crash the entire network when it happens.– Ron Maupin ♦Jan 13, 2021 at 17:09
-
2Running this in a flat L2 topology is insane... Could you mark the long links that need to stay put?– Zac67 ♦Jan 13, 2021 at 17:50
-
2More important than your timers is the switch priority and path costs. Also, how many vlans do you have, and how far do they extend in the network?– Ron TrunkJan 13, 2021 at 19:36
-
3This would have been a long obsolete and poor design 15 years ago. Is it absolutely critical that the same broadcast domain be shared throughout the network? Could you potentially break up these domains with routers? Turn it into five or six relatively simple topologies that can be individually worked on / remediated. Alternatively you could potentially rationalize this into a logical hub/spoke (or even leaf/spine) with cheap CWDM add/drops. Honestly tweaking STP parameters in this environment is going to have a huge amount of risk for not a lot of reward.– rnxrxJan 14, 2021 at 5:12
2 Answers
I guess you aren't familiar with the infamous "All Systems Down" article from 2003. [ archive ] I can't find the original CIO article, and it was never archived.
In simple terms, spanning-tree doesn't work beyond the designed limits. Your network is a timebomb. There's only one place where STP could detect a loop -- the 4 switch core. This really needs to be redesigned, but not knowing the physical topology, we can't begin to guess how to go about it. The first thing to consider is breaking the flat layer-2 domain into multiple routed layer-3 segments.
If you insist on playing with the timers, the limit is 17-18. [ the math ]
That topology far exceeds the xSTP design maximum. As you've stated, STP with default settings can handle a network with a maximum diameter of 7. As Ricky has shown so aptly, adjusting those settings may allow a diameter of up to 18.
Even if your network wasn't exceeding that already, a link disruption in the wrong place can substantially increase the diameter, likely rendering your network non-functional.
RPVST+ creates an independent spanning tree for each VLAN, so if your network doesn't exceed the maximum diameter within any VLAN, it might even work. However, I suspect that that isn't the case. Since losing links like CPS2-V7 or CPS3-SAT would seriously increase the network diameter, chances are high that your network might not recover from a single link loss ("train wreck waiting to happen"). Check out the link in Ricky's answer for how that would look like.
Actually, you'd be better off removing the closing loops S5-S6 and LIP-RUD and deactivating STP on those branches altogether. The redundancy those loops are supposed to create is just imaginary anyway.
Since merely tweaking STP settings can't solve your problem, there are three basic approaches:
- Using (proprietary!) RSTVP+, decrease effective diameter by shortening the VLAN reaches (=remove VLANs from switches and trunks wherever possible). Alternatively, switch to MSTP and create independent regions. In both cases, tweaking the STP settings is still required and might not even be enough.
- Connect all outer-ring switches to the inner-ring switches (CPS2, CPS3, HUS1, HUS2) directly. As you likely can't run extra fiber you could use WDM to leverage the already present strands.
- Change the inner ring to routed links instead of switched ones. That is the most substantial redo but offers the best in performance and growth potential. Also, it may not even require additional hardware.
-
RPVST+ is a Cisco thing. (if there are any non-cisco switches, you're stuck.) MST has the same diameter limits; segmenting into different regions won't help here as the loops are just too big. This should clearly be a L3 network, not the flat L2 domain from the 70's.– RickyJan 15, 2021 at 5:33
-
1