Network Engineering Stack Exchange is a question and answer site for network engineers. It only takes a minute to sign up.
Sign up to join this community
I have a problem that I can’t seem to resolve. Here’s what happens.
I have 6 switches, all connected in a circular way. ex: 1 -> 2 -> 3 -> 4 -> 5 -> 6 -> Back to 1
This created a local network for all my SAN storages.
However, if 1 switch get Rebooted (for ex: Switch 1 rebooted), it seems like the entire network get resets. All of my SAN Storages will lose connection and regain connection.
Each switch is connected to another using LAG
With a ring, STP (Spanning Tree Protocol) shuts off one port (usually in the switch with highest mac and the highest used port on that switch in that ring if not configured differently). This prevents loops. When you break the working path, STP has to figure what "it" thinks is the next best path. That takes time. It can take up to 50 seconds with STP. RSTP (Rapid Spanning Tree) is much faster but will still exhibit this behavior. Some model switches have faster/different ways of doing this. It is working as designed. You may need a different design if this isn't acceptable.
Here is a link that might help. There are many resources to understand STP out there of course. Understanding STP
With RSTP the tree needs to converge which takes appr. 4-6 seconds. If you need a faster converge rate you'd need to look at something like SPB or various proprietary alternatives.
Additionally, arranging the switches in a ring may save ports but isn't the best approach. The image linked by user4565 provides a better way with two redundant core switches (which you set up with the lowest STP priority) and the rest arranged into a fabric. This will also ensure the fastest possible converge rate with RSTP.
Still additionally: you should consider providing redundancy through L3/IP multipathing instead of through L2 spanning tree. For IP multipathing you provide (usually) two independent L2 and L3 paths through the fabric.
Without redundant bridge loops you can deactivate spanning tree(!), which saves dead reconvergence time and also has the benefit that all physical links can actually be utilized. In the cisco example, you'd connect all left switches to each other for one path and all right switches to each other for the other path.
Of course, you don't need to do the split in half on the physical switch level, you can also do it on the logical VLAN level.
Note that you haven't mentioned which models of switches you have in production. Can you post?
For a simple solution (to begin with) you may want to investigate the redesign of your datacentre switching paths to make it more resilient to individual switch reboots.
The image linked below is taken from Cisco but you can use any other manufacturer as long as they support what you want and are compatible with your current production switches:-
Placing a couple of Layer 3 distribution switches, uplinking your current switches and making these new distribution switches the spanning tree root will provide the resilience you seek.
