Theoretical: Bottleneck in a switch stacking environment

Quick question regarding stacking environments and their limitations regarding traffic bottlenecks.

Assuming the following use case:

• We have a stack of 8 switchs, forming a Half 10Gbps stack ring.
• Switch 1 master
• Each switch has 48 ports at max 100Mbps, so each switch could have a maximum of 4.8Gbps of traffic
• The uplink of 10Gbps is on the master

Assuming the members 2, 3, and 4 are at 4.8Gbps each, so am I true if I say there is a bottleneck between the member 2 and 1 stack cable in this case ? (4.8 x 3 = 14.4gbps)

Same question with a Full ring, I think we have no longer the bottleneck because of the traffic goes also from the other direction to reach the member 1 master ? (traffic of switch 2, 3 and 4 could pass through members 5, 6, 7, 8, and 1 ?)

Thanks.

• What is the switch model? Different switch models have different backplane stacking capacities. Also, while you can stack eight switches, you probably do not want more than about four in a stack. I have seen eight in a stack with 384 ports, but the network was `/24` (254 addresses), and the network engineers wondered why some hosts wouldn't work with more ports than addresses. Apr 4, 2018 at 14:41

You have not given us the switch model, but you tagged your question with the Cisco Catalyst tag, so I will use the Cisco Catalyst 3850 as an example.

The Cisco Catalyst 3850 has a pretty large stack switching capacity:

Cisco StackWise-480 technology provides scalability and resiliency with 480 Gbps of stack throughput

That means you could have 48 gigabit ports per switch for 10 switches (I believe that nine is the actual stack limit) and still have wirespeed switching in the stack.

Your bottleneck will be the uplink from the stack. Cisco recommends a 20:1 access to distribution bandwidth ratio. That means for every 20 gigabit access interfaces, you need one gigabit in the uplink.

• Thanks. In my question it was a theoretical example to understand the limitations. I understood your example. Another quick example, with 8 x 3750 stack with stackwise and stack ring speed at 32Gbps and uplink of 10Gbps on the member 1 master. With half ring, if switchs 2, 3 and 4 have 12Gbps of trafic each to the uplink, we will have a bottleneck between the member 2 and 1 ? True ? But in Full ring, we won't have bottleneck due to the two directions possible so no longer 32Gbps of stack but 64Gbps, true ? Apr 4, 2018 at 15:22
• According to Cisco: "StackWise Plus for ease of use and resiliency with 64 Gbps of throughput" When designing, you need to understand your hardware specifications, and design accordingly. There is no one-size-fits-all answer. For example, the 20:1 bandwidth ratio is for typical users. If you are running servers on the switch, then you will want to lower that. Also, understand that it will be nearly impossible for you to have all your hosts running at full interface speed all the time. In general, hosts will receive much more than they send. Apr 4, 2018 at 15:29

With a 10 Gbit/s uplink, 10 Gbit/s stack interlinks and only 100 Mbit/s edge ports the uplink bottleneck is rather theoretical.

48 FE ports on each switch can transport an aggregate, peak edge-to-core bandwidth of 4.8 Gbit/s, so up to two switches won't be able to saturate the uplink bandwidth. The most probable bottleneck is the 10G uplink with an oversubscription of just 3.8:1 - with a "standard" network this is far below the recommended subscription ratio of 20:1 and gives plenty of headroom.

However, mileage varies and with a workload of a full 100 Mbit/s on each edge port permanently, a single 10G uplink won't cut it. This is why you usually avoid rings/chains and use a central switch with the others grouped around (a "fat tree") or spine-leaf in a larger network.

The Cisco switch model makes a big difference in whether or not you hit a bottleneck on the stack ring - more than the raw bandwidth numbers will show.

The 3750(G) switches had a much different stack architecture (Stackwise) than the newer StackWise plus on the 3750(X) and the Stackwise-480 found on the Cisco 3850 switches. The 3750 stack will not do local switching - every Ethernet frame going from any interface to any other interface will go across the stack ring. This includes traffic going between interfaces on the same physical switch. Stackwise also does what is called source stripping, where the packet is removed from the ring only after it returns to the source switch.

The 3750X with StackWise Plus and the 3850 switch with Stackwise-480 can do local switching mainly because it has centralized design for all layer 2 and layer 3 protocols. The 3750 uses distributed design for the MAC address table and SPT. There are other inherent reasons in the different designs as well.

https://www.cisco.com/c/en/us/products/collateral/switches/catalyst-3850-series-switches/white-paper-c11-734429.html#_Toc417362558

How would this affect your theoretical bottleneck question? If you have a large amount of locally switched traffic (servers to database etc.. ) you can hit a bottleneck on your stack's ring before even getting close to your maximum upload capacity on your uplink.

The paper you want to read to understand the differences between these different stacking protocols and which also discusses throughput bottlenecks is the,

Cisco StackWise and StackWise Plus Technology white paper, easily found with a google search. If you search for stackwise plus white paper it will be the first Google search result.