0

First time poster and the more I read aboud RSTP, the more questions I have in relation to my situation.

I have found myself recently assigned to a project that has been essentially delivered on a site I will be managing once it becomes operational. This is for a critical piece of infrastructure and needs to be resiliant. I'm very concerned about the design and the fact the supplier has purposely implemented a single VLAN onn a campus that is quite large (2/3 km2), and is fully dependent on RSTP for redundancy. Added to this is during site testing there was discrepencies in convergence and the supplier then started pointing to the fact there was slower copper 100Mbs connections between some switches and started changing port priorities and port costs. However having revisited the design in more detail I have seen that they have also change timers from the defaults on devices that are not the root bridge. On reading up on RSTP I'm now also looking at the maximum diameter and have even more questions.

As the diagram shows, there are a total of 24 switches present on the same subnet and participating in the same RSTP. The core switches are HPE 1920s and field devices are MOXA L2 switches.

My questions are these.

  1. Is this a good design?
  2. Can/Should RSTP timers be changed from defaults and if so should it be done on non-root bridges
  3. What is the Max diameter of this topology and what diameter is advisible to use?enter image description here

In summary, what have I got here? Is it a resilient network topology?

Really look forward to any feedback.

Improve this question
2
  • 1
    Use trees, not loops, and the best practice is to never have one access switch connect to another access switch. An access switch should have two switch connections: one to each distribution switch. I would have fired the network designer.
    – Ron Maupin ♦
    Commented Jul 12, 2020 at 15:27
  • Did any answer help you? If so, you should accept the answer so that the question doesn't keep popping up forever, looking for an answer. Alternatively, you can post and accept your own answer.
    – Ron Maupin ♦
    Commented Dec 17, 2020 at 19:05

2 Answers 2

Reset to default
2

Your max diameter is currently 19. That's what happens if Field Switch 1 or 2 goes down, which they sometimes will for maintenance. Even during normal operation, your diameter is 11. That's alright -- RSTP default max age is 20 -- but I would make these rings a little smaller.

If I inherited that network, my priorities would be:

  • Remove all xSTP customizations. It's almost never helpful. Definitely not helpful when configured by some vendor who apparently doesn't know what they're doing!
  • Eliminate the 100Mb links. They're counter-productive.
  • Find out why it's all one VLAN; are all clients on the network being put directly into this big VLAN? More layer-3, and less layer-2, would be a big win.
  • Change to MSTP and make each ring its own region to reduce the scope of topology change events.
  • If additional fiber pairs are available, divide the longest rings so there are no more than 4 downstream switches per ring.
  • Stop using field switch 1 & 2 as a distribution layer. Connect all the campus rings directly onto the bigger core switches.
Improve this answer
5
  • Thanks @Jeff Wheeler. Really appreciate your feedback. Unfortunately, I cant remove the 100Mb links. There are additional Moxa devices on these copper links, that are transparent to STP BPDUs, I’m told. These devices are essentially ethernet to serial adapters allowing communication to devices attached which is the purpose of the network. Some of these devices are very sensitive to data loss which is why having a resilient network is critical. Also, MSTP not available. There is not a lot of network traffic but could these small Moxa switches come under pressure and cause problems?
    – celticherr
    Commented Jul 13, 2020 at 11:01
  • In that case, you do need to configure the STP cost for your 100Mb links to be higher than the 802.1D default (19) or else your network will use those links, instead of failing over to the ring's redundant long-path, because 1000Mb links have enough cost by default (4) that some switches will make a bad decision during certain outages. I'd just set the cost of all your 100Mb links to cost=1000 and call it a win.
    – Jeff Wheeler
    Commented Jul 13, 2020 at 16:16
  • @JeffWheeler Assuming he's using 802.1D costs with RSTP. What confuses me more is hanging crucial stuff off the 100Mbps links and having a Gbps in parallel with it. How's that meant to work? Configure the costs so that the Gbps is backup to the 100Mbps?
    – richardb
    Commented Jul 14, 2020 at 23:46
  • The OP explained the 100Mb devices are Moxa (Ethernet-over-Coaxial) switches serving some kind of downstream devices. He doesn't want, for example, Field Switch 15 to be a single point of failure for its Moxa downstreams; so those Moxa switches also uplinks to Field Switch 16. It sounds like some old television cabling has been repurposed to reduce the cost of network upgrades in some parts of the campus. Configuring cost=1000 on those 100Mb links will ensure they won't be mistakenly used as backups, yes. It also makes the intent obvious.
    – Jeff Wheeler
    Commented Jul 15, 2020 at 12:22
  • The whole implementation is a green field deployment. All new and everything by design. I should also point out where there is 100Mb links, the switches are in close proximity-few meters. Where just fibre it can be several 100 meters. I’ve also since found out the 100Mb RJ45 connections were crimped onsite manually and are not factory moulded/tested. Oh dear! I'm thinking any faulty cable connection could spell trouble for RSTP?
    – celticherr
    Commented Jul 15, 2020 at 14:54
1

This is for a critical piece of infrastructure and needs to be resiliant.

You can call a network with a single redundant link "resilient". The network architecture needs to reflect the degree of resilience that is required. Most often, a maximum downtime period or a minimum availability per year is defined.

I'm very concerned about the design and the fact the supplier has purposely implemented a single VLAN onn a campus that is quite large (2/3 km2),

That design is bad. Building Ethernet rings such a large L2 segment/broadcast domain must be avoided.

and is fully dependent on RSTP for redundancy.

That isn't the problem. RSTP/MSTP can provide a good degree of redundancy with very fast failover when used correctly.

However, a large number of redundant links increase the chance that some link isn't configured correctly and causes a broadcast loop, bringing down the network.

Added to this is during site testing there was discrepencies in convergence

RSTP is designed for a maximum of seven bridges in depth. Exceeding that requires changing the Maximum age parameter, so the tree has a chance to converge.

While the diagram shows a physical radius of only six, uneven data rates might shift the blocking point so that the radius exceeds seven. As a rule of thumb, never decrease link speed towards the center of a network, only towards the edge.

and the supplier then started pointing to the fact there was slower copper 100Mbs connections between some switches

Inbuilt bottlenecks make everything worse. 100 Mbit/s doesn't hurt RSTP though, the diameter/distance does. However, mixing different speeds in a ring shifts the STP-blocked port towards the lower-speed link, so the STP maximum depth can be exceeded more easily.

and started changing port priorities and port costs.

Patchwork, possibly working around the uneven link speed problem.

However having revisited the design in more detail I have seen that they have also change timers from the defaults on devices that are not the root bridge.

Yes. You likely need to tune max age. However, you should seriously consider rebuilding the network (see below).

As the diagram shows, there are a total of 24 switches present on the same subnet and participating in the same RSTP.

The number isn't necessarily a problem, the depth is (maximum distance between any two switches).

Is this a good design?

Definitely not. Building rings with Ethernet is a bad idea in general.

The 100M links look entirely useless. With a single L2 and STP, they'd just get blocked, even if aggregated.

Can/Should RSTP timers be changed from defaults and if so should it be done on non-root bridges

Yes, on all bridges, see above. Before doing that, you'll need to calculate what really is required.

What is the Max diameter of this topology and what diameter is advisible to use?

Best case, it's 12 (6-5-4-3-1-23-2-13-14-15-16-17), worst case it's 19.

So, you should consider a few changes:

  1. split up the one large VLAN into several ones, routed at the core (the 1920s support L3 switching)
  2. connect each field switch individually to each of the core switches - if that isn't possible, at least arrange the switches in a hierarchy (=physical tree), with distribution layer redundancy
  3. as a cheaper version of 2., interconnect the field switches in pairs, each of them connected to one of the cores
Improve this answer
5
  • The max diameter for RSTP is 40
    – Ron Trunk
    Commented Jul 12, 2020 at 12:01
  • ... but not with default parameters.
    – Zac67 ♦
    Commented Jul 12, 2020 at 12:16
  • Thanks @Zac67 Really appreciate your feedback. I’m afraid don’t follow how the different link speeds could increase the depth? I’ve also read that RSTP will automatically adjust timers on root bridges automatically if the diameter parameter is provided. Is this correct? Also, I've noticed the vendor changed timers on some field switches. What is the benifit of doing that if timers are set on the Root bridge?
    – celticherr
    Commented Jul 13, 2020 at 11:13
  • @celticherr If you've got 1G throughout, the loop splits/is blocked on the link farthest from the root due to equal root cost. A slower link in the ring increases the root cost significantly, so it becomes the split. E.g. with all 1G links, your upper loop splits/blocks between S6 and S7 (5x 20,000 each way). If the S3-S4 link was only 100M, the split moved to S3-S4 as the path S4-S5-...-S2-S23 had a lower cost (for S4-S3: 2x 20,000+200,000, S4-S5: 8x 20,000).
    – Zac67 ♦
    Commented Jul 13, 2020 at 11:34
  • TBH I would probably throw the 1920s away and get something decent. SMB switches have their place but at the core of an HA environment? It's just another sign that cost rather than quality was driving the network design.
    – richardb
    Commented Jul 13, 2020 at 14:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.