Connecting non-redundant switches to stacked switches

Question

We have stacked Force10 MXL switches with very basic setup. However we want to extend our network with Ceph using non-redundant (or stacked) switches.

What we aim to create a network like this;

NRS: Non-redundant basic switch

F10: Force10 Stacked

F10                     NRS     Servers
A,Port40 (Subnet A) <-> K  <->  Server A,Port 1 | Server B,Port 1
|
B,Port40 (Subnet A) <-> L  <->  Server A,Port 2 | Server B,Port 2

So what do you think can go wrong with this architecture ? And what sould we avoid while seting this up ?

Thank you

Answer 1

You're not providing enough details. What are the non-redundant, basic switches like? Unmanaged? What are your options on the servers?

Assuming the basic switches are simple, unmanaged ones you need to cope with the fail-over on the MAC level.

• Each server uses one IP address and one MAC address per link.
• Load balancing has to be done with round-robin DNS or similar.
• When the server detects a link failure it needs to move the failed side's MAC over to the other NIC (along with the IP).
• This doesn't cover a link failure between an F10 and a simple switch, or an F10 total failure - you'd need to set up some kind of detection for that which is beyond the scope of NE.

A better approach would be to use decent switches which give you a lot more options.

Edit: Since the basic switches do provide some choices, you should add an interlink between them. In normal operation it will be blocked be STP but in a far link failure (from host POV) it'll provide uplink to the otherwise isolated server NIC. [...] The 5524 in turn feature stacking so you should consider stacking them and trunking the F10 uplinks.

Answer 2

As drawn it's not completely clear. The big issue is actually around host connectivity, specifically connecting two links from the same server to the same subnet. You'll need either some kind of active-passive configuration (i.e. only one port at a time active on the server) or for the server and switches to support some kind of port channeling. There are other ways to approach the problem (certain flavors of NIC teaming, for example) but there are pretty substantial caveats around balancing either transmitted or (especially) received traffic an an active/active basis.

The right way to do this is supporting some kind of port channeling setup that's configured on both the switches and servers. The network sees a single logical path in, the server sees a single logical path out. This will both allow load to be shared and failover to happen cleanly. The problem is that the configuration you've drawn above most definitely doesn't support that. You could channel the two links from each server to one of the access/non-redundant switches (...assuming they support this, of course) or to a couple of ports on the stack. Channeling across two independent switches requires some kind of accommodation for multi-chassis link aggregation (ex: Arista MLAG, Cisco VPC, etc).

Answer 3

Couple of things I would be concerned about, if there is ever a cable that gets plugged in between switch K and switch L there would be a loop. If that happens you will have a spanning tree loop and sometimes basic switches don't have STP.

The other thing I would be concerned with is bandwidth going from Server A, Port 1 to Server B, Port 2. When this happens your traffic will need to go back up to your "spine". May not ever be an issue but I would watch your traffic load on the Force 10's interfaces.

The design you laid out is relatively close to how the Cisco Nexus fabric works only with some proprietary protocols to allow for non-looping of layer 2 traffic.