We have stacked Force10 MXL switches with very basic setup. However we want to extend our network with Ceph using non-redundant (or stacked) switches.

What we aim to create a network like this;

NRS: Non-redundant basic switch

F10: Force10 Stacked

F10                     NRS     Servers
A,Port40 (Subnet A) <-> K  <->  Server A,Port 1 | Server B,Port 1
B,Port40 (Subnet A) <-> L  <->  Server A,Port 2 | Server B,Port 2

So what do you think can go wrong with this architecture ? And what sould we avoid while seting this up ?

Thank you

3 Answers 3


You're not providing enough details. What are the non-redundant, basic switches like? Unmanaged? What are your options on the servers?

Assuming the basic switches are simple, unmanaged ones you need to cope with the fail-over on the MAC level.

  • Each server uses one IP address and one MAC address per link.
  • Load balancing has to be done with round-robin DNS or similar.
  • When the server detects a link failure it needs to move the failed side's MAC over to the other NIC (along with the IP).
  • This doesn't cover a link failure between an F10 and a simple switch, or an F10 total failure - you'd need to set up some kind of detection for that which is beyond the scope of NE.

A better approach would be to use decent switches which give you a lot more options.

Edit: Since the basic switches do provide some choices, you should add an interlink between them. In normal operation it will be blocked be STP but in a far link failure (from host POV) it'll provide uplink to the otherwise isolated server NIC. [...] The 5524 in turn feature stacking so you should consider stacking them and trunking the F10 uplinks.

  • NRS switches are managed, they are Dell 5524 Switches but I wanted to see as much comments as I can get to see what can goes wrong. That is why I didnt provide much info. Jun 26, 2018 at 5:46
  • That's not very effective. A lot of answers/comments might not be usable in your scenario.
    – Zac67
    Jun 26, 2018 at 6:15
  • I know however we have the option to switch things according to comments and this topic does not have one perfect solution as it includes many variables. Jun 26, 2018 at 7:34
  • If you want a solid answer you need to provide solid details. If you're looking for opinions this question is off-topic.
    – Zac67
    Jun 26, 2018 at 10:39
  • Actually you are wrong about being off-topic, it helped me a lot. And also you helped me a lot without knowing. Thank you. Jun 27, 2018 at 3:19

As drawn it's not completely clear. The big issue is actually around host connectivity, specifically connecting two links from the same server to the same subnet. You'll need either some kind of active-passive configuration (i.e. only one port at a time active on the server) or for the server and switches to support some kind of port channeling. There are other ways to approach the problem (certain flavors of NIC teaming, for example) but there are pretty substantial caveats around balancing either transmitted or (especially) received traffic an an active/active basis.

The right way to do this is supporting some kind of port channeling setup that's configured on both the switches and servers. The network sees a single logical path in, the server sees a single logical path out. This will both allow load to be shared and failover to happen cleanly. The problem is that the configuration you've drawn above most definitely doesn't support that. You could channel the two links from each server to one of the access/non-redundant switches (...assuming they support this, of course) or to a couple of ports on the stack. Channeling across two independent switches requires some kind of accommodation for multi-chassis link aggregation (ex: Arista MLAG, Cisco VPC, etc).

  • So if we setup LAG at F10 and LAG at NRS (like port-channeling) and LACP on the server side, there wouldnt be any problem. Right ? Jun 26, 2018 at 4:51
  • 2
    @HarunBarışBulut LAG (static or LACP) can't work with another switch in between.
    – Zac67
    Jun 26, 2018 at 5:27

Couple of things I would be concerned about, if there is ever a cable that gets plugged in between switch K and switch L there would be a loop. If that happens you will have a spanning tree loop and sometimes basic switches don't have STP.

The other thing I would be concerned with is bandwidth going from Server A, Port 1 to Server B, Port 2. When this happens your traffic will need to go back up to your "spine". May not ever be an issue but I would watch your traffic load on the Force 10's interfaces.

The design you laid out is relatively close to how the Cisco Nexus fabric works only with some proprietary protocols to allow for non-looping of layer 2 traffic.

  • That's not really anything Nexus-like. Stacked switches represent a single control plane as well as a merged data plane. In the Nexus line you might be thinking of Virtual Port Channels (VPC), which is two independent switches that are externally synchronized to present a common control protocol (usually LACP, but can be static). The closer analogy would be stacking in the Cat3K line or perhaps VSS in the Cat6K line.
    – rnxrx
    Jun 26, 2018 at 2:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.