12

I've been tasked with selecting two new switches intended to be our new iSCSI SAN backbone.

Currently, we are using two Dell N4064 improvised for the task since that's what we had when the need arrived, but now we are going to replace them with two smaller, 24 port 10Gbit Ethernet switches.

I would ask if any 10Gbit Ethernet will perform adequately as an iSCSI backbone or if there are any specs I should be looking for or avoiding in my selection.

Thank you in advance for your input.

5
  • Clarification - this is a stack of 2x switches dedicated for redundant iscsi connections, and not for general traffic of which some is iscsi and some is "frontside" traffic ?
    – Criggie
    Feb 21, 2020 at 0:02
  • 1
    Power consumption (aka heat generated) can become an issue. That's true for any device in your server room, of course. But if your boss thinks that he's found a great deal on old powerful (read: power hungry) switches, tell him that he might need to upgrade the AC as well.
    – Klaws
    Feb 21, 2020 at 14:51
  • 1
    Criggie. This are to be used as two independent, non stacked, switches. They will be used just for the iSCSI traffic. Feb 21, 2020 at 17:21
  • Latency and bandwith are most important. Make sure there is no backplane bottleneck and the switches work cut-through. You also may want to consider DAC cables. And I would use rather more than less ports, if you can avoid upstream links it's a big plus.
    – eckes
    Feb 22, 2020 at 1:30
  • Did any answer help you? If so, you should accept the answer so that the question doesn't keep popping up forever, looking for an answer. Alternatively, you can post and accept your own answer.
    – Ron Maupin
    Dec 17, 2020 at 15:40

4 Answers 4

11

In a word: deep buffers, preferably per-port buffers

iSCSI does not tolerate frame drops. Delays are sort of ok, but a switch dropping a frame will create all manner of issues.

(Finding buffer size -- and type -- in manufacturer specs can be a challenge. They aren't always provided, and what is provided isn't always true.)

2
  • I would actually argue the switch should do cut-through forwarding for low latency and avoid store+foreward and therefore should not buffer.
    – eckes
    Feb 22, 2020 at 1:27
  • 3
    If two or more machines try to send a frame at the same time, without a buffer, it gets dropped. Storage is about reliability, not low latency. In a SAN, it always takes measurable time to get the block you want; even an SSD is not instant.
    – Ricky
    Feb 22, 2020 at 4:00
8

Make sure the switch is non-blocking. That is, the switch backplane can support all ports at line rate with no oversubscription. For a 24-port switch, the minimum backplane bandwidth is 24 x 10G, or 240Gb.

4

Be sure to understand what your iSCSI implementation on the end systems has to say about LACP (Port Aggregation) and/or if it mandates path isolation, to have two completely independent network paths between any initiator and any target.

If either is true, then MLAG (Multichassis Link Aggregation) capabilities probably won't be required.

1
  • Thanks. In our solution port aggregation is not allowed for iSCSI thought. Feb 20, 2020 at 17:39
3

In complement to Ron's answer, you need to look for Jumbo Frame. That is not mandatory, but it will significantly improve the bandwidth.

A good point too is the latency. It should ideally be below the µs.

You may also go for a full SFP+ switch, because depending on the equipment, you will have a mix of interfaces.

3
  • 3
    "jumbo frames ... significantly improve the bandwidth" - that largely depends on your hardware. With decent offloading or hardware initiators the difference between standard and jumbo frames may be hardly noticable.
    – Zac67
    Feb 20, 2020 at 15:04
  • I agree, in a scenario where I switched on jumbo frames it did only improve a few percents because the drivers have been doing receive aggregating. That's at least a feature of the Intel server adapters.
    – eckes
    Feb 22, 2020 at 4:08
  • We actually tested this on VMware ESXi, and jumbo frames added single percentage points in performance, very similar to this article (boche.net/blog/2011/01/24/…), so we decided to just to stick with 1500 due to the configuration demand that jumbo frames requires in larger environments.
    – Stuggi
    Feb 25, 2020 at 11:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.