I need to set up a modest iSCSI SAN with a few ESX hosts and one storage array. Can I get by using a Catalyst 2960-S or do I need to upgrade? Are there any special tweaks I need to make to the switch in order to handle iSCSI? The switch will be dedicated to the SAN; no other traffic except management will be present.
asked May 8, 2013 at 15:08
-
1Speaking as someone who started working in an environment where 2960 switches were used for iSCSI - don't do it! We experienced regular target logouts due to full buffers. @nicotine 's answer below is a very good one.– pauskaCommented May 8, 2013 at 20:02
-
@JStretch: ¿ tag edit ? Thoughts on making these "cisco-catalyst" instead of the specific model numbers? I've been editing lower-level users' tags... but didn't want to step on your toes :)– Craig ConstantineCommented May 9, 2013 at 15:24
-
@Craig I'm no one special; feel free to hack up my posts. I've edited the tag, good idea!– Jeremy StretchCommented May 9, 2013 at 18:37
-
@pauska Given that there are many environments where a 2960S is an upgrade, is it always a mistake to use this class of switch for storage traffic? (even for small sites)– ewwhiteCommented May 16, 2013 at 10:35
-
@ewwhite it's hard to say - a small shop with a not-so-fast SAN and not many hosts could probably work just fine on them. Our troubles started when we added a EMC VNX to the mix, which was a much much faster SAN than our previous one. Maybe Cisco isn't the best way for those SMB customers. I know that newer Dell PowerConnect models have alot more buffering plus DCB features.– pauskaCommented May 16, 2013 at 10:53
Add a comment
|
4 Answers
Given that the Cat2960-S is a desktop/access switch, with very, very small buffers, you would likely experience a lot of output drops. A datacenter switch, such as a 4948E, would be a better choice for an iSCSI application.
To understand the reasoning behind this, you have to remember that an ethernet switch is either transmitting, or not transmitting on a specific port. If traffic arrives on port 1 for port 2, and port 3 is already sending traffic to port 2, the traffic from port 1 has to be buffered until there's a gap in the traffic from 3 to 2. If the buffer fills, additional traffic will be dropped. The term "microburst" is used to reference traffic that, over time, is well below the limit for the interface, but occasionally bursts to cause output drops.
I am not as familiar with the 2960-S platform, but enabling QoS (without extensive tuning, see comments below) on it would probably be a bad idea; that would actually increase the number of output drops. Enabling QoS splits your very small buffers into 4 even smaller buffers, and most traffic will hit only one of them.
-
3Agreed on everything except about turning on QoS. At least on the Catalyst 3560/3750 the buffers can be tuned to perform better with mls qos than without it. This document is a good reference but it's for the 3560. supportforums.cisco.com/docs/DOC-8093 Commented May 8, 2013 at 16:59
-
1Edited answer to reflect that. I'm unsure you'll actually get BETTER buffer performance with tuning -- the default when QoS is disabled tries to be as "fair" as possible and assumes all traffic is the same class. Enabling it and tuning it may allow you to allocate more TX buffer to your downstream port, but QoS is in and of itself "managed unfairness"– nicotineCommented May 8, 2013 at 19:35
http://packetlife.net/blog/2012/jan/16/catalyst-2960s-iscsi-optimization/
I followed Jeremy Stretch's post here a while back, moving our iSCSI SAN off a production switch onto a dedicated 3750G (stack of 2). While I can't speak specifically to the 2960S, the tweaks offered in that post seem to be doing well on the 3750G's. I've got a handful of OutDiscards (less than 100) but for 38 weeks of uptime, that seems pretty good for me.
This is running 4 ESXi hosts connected to 4 JetStor 1Gbps SANs and all links are aggregated.
I'll happily forward a "sh int counters" if you are interested.
(Ha! Just noticed it was Jeremy who asked this...sorry for linking your own blog post!)
-
2Funny, I just remembered having come across this question in the past and thought it would make a good question. Completely forgot about that post. Commented May 9, 2013 at 18:40
I've had experience where customers have chosen to use the lower end switches (3750, 2960) for iSCSI and they are pretty terrible.
The issue gets even worse when you have your storage connected at a higher speed (like 10gbit for the SAN and 1gbit for the end servers).
I would look towards more of the DC class switches, if you want to stick with Catalyst then there is the 4900 series, alternatively you could venture in to the Nexus product line but it really depends on your price constraints.
-
I'm not sure how the old 3750 performs, but the 3750X has a lot more buffers than 2960/2960-S.– pauskaCommented May 8, 2013 at 20:01
-
They do indeed, however the DC class switches make the 3750X look like a desktop switch. Commented May 8, 2013 at 21:15
The Best Practice Guide for HP Lefthand iSCSI Storage stated, "For optimal switch performance, its recommended that the switch have at least 512 KB of buffer cache per port. Consult your switch manufacturer specifications for the total buffer cache. For example, if the switch has 48x 1 Gb ports, the recommendation is to have at least 24 MB of buffer cache dedicated to those ports. If the switch aggregates cache among a group of ports (for example, 1 MB of cache per 8 ports), space your storage modules and servers appropriately to avoid cache oversubscription."
I couldn't tell you what buffer cache the 2960S has per port, as it doesn't seem to be stated on the datasheet.
