QoS (802.1p) for iSCSI Traffic on Cisco Nexus 3048

Question

We're building a new core for our University and one thing that bugs me is the iSCSI network.

We already considered Jumbo Frames and Flow Control on the iSCSI ports, but one thing remains for debate, and it's the QoS Feature.

Our new core consist of two Cisco Nexus 3048. There will be two different iSCSI VLAN's for MPIO, one VLAN will be in the first switch and the other one on the the second. The Jumbo Frames and Flow Control seems trivial, but QoS does not appear to be simple.

I was unable to find any guidelines for iSCSI on the Nexus 3048 models. There are a lot of information for the 5000 and 7000 series but many features are missing on the 3000 series, like the QoS traffic match with the iSCSI protocol.

Another thing that I'm unsure is about CoS (Class of Service) Classification. Is this a requirement requirement? I don't know where iSCSI should be placed.

Thanks in advance, V.

EDIT with Answers:

1. A couple of design questions: - Are you mixing iSCSI and other data in this cluster (I assume you will user VPC)? - Are you mixing iSCSI and other data in the same port with tagged frames?

Data is mixed on the switch. VM's Network and Access will reach the core. But the iSCSI traffic has dedicated ports with larger MTU in mind and specific VLANs with untagged data, complete Layer 2 segregation. VPC will not be used for iSCSI since we're deploying MPIO, so there's no need. The storage is connected via SFP+ on the 10GbE port and the hypervisor hosts on the downlink GbE ports. There's 5 hypervisor hosts in total, each one with two cards, one for each switch. VPC is only used in the VM's Network and the access uplink.

2. What is the actual issue you are experiencing? Is this a performance problem? Generally, it is not advised to have your ISCSI network on the same hardware that will provide connectivity to the rest of your network. You should really be using a dedicated switch (2 for redundency) for your ISCSI network and segrate that from the normal user data.

We don't have any issues at this moment since this core is not in production, we just want to make it right before putting it on production. I'm aware that separate fabrics should be deployed for SAN traffic. With cheaper and common switches this is true and we rolled iSCSI this way on older equipment. But this switch is a unified fabric model, so it's developed to have mixed SAN and LAN traffic on the same box. The model we choose, the Nexus 3048 is capable of delivering extremely low latency and lossless iSCSI and here is where we need to configure it properly to make everything work as expected.

Thanks for the questions guys. Hope this helps.

Answer 1

If the clasification isn't available on the switch model, then you need to look at something like ACL-based QoS classification (Layers 2, 3, and 4) so that you can mark what traffic, by IP address range for example and tell the hardware how you want to prioritise the traffic with mappings.

CoS is for traffic that is layer 2 only. If you need to prioritise traffic that is going to get routed, then you would use DSCP which is layer 3.

So CoS will put the marking on an Ethernet frame header and DSCP would do it within the space on a IP header. If the traffic is getting routed, the Ethernet frame get's stripped so you would loose your CoS marking and would rely on DSCP within the IP header.

You achieve more granuality with DSCP compared to CoS as you can set more options because of the difference in bit size since CoS uses only 3 bits to set markings while DSCP uses 6 bits.

All in all, I would advise you seperate your ISCSI network from your core infrastrucure and then you wouldn't even need to worry about QoS.