We're building a new core for our University and one thing that bugs me is the iSCSI network.
We already considered Jumbo Frames and Flow Control on the iSCSI ports, but one thing remains for debate, and it's the QoS Feature.
Our new core consist of two Cisco Nexus 3048. There will be two different iSCSI VLAN's for MPIO, one VLAN will be in the first switch and the other one on the the second. The Jumbo Frames and Flow Control seems trivial, but QoS does not appear to be simple.
I was unable to find any guidelines for iSCSI on the Nexus 3048 models. There are a lot of information for the 5000 and 7000 series but many features are missing on the 3000 series, like the QoS traffic match with the iSCSI protocol.
Another thing that I'm unsure is about CoS (Class of Service) Classification. Is this a requirement requirement? I don't know where iSCSI should be placed.
Thanks in advance, V.
EDIT with Answers:
- A couple of design questions: - Are you mixing iSCSI and other data in this cluster (I assume you will user VPC)? - Are you mixing iSCSI and other data in the same port with tagged frames?
Data is mixed on the switch. VM's Network and Access will reach the core. But the iSCSI traffic has dedicated ports with larger MTU in mind and specific VLANs with untagged data, complete Layer 2 segregation. VPC will not be used for iSCSI since we're deploying MPIO, so there's no need. The storage is connected via SFP+ on the 10GbE port and the hypervisor hosts on the downlink GbE ports. There's 5 hypervisor hosts in total, each one with two cards, one for each switch. VPC is only used in the VM's Network and the access uplink.
- What is the actual issue you are experiencing? Is this a performance problem? Generally, it is not advised to have your ISCSI network on the same hardware that will provide connectivity to the rest of your network. You should really be using a dedicated switch (2 for redundency) for your ISCSI network and segrate that from the normal user data.
We don't have any issues at this moment since this core is not in production, we just want to make it right before putting it on production. I'm aware that separate fabrics should be deployed for SAN traffic. With cheaper and common switches this is true and we rolled iSCSI this way on older equipment. But this switch is a unified fabric model, so it's developed to have mixed SAN and LAN traffic on the same box. The model we choose, the Nexus 3048 is capable of delivering extremely low latency and lossless iSCSI and here is where we need to configure it properly to make everything work as expected.
Thanks for the questions guys. Hope this helps.