4

I'm pretty sure the following should be OK, but I don't have a way of testing before I get on site, so it would be great if anyone can tell me if there is anything specific that I'll need to change in my setup to make this work.

Setup 1 (Current lab setup)

  • We generate traffic by running dpdk pktgen and send it out of a QSFP+ port on an intel NIC: XL710-QDA2
  • This runs via a QSFP+ DAC cable to our analytics machine which uses the same NIC, in promiscuous mode
  • This analytics machine is also running dpdk and receives the traffic for downstream analytics

...all works fine
Setup 2 (on site setup)
We want to test this on site at a location that wants to:

  • Feed traffic from a tap to an Arista switch and send to 4 x 10G SFP+ ports
  • Use a 4x10G DAC to 1 x 40G QSFP+ DAC cable (specifically CAB-Q-S-3M-C)
  • Have our appliance working as above, connecting to our QSFP+ interface & running the above intel NIC in promiscuous mode
  • Then feed into dpdk and onwards for downstream analytics (as in setup 1)

Everything I've read in similar posts seems to say this shouldn't be an issue, actually not even detectable at the analytics machine, but is there anything specific that I need to set up to make the second configuration work? I've read elsewhere some issues people had with QSFP+ not allowing breakouts, but this was for optical not DAC...I just want to avoid getting on site and everything being stalled and I can't mirror their setup in our lab.

Thanks in advance!

  • I've never done the second connection, but I'm curious how four physical ports on one side of a link can be treated like one physical port on the other side. I suspect that if it works, the 40G interface will have to somehow be 4x 10G "virtual" interfaces. Then if you want to treat the whole connection as one interface, you may have to use link aggregation like LACP. And 4x 10G aggregated links will behave a bit differently from 1x 40 GB link. – Todd Wilcox Oct 21 '17 at 4:48
  • Hi Todd, It happens the other way round...the connection has 4 SFP+ lanes bundled up into a single connector (hence Quad SFP+). they can then bundle these to act as a single 40G link or not bundle and have them act as 4 separate 10G links...What I was worrying about was how to make the NIC aware which configuration is being used...but I think I've found a useful document - see my comment to Zac below. – Richard_xd Oct 25 '17 at 11:21
  • Did any answer help you? If so, you should accept the answer so that the question doesn't keep popping up forever, looking for an answer. Alternatively, you can provide and accept your own answer. – Ron Maupin Feb 19 '18 at 19:47
5

Many QSFP+ ports can run in two modes: a single 40GE link or 4x 10GE links (four distinct ports in one connector). The former is not the same as four aggregated 10GE ports as it can transport a single 40G stream which aggregated ports can't.

This is possible since a 40G port uses 4 separate lanes. These can be defined as a single link or four separate links.

  • Thanks - this gave me a clue to google port modes and I found the guide I needed link I'll start to look into this and will report back for anyone hitting this issue in future. – Richard_xd Oct 25 '17 at 11:09
  • Just adding a final comment regarding the outcome of the exercise - everything went OK, thanks for the pointer, it really saved me. However, when we went on site, their network admin had set up the 4x10 end of the cable to logically look like a 1x40, so we spent a while trying to work out why the traffic wasn't appearing...but we got there after a little while. That was the only hiccup other than the networking card initially not responding as it had shaken a little loose in transit... – Richard_xd Nov 8 '17 at 15:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.