9

Recently I came across a new kind of NIC capable of 400Gbps (It kind of blew my mind). Does it mean it's electric circuitry is capable of serializing/deserializing (SERDES) 400G bits per second of data onto a wire while maintaining a relativley clean signal (low SNR)? So bandwidth is really the speed at which bits are serialized ,not traveled across, a link? Does it also indicate the time between 1 and 0 pulses? (But the whole serialized packet moves at the ~speed of light) Or is it something else?

Thanks!

Improve this question
2
  • 5
    Yes, it will really serialize/deserialize 400,000,000,000 bits per second, but not as 1 and 0 pulses. The bits are encoded.
    – Ron Maupin ♦
    Dec 20, 2019 at 15:33
  • 1
    Bandwidth is indeed the speed at which bits are serialized into a link, which is directly related to the time between 1s and 0s multiplied by how many "channels" are in a link to stuff 1s and 0s into. The time it takes for the bits to come out the other side is latency, and does not directly relate to bandwidth (in discussions at this level).
    – Mooing Duck
    Dec 21, 2019 at 1:40

1 Answer 1

Reset to default
15

Does it mean it's electric circuitry is capable of serializing/deserializing (SERDES) 400G bits per second of data onto a wire while maintaining a relativley clean signal (low SNR)?

Yes, that's what 400GE is designed for. The physical coding sublayer (PCS) uses forward error correction (FEC) to achieve a block error rate of 10-13 or better. The acceptable SNR varies with the different PHYs.

400G Ethernet uses multiple 25, 50, or 100 Gbit/s lanes (with up to 53 GBd using PAM-4), so it requires multiple fiber strands or wavelengths (or differential pairs in a backplane). 400GBASE-LR4 currently provides up to 10 km reach (nominally), the upcoming 400GBASE-ER8 is going to support up to 40 km.

Propagation in fiber is generally limited by the fiber's velocity factor, ca. .67 (the reciprocal of the refractive index), so .67 x c0 ≈ 200,000 km/s. Accordingly, on the fiber the PAM-4 symbols at 53 GBd are 3.8 mm "long".

For Ethernet, the nominal bandwidth is present at the top of the physical layer. It includes "high-level" signaling like the preamble and inter-packet gap (IPG) and of course, L2 frame header and footer. However, it excludes line code overhead from PCS.

That way, the exact, usable bandwidth can be very easily calculated: for maximum-sized frames there's 1500 bytes L3 payload and 38 bytes overhead for L1 & L2 in total. Thus, 400GE provides a usable bandwidth for L3 of 400 Gbit/s / 8 bit/byte / 1538 bytes * 1500 bytes = 48,76 GB/s.

Incidentally, Broadcom just announced their new Tomahawk 4 switch chip, sporting 64x 400 Gbit/s ports (or 256x 100 Gbit/s) for a total backplane capacity of 25.6 Tbit/s. The chip with 31 billion transistors must be huge, even in 7 nm. Truly mindblowing...

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.