I do not know if it is the right place to ask. It may be a very silly question. I assume that some processor has to process data frames for switching/routing. Modern processors have speed of few GHz. How do they handle data that is coming at a rate faster than they operate?
You are totally correct, if we have to use an instruction cycle per bit then 10Gbps would be unachievable. So the first thing to note is that we handle a word per CPU instruction -- 64 bits.
Even then the worst thing we can do for performance is to have the CPU access all of the words of a packet. Thus the focus on "zero-copy" handling of packets. Some of that trickery is in the interfaces themselves: they have DMA ("Direct memory access") so that the ethernet controller chip copies the data into RAM; they calculate the checksums so that the CPU doesn't have to access all the words in the packet to do so. Some of it is in the data structure design: we are careful to align packet buffers so we can move them by changing the owndership of a page table entry. Some of it is just careful programming to ensure that packet data is accessed the least number of times, and preferrably not accessed at all until the receiving application program.
Once we've done all of this the next limitation is the overhead of handling packets one at a time. Thus there are a heap of "segmentation offload" features both in the ethernet controller and in the kernel so that we handle groups of packets. We even delaying retrieving data from the ethernet controller so that these groups are larger.
Finally we have special-case shortcuts, such as the kernel's sendfile() call which is an express path from disk to network using the minimal amount of work.
We can even special-case routing (the forwarding of packets from one interface to another) using the hardware features of the network interface cards and treating the PCI bus as a bus between the cards rather than getting the CPU involved. That can't be done in general purpose operating systems, but vendors like Intel provide software libraries to implement such features on their ethernet controllers.
Moving away from CPUs altogether, we can even build special-purpose routers where all forwarding tasks happen in hardware. Since the PCI bus would then be a limitation they run multiple parallel busses; or even multiple parallel busses to multiple parallel crossbar switch assemblies. At one end of the market a small TCAM-based ethernet switch would be one example; at the other end of the market the Juniper M40 would be a canonical design.
A typical switch will start to receive a packet, look up the destination address in the TCAM, attach a tag with the egress-port to the packet, and then DMA the still-incoming packet to the egress port's controller. Note that if the output port is congested then all that can be done on this simple switch is to throw away the ingress packet. Thus simple switches do not make a good choice for when links change speed and some queueing is desirable. Of course more sophisticated switches exist, for which you pay more.
A typical router will receive a packet and hold it in a short queue. The destination IP address will be looked up in static RAM, the packet will then be exploded into cells to reduce latency, and each cell send to a cross-bar switch to the egress card. That card will reassemble the cells into a packet, and queue the packet out the egress interface. The queuing on the egress interface can be sophisticated.
Great answer. Can you elaborate on
the packet will then be exploded into cells to reduce latency?– EddieMay 17, 2016 at 14:46
In a router design you could simply send a packet from one line-card to another via the cross-bar switching (or some other sort of inter-card bus). But then the latency is bound by the length of the packet -- you would have another complete packet transmit delay as the packet is sent through the crossbar switching. To avoid that we can have multiple parallel links into the crossbar switching and split the packet across those links. Then the latency for large packets is much reduced. A part of a split-up packet is called a 'cell'.– vk5tuMay 17, 2016 at 23:37
I know this answer is from two years ago, but THANK YOU SO MUCH. I had the same question as the original poster, and found your answer in my search. It is VERY well written and thorough. Thanks!– loneboatMay 10, 2018 at 23:54
Today, almost all switching, and much of routing, is handled in hardware, so the processor speed comes into play for exceptions. For things like servers, it is possible that the processor isn't fast enough. This has been the case in the past. When 1 Gbps ethernet first came out, the bus used in PCs and servers could only handle 400 Mbps.
What happens when the processor isn't fast enough is that traffic gets dropped. A lot of traffic may be dropped, anyway, since that is how congestion is handled, if done correctly. RED (Random Early Detection) is a method used to randomly drop packets in queues in order to prevent them from filling and tail-dropping packets. This can help to prevent TCP synchronization. A lot of drops occur on switches, where multiple ports of a speed may need to send to another single port of the same speed.
1This confuses ingress discards with egress drops. Discards typically happen when the ring-buffer of the ethernet interface is over-filled -- as happens when a CPU can't keep pace with the arrival rate of data. That ring-buffer is too small for RED to be implemented.– vk5tuMay 17, 2016 at 12:32
@vk5tu, you missed my point that traffic gets dropped all the time, whether ingress or egress. If any part of the system can't handle the amount of traffic, some will be dropped, and some is dropped on purpose.– Ron Maupin ♦May 17, 2016 at 14:24