I'm benchmarking latency of sending a TCP packet from a client to a server. The payload of the packet is the nanosecond timestamp (using
clock_gettime() on Linux) right before the
send() socket function. Basically, my client/server pair measures the time from the client calling
send(), to when the server have
The client/server programs are run on 2 PCs in the same room, same network/router, and both use Solarflare X2522 25Gigabit/s network cards. The server use Solarflare EF_VI kernel bypass, the client uses BSD C Socket with OpenOnload kernel bypass. Both programs use memory pinning + CPU pinning + core isolation + locked CPU clock.
Currently, my latency is ~1.8 microsecond. Each of my TCP packet has size exactly 94 bytes, and I use
TCP_NODELAY option in C socket to force send a packet as soon as
send() is called.
Problem is, IDK how much time is spent in each OSI layer (or software/hardware). So I don't know if performance is limited by hardware (NIC, routers, optical fibres, etc) or software (network stack). Is there anyway to benchmark this?