What could be the bottleneck of my load-testing case?

Question

I need to bring up a bunch of my devices in the network to hit some other service, for testing purposes. And I have used GNS3 to simulate that.

As you may find in my GNS3 project, I have such following connections:

Cloud <-> Cisco Router <-> 4 Ethernet Switch (ES) <--> 4 rows of Devices connecting to each separate ES.

As of now, 3 rows of such devices are up are running, byways of the "QEMU" command, and they are fully networked as well. And from the htop command of the GNS3 VM, that holds this project, I can see the current loading is at a low percentage.

But somehow, when I was to start the 4th row of my devices, I always don't see they are fully networked, and sometimes I got a timeout warning on the GNS3 side as if something is bottlenecked.

So other than the CPU and MEM issue, What could be the reason for such failure ?

Thanks,

Jack

Answer 1

That may be a simulation artifact but it might also indicate heavy congestion.

In your topology, the bottlenecks are the switch uplinks to the router and possibly the router itself.

You haven't told us what the link bandwidths and the router capabilities are and what flows you're testing. If there are many peer-to-peer connections, the interconnects are heavily congested because of their oversubscription.

In a real-life scenario, that kind of topology might be sufficient for a low-to-medium-traffic access layer. Servers usually carry more traffic and are often connected closer to the core. A large server farm most often requires implementing a fat tree with the uplinks being much faster than the individual downlinks (to reduce or even eliminate oversubscription).

After all, there's no single topology to match all use cases but it needs to be modeled for the requirements.