Switch SPAN Port Overload

Question

I would like to monitor all traffic that passes between devices connected to a switch (no particular switch make or model - I would need to do this for many types of switches encountered on many different networks.) I originally intended to configure the switch to mirror traffic on all ports to the SPAN port, which is then connected to my monitoring device. However, I recall hearing that switches sometimes have load issues when asked to mirror many ports at once.

Is this a genuine concern that could prevent this from working? If so, is there any alternative method I could employ to accomplish this?

Thank you.

Answer 1

So a few points to this-

1. SPAN is actually Cisco-specific terminology. The generic is just called port mirroring.

2. There is huge variability in terms of the behavior of port mirroring implementations across different hardware. As an example - in a surprising number of platforms (across different vendors) oversubscribing the amount of traffic being mirrored to a given port can actually cause packet loss on otherwise undersubscribed ports.

3. Keep in mind that mirrored traffic often won't include packets with errors (..they get dropped at ingress on most switches) and that there's also no guarantee that packets will be delivered in order or that duplication will always be suppressed.

4. There are also limits as to the number of mirroring sessions, the number of ports/VLAN's being mirrored and whether the sources are monitored bidirectionally (ex: some switches might allow mirroring input from all 24 ports on a switch not input+output).

So - depending on the switch platform in use there are a bunch of things that can be used to control load and otherwise make the problem a lot more approachable. As Ron mentions, something like Netflow or sFlow can be a great way to go but won't provide a packet-by-packet level of analysis or allow diving into packet payloads.

Being careful with the application of mirroring through the use of ACL's can explicitly exclude/include traffic based on the problem being solved. Some platforms also allow for the truncation of mirrored packets, which can also vastly reduce the sheer volume of information being managed.

Another interesting approach is the use of ACL's to copy/redirect interesting traffic on an in-line basis. This is very different from port mirroring and ends up looking a lot more like policy routing but, again, like much of the above is incredibly hardware dependent.

There are also item like in-line taps that can physically copy the traffic on a given link to your analyzer. This is almost invariably a better solution to a permanent requirement for full traffic visibility but is obviously a lot more expensive and a lot less flexible.

In terms of actual use I tend to see larger organizations focusing first on natural choke points - switch uplinks, routers, firewalls, etc and then adding a capability to look at edge ports on an as-needed basis. Obviously use-cases vary but the sheer volume of data that can be generated can be overwhelming.

Answer 2

If the aggregate bandwidth of the traffic from all the other switch interfaces exceeds the bandwidth of the monitor interface, you will not be able to capture all the traffic, some will need to be dropped, and you will not have any control over what gets dropped.

For example, if you have a switch with twenty-four 1 Gbps access interfaces, you will likely lose some traffic (maybe more than half), even with a 10 Gbps monitor interface, but you will lose a lot of traffic if you only have a 1 Gbps monitor interface.

SPAN is really designed to be used for network troubleshooting. Configure it when you have a problem, and remove it when you have fixed the problem.

If you are simply looking to analyse traffic statistics, many enterprise-grade network devices support NetFlow, or one of its variants. You can set up a NetFlow collector, and use software to create reports. The NetFlow collector doesn't even need to be on the same switch or even network.