How do you know if a Cisco switch is handling traffic well?

Question

I have just read Jeremy's question about 2960-S and iSCSI traffic handling, which drove me to ask this question: We all have lots of switches deployed in distinct infrastructures, how can you know that the traffic is treated well?

I usually check CPU, RAM and system logs but didn't think about buffers.

I used show buffers failures and here is an example of the data:

show buffers failures 
Caller       Pool          Size      When
0x1279AE0   Middle           445    3d19h
0x1279AE0   Middle           446    3d19h
0x1279AE0   Middle           444    3d19h

How can I know on which interface this occurred?

Are there other performance metrics that should be reviewed?

Answer 1

That depends on the switch model you have and it's not that easy.

Smaller Catalyst switches in general use at least two forms of buffer - there's usually a interface-lavel buffer, on the smaller Catalyst (2k/3k) visible in 'show buffers' under the section named 'Interface buffer pools:':

Interface buffer pools:
Syslog ED Pool buffers, 600 bytes (total 132, permanent 132):
   100 in free list (132 min, 132 max allowed)
   11151 hits, 0 misses
RxQ1 buffers, 2040 bytes (total 128, permanent 128):
   4 in free list (0 min, 128 max allowed)
   244594209 hits, 4559839 fallbacks
RxQ2 buffers, 2040 bytes (total 128, permanent 128):
   1 in free list (0 min, 128 max allowed)
   202559241 hits, 1582494 fallbacks, 0 trims, 0 created
   1582494 failures (0 no memory)
...

...and there's "public" area, where CPU is using the buffers as they're punted towards it and need servicing. The "Rx"-level buffers are part of the shared buffer to service all interfaces (on either old, legacy switches like 2950 or newest 2960S/3560X/etc) or a subset of interfaces, belonging to specific port ASIC (like 2960 or 3560/3750/3560E/3750E).

On the 4500 and 6500 in particular it gets messy, as there is a number of pools that packet can go through - input interface (ASIC) buffer, the pool at linecard level (on the 6500 at DFC), at the switch-fabric level and at the end the buffer at Supervisor level. They don't have to be physically separate memory pools, but are often mapped in different commands to different names to ease the troubleshooting process (at which step of the packet walk-through was the packet dropped for example).

Answer 2

You've got great replies so far.

Like Lukasz says the buffer setup is a bit complex on the Catalyst switches. There is a pool that you assign buffers from for the 4 egress queues. You can choose to reserve a certain amount of buffers and let the rest sit in a pool and use it as needed. The buffers that are reserved can't be touched by other queues/thresholds.

To see drops on different queues/thresholds use this command:

show platform port-asic stats drop gix/x

Answer 3

If you are getting buffer failures, I would use a command like the following:

show interface | i (line protocol|no buffer)

You will need to manually sort through the output a bit, but you are looking for lines were you have more than 0 no buffer errors like this:

FasttEthernet0/24 is up, line protocol is up (connected)
     1557332164 packets input, 3733280910 bytes, 6718 no buffer