Unbalanced port-channel utilization

Question

I have below setup:

+------------------+         vPC           +------------------+
|                  |                       |                  |
|  NX5K-SW01       +-----------------------+       NX5K-SW02  |
|                  |                       |                  |
+------------------+--+                 +--+------------------+
                      |                 |
                      |                 |
                      |                 |
               FET1   |                 |  FET2
                      |                 |
                      |                 |
                      |                 |
                 +----+-----------------+----+
                 |                           |
                 |             NX2K          |
                 |                           |
                 +---------+-----+-----------+
                           |     |
                           |     |
                           |     |
                           |     Eth101/1/37
                           |     +
                   Eth101/1/38   |
                           +     |
                           |     |
                  +--------+-----+--------+
                  |                       |
                  |     Cisco ASA         |
                  |                       |
                  +-----------------------+

Eth101/1/37 and Eth101/1/38 are members of port-channel 22, which is utilized for voice traffic (RTP). Now, when I look at the link utilization, I see below numbers (it's the same on NX5k-SW02 obviously):

NX5K-SW01# show port-channel traffic interface port-channel 22
ChanId      Port Rx-Ucst Tx-Ucst Rx-Mcst Tx-Mcst Rx-Bcst Tx-Bcst
------ --------- ------- ------- ------- ------- ------- -------
22  Eth101/1/38    0.0%    0.0%  50.00%   1.62%    0.0%    0.0%
22  Eth101/1/37    0.0% 100.00%  49.99%  98.37%    0.0% 100.00%

loadbalancing is default on NX5k switches:

NX5K-SW01# show port-channel load-balance

Port Channel Load-Balancing Configuration:
System: source-dest-ip

Port Channel Load-Balancing Addresses Used Per-Protocol:
Non-IP: source-dest-mac
IP: source-dest-ip source-dest-mac

and on the ASA it is: src-dst-ip

The ASA has other port-channel connections to NX2k, which are showing normal loadbalancing 50/50.

Any idea why this port-channel is behaving like this?

Answer 1

The load balancing on a port channel is per flow. All the traffic in a single flow will use the same port channel member. Each flow is limited to a single physical link, but different flows will use different links. The way you have it configured, a flow is determined by the source and destination IP and MAC addresses on the Nexus, meaning that any traffic with the same values for those will always use the same port channel member. There is a hash algorithm that determines which flows use which port channel members, and it is only when you have multiple flows between different devices that you get load balancing across the different links.

You seem to be sending traffic on that port channel from a single source to a single destination, and that traffic will always use the same port channel member. It is also possible that you have multiple flows that just happen to hash to the same port channel member.

If you were to load balance individual frames, rather than flows, you will end up with problems created by out-of-order packet delivery. That would be especially and extremely detrimental to VoIP.