7

Maybe my question will be a bit unusual because I will not ask why something doesn't work. Instead, I will ask why things seem to work okay.

I have ASR-920-24SZ-IM connected to upstream ASR9ks with 2x10G links. Downstream device is cisco 4948E acting as an access device (connected via another 10G link). Services handed off the 4948E are Internet access, a bunch of E-LINEs and IPTV.

Uplinks are moderately utilized - ~20% and 10%. However, I observe output drops on the interface towards 4948E.

 Last clearing of "show interface" counters 01:52:25
 Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 122398
 Queueing strategy: fifo

Since there is no QoS configuration on ASR920 interface towards 4948E, there should be 120kB of buffer available on that port. If my math is correct, this would mean that ASR920 is able to buffer ~0,1 ms worth of traffic during line-rate burst coming from upstream 10G links.

What is interesting, IPTV customers nor monitoring system don't report any issues with multicast traffic, which is sensitive to packet drops. Each IPTV channel is 10-20 Mbps stream (variable or constant bitrate) with 1358 byte packet size.

How is it possible that multicast doesn't seem to suffer despite output drops?

EDIT:

After 48 hours counters look like below:

Last clearing of "show interface" counters 2d05h
Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 1217201
Queueing strategy: fifo
Output queue: 0/40 (size/max)
30 second input rate 576849000 bits/sec, 243662 packets/sec
30 second output rate 3706610000 bits/sec, 374245 packets/sec
 29523227831 packets input, 8591353468212 bytes, 0 no buffer
 Received 44508 broadcasts (0 IP multicasts)
 0 runts, 0 giants, 0 throttles 
 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
 0 watchdog, 977069 multicast, 0 pause input
 50286674450 packets output, 62508590137876 bytes, 0 underruns

Unfortunately I don't know what codec is it, but I'll try to find out.

Test stream is constant bitrate and interval between packets is 500 us.

Timestamp: 0.523377 Diff: 0.000544 Sender: 10.200.200.207:34620 Size:1316
Timestamp: 0.523866 Diff: 0.000489 Sender: 10.200.200.207:34620 Size:1316
Timestamp: 0.524424 Diff: 0.000558 Sender: 10.200.200.207:34620 Size:1316
Timestamp: 0.524935 Diff: 0.000511 Sender: 10.200.200.207:34620 Size:1316
Timestamp: 0.525474 Diff: 0.000539 Sender: 10.200.200.207:34620 Size:1316
Timestamp: 0.525977 Diff: 0.000503 Sender: 10.200.200.207:34620 Size:1316

There is only one explanation that comes to my mind at the moment: bursts are shorter than 500 us. I know that output drops are there and it takes > 100 us to tail drop. If the bursts are 200-300 us long it will cause output drops, but should not affect multicast.

Below I provide some outputs as requested.

ASR920#show interfaces te0/0/27 stats
TenGigabitEthernet0/0/27
      Switching path    Pkts In   Chars In   Pkts Out  Chars Out
           Processor          0          0      22354    8324046
         Route cache          0          0          0          0
   Distributed cache          0          0          0          0
               Total          0          0      22354    8324046

Command sh interfaces te0/0/27 switch doesn't seem to be supported on this platform.

11
  • 1
    Which codec are you using? h.264 SVC by any chance?
    – manish ma
    Sep 2 '17 at 9:22
  • 1
    The command output suggests ~122k packets were dropped in not quite 2hrs. That might not actually be that much. How many packetd were transmittted during the same time period? Sep 2 '17 at 10:10
  • 4
    217201/50286674450 = 24E-6 or 24 out of 1,000,000 packets. I suspect you wouldn't notice it.
    – Ron Trunk
    Sep 3 '17 at 23:04
  • Next week I will carry out additional tests with more "heavy" stream and I'll see if the drops make any impact. However, I'm sure this requires a bit of tuning.
    – mkurek
    Sep 5 '17 at 6:05
  • Which version of code are you using?
    – YLearn
    Sep 6 '17 at 3:47
4
+50

As pointed out in the comments while the drop count looks high, when compared to the total traffic it is actually quite low. The output drop rate is 2.4e-5 or 0.0024% so if the drops occur at regular intervals your test stream would experience a missed packet approximately every 41.7k packets sent. Even multicast should have no trouble recovering from a drop rate that low and an end user will likely not notice anything to complain about. That is also assuming any or all the drops are multicast.

You also seem to be trying to understand how/why the drops are occurring and are looking at bursts as a source of the drops. Is there any reason you believe this to be the case? You haven't provided your version of code or the configuration from the ASR, but I would lean more towards something like a bug such as CSCuw45886 to be the source of your problems.

3
  • Thanks for your reply. I'm running 15.5(3)S4 at the moment. I think that the bug you mentioned should be fixed in this release. I suspect the drops might be caused by bursty traffic because we have many residential customers there and default buffer size on that platform is relatively small. After 5 days output drop rate is 0,04%. Why are you assuming that drops occur at regular intervals?
    – mkurek
    Sep 6 '17 at 9:58
  • @mkurek, yes it should be fixed (unless it was accidentally reintroduced), but there can be other bugs. I am making no assumptions about your issue, but you haven't provided any information on the nature of the drops, whether they are occurring regularly or in bursts. However going from ~30-35% (your reported uplinks plus a %) of a 10G link being utilized to bursting over 100% for only microseconds at a time seems a bit of a stretch. Maybe if you were pushing 60% or more....
    – YLearn
    Sep 6 '17 at 18:53
  • 1
    As for drops happening constantly/at intervals/dependent on time-of-day ... Do you happen to have an NMS system running that SNMP-collects and graphs drop count (or delta-of-drop-counts over a given polling interval)? The graph might help to reveal if the occurrence of drops is actually constant, related to office hours (i.e. user activity related, possibly non-multicast), or if it relates to other patterns (i.e. multicast traffic above a given threshold). Sep 6 '17 at 19:56
-2

Drop rate was so low that can be ignored

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.