I have a problem on my Cisco 6509, each entry in my BGP table occupies two entries in the TCAM. If I show capacity forwarding, I see MPLS entries in L3 forwarding resources. But, I do not use MPLS on my chassis !

#show run | i mpls
mls cef maximum-routes mpls 508
no mpls ldp advertise-labels
no mpls ip

And my L3 Forwading :

L3 Forwarding Resources
             FIB TCAM usage:                     Total        Used       %Used
                  72 bits (IPv4, MPLS, EoM)     1032192      899612         87%
                 144 bits (IP mcast, IPv6)        8192           7          1%

                     detail:      Protocol                    Used       %Used
                                  IPv4                      450051         44%
                                  MPLS                      449560         44%
                                  EoM                            1          1%

                                  IPv6                           1          1%
                                  IPv4 mcast                     3          1%
                                  IPv6 mcast                     3          1%

            Adjacency usage:                     Total        Used       %Used
                                               1048576      448758         43%

Any idea? Could it be that the routes are in a VRF?

  • +1 Interesting question. Can you add in your version of IOS for comparison with Bigmstone's answer? – jwbensley May 19 '13 at 10:10
  • Oups, my IOS version is s72033_rp-ADVENTERPRISEK9_WAN-M - Version 12.2(33)SXH3a – Johann M. Jun 11 '14 at 17:49

It seems that the 6500 generates MPLS labels for every route if BGP is run in VRF. The fact that your IPv4 and MPLS TCAM usage is almost identical seems to indicate this as well. Can you try this command:

show bgp vpnv4 uni all labels

There seems to be a hidden command that makes IOS allocate labels per VRF instead of per prefix.

mpls label mode all-vrfs protocol bgp-vpnv4 per-vrf

This is a hidden command so IOS won't show it. Also before running that can you try to run:

show ip vrf detail
| improve this answer | |
  • 1
    Yes, I have one label per prefix BGP ! #mpls label mode all-vrfs protocol bgp-vpnv4 per-vrf Hum good, but a warning. I see now "IPv4 VRF Aggr:16" for all prefix :) Wait a moment and... IPv4 449979 44% MPLS 8 1% GOOD! Thank you :-) – Johann M. May 18 '13 at 20:40

Oh the 6500. I run a small service provider network and run the 6500 as a PE router. Worst decision of my life. (That was an embellished statement, but you get my point.)

I run full BGP routes in a VRF and have experienced a lot of problem surrounding this.

You're example is not very surprising. As Daniel said in his post there is an LFIB entry for each VRF prefix as well as a VPNv4 entry. This can be changed by adding the command mpls label mode vrf Internet protocol all-afs per-vrf as was stated; however, this does not get you out of the woods. If you change to per VRF prefixes it does remove the LFIB entry (yay!) but adds an entry for every single prefix into the Adjacency table (wait, what?!). Since the 6500 forwarding hardware is shared between L2 and L3 forwarding this doesn't change your hardware memory usage at all. If anything it makes the problem harder to find.

If you look at your usage once you've changed to per VRF usage (using show platform hardware cef resource-level) it looks as though you've fixed the issue. However if you use the command show platform hardware cef adjacencies resource-level it reveals the problem has just moved to a different location.

Below are the outputs from one of my 6500's resource-level and adjacency usage. Outlining what I'm talking about.


Global watermarks: apply to Fib shared area only.
Protocol watermarks: apply to protocols with non-default max-routes

Fib-size: 1024k (1048576), shared-size: 1016k (1040384), shared-usage: 458k(469769)

Global watermarks:
            Red_WM: 95%,   Greem_WM: 80%,   Current usage: 45%

Protocol watermarks:

 Protocol           Red_WM(%)      Green_WM(%)     Current(%)
 --------           ---------      ----------      ----------
 IPV4                --             --              42% (of shared)
 IPV4-MCAST          --             --              0 % (of shared)
 IPV6                --             --              2 % (of shared)
 IPV6-MCAST          --             --              0 % (of shared)
 MPLS                --             --              0 % (of shared)
 EoMPLS              --             --              0 % (of shared)
 VPLS-IPV4-MCAST     --             --              0 % (of shared)
 VPLS-IPV6-MCAST     --             --              0 % (of shared)

Adjacency Usage

Watermarks apply to regions available for allocation and not pre-reserved
Stats region size for alloc:        444160
Non-stats region size for alloc:    376832

Adjacency Mgr watermarks:

 Type             Red_WM(%)      Green_WM(%)     Current usage(%)
 ----             ---------      ----------      ----------------
 Stats_WM         95%            80%             97%
 Non-Stats_WM     95%            80%             14%

Ivan's post on this was base on my findings here. I am currently working with Cisco to attempt to fix this issue, but unfortunately right now there is no way to fix this.

Your milage may vary since you've got no MPLS adjacencies. Would be interested to see your adjacency usage now that you've made the change.

| improve this answer | |
  • +1 A great addition to Daniels answer. I was thinking of Ivan's post as I was reading your answer, then saw you had linked to it :) You said you ware working on a solution with Cisco, which I presume is a TAC case. Can you add in to your post your IOS version? – jwbensley May 19 '13 at 10:09
  • Great comment! But strangely show platform hardware cef [...] does not exist in my C6509. But if I see show cef fib, it's scary : Totals : 96942392/97131416 ( 99%) [4296] and ADJ: adjacency : 132616/132792 ( 99%) [4] – Johann M. May 19 '13 at 16:29
  • I'm SUP2T. I'm guessing you're SUP720? – bigmstone May 19 '13 at 17:56
  • @javno, I believe 15.1(1)SY. Too lazy to VPN with this crappy Airport wireless. I'll confirm that and edit if it needs to be changed...but I'm pretty sure that's what I'm running. Yeah, I've got a TAC case that has been open for ~6 months now. Working with a couple of engineers to see how best to get it addressed. I'm trying to convince them to implement per next-hop labels...we'll see. – bigmstone May 19 '13 at 18:22
  • @bigmstone : Yes, I'm SUP720 (3BXL) – Johann M. May 19 '13 at 18:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.