With regards to Juniper Networks , how does 'targeted-distribution' knob works?

Question

Can someone please explain how does the knob 'targeted-distribution' works in Juniper MX routers.

We created aggregated Ethernet (AE) links on our Juniper MX router acting as BNG. However , after creating bundle links and terminate subscribers on it , it was revealed that subscribers are getting more bandwidth than what was being policed. JTAC suggested to apply knob 'targeted-distribution' under dynamic profile to address the issue. After applying the knob , the problem got solved.

I have tried to learn from Juniper documentation but here is what i understand.

As per my understanding , this knob distributes the dynamic interfaces (created for BNG subscribers) among the member links in a bundle. Some subscribers may be on one member link while other subscribers on other member link. The policier which is being pushed by AAA/SRC against any subscriber will then rate limit traffic on that member link at which subscriber resides because without this knob , subscriber traffic will be hashed on all member links in a bundle and rate limit will be active for each individual member link in a bundle , hence all subscribers getting more speed than their profile on BNG.

Please correct me if I am missing something or I am wrong altogether. And How does it differs from 'shared-bandwidth-policer' knob

Here are dynamic-profile configurations:-

dynamic-profile configurations:


set dynamic-profiles auto-configure-hsi interfaces "$junos-interface-ifd-name" unit "$junos-interface-unit" vlan-id "$junos-vlan-id"
set dynamic-profiles auto-configure-hsi interfaces "$junos-interface-ifd-name" unit "$junos-interface-unit" family pppoe duplicate-protection
set dynamic-profiles auto-configure-hsi interfaces "$junos-interface-ifd-name" unit "$junos-interface-unit" family pppoe dynamic-profile dp-default
set dynamic-profiles auto-configure-hsi interfaces "$junos-interface-ifd-name" unit "$junos-interface-unit" family pppoe short-cycle-protection lockout-time-min 2
set dynamic-profiles auto-configure-hsi interfaces "$junos-interface-ifd-name" unit "$junos-interface-unit" family pppoe short-cycle-protection lockout-time-max 300

set dynamic-profiles dp-default routing-instances "$junos-routing-instance" interface "$junos-interface-name"
set dynamic-profiles dp-default routing-instances "$junos-routing-instance" routing-options access route $junos-framed-route-ip-address-prefix next-hop "$junos-framed-route-nexthop"
set dynamic-profiles dp-default routing-instances "$junos-routing-instance" routing-options access route $junos-framed-route-ip-address-prefix metric "$junos-framed-route-cost"
set dynamic-profiles dp-default routing-instances "$junos-routing-instance" routing-options access route $junos-framed-route-ip-address-prefix preference "$junos-framed-route-distance"
set dynamic-profiles dp-default routing-instances "$junos-routing-instance" routing-options access route $junos-framed-route-ip-address-prefix tag "$junos-framed-route-tag"
set dynamic-profiles dp-default routing-instances "$junos-routing-instance" routing-options access-internal route $junos-subscriber-ip-address qualified-next-hop "$junos-interface-name"
set dynamic-profiles dp-default interfaces pp0 unit "$junos-interface-unit" no-traps
set dynamic-profiles dp-default interfaces pp0 unit "$junos-interface-unit" ppp-options pap
set dynamic-profiles dp-default interfaces pp0 unit "$junos-interface-unit" pppoe-options underlying-interface "$junos-underlying-interface"
set dynamic-profiles dp-default interfaces pp0 unit "$junos-interface-unit" pppoe-options server
set dynamic-profiles dp-default interfaces pp0 unit "$junos-interface-unit" keepalives interval 30
set dynamic-profiles dp-default interfaces pp0 unit "$junos-interface-unit" family inet rpf-check
set dynamic-profiles dp-default interfaces pp0 unit "$junos-interface-unit" family inet filter input default-fw-v4-in
set dynamic-profiles dp-default interfaces pp0 unit "$junos-interface-unit" family inet filter input precedence 200
set dynamic-profiles dp-default interfaces pp0 unit "$junos-interface-unit" family inet filter output default-fw-v4-out
set dynamic-profiles dp-default interfaces pp0 unit "$junos-interface-unit" family inet filter output precedence 200
set dynamic-profiles dp-default interfaces pp0 unit "$junos-interface-unit" family inet unnumbered-address "$junos-loopback-interface"
set dynamic-profiles dp-default firewall family inet filter default-fw-v4-in interface-specific
set dynamic-profiles dp-default firewall family inet filter default-fw-v4-in term first from service-filter-hit
set dynamic-profiles dp-default firewall family inet filter default-fw-v4-in term first then accept
set dynamic-profiles dp-default firewall family inet filter default-fw-v4-in term rl then policer default-rlp-IN
set dynamic-profiles dp-default firewall family inet filter default-fw-v4-in term rl then forwarding-class best-effort
set dynamic-profiles dp-default firewall family inet filter default-fw-v4-in term rl then accept
set dynamic-profiles dp-default firewall family inet filter default-fw-v4-out interface-specific
set dynamic-profiles dp-default firewall family inet filter default-fw-v4-out term first from service-filter-hit
set dynamic-profiles dp-default firewall family inet filter default-fw-v4-out term first then accept
set dynamic-profiles dp-default firewall family inet filter default-fw-v4-out term rl then policer default-rlp-OUT
set dynamic-profiles dp-default firewall family inet filter default-fw-v4-out term rl then accept
set dynamic-profiles dp-default firewall policer default-rlp-IN logical-interface-policer
set dynamic-profiles dp-default firewall policer default-rlp-IN if-exceeding bandwidth-limit 1m
set dynamic-profiles dp-default firewall policer default-rlp-IN if-exceeding burst-size-limit 13k
set dynamic-profiles dp-default firewall policer default-rlp-IN then discard
set dynamic-profiles dp-default firewall policer default-rlp-OUT logical-interface-policer
set dynamic-profiles dp-default firewall policer default-rlp-OUT if-exceeding bandwidth-limit 3m
set dynamic-profiles dp-default firewall policer default-rlp-OUT if-exceeding burst-size-limit 100k
set dynamic-profiles dp-default firewall policer default-rlp-OUT then discard

Please note that we will be adding 'targeted-distribution' knob under dynamic-profile dp-default.

set dynamic-profiles dp-default interfaces pp0 unit "$junos-interface-unit" targeted-distribution 

set dynamic-profiles dp-block variables filter-ports uid
set dynamic-profiles dp-block interfaces pp0 unit "$junos-interface-unit" family inet filter output "$filter-ports"
set dynamic-profiles dp-block interfaces pp0 unit "$junos-interface-unit" family inet filter output precedence 50
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" interface-specific
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_PERMIT from source-address 182.184.25.0/29
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_PERMIT from protocol tcp
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_PERMIT from destination-port 30005
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_PERMIT from destination-port 7547
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_PERMIT from destination-port 58603
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_PERMIT then accept
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term TEMP_PERMIT from source-address 182.180.61.124/32
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term TEMP_PERMIT from protocol tcp
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term TEMP_PERMIT from destination-port 80
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term TEMP_PERMIT from destination-port 8080
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term TEMP_PERMIT from destination-port 443
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term TEMP_PERMIT from destination-port 21
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term TEMP_PERMIT from destination-port 23
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term TEMP_PERMIT then accept
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_TCP from protocol tcp
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_TCP from destination-port 30005
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_TCP from destination-port 7547
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_TCP from destination-port 58603
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_TCP from destination-port 21
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_TCP from destination-port 20
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_TCP from destination-port 80
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_TCP from destination-port 22
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_TCP from destination-port 23
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_TCP from destination-port 2869
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_TCP from destination-port 5431
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_TCP from destination-port 443
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_TCP from destination-port 8080
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_TCP from destination-port 53
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_TCP from destination-port 24
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_TCP from destination-port 25
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_TCP from destination-port 37215
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_TCP then discard
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_UDP from protocol udp
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_UDP from destination-port 53
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_UDP from destination-port 161
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_UDP from destination-port 1900
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_UDP from destination-port 162
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_UDP from destination-port 10301
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_UDP from destination-port 10302
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term ACS_OUT_BLOCK_UDP then discard
set dynamic-profiles dp-block firewall family inet filter "$filter-ports" term default-action then service-filter-hit

set dynamic-profiles dp-ott variables ott1v4
set dynamic-profiles dp-ott variables policer1 mandatory
set dynamic-profiles dp-ott variables policer2 mandatory
set dynamic-profiles dp-ott variables fw_ip-v4-IN uid
set dynamic-profiles dp-ott variables fw_ip-v4-OUT uid
set dynamic-profiles dp-ott variables burst-size-in equals "round($policer1 * 0.125)"
set dynamic-profiles dp-ott variables burst-size-in-k equals "$burst-size-in##'k'"
set dynamic-profiles dp-ott variables Bw-IN-K equals "$policer1##'k'"
set dynamic-profiles dp-ott variables burst-size-out equals "round($policer2 * 0.125)"
set dynamic-profiles dp-ott variables burst-size-out-k equals "$burst-size-out##'k'"
set dynamic-profiles dp-ott variables Bw-OUT-K equals "$policer2##'k'"
set dynamic-profiles dp-ott variables rlp-IN uid
set dynamic-profiles dp-ott variables rlp-OUT uid
set dynamic-profiles dp-ott interfaces "$junos-interface-ifd-name" unit "$junos-interface-unit" family inet filter input "$fw_ip-v4-IN"
set dynamic-profiles dp-ott interfaces "$junos-interface-ifd-name" unit "$junos-interface-unit" family inet filter input precedence 90
set dynamic-profiles dp-ott interfaces "$junos-interface-ifd-name" unit "$junos-interface-unit" family inet filter output "$fw_ip-v4-OUT"
set dynamic-profiles dp-ott interfaces "$junos-interface-ifd-name" unit "$junos-interface-unit" family inet filter output precedence 90
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-IN" interface-specific
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-IN" term ott from destination-address $ott1v4
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-IN" term ott then hierarchical-policer "$rlp-IN"
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-IN" term ott then forwarding-class VoD
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-IN" term ott then service-filter-hit
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-IN" term ott then force-premium
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-IN" term ott then accept
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-IN" term 1 then hierarchical-policer "$rlp-IN"
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-IN" term 1 then service-accounting-deferred
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-IN" term 1 then service-filter-hit
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-IN" term 1 then accept
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-OUT" interface-specific
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-OUT" term ott from source-address $ott1v4
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-OUT" term ott then hierarchical-policer "$rlp-OUT"
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-OUT" term ott then forwarding-class VoD
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-OUT" term ott then service-filter-hit
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-OUT" term ott then force-premium
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-OUT" term ott then accept
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-OUT" term 1 then hierarchical-policer "$rlp-OUT"
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-OUT" term 1 then service-accounting-deferred
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-OUT" term 1 then service-filter-hit
set dynamic-profiles dp-ott firewall family inet filter "$fw_ip-v4-OUT" term 1 then accept
set dynamic-profiles dp-ott firewall hierarchical-policer "$rlp-OUT" logical-interface-policer
set dynamic-profiles dp-ott firewall hierarchical-policer "$rlp-OUT" aggregate if-exceeding bandwidth-limit "$Bw-OUT-K"
set dynamic-profiles dp-ott firewall hierarchical-policer "$rlp-OUT" aggregate if-exceeding burst-size-limit "$burst-size-out-k"
set dynamic-profiles dp-ott firewall hierarchical-policer "$rlp-OUT" aggregate then discard
set dynamic-profiles dp-ott firewall hierarchical-policer "$rlp-OUT" premium if-exceeding bandwidth-limit "$Bw-OUT-K"
set dynamic-profiles dp-ott firewall hierarchical-policer "$rlp-OUT" premium if-exceeding burst-size-limit "$burst-size-out-k"
set dynamic-profiles dp-ott firewall hierarchical-policer "$rlp-OUT" premium then discard
set dynamic-profiles dp-ott firewall hierarchical-policer "$rlp-IN" logical-interface-policer
set dynamic-profiles dp-ott firewall hierarchical-policer "$rlp-IN" aggregate if-exceeding bandwidth-limit "$Bw-IN-K"
set dynamic-profiles dp-ott firewall hierarchical-policer "$rlp-IN" aggregate if-exceeding burst-size-limit "$burst-size-in-k"
set dynamic-profiles dp-ott firewall hierarchical-policer "$rlp-IN" aggregate then discard
set dynamic-profiles dp-ott firewall hierarchical-policer "$rlp-IN" premium if-exceeding bandwidth-limit "$Bw-IN-K"
set dynamic-profiles dp-ott firewall hierarchical-policer "$rlp-IN" premium if-exceeding burst-size-limit "$burst-size-in-k"
set dynamic-profiles dp-ott firewall hierarchical-policer "$rlp-IN" premium then discard


Hardware Models Summary


Hardware inventory:
Item             Version  Part number  CLEI code         FRU model number
Midplane         REV 09   750-047862   COMLM10BRA        CHAS-BP3-MX480-S
FPM Board        REV 02   710-017254                     CRAFT-MX480-S
PEM 0            Rev 05   740-027736                     PWR-MX480-2400-DC-S
PEM 1            Rev 05   740-027736                     PWR-MX480-2400-DC-S
PEM 2            Rev 05   740-027736                     PWR-MX480-2400-DC-S
PEM 3            Rev 05   740-027736                     PWR-MX480-2400-DC-S
Routing Engine 0 REV 10   740-031116   COUCASYBAA        RE-S-1800X4-16G-S
Routing Engine 1 REV 10   740-031116   COUCASYBAA        RE-S-1800X4-16G-S
CB 0             REV 27   750-031391   COUCATXBAA        SCBE-MX-S
CB 1             REV 27   750-031391   COUCATXBAA        SCBE-MX-S
FPC 1            REV 21   750-038493   COUIBEWBAB        MX-MPC2E-3D-Q
  MIC 0          REV 31   750-028387   COUIA16BAB        MIC-3D-4XGE-XFP
  MIC 1          REV 31   750-028387   COUIA16BAB        MIC-3D-4XGE-XFP
FPC 2            REV 21   750-038493   COUIBEWBAB        MX-MPC2E-3D-Q
  MIC 0          REV 31   750-028387   COUIA16BAB        MIC-3D-4XGE-XFP
  MIC 1          REV 31   750-028387   COUIA16BAB        MIC-3D-4XGE-XFP
Fan Tray                                                 FFANTRAY-MX480-HC-S

show subscriber summary port
user@R1> show subscribers summary port 

Interface           Count 
ae71: xe-2/0/0      6982               
ae71: xe-2/0/1      6982               
ae72: xe-1/0/0      8564               
ae72: xe-1/0/1      8564               
ae73: xe-2/1/0      8141               
ae73: xe-2/1/1      8141               
ae74: xe-1/1/0      6152               
ae74: xe-1/1/1      6152               

Total Subscribers: 29839

Thanks in advance.

Answer 1

It helps to explain the problems that targeted distribution is solving, so let's take a look at that. Basically, without targeted distribution, you have 2 ways to allocate shaping resources to a subscriber. For the sake of example, let's say you have a 4x1G LAG with a 100M subscriber connected.

1. Split the 100M shaper over the 4 LAG members (25M each). This works fine, but large flows will max out at 25M. Not a great experience for your customers. This is effectively what a shared-bandwidth-policer does, it will statically distribute the configured shaping rate across n number of interfaces that a subscriber can use for access (LAG members). You can check out this KB to get a bit more context with regard to subscriber management.

2. Replicate the 100M shaper to all 4 LAG members (100M each). This does fix the first problem, but it also presents a new one in that even though the customer pays for 100M, they could get up to 400M depending on what their traffic patterns. This is what you're seeing, I think. This over subscription may also cause congestion for other subscribers.

Finally, both options consume more hardware scheduling resources than is really required.

In comes targeted-distribution. Essentially, instead of having to choose per-flow OR per-subscriber shaping, you get both. It does this by assigning the subscriber (and corollary resources) to a single LAG member. In addition there are redundancy options where resources are replicated to a backup link within the LAG, either on the same FPC (link level redundancy) or another FPC (FPC redundancy). This makes link failures more transparent to the end customer.

As subscribers connect, they will be assigned to the least loaded LAG member based on the number of current subscriber sessions on that LAG member (not bandwidth). Furthermore, you can assign a distribution-list to ensure that certain LAG members only get subscribers associated with a specific list.

NOTE: In your particular case, you have 2 LAG members per AE. These could be primary and backup if you decide to deploy it this way. If your network eventually demanded increasing the size of those LAGs, then the other links would not become primary or backup until failure conditions arose, this is how the resource allocations are more efficient with targeted distribution.