1

I have the following topology set on AWS where there are 4 CSR 1000v Routers deployed and Segment Routing enabled in which the Segment Routing Headend is R1 and the destination is R3.

I aim to send iperf traffic from Endpoint1(Client) to Endpoint2 (Server) with the following SR path = {R1, R2, R3, R4, R3, R2, R1, R2, and R3}. enter image description here First of all, the traffic generated by iperf from Endpoint1 a linearly increasing UDP traffic until I stop manually. Using SNMP and Cacti I could get throughput across the links (GRE Tunnels) which are summarized below.

I set all tunnels' bandwidth as 10Mbps, I was expecting to see packet loss but I didn't. It is obvious from the plots that the throughput has exceeded the tunnels' bandwidth. However, no packet loss is recorded.

enter image description here enter image description here

here's the tunnels' configuration:

interface Tunnel0
 bandwidth 10000
 tunnel bandwidth transmit 10000
 tunnel bandwidth receive 10000
 ip address 10.10.2.1 255.255.255.252
 ip router isis aws
 load-interval 30
 mpls traffic-eng tunnels
 keepalive 2 3
 tunnel source GigabitEthernet1
 tunnel destination 52.27.173.12
 tunnel path-mtu-discovery
 isis metric 1

After such gib amount of traffic, the txload is reached the overload which supposed to happen seeing drops on the tunnel. However, there are no losses 0 output errors.

ip-172-1-1-13#sh int t0                 
Tunnel0 is up, line protocol is up 
  Hardware is Tunnel
  Internet address is 10.10.2.1/30
  MTU 9976 bytes, BW 10000 Kbit/sec, DLY 50000 usec, 
     reliability 255/255, txload 255/255, rxload 200/255
  Encapsulation TUNNEL, loopback not set
  Keepalive set (2 sec), retries 3
  Tunnel linestate evaluation up
  Tunnel source 172.1.1.13 (GigabitEthernet1), destination 52.38.167.137
   Tunnel Subblocks:
      src-track:
         Tunnel0 source tracking subblock associated with GigabitEthernet1
          Set of tunnels with source GigabitEthernet1, 2 members (includes iterators), on interface <OK>
  Tunnel protocol/transport GRE/IP
    Key disabled, sequencing disabled
    Checksumming of packets disabled
  Tunnel TTL 255, Fast tunneling enabled
  Path MTU Discovery, ager 10 mins, min MTU 92
  Tunnel transport MTU 1476 bytes
  Tunnel transmit bandwidth 10000 (kbps)
  Tunnel receive bandwidth 10000 (kbps)
  Last input 00:00:05, output 00:00:00, output hang never
  Last clearing of "show interface" counters 01:18:49
  Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/0 (size/max)
  30 second input rate 32000 bits/sec, 5 packets/sec
  30 second output rate 59000 bits/sec, 8 packets/sec
     1221047 packets input, 936187461 bytes, 0 no buffer
     Received 0 broadcasts (0 IP multicasts)
     0 runts, 0 giants, 0 throttles 
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     2938865 packets output, 2309121792 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out

I tried to configure a Policing bandwidth on the interface's tunnel which is the following:

    class-map match-all acgroup2
 match access-group 2
Policy Map police
        Class acgroup2
         police cir 10000000 bc 250000 be 250000
           conform-action transmit 
           exceed-action drop 
           violate-action drop

and have it enabled on the tunnels interfaces service-policy input police. However, nothing is shown in Cacti as dropped packets.

Here it is the full configuration of the routers:

R1 

    segment-routing mpls
     global-block 17000 18000
     !
     connected-prefix-sid-map
      address-family ipv4
       11.11.11.11/32 index 11 range 1 
      exit-address-family
    !
    class-map match-all acgroup2
 match access-group 2
        !
        Policy Map police
        Class acgroup2
         police cir 10000000 bc 250000 be 250000
           conform-action transmit 
           exceed-action drop 
           violate-action drop 
        !
        interface Loopback0
         ip address 11.11.11.11 255.255.255.255
        !
        interface Tunnel0
     bandwidth 10000
     tunnel bandwidth transmit 10000
     tunnel bandwidth receive 10000
         ip address 10.10.1.1 255.255.255.252
         ip router isis aws
         load-interval 30
         mpls traffic-eng tunnels
         keepalive 2 3
         tunnel source GigabitEthernet1
         tunnel destination 52.27.173.12
         tunnel path-mtu-discovery
         isis metric 1
         service-policy output police
        !
        interface Tunnel1
     bandwidth 10000
     tunnel bandwidth transmit 10000
     tunnel bandwidth receive 10000
         ip flow monitor NTAMonitor input
         ip flow monitor NTAMonitor output
         ip address 10.10.4.2 255.255.255.252
         ip router isis aws
         load-interval 30
         mpls traffic-eng tunnels
         keepalive 2 3
         tunnel source GigabitEthernet1
         tunnel destination 54.70.66.102
         tunnel path-mtu-discovery
         isis metric 1
         service-policy output police
        !
        interface Tunnel4
         description MPLS TE Tunnel1 to the destination for path SR1
         ip unnumbered Loopback0
         tunnel mode mpls traffic-eng
         tunnel destination 33.33.33.33
         tunnel mpls traffic-eng autoroute announce
         tunnel mpls traffic-eng path-option 1 explicit name SR1 segment-routing verbatim
        !
        interface Tunnel5
         description MPLS TE Tunnel2 to the destination for path SR2
         ip unnumbered Loopback0
         tunnel mode mpls traffic-eng
         tunnel destination 33.33.33.33
         tunnel mpls traffic-eng autoroute announce
         tunnel mpls traffic-eng path-option 1 explicit name SR2 segment-routing verbatim
        !         
        interface VirtualPortGroup0
         vrf forwarding GS
         ip address 192.168.35.101 255.255.255.0
         ip nat inside
         no mop enabled
         no mop sysid
        !
        interface GigabitEthernet1
         ip flow monitor NTAMonitor input
         ip flow monitor NTAMonitor output
         ip address dhcp
         ip nat outside
         load-interval 30
         negotiation auto
         ipv6 address dhcp
         ipv6 enable
         no mop enabled
         no mop sysid
         service-policy output police
        !
        router isis aws
         net 49.0001.0000.0000.0011.00
         metric-style wide
         segment-routing mpls
         segment-routing prefix-sid-map advertise-local
         passive-interface Loopback0
         mpls traffic-eng router-id Loopback0
        !
        iox
        ip forward-protocol nd
        ip tcp window-size 8192
        ip http server
        ip http authentication local
        ip http secure-server
        ip nat inside source list GS_NAT_ACL interface GigabitEthernet1 vrf GS overload
        ip route 0.0.0.0 0.0.0.0 GigabitEthernet1 172.0.1.1
        ip route 33.33.33.33 255.255.255.255 Tunnel4
        ip route 33.33.33.33 255.255.255.255 Tunnel5
        ip route 172.2.1.5 255.255.255.255 Tunnel4
        ip route 172.2.1.7 255.255.255.255 Tunnel5
        ip route vrf GS 0.0.0.0 0.0.0.0 GigabitEthernet1 172.0.1.1 global
        !
        ip explicit-path name SR1 enable
         index 1 next-label 17022
         index 2 next-label 17033
         index 3 next-label 17044
         index 4 next-label 17033
         index 5 next-label 17022
         index 6 next-label 17011
         index 7 next-label 17022
         index 8 next-label 17033
        !
        ip ssh rsa keypair-name ssh-key
        ip ssh version 2
        ip ssh pubkey-chain
        ip scp server enable
        !
        access-list 100 permit ip any any
        ipv6 route ::/0 GigabitEthernet1 FE80::83F:37FF:FE14:2840
        !
        !
        snmp-server community public RO
        snmp-server community private RW
        !
        !
        control-plane
        !
        line con 0
         stopbits 1
        line vty 0 4
         login local
         transport input ssh
        !
        app-hosting appid guestshell
         app-vnic gateway1 virtualportgroup 0 guest-interface 0
          guest-ipaddress 192.168.35.102 netmask 255.255.255.0
         app-default-gateway 192.168.35.101 guest-interface 0
         name-server0 8.8.8.8
        end

R2 

segment-routing mpls
 global-block 17000 18000
 !
 connected-prefix-sid-map
  address-family ipv4
   22.22.22.22/32 index 22 range 1 
  exit-address-family
 !
!
spanning-tree extend system-id
!
netconf-yang
!
restconf
!
redundancy    
!
interface Loopback0
 ip address 22.22.22.22 255.255.255.255
!
interface Tunnel0
 bandwidth 10000
 tunnel bandwidth transmit 10000
 tunnel bandwidth receive 10000
 ip flow monitor NTAMonitor input
 ip flow monitor NTAMonitor output
 ip address 10.10.2.1 255.255.255.252
 ip router isis aws
 load-interval 30
 mpls traffic-eng tunnels
 keepalive 2 3
 tunnel source GigabitEthernet1
 tunnel destination 52.38.167.137
 tunnel path-mtu-discovery
 isis metric 1
!
interface Tunnel1
 bandwidth 10000
 ip flow monitor NTAMonitor input
 ip flow monitor NTAMonitor output
 ip address 10.10.1.2 255.255.255.252
 ip router isis aws
 load-interval 30
 mpls traffic-eng tunnels
 keepalive 2 3
 tunnel source GigabitEthernet1
 tunnel destination 35.167.193.103
 tunnel path-mtu-discovery
 isis metric 1
!
interface VirtualPortGroup0
 vrf forwarding GS
 ip address 192.168.35.101 255.255.255.0
 ip nat inside
 no mop enabled
 no mop sysid
!
interface GigabitEthernet1
 ip flow monitor NTAMonitor input
 ip flow monitor NTAMonitor output
 ip address dhcp
 ip nat outside
 load-interval 30
 negotiation auto
 ipv6 address dhcp
 ipv6 enable
 no mop enabled
 no mop sysid
!
router isis aws
 net 49.0001.0000.0000.0022.00
 metric-style wide
 segment-routing mpls
 segment-routing prefix-sid-map advertise-local
 passive-interface Loopback0
 mpls traffic-eng router-id Loopback0
!
iox
ip forward-protocol nd
ip tcp window-size 8192
ip http server
ip http authentication local
ip http secure-server
ip nat inside source list GS_NAT_ACL interface GigabitEthernet1 vrf GS overload
ip route 0.0.0.0 0.0.0.0 GigabitEthernet1 172.1.1.1
ip route vrf GS 0.0.0.0 0.0.0.0 GigabitEthernet1 172.1.1.1 global
!
ip ssh rsa keypair-name ssh-key
ip ssh version 2
ip scp server enable
!
!
ip access-list standard GS_NAT_ACL
 permit 192.168.35.0 0.0.0.255
ipv6 route ::/0 GigabitEthernet1 FE80::83E:87FF:FEAA:8604
!      
snmp-server community public RO
snmp-server community private RW
!
!
control-plane
!
line con 0
 stopbits 1
line vty 0 4
 login local
 transport input ssh
!
app-hosting appid guestshell
 app-vnic gateway1 virtualportgroup 0 guest-interface 0
  guest-ipaddress 192.168.35.102 netmask 255.255.255.0
 app-default-gateway 192.168.35.101 guest-interface 0
 name-server0 8.8.8.8
end

R3 

segment-routing mpls
 global-block 17000 18000
 !
 connected-prefix-sid-map
  address-family ipv4
   33.33.33.33/32 index 33 range 1 
  exit-address-family
 !
spanning-tree extend system-id
!
netconf-yang
!
restconf
!
!
redundancy
!
interface Loopback0
 ip address 33.33.33.33 255.255.255.255
!
interface Tunnel0
 bandwidth 10000
 tunnel bandwidth transmit 10000
 tunnel bandwidth receive 10000
 ip flow monitor NTAMonitor input
 ip flow monitor NTAMonitor output
 ip address 10.10.3.1 255.255.255.252
 ip router isis aws
 load-interval 30
 mpls traffic-eng tunnels
 keepalive 2 3
 tunnel source GigabitEthernet1
 tunnel destination 54.70.66.102
 tunnel path-mtu-discovery
 isis metric 1
!
interface Tunnel1
 bandwidth 10000
 ip flow monitor NTAMonitor input
 ip flow monitor NTAMonitor output
 ip address 10.10.2.2 255.255.255.252
 ip router isis aws
 load-interval 30
 mpls traffic-eng tunnels
 keepalive 2 3
 tunnel source GigabitEthernet1
 tunnel destination 52.27.173.12
 tunnel path-mtu-discovery
 isis metric 1
!
interface VirtualPortGroup0
 vrf forwarding GS
 ip address 192.168.35.101 255.255.255.0
 ip nat inside
 no mop enabled
 no mop sysid
!
interface GigabitEthernet1
 ip flow monitor NTAMonitor input
 ip flow monitor NTAMonitor output
 ip address dhcp
 ip nat outside
 load-interval 30
 negotiation auto
 ipv6 address dhcp
 ipv6 enable
 no mop enabled
 no mop sysid
 service-policy input police
!
router isis aws
 net 49.0001.0000.0000.0033.00
 metric-style wide
 segment-routing mpls
 segment-routing prefix-sid-map advertise-local
 passive-interface Loopback0
 mpls traffic-eng router-id Loopback0
!
iox
ip forward-protocol nd
ip tcp window-size 8192
ip http server
ip http authentication local
ip http secure-server
ip nat inside source list GS_NAT_ACL interface GigabitEthernet1 vrf GS overload
ip route 0.0.0.0 0.0.0.0 GigabitEthernet1 172.2.1.1
ip route 172.0.1.0 255.255.255.240 11.11.11.11
ip route vrf GS 0.0.0.0 0.0.0.0 GigabitEthernet1 172.2.1.1 global
!
ip ssh rsa keypair-name ssh-key
ip ssh version 2
ip ssh pubkey-chain
ip scp server enable
!
ip access-list standard GS_NAT_ACL
 permit 192.168.35.0 0.0.0.255
ipv6 route ::/0 GigabitEthernet1 FE80::893:B3FF:FED5:7104      
!
snmp-server community public RO
snmp-server community private RW
!
control-plane
!
line con 0
 stopbits 1
line vty 0 4
 login local
 transport input ssh
!
app-hosting appid guestshell
 app-vnic gateway1 virtualportgroup 0 guest-interface 0
  guest-ipaddress 192.168.35.102 netmask 255.255.255.0
 app-default-gateway 192.168.35.101 guest-interface 0
 name-server0 8.8.8.8
end

R4

segment-routing mpls
 global-block 17000 18000
 !
 connected-prefix-sid-map
  address-family ipv4
   44.44.44.44/32 index 44 range 1 
  exit-address-family
 !
!
spanning-tree extend system-id
!
netconf-yang
!
restconf
!

!
redundancy
!
interface Loopback0
 ip address 44.44.44.44 255.255.255.255
!
interface Tunnel0
 bandwidth 10000
 tunnel bandwidth transmit 10000
 tunnel bandwidth receive 10000
 ip flow monitor NTAMonitor input
 ip flow monitor NTAMonitor output
 ip address 10.10.4.1 255.255.255.252
 ip router isis aws
 load-interval 30
 mpls traffic-eng tunnels
 keepalive 2 3
 tunnel source GigabitEthernet1
 tunnel destination 35.167.193.103
 tunnel path-mtu-discovery
 isis metric 1
!
interface Tunnel1
 bandwidth 10000
 tunnel bandwidth transmit 10000
 tunnel bandwidth receive 10000
 ip flow monitor NTAMonitor input
 ip flow monitor NTAMonitor output
 ip address 10.10.3.2 255.255.255.252
 ip router isis aws
 load-interval 30
 mpls traffic-eng tunnels
 keepalive 2 3
 tunnel source GigabitEthernet1
 tunnel destination 52.38.167.137
 tunnel path-mtu-discovery
 isis metric 1
!
interface VirtualPortGroup0
 vrf forwarding GS
 ip address 192.168.35.101 255.255.255.0
 ip nat inside
 no mop enabled
 no mop sysid
!
interface GigabitEthernet1
 ip flow monitor NTAMonitor input
 ip flow monitor NTAMonitor output
 ip address dhcp
 ip nat outside
 load-interval 30
 negotiation auto
 ipv6 address dhcp
 ipv6 enable
 no mop enabled
 no mop sysid
!
router isis aws
 net 49.0001.0000.0000.0044.00
 metric-style wide
 segment-routing mpls
 segment-routing prefix-sid-map advertise-local
 passive-interface Loopback0
 mpls traffic-eng router-id Loopback0
!
iox
ip forward-protocol nd
ip tcp window-size 8192
ip http server
ip http authentication local
ip http secure-server
ip nat inside source list GS_NAT_ACL interface GigabitEthernet1 vrf GS overload
ip route 0.0.0.0 0.0.0.0 GigabitEthernet1 172.3.1.1
ip route vrf GS 0.0.0.0 0.0.0.0 GigabitEthernet1 172.3.1.1 global
!
ip ssh rsa keypair-name ssh-key
ip ssh version 2
ip ssh pubkey-chain
ip scp server enable
!
ip access-list standard GS_NAT_ACL
 permit 192.168.35.0 0.0.0.255
ipv6 route ::/0 GigabitEthernet1 FE80::D:1CFF:FE1E:97C2
!       
snmp-server community private RW
snmp-server community public RO
!
control-plane
!
line con 0
 stopbits 1
line vty 0 4
 login local
 transport input ssh
!
app-hosting appid guestshell
 app-vnic gateway1 virtualportgroup 0 guest-interface 0
  guest-ipaddress 192.168.35.102 netmask 255.255.255.0
 app-default-gateway 192.168.35.101 guest-interface 0
 name-server0 8.8.8.8
end

EDIT

This is the GRE tunnel description: 

ip-172-0-1-8#sh int t0
Tunnel0 is up, line protocol is up 
  Hardware is Tunnel
  Internet address is 10.10.1.1/30
  MTU 9976 bytes, BW 10000 Kbit/sec, DLY 50000 usec, 
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation TUNNEL, loopback not set
  Keepalive set (2 sec), retries 3
  Tunnel linestate evaluation up
  Tunnel source 172.0.1.8 (GigabitEthernet1), destination 52.27.173.12
   Tunnel Subblocks:
      src-track:
         Tunnel0 source tracking subblock associated with GigabitEthernet1
          Set of tunnels with source GigabitEthernet1, 2 members (includes iterators), on interface <OK>
  Tunnel protocol/transport GRE/IP
    Key disabled, sequencing disabled
    Checksumming of packets disabled
  Tunnel TTL 255, Fast tunneling enabled
  Path MTU Discovery, ager 10 mins, min MTU 92
  Tunnel transport MTU 1476 bytes
  Tunnel transmit bandwidth 10000 (kbps)
  Tunnel receive bandwidth 10000 (kbps)
  Last input 00:00:03, output 00:00:00, output hang never
  Last clearing of "show interface" counters 00:03:26
  Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/0 (size/max)
  30 second input rate 18000 bits/sec, 2 packets/sec
  30 second output rate 0 bits/sec, 0 packets/sec
     495 packets input, 455872 bytes, 0 no buffer
     Received 0 broadcasts (0 IP multicasts)
     0 runts, 0 giants, 0 throttles 
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     179 packets output, 77152 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out
Improve this question
3
  • 2
    The bandwidth command is simply a label for things like routing protocols and QoS. It does not change the bandwidth on the interface.
    – Ron Maupin ♦
    Sep 17, 2019 at 11:53
  • @RonMaupin is there any way to see drops via SNMP? I have tried traffic policing but it did not work.
    – Khalil Mebarkia
    Sep 17, 2019 at 11:57
  • Your policy is actually allowing bursts above 10 Mbps. You need to just allow a 10 Mbps CIR and drop everything above that.
    – Ron Maupin ♦
    Sep 17, 2019 at 13:33

1 Answer 1

Reset to default
5

The bandwidth command is simply a label for things like routing protocols and QoS. It does not change the bandwidth on the interface.

If you want it to drop packets above 10 Mbps, you can create a QoS policy to police traffic above 10 Mbps. Something like (created on the fly and not tested):

policy-map Police_10Mbps
 description Police above 10 Mbps
  class class-default
    police cir 10000 conform-action transmit exceed-action drop
!
interface Tunnel0
 service-policy output Police_10Mbps
!
Improve this answer
7
  • Thanks Ron! Well my first observation you wrote police cir 10000 which is 0.01Mbps according to #police cir ? <8000-10000000000> Target Bit Rate (biTs per second) (postfix k, m, g optional; decimal point allo I fixed for the value 10000000 On the other hand, I want to policy map to drop packet when exactly the load is 10Mbps. Because now, when the load is for example 8900kbps, there are drops. Is there any way to fix this?
    – Khalil Mebarkia
    Sep 17, 2019 at 16:21
  • 1
    That depends on which layer you are measuring. For example, if you are measuring the data throughput, you will lose bandwidth to the data-link, network, and transport protocol headers. Also, you must take into account the reduced MTU of the tunnel, and if you try to use 1500 byte packets, they will need to be fragmented, creating multiple packets from a single packet, with even more network protocol overhead.
    – Ron Maupin ♦
    Sep 17, 2019 at 16:33
  • thanks for clarifying things. I apologize, but how am I supposed to fix that? Just modifying the MTU value? I just have modified the question including the Tunnel description.
    – Khalil Mebarkia
    Sep 18, 2019 at 9:44
  • 1
    Your tunnel as a 10 Mbps virtual link. The physical ethernet connection has the data-link seven bytes of Preamble, one byte of SoF Delimiter, 14 bytes of ethernet header, four bytes of FCS, and 12 bytes of Inter-frame Gap between each ethernet frame. Then, you have 20 bytes of IPv4 header, and the tunnel header, plus 20 bytes of IP header and eight bytes of UDP header inside the tunnel. The tunnel MTU will be smaller than the physical MTU, otherwise you will fragment at the physical interface. (Ethernet MTU) - (IP header) - (tunnel header). You seem to want throughput, not bandwidth.
    – Ron Maupin ♦
    Sep 18, 2019 at 10:54
  • That's true. I aim to measure the throughput on the tunnels using SNMP. So according to what you just wrote, what shall the tunnel MTU value?
    – Khalil Mebarkia
    Sep 18, 2019 at 11:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.