I have the following topology set on AWS where there are 4 CSR 1000v Routers deployed and Segment Routing enabled in which the Segment Routing Headend is R1 and the destination is R3.
I aim to send iperf traffic from Endpoint1(Client) to Endpoint2 (Server) with the following SR path = {R1, R2, R3, R4, R3, R2, R1, R2, and R3}.
First of all, the traffic generated by iperf from Endpoint1 a linearly increasing UDP traffic until I stop manually. Using SNMP and Cacti I could get throughput across the links (GRE Tunnels) which are summarized below.
I set all tunnels' bandwidth as 10Mbps, I was expecting to see packet loss but I didn't. It is obvious from the plots that the throughput has exceeded the tunnels' bandwidth. However, no packet loss is recorded.
here's the tunnels' configuration:
interface Tunnel0
bandwidth 10000
tunnel bandwidth transmit 10000
tunnel bandwidth receive 10000
ip address 10.10.2.1 255.255.255.252
ip router isis aws
load-interval 30
mpls traffic-eng tunnels
keepalive 2 3
tunnel source GigabitEthernet1
tunnel destination 52.27.173.12
tunnel path-mtu-discovery
isis metric 1
After such gib amount of traffic, the txload
is reached the overload which supposed to happen seeing drops on the tunnel. However, there are no losses 0 output errors
.
ip-172-1-1-13#sh int t0
Tunnel0 is up, line protocol is up
Hardware is Tunnel
Internet address is 10.10.2.1/30
MTU 9976 bytes, BW 10000 Kbit/sec, DLY 50000 usec,
reliability 255/255, txload 255/255, rxload 200/255
Encapsulation TUNNEL, loopback not set
Keepalive set (2 sec), retries 3
Tunnel linestate evaluation up
Tunnel source 172.1.1.13 (GigabitEthernet1), destination 52.38.167.137
Tunnel Subblocks:
src-track:
Tunnel0 source tracking subblock associated with GigabitEthernet1
Set of tunnels with source GigabitEthernet1, 2 members (includes iterators), on interface <OK>
Tunnel protocol/transport GRE/IP
Key disabled, sequencing disabled
Checksumming of packets disabled
Tunnel TTL 255, Fast tunneling enabled
Path MTU Discovery, ager 10 mins, min MTU 92
Tunnel transport MTU 1476 bytes
Tunnel transmit bandwidth 10000 (kbps)
Tunnel receive bandwidth 10000 (kbps)
Last input 00:00:05, output 00:00:00, output hang never
Last clearing of "show interface" counters 01:18:49
Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/0 (size/max)
30 second input rate 32000 bits/sec, 5 packets/sec
30 second output rate 59000 bits/sec, 8 packets/sec
1221047 packets input, 936187461 bytes, 0 no buffer
Received 0 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
2938865 packets output, 2309121792 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 output buffer failures, 0 output buffers swapped out
I tried to configure a Policing bandwidth on the interface's tunnel which is the following:
class-map match-all acgroup2
match access-group 2
Policy Map police
Class acgroup2
police cir 10000000 bc 250000 be 250000
conform-action transmit
exceed-action drop
violate-action drop
and have it enabled on the tunnels interfaces service-policy input police
. However, nothing is shown in Cacti
as dropped packets.
Here it is the full configuration of the routers:
R1
segment-routing mpls
global-block 17000 18000
!
connected-prefix-sid-map
address-family ipv4
11.11.11.11/32 index 11 range 1
exit-address-family
!
class-map match-all acgroup2
match access-group 2
!
Policy Map police
Class acgroup2
police cir 10000000 bc 250000 be 250000
conform-action transmit
exceed-action drop
violate-action drop
!
interface Loopback0
ip address 11.11.11.11 255.255.255.255
!
interface Tunnel0
bandwidth 10000
tunnel bandwidth transmit 10000
tunnel bandwidth receive 10000
ip address 10.10.1.1 255.255.255.252
ip router isis aws
load-interval 30
mpls traffic-eng tunnels
keepalive 2 3
tunnel source GigabitEthernet1
tunnel destination 52.27.173.12
tunnel path-mtu-discovery
isis metric 1
service-policy output police
!
interface Tunnel1
bandwidth 10000
tunnel bandwidth transmit 10000
tunnel bandwidth receive 10000
ip flow monitor NTAMonitor input
ip flow monitor NTAMonitor output
ip address 10.10.4.2 255.255.255.252
ip router isis aws
load-interval 30
mpls traffic-eng tunnels
keepalive 2 3
tunnel source GigabitEthernet1
tunnel destination 54.70.66.102
tunnel path-mtu-discovery
isis metric 1
service-policy output police
!
interface Tunnel4
description MPLS TE Tunnel1 to the destination for path SR1
ip unnumbered Loopback0
tunnel mode mpls traffic-eng
tunnel destination 33.33.33.33
tunnel mpls traffic-eng autoroute announce
tunnel mpls traffic-eng path-option 1 explicit name SR1 segment-routing verbatim
!
interface Tunnel5
description MPLS TE Tunnel2 to the destination for path SR2
ip unnumbered Loopback0
tunnel mode mpls traffic-eng
tunnel destination 33.33.33.33
tunnel mpls traffic-eng autoroute announce
tunnel mpls traffic-eng path-option 1 explicit name SR2 segment-routing verbatim
!
interface VirtualPortGroup0
vrf forwarding GS
ip address 192.168.35.101 255.255.255.0
ip nat inside
no mop enabled
no mop sysid
!
interface GigabitEthernet1
ip flow monitor NTAMonitor input
ip flow monitor NTAMonitor output
ip address dhcp
ip nat outside
load-interval 30
negotiation auto
ipv6 address dhcp
ipv6 enable
no mop enabled
no mop sysid
service-policy output police
!
router isis aws
net 49.0001.0000.0000.0011.00
metric-style wide
segment-routing mpls
segment-routing prefix-sid-map advertise-local
passive-interface Loopback0
mpls traffic-eng router-id Loopback0
!
iox
ip forward-protocol nd
ip tcp window-size 8192
ip http server
ip http authentication local
ip http secure-server
ip nat inside source list GS_NAT_ACL interface GigabitEthernet1 vrf GS overload
ip route 0.0.0.0 0.0.0.0 GigabitEthernet1 172.0.1.1
ip route 33.33.33.33 255.255.255.255 Tunnel4
ip route 33.33.33.33 255.255.255.255 Tunnel5
ip route 172.2.1.5 255.255.255.255 Tunnel4
ip route 172.2.1.7 255.255.255.255 Tunnel5
ip route vrf GS 0.0.0.0 0.0.0.0 GigabitEthernet1 172.0.1.1 global
!
ip explicit-path name SR1 enable
index 1 next-label 17022
index 2 next-label 17033
index 3 next-label 17044
index 4 next-label 17033
index 5 next-label 17022
index 6 next-label 17011
index 7 next-label 17022
index 8 next-label 17033
!
ip ssh rsa keypair-name ssh-key
ip ssh version 2
ip ssh pubkey-chain
ip scp server enable
!
access-list 100 permit ip any any
ipv6 route ::/0 GigabitEthernet1 FE80::83F:37FF:FE14:2840
!
!
snmp-server community public RO
snmp-server community private RW
!
!
control-plane
!
line con 0
stopbits 1
line vty 0 4
login local
transport input ssh
!
app-hosting appid guestshell
app-vnic gateway1 virtualportgroup 0 guest-interface 0
guest-ipaddress 192.168.35.102 netmask 255.255.255.0
app-default-gateway 192.168.35.101 guest-interface 0
name-server0 8.8.8.8
end
R2
segment-routing mpls
global-block 17000 18000
!
connected-prefix-sid-map
address-family ipv4
22.22.22.22/32 index 22 range 1
exit-address-family
!
!
spanning-tree extend system-id
!
netconf-yang
!
restconf
!
redundancy
!
interface Loopback0
ip address 22.22.22.22 255.255.255.255
!
interface Tunnel0
bandwidth 10000
tunnel bandwidth transmit 10000
tunnel bandwidth receive 10000
ip flow monitor NTAMonitor input
ip flow monitor NTAMonitor output
ip address 10.10.2.1 255.255.255.252
ip router isis aws
load-interval 30
mpls traffic-eng tunnels
keepalive 2 3
tunnel source GigabitEthernet1
tunnel destination 52.38.167.137
tunnel path-mtu-discovery
isis metric 1
!
interface Tunnel1
bandwidth 10000
ip flow monitor NTAMonitor input
ip flow monitor NTAMonitor output
ip address 10.10.1.2 255.255.255.252
ip router isis aws
load-interval 30
mpls traffic-eng tunnels
keepalive 2 3
tunnel source GigabitEthernet1
tunnel destination 35.167.193.103
tunnel path-mtu-discovery
isis metric 1
!
interface VirtualPortGroup0
vrf forwarding GS
ip address 192.168.35.101 255.255.255.0
ip nat inside
no mop enabled
no mop sysid
!
interface GigabitEthernet1
ip flow monitor NTAMonitor input
ip flow monitor NTAMonitor output
ip address dhcp
ip nat outside
load-interval 30
negotiation auto
ipv6 address dhcp
ipv6 enable
no mop enabled
no mop sysid
!
router isis aws
net 49.0001.0000.0000.0022.00
metric-style wide
segment-routing mpls
segment-routing prefix-sid-map advertise-local
passive-interface Loopback0
mpls traffic-eng router-id Loopback0
!
iox
ip forward-protocol nd
ip tcp window-size 8192
ip http server
ip http authentication local
ip http secure-server
ip nat inside source list GS_NAT_ACL interface GigabitEthernet1 vrf GS overload
ip route 0.0.0.0 0.0.0.0 GigabitEthernet1 172.1.1.1
ip route vrf GS 0.0.0.0 0.0.0.0 GigabitEthernet1 172.1.1.1 global
!
ip ssh rsa keypair-name ssh-key
ip ssh version 2
ip scp server enable
!
!
ip access-list standard GS_NAT_ACL
permit 192.168.35.0 0.0.0.255
ipv6 route ::/0 GigabitEthernet1 FE80::83E:87FF:FEAA:8604
!
snmp-server community public RO
snmp-server community private RW
!
!
control-plane
!
line con 0
stopbits 1
line vty 0 4
login local
transport input ssh
!
app-hosting appid guestshell
app-vnic gateway1 virtualportgroup 0 guest-interface 0
guest-ipaddress 192.168.35.102 netmask 255.255.255.0
app-default-gateway 192.168.35.101 guest-interface 0
name-server0 8.8.8.8
end
R3
segment-routing mpls
global-block 17000 18000
!
connected-prefix-sid-map
address-family ipv4
33.33.33.33/32 index 33 range 1
exit-address-family
!
spanning-tree extend system-id
!
netconf-yang
!
restconf
!
!
redundancy
!
interface Loopback0
ip address 33.33.33.33 255.255.255.255
!
interface Tunnel0
bandwidth 10000
tunnel bandwidth transmit 10000
tunnel bandwidth receive 10000
ip flow monitor NTAMonitor input
ip flow monitor NTAMonitor output
ip address 10.10.3.1 255.255.255.252
ip router isis aws
load-interval 30
mpls traffic-eng tunnels
keepalive 2 3
tunnel source GigabitEthernet1
tunnel destination 54.70.66.102
tunnel path-mtu-discovery
isis metric 1
!
interface Tunnel1
bandwidth 10000
ip flow monitor NTAMonitor input
ip flow monitor NTAMonitor output
ip address 10.10.2.2 255.255.255.252
ip router isis aws
load-interval 30
mpls traffic-eng tunnels
keepalive 2 3
tunnel source GigabitEthernet1
tunnel destination 52.27.173.12
tunnel path-mtu-discovery
isis metric 1
!
interface VirtualPortGroup0
vrf forwarding GS
ip address 192.168.35.101 255.255.255.0
ip nat inside
no mop enabled
no mop sysid
!
interface GigabitEthernet1
ip flow monitor NTAMonitor input
ip flow monitor NTAMonitor output
ip address dhcp
ip nat outside
load-interval 30
negotiation auto
ipv6 address dhcp
ipv6 enable
no mop enabled
no mop sysid
service-policy input police
!
router isis aws
net 49.0001.0000.0000.0033.00
metric-style wide
segment-routing mpls
segment-routing prefix-sid-map advertise-local
passive-interface Loopback0
mpls traffic-eng router-id Loopback0
!
iox
ip forward-protocol nd
ip tcp window-size 8192
ip http server
ip http authentication local
ip http secure-server
ip nat inside source list GS_NAT_ACL interface GigabitEthernet1 vrf GS overload
ip route 0.0.0.0 0.0.0.0 GigabitEthernet1 172.2.1.1
ip route 172.0.1.0 255.255.255.240 11.11.11.11
ip route vrf GS 0.0.0.0 0.0.0.0 GigabitEthernet1 172.2.1.1 global
!
ip ssh rsa keypair-name ssh-key
ip ssh version 2
ip ssh pubkey-chain
ip scp server enable
!
ip access-list standard GS_NAT_ACL
permit 192.168.35.0 0.0.0.255
ipv6 route ::/0 GigabitEthernet1 FE80::893:B3FF:FED5:7104
!
snmp-server community public RO
snmp-server community private RW
!
control-plane
!
line con 0
stopbits 1
line vty 0 4
login local
transport input ssh
!
app-hosting appid guestshell
app-vnic gateway1 virtualportgroup 0 guest-interface 0
guest-ipaddress 192.168.35.102 netmask 255.255.255.0
app-default-gateway 192.168.35.101 guest-interface 0
name-server0 8.8.8.8
end
R4
segment-routing mpls
global-block 17000 18000
!
connected-prefix-sid-map
address-family ipv4
44.44.44.44/32 index 44 range 1
exit-address-family
!
!
spanning-tree extend system-id
!
netconf-yang
!
restconf
!
!
redundancy
!
interface Loopback0
ip address 44.44.44.44 255.255.255.255
!
interface Tunnel0
bandwidth 10000
tunnel bandwidth transmit 10000
tunnel bandwidth receive 10000
ip flow monitor NTAMonitor input
ip flow monitor NTAMonitor output
ip address 10.10.4.1 255.255.255.252
ip router isis aws
load-interval 30
mpls traffic-eng tunnels
keepalive 2 3
tunnel source GigabitEthernet1
tunnel destination 35.167.193.103
tunnel path-mtu-discovery
isis metric 1
!
interface Tunnel1
bandwidth 10000
tunnel bandwidth transmit 10000
tunnel bandwidth receive 10000
ip flow monitor NTAMonitor input
ip flow monitor NTAMonitor output
ip address 10.10.3.2 255.255.255.252
ip router isis aws
load-interval 30
mpls traffic-eng tunnels
keepalive 2 3
tunnel source GigabitEthernet1
tunnel destination 52.38.167.137
tunnel path-mtu-discovery
isis metric 1
!
interface VirtualPortGroup0
vrf forwarding GS
ip address 192.168.35.101 255.255.255.0
ip nat inside
no mop enabled
no mop sysid
!
interface GigabitEthernet1
ip flow monitor NTAMonitor input
ip flow monitor NTAMonitor output
ip address dhcp
ip nat outside
load-interval 30
negotiation auto
ipv6 address dhcp
ipv6 enable
no mop enabled
no mop sysid
!
router isis aws
net 49.0001.0000.0000.0044.00
metric-style wide
segment-routing mpls
segment-routing prefix-sid-map advertise-local
passive-interface Loopback0
mpls traffic-eng router-id Loopback0
!
iox
ip forward-protocol nd
ip tcp window-size 8192
ip http server
ip http authentication local
ip http secure-server
ip nat inside source list GS_NAT_ACL interface GigabitEthernet1 vrf GS overload
ip route 0.0.0.0 0.0.0.0 GigabitEthernet1 172.3.1.1
ip route vrf GS 0.0.0.0 0.0.0.0 GigabitEthernet1 172.3.1.1 global
!
ip ssh rsa keypair-name ssh-key
ip ssh version 2
ip ssh pubkey-chain
ip scp server enable
!
ip access-list standard GS_NAT_ACL
permit 192.168.35.0 0.0.0.255
ipv6 route ::/0 GigabitEthernet1 FE80::D:1CFF:FE1E:97C2
!
snmp-server community private RW
snmp-server community public RO
!
control-plane
!
line con 0
stopbits 1
line vty 0 4
login local
transport input ssh
!
app-hosting appid guestshell
app-vnic gateway1 virtualportgroup 0 guest-interface 0
guest-ipaddress 192.168.35.102 netmask 255.255.255.0
app-default-gateway 192.168.35.101 guest-interface 0
name-server0 8.8.8.8
end
EDIT
This is the GRE tunnel description:
ip-172-0-1-8#sh int t0
Tunnel0 is up, line protocol is up
Hardware is Tunnel
Internet address is 10.10.1.1/30
MTU 9976 bytes, BW 10000 Kbit/sec, DLY 50000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation TUNNEL, loopback not set
Keepalive set (2 sec), retries 3
Tunnel linestate evaluation up
Tunnel source 172.0.1.8 (GigabitEthernet1), destination 52.27.173.12
Tunnel Subblocks:
src-track:
Tunnel0 source tracking subblock associated with GigabitEthernet1
Set of tunnels with source GigabitEthernet1, 2 members (includes iterators), on interface <OK>
Tunnel protocol/transport GRE/IP
Key disabled, sequencing disabled
Checksumming of packets disabled
Tunnel TTL 255, Fast tunneling enabled
Path MTU Discovery, ager 10 mins, min MTU 92
Tunnel transport MTU 1476 bytes
Tunnel transmit bandwidth 10000 (kbps)
Tunnel receive bandwidth 10000 (kbps)
Last input 00:00:03, output 00:00:00, output hang never
Last clearing of "show interface" counters 00:03:26
Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/0 (size/max)
30 second input rate 18000 bits/sec, 2 packets/sec
30 second output rate 0 bits/sec, 0 packets/sec
495 packets input, 455872 bytes, 0 no buffer
Received 0 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
179 packets output, 77152 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 output buffer failures, 0 output buffers swapped out