0

I have really bizarre problem in clients network.

enter image description here

Clients have access to the internet via NAPT - I'm using several public ip addreses for this purpose. (I'll change ip addresses for obv reasons)

ip nat pool NAT 3.3.3.175 3.3.3.190 netmask 255.255.255.192
ip nat inside source list NAT pool NAT vrf NAT overload

Other NAT settings

ip nat settings pap 
ip nat log translations flow-export v9 udp destination 192.168.55.10 9995
ip nat log translations flow-export v9 vrf NAT on
ip nat translation timeout 3600
ip nat translation tcp-timeout 1200
ip nat translation udp-timeout 120

ip route vrf NAT 0.0.0.0 0.0.0.0 3.3.3.1

Interfaces

interface GigabitEthernet0/0/0.220
 description IP NAT OUTSIDE
 encapsulation dot1Q 220
 ip vrf forwarding NAT
 ip address 3.3.3.5 255.255.255.192
 ip nat outside
!

interface GigabitEthernet0/0/2.202
 description IP NAT INSIDE
 encapsulation dot1Q 202
 ip vrf forwarding NAT
 ip flow monitor FLOW-MONITOR input
 ip flow monitor v3.12 input
 ip flow monitor FLOW-MONITOR output
 ip flow monitor v3.12 output
 ip address 10.128.1.2 255.255.255.248
 ip nat inside

access-list NAT has about 477 entries

I am experiencing problems with accessing certain sites:

  1. google.com gives always gives me 500 error
  2. Can't access others several sites - just infinite load and white screen

Nevertheless, all this happens, when my global IPs from the pool above are .179 or .180

I tried to change my global ip with static nat to .189 (end of the pool) and all above described problems were gone: google's 500 problem is none and sites, that weren't loading load successfully.

I am really lost in trying to find the root of the problem. When I dealt with google problems I was thinking on botnet/viruses activity, or maybe bad TLS handshake due to lost/re-transmitted packets, but I would experience those problems on many resources then.

I don't even know if it's a NAT problem. I guess the only way to find out (now if I think of it) is reassemble NAT pool (make it smaller, exclude "bad IPs") and try to statically assign it to hosts and find out. It will tell (may be the problem is indeed in to many or bad requests from one IP).

Maybe to many request were handling from one global IP but to deal with it I tried ip nat settings pap

Current translations:

asr1.core#sh ip nat translations total 
Total number of translations: 60199

And the router is ASR1000X

asr1.core#sh ver
Cisco IOS XE Software, Version 16.05.01b

cisco ASR1001-X (1NG) processor (revision 1NG) with 3763831K/6147K bytes of memory.
6 Gigabit Ethernet interfaces
2 Ten Gigabit Ethernet interfaces
32768K bytes of non-volatile configuration memory.
8388608K bytes of physical memory.
6594559K bytes of eUSB flash at bootflash:.

I am asking advice in the approaches to troubleshoot such problems and maybe some real life experience with NAT/PAT problems?

EDIT ============================================

asr1.core#sh ip route 3.3.3.179
Routing entry for 3.3.3.128/26
  Known via "static", distance 1, metric 0
  Routing Descriptor Blocks:
  * 3.3.3.5
      Route metric is 0, traffic share count is 1


asr1.core#sh ip route 3.3.3.189
Routing entry for 3.3.3.128/26
  Known via "static", distance 1, metric 0
  Routing Descriptor Blocks:
  * 3.3.3.5
      Route metric is 0, traffic share count is 1

So route to both just pointing to nat outside interface.

In NAT routing table there is nothing to global IPs, but it's understandable cause routing happens after de-nating to private IPs

asr1.core#sh ip route vrf NAT 3.3.3.189
Routing Table: NAT
% Subnet not in table

asr1.core#sh ip route vrf NAT 3.3.3.179
Routing Table: NAT
% Subnet not in table

Default routes:

Global routing table (leads to nat inside interface)

Gateway of last resort is 10.128.1.2 to network 0.0.0.0

S*    0.0.0.0/0 [1/0] via 10.128.1.2
      1.0.0.0/8 is variably subnetted, 2895 subnets, 14 masks
B        1.0.0.0/24 [20/0] via BGP1, 2d10h
B        1.0.4.0/22 [20/0] via BGP1, 2d10h
B        1.0.4.0/24 [20/0] via BGP1, 2d10h
B        1.0.5.0/24 [20/0] via BGP1, 2d10h
B        1.0.6.0/24 [20/0] via BGP1, 2d10h
B        1.0.7.0/24 [20/0] via BGP1, 2d10h
B        1.0.16.0/24 [20/0] via BGP2, 2d10h
B        1.0.64.0/18 [20/0] via BGP2, 2d10h
.....etc

vrf NAT routing table

Routing Table: NAT

Gateway of last resort is 3.3.3.1 to network 0.0.0.0

S*    0.0.0.0/0 [1/0] via 3.3.3.1
      10.0.0.0/8 is variably subnetted, 889 subnets, 10 masks
S        10.0.0.0/8 [1/0] via 10.128.1.3
.....etc
  • I would look at the route on router 3.3.3.1 for the 3.3.3.128/26 block. If that route was (guessing) 3.3.3.188/30 it would result in exactly these symptoms. Does router 3.3.3.1 know where to send packets destined to 3.3.3.180? – Darrell Root Feb 7 at 6:20
  • Please share your router routing table – Mahdi Feb 7 at 7:00
  • @DarrellRoot I edited the main post – vudex Feb 7 at 13:59
  • Also need the routes back to core from the router beyond (the router with ip 3.3.3.1). My guess is this is a “return route” problem. – Darrell Root Feb 7 at 14:21
  • @Darrell Root this commands asr1.core#sh ip route vrf NAT 3.3.3.189 asr1.core#sh ip route 3.3.3.189 are from the router with 3.3.3.1 interface – vudex Feb 7 at 14:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.