Cisco IOS pick default route from two DHCP interfaces with track

Question

I plan to have a portable IOS cisco router which will have two DHCP assigned ISP addresses from unknown ranges. A tunnel is then built through the preferred ISP to another outer.

I would like to set the default gateway based on a precedence, but more importantly based on reachability. Often you may get a DHCP address, but have no connectivity. The idea is a non-technical person can drop this in, plug it in (port on the local router already setup), and get connectivity based on a local ISP if possible, or a cellular one if not.

When you know the address of the next hop at config time, this is simple - the IP ROUTE ... TRACK will work nicely, combined with a suitable metric for precedence.

When you get a DHCP address, you can do an IP ROUTE ... DHCP but it will not accept a TRACK statement with it.

In the interface IP DHCP command you can set a default-router distance, so you can supply a precedence, but not test for reachability. So if everything is working you can get the best one, but not if one ISP is flakey (or becomes so).

Below are relevant portions of a configuration that is close. If either DHCP assigned interface fails to get an address, everything works, and it uses the other. It also actively tests to pick the right tunnel source, so if connectivity is down for one, it will shift to the other as the tunnel source. However, if both are up, then there is a failure of connectivity through gi0/7:

• The metric originally assigned gives gi0/7's next hop the best metric (no connectivity), and that metric will not go away until the interface rebinds (and maybe not then if it gets a new address but still lacks connectivity).
• the track in the ikev2 client setup will fail to pick 7, but also fail to get 6 (no working gateway as the gi0/7 metric still has it as best)

If I could force a very short lease the interface would rebind (if the dhcp server was also unreachable), but the servers usually override that request and put in 1 day or something to long to be useful. And the dhcp server might be semi-local and unaffected by the outage.

In theory it could work to put a route with destination of the interface (instead of next hop) with a TRACK, but that depends on the ISP providing proxy arp (I think that's the term), which is far from sure. And if proxy-arp is available only on the GSM router it might force it to be always up.

The question is this: Is there a way in current router IOS CLI (I'm on 15.6 to test with) to get, and update, a correct default route based on reachability when there are two DHCP ISP interfaces?

I've also tried various ways to get policy based routing to force the interface for the SLA (still using the DHCP provided next hop) without any success, but so far without success.

Thanks for any ideas.

track 6 ip sla 6 reachability
track 7 ip sla 7 reachability
!         
crypto ikev2 client flexvpn FLEXVPN_IKEV2_CLIENT
  peer 1 9.9.9.2
  source 1 GigabitEthernet0/7 track 7
  source 2 GigabitEthernet0/6 track 6
  client connect Tunnel31
  !
interface Tunnel31
  ip unnumbered Loopback192
  ip nat inside
  tunnel source dynamic
  tunnel mode ipsec ipv4
  tunnel destination dynamic
  tunnel protection ipsec profile IKEV2_IPSEC_PROFILE
!         
interface GigabitEthernet0/6
 description Connection to GSM modem (secondary internet if primary down)
 ip dhcp client default-router distance 200
 ip dhcp client lease 0 0 5
 ip address dhcp
 ip nat outside
!
interface GigabitEthernet0/7
 description Connection to local intenet (if any) 
 ip dhcp client default-router distance 100
 ip dhcp client lease 0 0 5
 ip address dhcp
 ip nat outside

ip sla 6
 icmp-echo 75.75.75.75 source-interface GigabitEthernet0/6
 tag Cellular connection up test 
 frequency 10
ip sla schedule 6 life forever start-time now

ip sla 7
 icmp-echo 75.75.75.75 source-interface GigabitEthernet0/7
 tag Local ISP connection up test 
 frequency 10

PS. I left out most of the tunnel stuff as this is really about the routing, not the tunnel; the tunnel is working properly when it has the right default gateway.

Answer 1

I'm going to post this as a possible solution for people to shoot holes in, and/or suggest how to clean it up. it works in the GNS3 simulation with IOSv images.

Let me first explain what happens to me a bit more clearly, and the rationale for all the complexity.

First for clarity -- I am trying to track against a distant IP, not the next hop, because (a) on DHCP don't know the next hop, and (b) sometimes connectivity problems are after the next hop.

There are simple techniques to achieve this if you do not have DHCP -- TRACK the distant item on separate statements with each next-hop IP explicit, and everything works, and you set precedence with distance.

There is no syntax I have found to use IP ROUTE...TRACK with DHCP and distinguish next-hop's from different interfaces. All the variations I have found end up in a chicken and egg problem - the track can't operate as the SLA has no route to let it reach a distant IP. You can specify ROUTE TRACK with the interface DHCP CLIENT statement, but I cannot figure out any way to make the SLA use that particular interface's next hop. Same with the verify-reachability, at least with all syntax I have found.

If the two DHCP interfaces have unequal metrics, to give one a preference, what I find happens is this (in a straightforward setup): both TRACK statements come up, but despite the SOURCE INTERFACE on them (which is just setting an address), both are going out the preferred default route. Thus when the preferred route is broken (i.e. I stop traffic on it), the route stays as preferred, and BOTH tracks go down. Source interface is not specifying the egress interface

This can partially be fixed by giving them equal preference; then both routes to 0.0.0.0 are installed in the rib, and sometimes (but not always) the two TRACK's will use their interface instead of the opposite and operate independently. Unfortunately not always; not sure why, but by observation sometimes both tracks as written above go out the same interface even with equal preference on the then default gateway.

So this proposed solution uses a route-map to force the traffic for each SLA (using different target IP's) onto a specific egress interface. If this is done with unequal preference default gateways it also fails -- the route map itself fails on the packet for the non-preferred interface; I am unclear why, it just rejects it and says "normal forwarding". I assume it's validating against the rib? But with equal preference sla6 is forced to gi0/6, and sla7 is forced to gi0/7, and work independently.

What I do not know is whether this use of a SET INTERFACE will work generally in this specific cast, i.e. is the packet using the proper next hop, or a proxy arp. I tried looking at it with wireshark but had some issues getting it to run on the simulation. I'll confirm later. I'm hoping it uses the rib for the next hop with the selection of which path made by the route map.

But it's complicated, a bit contrived, and I really would expect there to be an easier solution. The other problem is that it only works, as written, to provide preference for the tunnel, not general internet traffic from inside (that's OK in this case as that was not the intent, but I want to experiment further with additional route map clauses, perhaps based on tracking there, to prefer general traffic also).

I would sure like someone to provide a better way. And/or shoot holes in this one if it won't work.

As above, only the route preference related items shown.

PS. All the non-private addresses are simulated, not used in reality.

track 6 ip sla 6 reachability
track 7 ip sla 7 reachability

crypto ikev2 client flexvpn FLEXVPN_IKEV2_CLIENT
  peer 1 9.9.9.2
  source 1 GigabitEthernet0/7 track 7       <<< precedence established here, only for tunnels
  source 2 GigabitEthernet0/6 track 6
  client connect Tunnel31

interface Tunnel31
 ip unnumbered Loopback192
 ip nat inside
 tunnel source dynamic
 tunnel mode ipsec ipv4
 tunnel destination dynamic
 tunnel protection ipsec profile IKEV2_IPSEC_PROFILE
!

interface GigabitEthernet0/6
 ip dhcp client default-router distance 100  << must be the same as below or omitted
 ip address dhcp
 ip nat outside
!
interface GigabitEthernet0/7
 ip dhcp client default-router distance 100     << must be the same as above or omitted
 ip address dhcp
 ip nat outside
!
ip local policy route-map CHOOSE-ISP

ip sla 6
 icmp-echo 74.74.74.74 source-interface GigabitEthernet0/6
 tag Cellular connection up test 
 frequency 10
ip sla schedule 6 life forever start-time now

ip sla 7
 icmp-echo 75.75.75.75 source-interface GigabitEthernet0/7
 tag Local ISP connection up test 
 frequency 10
ip sla schedule 7 life forever start-time now
!
route-map CHOOSE-ISP permit 10
 match ip address 106
 set interface GigabitEthernet0/6   <<< give warning 
!
route-map CHOOSE-ISP permit 20
 match ip address 107
 set interface GigabitEthernet0/7   <<< give warning 
!
access-list 6 permit 74.74.74.74
access-list 7 permit 75.75.75.75

Answer 2

I spent the past few weeks researching how to implement WAN failover with DHCP and found a number of helpful articles and configuration guides, but nothing worked. After digging in and continuing to search, I came across your question and your creative solution. Still, I couldn't imagine crypto maps was the only way to solve this.

I found this Cisco article Configure "ip dhcp client route track" for Dual ISP Failover on ISR Routers describing how to implement dual wan failover. Despite being a Tech Note published by Cisco not only did it not work for me, it failed in the same manner as you describe in your question above!

After playing around, I found that I had to leave out the interface in the default gateway primary route. Here's a simplified configuration:

track 40 ip sla 30 reachability

ip sla 30
 icmp-echo 8.8.4.4 source-interface GigabitEthernet0/0
 timeout 6000
 frequency 10

ip sla schedule 30 life forever start-time now

interface GigabitEthernet0/0
 description DHCP WAN, primary
 ip dhcp client route track 40
 ip address dhcp

interface GigabitEthernet0/1
 description DHCP WAN, secondary
 ip address dhcp

ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/1 dhcp 200
ip route 0.0.0.0 0.0.0.0 dhcp

I've tested this a number of times by power cycling my modem and also shut/no shut on interface 0/0. As soon as the router notices the gateway won't connect to the remote IP (sh track and sh ip sla summary) I can run sh ip route | i 0\.0\.0\.0 and see the secondary default route take over. Then I wait a little bit and the primary link pops back up. However, if I instead specify the primary default gateway as ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp - that is, with the interface - the track and ip sla remain down and the primary route never comes back on its own.

EDIT

I don't know why specifying the interface in the default gateway route prevents the SLA from resetting and not specifying it allows the reset. However, I suspect it's because with the interface, the routing rule is very specific and matches the SLA directly. I also suspect that internally the rule is not removed from the router so much as marked to be ignored or skipped. Then, if the rule is there, because it exactly matches the SLA icmp-echo, the action is dropped. Whereas, when it's the default gateway rule without the interface, the dhcp is interpreted with the secondary interface's address even though it's marked as ignore, and so none of the routing table rules match exactly and the icmp-echo goes through.

Perhaps someone with cisco debugging and logging skills (ie. not me) might be able to tell us exactly what's going on here.