I am in search of a network protocol recommendation for the following scenario:

I would like to have a load balance routing protocol that can alter the cost of a router based on its current load.


Router A can reach Router D via Router B and Router C

Router B becomes saturated and using a load balance routing protocol Router B sends an update to all its peers about its current load.

Router A then prioritizes Router C over Router B because it has less load.

Can this be possible ? maybe BGP or MPLS-TE?


1 Answer 1


Yes - such features exist in a couple of protocols, and I'll give a couple of examples. They're not in wide-use, though, because in actual usage they're virtually always a terrible idea.

First off - there isn't really a notion of a router individually being busy, but rather links between particular sets of routers being busy. The idea has generally been to respond to a busy link by causing some number of upstream links to choose paths that are, ostensibly both less utilized and (normally) less preferred. Keep in mind that I used the term preferred rather than direct, as the means of choosing links may include any number of potential factors.

So - let's look at a couple of easy examples: Cisco's EIGRP and MPLS auto-bandwidth

In the case of EIGRP, as you may know the DUAL algorithm actually makes final metric determination (and thus routing decisions) based on several values known as K1-K5. These are actually various multipliers of values that the protocol is carrying. The values themselves are (minimum path) bandwidth, (additive) link delay, (minimum) link reliability and (maximum) link utilization. This is all explained in good detail here. Again, though, these K values are integer multipliers of these values, not the values themselves. What's more, by default three of these five values are actually left at zero while the other two are left at 1. The net effect here is that link reliability and link utilization are ignored, while link bandwidth and delay are actually used. Why these attributes? Because they are statically defined on links. In early deployments utilizing all of these values it was quickly discovered that the variability of these values could cause rippling instability in larger networks. What's worse, utilization in particular created an oscillation condition, where traffic would tend to be steered away from a busy link only to cause another set of links to become busy which would, in turn, redirect back to the original link. The issue here is that the network is potentially never in a completely settled/predictable state. This basically means that a busy link can actually commonly lead to network instability - which means that slowness for one set of users just resulted in unpredictable performance for potentially a great many users. Similarly, link quality determination tended to magnify a circuit taking a few CRC errors into a network-wide convergence event. At a minimum this churned CPU, but worst-case it could (and did) yield outages unto itself.

In the MPLS case the implementation is quite different but many of the issues are the same. In this case RSVP-TE is used to set up TE tunnels / LSP's (label switched paths) - which, for the unfamiliar, are specially signaled paths across an MPLS network. These paths are unidirectional and in nature built between an ingress and egress router based on a number of factors learned about the intervening network via both traditional metric information and extended TE-specific data carried a link-state routing protocol (IS-IS or OSPF). These factors can include bandwidth, link affinity/coloring attributes, explicit paths (i.e. which intervening nodes should be waypoints) as well as traditional unidimensional IGP metrics (i.e. standard link-cost).

RSVP actually provides the mechanism by which these TE LSP's are signaled through the network. Based on the criteria the ingress router requests the routers on the way to the ingress will either signal back their acceptance of the path or provide admission control to reject a proposed path.

As mentioned there a bunch of criteria that can be used but if we focus on RSVP bandwidth we can see your use-case in action. As an aside, RSVP bandwidth is a static attribute set on a per-link basis. If a given LSP is configured to ask for a bandwidth reservation then this value is used for admission control by the intermediate routers. If a given requested link has bandwidth then it is allowed and the total reservable bandwidth on that link is decremented by the reservation amount. If the remaining reservable amount is insufficient then the request is denied and a different path is attempted.

As an example - consider a diamond/square topology of four routers: A, B, C and D. A connects to B and C and D connects to B and C. The path A -> B -> D is all 10G, while the path A -> C -> D is all 1G. Put another way, under normal circumstances a standard IGP will steer all traffic via the A->B link, leaving the A->C link in play as a backup. In this case, though, there are already LSP's running via A->B that have reserved 9.5G out of 10G. When A tries to set up a new LSP to D it will find that it can't signal a new 750M LSP via the preferred path so will, instead, signal via the lower-speed path via C.

So... A couple of side notes are in order: First, the bandwidth requested on a given LSP is generally a static value. Second, there's nothing (by default) either restricting a given LSP from transmitting more than its reservation or, conversely, making unused bandwidth available to other paths. Indeed, the bandwidth value itself on the interface is just an integer. It's convenient to consider it in terms of kilobits, but it's not absolutely necessary. It's also the case that while the simple case is that bandwidth on the link is allocated on a first-come/first-serve basis it's possible to both allow overbooking for certain types of traffic (i.e. allow, say, 8 gigabits of reservations for certain LSP's on a 1 gigabit link) and also to give relative priority to different LSP's (...so one LSP can bump another, possibly even knocking it offline if no other path is available).

So - with all this in mind (...and there's LOTS more to be written about TE) the idea with auto-bandwidth is that the ingress router starts off with an initial reservation but actually measures the amount of bandwidth entering the tunnel. As the ingress sees the actual usage it will actually periodically resignal the path. If the actual utilization is less than the initial reservation then the intervening nodes know that more bandwidth is now available on the links our LSP traverses. If the actual utilization is greater then the reservations are either updated or, if insufficient bandwidth is available, alternate links are selected.

This all sounds nice in principal but, again, this doesn't map very well to the real world. One of the initial issues is that there's no guarantee of optimally efficient bandwidth distribution and, in fact, it's common for vendor implementations to actually have options to look amongst otherwise equivalent candidate paths and choose the least utilized, the otherwise most utilized or just random choice. There are actually formal proofs for the fact that these methods end up being essentially equivalent and statistically likely to be suboptimal. Much like the EIGRP case mentioned above, the problem is that as the network tries to move traffic around that the load condition on intervening links will tend to lead to ongoing instability and oscillation. The periodicity of this oscillation can be adjusted (re-signaling timing can be adjusted to, say, once an hour or once a day) but this hasn't really yielded a substantially better experience in most networks and has led to less determinism and greater instability in plenty of cases.

It's been argued that using a central controller that can more intelligently respond to load conditions by forcing the re-signaling of LSP's can work really well. This tends to follow a much more specific and bounded set of use-cases (...say, redirecting spiking backup traffic via links through an underutilized user site), rather than allowing a generalized algorithm to just go nuts across your entire network. This is probably one of the better use-cases for so-called SDN, even though it predates our present use of the term by 10-15 years.

There's a good presentation on MPLS-TE and auto-bandwidth here but, as mentioned, the overall topic of TE can get pretty deep and will also vary based on vendor implementation (..and even within vendors based on hardware platform capabilities, etc).

The point, though, is that while it sounds like a good idea to be dynamically re-signaling networks based on observed traffic loads that in practice the network will rarely benefit and will often be destabilized by such approaches. Engineer based on observed loading. Make careful changes based on observed data. A slight improvement in performance is useless when it brings a substantial decrease in stability.

  • Thanks for the detailed answer, very specific with real life use cases. I will take a look at RSVP-TE as it seems like is what I'm looking for.
    – felartu
    Jun 18, 2017 at 3:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.