Impact of IOS BGP soft-reconfiguration-inbound and peering options

Question

This scenario is a network with four routers in a square; each has a connection to a different transit provider with a full BGP table from each.

R1----R2
|     |
|     |
R3----R4

How do full BGP feeds affect the memory requirement of the routers in the ring, other than the router receiving the full feed?

All four routers here take full feeds from different providers, and are iBGP meshed (you can imagine diagonal lines join R1 & R4, and R2 & R3, my ASCII art isn't brilliant).

iBGP peers send each other all eBGP routes so, R1 receives full table from upstream1. R2 receives a full feed from upstream2. R2 will send R1 all its external routes and vice versa. For any received routes that are better via upstream2, R1 will place into it's FIB, and send traffic to R2. And so on and so fourth, for all four routers and upstream providers. Now, for traffic that has a better path via upstream3, it will be sent via R3, even when it comes from a device attached to R1. That is the beauty of BGP.

Lets just use R1 and R2 to keep it simple for now, and they are booth bringing up BGP peerings for the first time;

1. When R1's BGP session with upstream1 first establishes, it receives BGP updates from upstream1, it processes the routes and they progress from BGP RIB into the FIB.

2. All of these routes on R1 are now passed to R2 via iBGP, and R2 pushes them all from its BGP RIB into FIB (these are the only external routes it has at present).

3. R2's session with upstream2 comes up and upstream2 sends BGP updates to R2. At this point, does R2 compare the routes in it's BGP RIB, keeping the better ones, drop the lesser ones, than it has from upstream1, and push the replaced better ones into FIB (assuming no better routes in FIB from other routing protocols or sources). Does R2's BGP RIB still only contains one full BGP table at this point, but its a mix of both upstream providers table views? Is the correct?

4. The same now happens in reverse, with R2 sending updates to R1. R1 ends up with the same BGP view.

Am I correct that unless soft-reconfiguration inbound or something similar is configured, any router in this square only holds the equivalent of one full table? What change in memory requirements would enabling soft-reconfiguration inbound here make, would each router hold 4 full tables and need 4 times as much memory?

Answer 1

Just like BGP attributes are packed when sending BGP UPDATEs, they are stored in memory in a rather compact format where each prefixes only holds references to the attributes that apply to that route. The AS path and the communities applied to a route are typically the attributes that are largest in size and as these are often the same over a large amount of routes, a great deal of memory can be saved by just storing an AS path once and then keeping references to all routes for which this AS path applies. The exact in-memory format changes with different versions of IOS/JUNOS.

A route for the same prefix received via different eBGP sessions will likely share many attributes (especially if the eBGP sessions are to the same upstream) and so they can be "packed" quite efficiently in memory.

Your explanation of BGP behaviour is mostly correct. R1 has sent the content of its entire BGP RIB to R2. R2s eBGP session is enablde and for every route received, R2 will compare it to the one in its BGP RIB (which mostly contains routes from R1). When R2 finds a prefix from its new eBGP peer that is better than the one in BGP RIB, it will use the new eBGP route and announce it to R1. If R1 finds the route received from R2 to be better than its own route (received from its eBGP session) then R2 will withdraw the route that it announced to R2. The routers will only announce their best route and if the best route is not from eBGP but from another iBGP neighbor, it will not announce it unless you have some route-reflection going on as well. You can easily end up in a scenario where both R1 and R2 each select all the routes learnt from their respective eBGP session as the best routes and that means they will always announce a full table to their iBGP buddies.

There's also something called "best external" which means you always announce your best eBGP route regardless if that's the one you have chosen as best path or not. Best external enhances convergence time by not having to go through "BGP path hunting" when something breaks. Obviously it consumes more memory.

Personally I don't think BGP Soft reset enhancement is much of an enhancement. RAM is cheap, so keeping the routes, exactly as received from your neighbor, in memory is quite easy, especially on a modern router. JUNOS doesn't even offer you the option to not store it - it always keeps the Adj-RIB-IN, but JUNOS was designed in the late 90s when RAM was plentiful and not in the 80s when IOS made the scene. If someone out there designing a router thinks "let's save on RAM because we have BGP soft reset enhancement", I'd like to shoot that person ;) On the other hand, if you are running a network with old routers running low on memory I totally understand if you don't enable soft-reconf.

Bottom line, it's difficult to say and depends on your environment.

Answer 2

Cisco constantly works on the BGP stack to achieve better performance and, so, memory consumption should vary between different versions. Check the this document for details: BGP – From Dinosaur to racecar

That said the use of soft-reconfiguration inbound has been discouraged since the introduction of the BGP Soft Reset Enhancement.

To make a long story short, with soft-reconfiguration inbound enabled IOS first saves a local copy of the received NLRIs for each of its peers and then recursively applies ingress routing policy to them to calculate the BGP actual BGP RIB.

With the BGP Soft reset Enhancement the router is able to ask to its peers to re-send the entire BGP feed avoiding the need to store in memory multiple copies of the same information.

Furthermore, to answer your question at point #2, memory consumption of BGP informations in Cisco IOS isn't linear, because a single NLRI is associated to differents next-hop/policy to achieve best memory utilization. So two FRTs consume less then the double of what one does.