Limitations of Bilateral peering at an IX

Question

Are there any disadvantages to setting up peering with every other (open) ISP at an IX? All I'm sure is that this could make peering cheaper since the reliance on Tier 1 providers decreases.

Answer 1

Although there can be huge benefits for peering, there are some downsides for sure:

Peering doesn't have to be cheaper. In fact, if the utilization of your peering port is low, it's very likely that you'd be better off buying transit if you look at the price per mbit/sec. Peering ports become financially efficient if they're actively used, while on transit ports you can often commit to a lower usage on a high bandwith port.

Also, peering routes don't have to be better than transit routes. It can very well be that your transit providers have paths available that are either shorter or have more capacity (or both). This is especially the case when your peers do 'remote peering', where they backhaul traffic from the IXP back to their network over longer distances. For example: if a network which hosts its services in Europe leases a circuit to the USA and connects to an IXP there, networks with a presence in the US could decide that that is the best place to exchange traffic, possibly sending traffic from their network in Europe to the USA, to be sent back to Europe again, with high latency on that path as a result. An IP transit provider would often be able to keep that traffic within Europe.

Then there's time. Setting up and maintaining peering sessions takes up time. Parts of this can be automated, yet that also takes time. Setting up proper filters on every peering session so you don't just accept any route advertised to you can take up quite some time as well. And you don't want to be victim of the misconfiguration of some peer which caused a route leak...

Another big difference is the absence of an SLA. Of course, many IXPs do offer some sort of SLA on availability of the peering fabric, but no peer will offer you one on uptime of the peering session. Peers dropping off IXPs for longer periods of time is not uncommon. So you need to be able to offload traffic elsewhere.

Also, most open peering agreements can be changed or terminated without any prior announcement. So if a peer decided to leave the IXP or change their peering policy and depeer you, there's not much you can do and traffic needs to shift elsewhere.

A last downside on peering on an IXP is that you often have to rely on BGP timeouts (which are typically LONG) when a peer drops off the IXP. Until routes are updated, traffic will be blackholed. On a transit session you typically have a layer 1 connection which you can monitor (and when it drops BGP doesn't need to wait for a timeout), or protocols like BFD can be used to detect problems with the peer.