Is it common for ISPs to connect to each other directly?

Question

I'm reading up on the relationship between latency and ISP interconnectivity (that higher ISP interconnectivity results in lower latency, which makes sense to me).

My understanding is that IXPs provide the primary means of ISPs to connect with each other (taken from this article on edge servers by cloudflare). But why not, as an ISP, connect directly to another ISP?

Does this happen? And, in terms of terminology, would the connection then be referred to as an IXP?

Answer 1

Yes, this does happen quite a lot, and it is called private peering. It has some benefits over peering over an IXP:

• dedicated bandwidth, you can be sure you can use the full capacity of the interconnecting link for traffic to and from the other ISP
• no dependency on the IXP, an IXP connects two ISPs on their switch(es), you're not suffering from any outages of the IXP. Also, you're in direct contact with the other ISP when solving problems.
• possibly lower costs, if an ISP does a lot of traffic with one specific other ISP, it can be cost efficient not to pay an IXP to provide the connectivity, but instead just use a direct connection

However, there can be downsides too:

• cost and availability router ports, routers often have a very limited number of ports, and port can be very costly (especially for high speed connections). By connecting to an IXP, you can reduced the number of private peering connections and thus lowering costs.
• localisation, not every ISP is present in every datacenter. IXPs often provide a peering LAN which stretches over multiple datacenters spanning a city (or sometimes a country or even a continent). Buying fiber paths to every other ISP can become very, very costly, especially if the distances are longer.
• operational costs, having many interconnections means more configurations, outages, links and ports to monitor, etc. Doing this for every single ISP can be very cost inefficient.
• connectivity between inequal peers, not every ISP wants to do private peering with all other peers, especially if there's a large difference in size. IXPs may enable them to peer with smaller peers, because the operational costs are much lower. Also, IXPs often offer route servers, which can function as an intermediate between ISPs, so the do not have to setup peering sessions with each other peer on the IXP.

Answer 2

My understanding is that IXPs provide the primary means of ISPs to connect with each other

I think that is a massive oversimplification.

In general connections between autonomous systems (this includes ISPs but also other major networks) can be split into two main catogories (there are also intermediate cases). In a "transit" connection a customer AS pays a provider ISP for routes to the internet in general. In a "peering" connection the two ASs exchange routes to each other's customers but not to the internet in general.

As a general rule small networks and networks that don't sell transit services want to peer widely (why pay for transit when you don't have to) while large networks are often more restrictive about who they will peer with (why give people something for free when you could be charging them for it).

At the top of the pile are a small number of "teir 1" providers who don't buy any transit at all relying only on peering with each other (but usually not with the little guys) to get a (hopefully) full view of the Internet.

---

Transit connections normally go over private links (though some IXPs do allow transit), a transit customer usually exchanges a large amount of traffic with a handful of transit providers.

For peering it generally depends on traffic volumes.

Interconnections cost money to maintain. They also cost money to set up/tear down. They cost that money regardless of whther they are fully utilised or not. So putting in place private links for low traffic or speculative peering relationships is prohibitively expensive.

That is where IXPs come in, a network can rent a small number of connections to the IXP's switching infrastructure and use it to establish peering with tens or even hundreds of providers. The IXP will also normally provide a route server allowing routes to be exchanged between participating members (usually the smaller ones) without needing direct BGP sessions.

However the IXPs switching infrastructure does not come for free, so if two networks are consistently exchanging a substantial fraction of a link's traffic then it is more efficient to do it over a direct link.