It is impractical for every ISP to connect directly to every other ISP. It's also rather impractical to have one central point of interconnection, noone likes it when local traffic crosses the ocean twice.
Relationships between ISPs (and other networks that have an AS number and participate in internet routing) can be broadly divided into two categories (yes there do exist intermediate cases). One is "peering", ISP A and ISP B will interconnect with each other so that ISP A and their customers get access to ISP B and their customers. This is not transitive though, if ISP B has peering relationships with ISPs A and C then C does not get a path to A. Peering is usually but not always "settlement free", that is neither provider pays the other for the interconnection (though someone has to pay for the physical link between them).
Peering may be established over either a direct link between the providers or over an exchange point. The exchange point provides a connection fabric (usually presented as an Ethernet network) allowing lots of providers to peer with each other without the cost of dedicated links.
The other is "transit", the transit provider provides the transit customer with access to the internet in general. They will obviously charge for this service.
Most ISPs buy transit from other ISPs, there are serveral reasons for this.
- Large ISPs usually won't peer with smaller ones (at least not for free), because they see the relationship as having more benefit to the small ISP than to the large one.
- There are a heck of a lot of ISPs in the world.
- There are also a heck of a lot of interconnection locations. Smaller ISPs will not want to build out their networks to distant locations.
At the top of the pile are a small number of ISPs who do not buy transit from anyone and rely entirely on peering to reach each other. These are known as transit free or "teir 1"*
* The traditional definition of teir 1 also required all peering to be settlement free, but it's impossible to determine from the outside if peering is settlment free or not.