Why are IPv4 addresses running out?

Question

I understand that we are running out (or ran out already?) of IPv4 addresses, but I don't really understand why that is. Right now, every home has its own IPv4 address (dynamically assigned, but still, each has an address). Why can't a city (for example) have just one IPv4 address and all homes in this city would just be on a private network of that city? Then this one city would be able to assign addresses from range 0.0.0.1 to 255.255.255.254.

I'm sure that my understanding is wrong somehow otherwise IPv4 addresses would not run out. What's wrong with my understanding?

Answer 1

The IPv4 Address Shortage

According to Vint Cerf (the father of IP), the IPv4 32-bit address size of was chosen arbitrarily. IP was a government/academic collaborative experiment, and the current public Internet was never envisioned. The IP paradigm was that each connected device would have a unique IP address (all packets sent between IP devices would be end-to-end connected from the source IP address to the destination IP address), and many protocols using IP depend on each device having a unique IP address.

Assuming we could use every possible IPv4 address*, there are only 4,294,967,296 possible IPv4 addresses, but (as of September 2018) the current world population is 7,648,290,361. As you can see, there are not enough possible IPv4 addresses for every person to have even one, but many people have a computer, printer, cell phone, tablet, gaming console, smart TV, etc., each requiring an IP address, and that doesn’t even touch on the business needs for IP addresses. We are also on the cusp of the IoT (Internet of Things), where every device needs an IP address: light bulbs, thermostats, thermometers, rain gauges and sprinkler systems, alarm sensors, appliances, vehicles, garage door openers, entertainment systems, pet collars, and who knows what all else. All this adds up to the fact that IPv4 simply cannot handle the addressing needs of the modern world.

---

*There are blocks of IPv4 addresses that cannot be used for host addressing. For example, multicast has a block of 268,435,456 addresses that cannot be used for host addressing. IANA maintains the IANA IPv4 Special-Purpose Address Registry at https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml to document all the special address blocks and their purposes.

---

IANA (Internet Assigned Numbers Authority) ran out of IPv4 address blocks to assign to the RIRs (Regional Internet Registries) to be assigned in their respective regions, and the RIRs have now also run out of IPv4 addresses to assign in each region. ISPs (Internet Service Providers) and companies that want or need IPv4 addresses can no longer get IPv4 addresses from their RIRs and now must try to buy IPv4 addresses from businesses that may have extra (as the IPv4 address shortage deepens, the price of IPv4 addresses goes up).

Even if all the IPv4 addresses that are reserved for special purposes and cannot be used for host addressing were made available for use, we would still be in the same position because there are simply not enough IPv4 addresses due to the limited size of IPv4 addresses.

Mitigating the IPv4 Address Shortage

IANA and the RIRs would have run out of IPv4 addresses many years before they did if IANA and the IETF (Internet Engineering Task Force) had not adopted mitigations for the IPv4 address shortage. One important mitigation was the deprecation of IPv4 network classes in favor of CIDR (Classless Inter-Domain Routing). Classful addressing only allows for three assigned network sizes (16,777,216, 65,536, or 256 total host addresses per network), meaning that many addresses are wasted (a business needing only 300 host addresses would need to be allocated a classful network that has 65,536 possible host addresses, wasting over 99% of the addresses in the classful network), but CIDR allows for network sizes to fit more closely with network address requirements (a business needing only 300 host addresses could be allocated a CIDR /23 network that has only 510 usable host addresses), wasting far fewer addresses and still providing some room for growth.

By far, the mitigation that has had the biggest impact on extending the life of IPv4 is the use of Private Addressing and a variant of NAT (Network Address Translation) called NAPT (Network Address Port Translation), which is what most people mean when they refer to NAT or PAT (PAT is a vendor-specific term for NAPT). Unfortunately, NAPT is an ugly workaround that breaks the IP end-to-end paradigm, and that breaks protocols that depend on unique IP addressing, requiring even more ugly workarounds.

NAT/NAPT

The concept of NAT is pretty simple: it replaces either or both the source and destination IPv4 addresses in a packet header as the packet passes through the NAT device. In practice, it requires computation because the IPv4 header has a computed field to check the integrity of the IPv4 header, and any change made to the IPv4 header requires recalculation of the field, and some transport protocols in the packet payload also have their own computed fields that must be recalculated, using computing resources in the NAT device that could be used for packet forwarding.

In Basic NAT, the NAT device has a pool of IPv4 addresses that it uses to replace the source IPv4 addresses of the packet headers for IPv4 packets sent from an inside network to an outside network, and it maintains a translation table in order to translate the destination IPv4 addresses of traffic returning from the outside network in order to deliver the packets back to the correct hosts on the inside network. This also requires resources on the NAT device to build and maintain the translation table, and to perform table lookups. This resource utilization can slow the forwarding of packets because the resources used by NAT are taken from the resources that could be used for packet forwarding.

NAPT takes Basic NAT further by also translating the transport protocol addresses (ports) for TCP and UDP, and the Query IDs for ICMP. By also translating the transport-layer addresses, NAPT allows the use of a single outside IPv4 address for many inside host IPv4 addresses. NAPT is even more resource intensive than Basic NAT because it requires a separate table for each transport-layer protocol, and it must also perform the integrity calculations for the transport protocols.

The use of Private IPv4 addressing, that can be reused on multiple networks (you may have noticed that most home/residential networks default to use the same 192.168.1.0/24 network, which is in one of the IANA allocated Private IPv4 address ranges), along with NAPT, allows business and home users to each use a single outside (public) address for a large inside (privately addressed) network. This saves many, many IPv4 addresses (several times the total number of possible IPv4 addresses) and has extended the life of IPv4 far beyond the point at which it would have collapsed without NAPT. NAPT does have some serious drawbacks:

• NAPT breaks the IP end-to-end paradigm, and it only works with TCP, UDP, and ICMP, breaking other transport protocols. There are also application-layer protocols that use TCP or UDP that are broken by NAPT, even though TCP and UDP nominally work with NAPT. Other mitigations, e.g. STUN/TURN, may be available for some application-layer protocols, but they can add cost and complexity.
• NAPT is very resource intensive, slowing packet forwarding compared to what is possible without using any form of NAT. Some vendors add dedicated hardware to mitigate the need to steal resources from packet forwarding, but this comes at added expense, size, complexity, and power usage.
• When using NAPT, traffic initiated from outside the NAPT network cannot be delivered to the inside network because there is no translation entry in the translation table, which is added by inside-initiated traffic. The single outside (public) address is configured on the NAT device, and any packets with that destination IPv4 address and no entry for the source IPv4 address in the translation table for the transport protocol is assumed to be for the NAPT device, itself, not the inside network. There is a mitigation, called Port Forwarding, for this problem.
• Port Forwarding basically configures, manually, a permanent entry in a translation table to allow outside-initiated traffic that is destined to a particular transport protocol and address for the protocol to be delivered to a particular inside host. This does have the drawback of only allowing one inside host to be the target for a particular transport protocol and address. For example, if there are multiple web servers on the inside network, only one of the web servers can be exposed on TCP port 80 (the default for web servers).
• Because the IPv4 address shortage is so severe, the ISPs (Internet Service Providers) are running out of public addresses to assign to their customers. The ISPs can no longer get any more public addresses, so they have adopted some mitigations that especially hurt home/residential users. The ISPs want to reserve their precious public address pool for their business customers that are willing to pay for the privilege of getting public addresses. To do that, the ISPs are now starting to assign Private or Shared addresses to their home/residential customers, and the ISPs use NAPT on their own routers to facilitate the use of multiple Private or Shared addresses on a single public address. That creates a situation where a home/residential network is behind two NAPT translations (ISP NAPT to customer NAPT), and port forwarding configured by the customer on the home/residential router no longer works because it is broken by the ISP NAPT, which is not configured to forward the port to the customer router.
• Many people make the mistake of equating NAPT and security because the inside hosts cannot be directly addressed from outside. This is a false sense of security. Because a firewall connecting a network to the public Internet is a convenient place to run NAPT, that simply confuses the situation. It creates a dangerous perception that that NAPT, itself, is the firewall, and a real firewall is unnecessary. Network security comes from firewalls, which block all outside-initiated traffic by default, only allowing traffic it is explicitly configured to permit, possibly doing a deep inspection on the packet contents to drop dangerous packet payloads. What some people fail to realize is that, without a firewall, either in hardware or software, on the outside of or built into the NAPT device, to protect the NAPT device, the NAPT device itself is vulnerable. If the NAPT device is compromised, it, and by extension an attacker, has full access to the privately addressed inside network. Outside-initiated packets that do not match a translation table are destined to the NAPT device, itself, because it is the device that is actually addressed with the external address, so the NAPT device can be directly attacked.

The Solution to the IPv4 Address Shortage

The IETF predicted the IPv4 address shortage, and it created the solution: IPv6, which uses 128-bit addresses, meaning there are 340,282,366,920,938,463,463,374,607,431,768,211,456 (340 undecillion) possible IPv6 addresses. The almost unimaginable number of IPv6 addresses removes the need for NAPT (IPv6 doesn’t have any NAT standards, the way IPv4 does, and the experimental IPv6 NAT RFC specifically forbids NAPT), restoring the original IP end-to-end paradigm. The mitigations for the IPv4 address shortage are meant to extend the life of IPv4 until IPv6 is ubiquitous, at which point IPv4 should fade away.

Humans cannot really comprehend numbers of the size used for IPv6. For example, a standard IPv6 network uses 64 bits for each of the network and host portions of the network address. That is 18,446,744,073,709,551,616 possible IPv6 standard /64 networks, and that same (huge) number of host addresses for each of those networks. To try to understand a number that large, consider tools that scan all the possible addresses on a network. If such a tool could scan 1,000,000 addresses per second (unlikely), it would take over 584,542 years to perform the scan on a single /64 IPv6 network. Currently, only 1/8 of the total IPv6 address space is allocated for global IPv6 addresses, which works out to 2,305,843,009,213,693,952 standard IPv6 /64 networks, and if the world population is 21 billion in the year 2100 (a somewhat realistic number), every one of those 21 billion people could have 109,802,048 standard IPv6 /64 networks, each network having 18,446,744,073,709,551,616 possible host addresses. Unfortunately, the (decades of) IPv4 address shortage has so ingrained address conservation in people, that many people simply cannot let it go, and they try to apply it to IPv6, which is pointless and actually detrimental. IPv6 is actually designed to waste addresses.

The IETF also had the advantage of hindsight, and it improved IP (in IPv6) by removing features of IPv4 that didn’t work well, improving some IPv4 features, and adding features that IPv4 didn’t have, creating a new and improved IP. Because IPv6 is a completely separate protocol from IPv4, it can be run in parallel with IPv4 as the transition is made from IPv4 to IPv6. Hosts and network devices can run both IPv4 and IPv6 on the same interface at the same time (dual-stacked), and each is invisible to the other; there is no interference between the two protocols.

The problem with IPv6 is that it is actually a completely different protocol that is incompatible with the ubiquitous IPv4, and the mitigations for the IPv4 address shortage are seen by many people to be “good enough.” The result is that it has been over 20 years since IPv6 was standardized, and we are just now getting some real traction in using IPv6 (Google reports, as of September 2018, worldwide IPv6 adoption of over 20%, and the IPv6 adoption rate in the U.S. is over 35%). The reason we are finally moving to IPv6 is that there are simply no more unused IPv4 addresses to be assigned.

There are other obstacles, all part of the IPv4 culture, that are simply hard for people to look past. Many people are also scared of IPv6, having grown up and being comfortable with IPv4, warts and all. For example, the IPv6 addresses appear to be large and ugly compared to IPv4 addresses, and that seems to put many people off. The reality is that IPv6 is often easier and more flexible than IPv4, especially for addressing, and the lessons learned in IPv4 have been applied to IPv6 from the beginning.

Answer 2

Ron Maupin's answer gives a brilliant overview of the IPv4 shortage, but I'd like to address this part of your question:

Why can't a city (for example) have just one IP address and all homes in this city would just be on a private network of that city. Then this one city would be able to assign addresses from range 0.0.0.1 to 255.255.255.254.

On the face of it, this is exactly how "NAT" (or, more specifically, "IP address masquerading") works: a private network is set up which looks to the outside internet like a single host, and routes traffic internally to many different users. But there are some important limits you've overlooked in your example:

• Users on the inside still need to be able to address the outside internet; if you assign the address 151.101.1.69 to an internal user on your network, you won't be able to access networkengineering.stackexchange.com, because that's the address where it's hosted. So in practice, you can only use internal addresses which are reserved for this use.
• Less obviously, hosts on the outside need some way to pass traffic to the users on the inside, even if they're just browsing the web. This is because IP connections aren't like tunnels: sending a packet to a web server doesn't reserve a cable for the conversation, it just asks the server to send some packets back your way. If the web server is just sending its responses to one public address, something needs to keep track of which internal computer actually requested it. If more than one person on the internal network accesses the same web server, the only way to keep track is to assign each connection a unique dynamic port number for the replies to go to.

There are around 18 million private-use IPv4 addresses, but only 65536 port numbers. You don't actually need a unique port for every connection, because you can have a lookup table which includes the remote address as well, but there's still a limit to how far you can scale without problems.

That said, NAT is indeed one of the biggest reasons why the IPv4 network hasn't completely collapsed due to address shortages. Assigning an IP address to each household or office, and issuing them with a device to perform NAT, allows many more devices to be connected than the original design of IPv4 would allow. To scale further, carrier-grade NAT is used, where an ISP has fewer public addresses than connected households, possibly using two layers of NAT to manage the routing of packets to their eventual destination.

In the end, squeezing every possible route out of the few remaining addresses is just life support for IPv4, and at some point, every address will either be reserved for internal use, the public face of some NAT'd network, or the public address of a server accepting unsolicited connections.

Answer 3

Right now, every home has its own IP address. Why can't a city (for example) have just one IP address and all homes in this city would just be on a private network of that city?

Exactly this is already done by many internet service providers since the end of the 1990s.

In the 1990s there were different reasons (not the IPv4 shortage) for doing this. However in 2012 the internet service provider where I am customer started to do this because there were not enough IPv4 addresses any longer:

My provider uses "DS-Lite" which means that new customers get a range of global IPv6 addresses (dynamically assigned) and they only have a private IPv4 address as you describe it.

Other internet service providers use "CGNAT" what is exactly what you describe (without IPv6).

What's wrong with my understanding?

You have to see what is not possible using NAT:

If you want to operate some server you definitely require an unique IP address. And please note that some devices at your home you can access from your smartphone are "servers".

There also was a research done by Facebook Inc. that showed that the connection speed is influenced when multiple customers share one IP address.

And there is another problem:

Because of the port numbers the number of connections one IP address can establish to a certain server the same time is limited to about 60000.

I've seen video streaming pages that opened 10 connections the same time. 6000 people using the same page and the 60000 limit is reached.

And 60000 is the theoretical limit; the real limit is less.

Then this one city would be able to assign addresses from range 0.0.0.1 to 255.255.255.254.

This would definitely not work:

Suppose you want to access this website (networkengineering.stackexchange.com). Then you'll have to establish a connection to 151.101.129.69.

If there was a computer having the address 151.101.129.69 inside the city-wide local network a connection to "networkengineering.stackexchange.com" would probably not work because it is not clear which of the two computers having the same IP address is meant: The one inside the city network or the one outside the network?

This means that the city-wide network must know that 151.101.129.69 is an address outside that network.

Therefore only addresses not being used in the worldwide internet can be used.

By the way:

Even in a private network not being connected to the internet we cannot use the whole range 0.0.0.1-255.255.255.254:

In such networks we can use addresses like 151.101.129.69.

However the ranges 0, 127 and 224-255 cannot be used because these ranges are handled specially by nearly all operating systems and devices. So no common OS will allow a computer to have the address 127.10.11.12 for example.

Answer 4

There are some objective and some opinion-based reasons, and of course the most important reason -- money -- for that.

First of all, IP is designed to be end-to-end with hosts having unique addresses. Which basically means -- in theory -- that every device (server, pc, phone) needs to have its own individual IP address.

There are 4 billion addresses available, which would in principle be enough for everybody since although there are almost twice as many people on the planet, the overwhelming majority is struggling to get enough to eat and couldn't possibly afford a computer in their lifetime anyway. However, among the minority which can afford, it's more like 15-20 devices per capita always-online, including some cameras, a refridgerator, remote-controllable illumination and heating, and... your toaster. Which, for a reason unbeknownst to me, must be connected to the internet.
It's not like Internet-of-Things is truly something that's necessary, but it has been very successfully marketed and widely adopted during the last decade, so it is a reality, and thus a real issue. Plus, a considerable number of addresses is wasted for good technical reasons, and a huge number of addresses is wasted for no good reason at all (I'll come back to that later).

In practice, there is that thing called NAT which does exactly (or almost so) what you suggest. Technically it violates the design principles of IP, which is however not such a big issue. Nevertheless, while NAT works fine and is actually a good thing in may respects, it has its limits (notably, it can in theory only extend the number of addresses by a factor of at most 65535, and in practice much less than that) there is a strong desire to have NAT die out. The most urgent one is the fact that NAT obscures the 1:1 relation between address and device. Which is actually a good thing because oh heck, you actually don't want that, but it's not so good for certain people who have a say (such as law enforcement). Another reason is that NAT (which is usually firewalled, too) makes certain applications harder. In particular, one host behind NAT talking to another one behind NAT is troublesome (not impossible, just harder to implement than absolutely necessary).

Now, for a "typical" enterprise (or other organization) with some people sitting in cubicles and communicating in a corporate network or via VPN, and possibly with internet access of sorts, it would make perfect sense to have e.g. one IP address per floor and NAT them all through. Not only would it be much cheaper, it would also make it much less obvious to an external observer who's who. Plus, it would make it much more obvious to the network admin. Both of which is a good thing. Alas, reality has it that several large companies and US universities have reserved huge ranges of IP addresses just so every machine that they might possibly ever have has its own individual address. Ask why? I couldn't say.

The most important reason why we are "running out" is that there is a strong motivation to promote IPv6. While IPv6 not only removes some troubles and brings some (though few) desirable features, it also adds measurable overhead on the wire (especially if several layers such as IP-over-PPPoE-over-ATM are involved such as is the case for many home internet connections), and customers pay for bandwidth measured in bytes per second or pay for volume per-byte. So... either way moving away from IPv4 is very desirable because as a provider you either need less backbone, or you can charge your customers more for the same stuff.
Which lead to the crisis being actively promoted, giving out addresses like candy (until almost none were left). I can remember, not very long ago, couple of years, you'd get something like 5 addresses just like that when renting a $50 server.

Answer 5

We are not running out of IPv4 addresses.

What is the cost of an IPv4 address today? It seems it's about $18: http://ipv4marketgroup.com/ipv4-pricing/

A single IPv4 is much cheaper than a barrel of oil. I need 8 new barrels of oil per year for my car. I don't need 8 new IPv4 addresses every year -- in fact, a single IPv4 address is enough for me, and currently I'm not running any servers so I could perfectly well share the address via carrier grade NAT.

In fact, even though oil (in typical quantities consumed) is much, much more expensive than IPv4 addresses (in typical quantities needed), we are not even running out of oil.

If you buy an IPv4 address, you'll have it forever (or until the IPv6 believers have displaced IPv4 as the protocol of choice). At a discount rate of 5%, it's $0.9 per year. I pay about $300 per year for my mobile connectivity (that is behind carrier grade NAT) and about $300 per year for my fixed connectivity (that is not behind a carrier grade NAT). The yearly cost of an IPv4 address is 0.3% of the yearly cost of Internet connectivity via either fixed or mobile services.

There are technologies that such as TLS and HTTP proxy that allow running many servers behind a single IPv4 address, identified by their domain names. Thus, for example Internet of Things (IoT) can perfectly well result in deployment of billions of servers, far more than what we have IPv4 addresses. Most new unencrypted protocols today are built on top of HTTP; most new encrypted protocols today are built on top of TLS. Both have means of specifying the DNS name of the server, so a proxy can work and hide multiple servers behind a single IPv4 address. Protocols such as SSH can run servers behind non-standard ports, and OpenSSH has a "proxy command", so a SOCKS / HTTP CONNECT proxy can also support multiple machines you can SSH to behind a single IPv4 address. Would I pay $18 extra for the convenience of not having to specify a non-standard port or a proxy command in my OpenSSH config? I wouldn't.

Techniques such as endpoint independent mappings allow UDP and TCP hole punching that can setup a point-to-point connection between two hosts behind a NAT, just needing an external server for determining what TCP/UDP port the NAT assigned for each endpoint. NAT doesn't break end-to-end connectivity, it just requires a separate helper server to determine what port number the NAT assigned. Will the world need 4 billion such helper servers? No it doesn't.

IPv6 is a religion. Many people find it hard to believe Internet of tomorrow can work without IPv6, just like they find it hard to believe a society can work without religion.