If you are buliding a network with a stable heirachical structure, centralised up-front design and a plentiful supply of addresses you can keep routing tables pretty small. Routers towards the bottom of the heiracty only need to know about their local environment and which ports led upstream. Routers towards the top of the heiracty only need to know about the broad structure of the network.
The Internet is not that network. It's a loose jumble of interconnected networks run by different entities with constantly shifting relationships. While smaller customers do get addresses allocated by their providers most larger networks get their addresses allocated in a way that has nothing to do with network structure. Worse on the IPv4 internet the long history and constrained address supply mean that many networks have acquired their addresses in multiple seperate allocations.
The result is that the Internet routing table is large and growing. Some routers can avoid having the full table relying on default routes to their upstreams to do the heavy lifting for them but ultimately default routes can only take you so far, sooner or later you need an explicit route to tell you where the destination is.