When IP uses a MAC-based L2 for local transport, the L2 transport needs to be told the destination node. For this, the next hop IP address needs to be translated to a MAC address. With IPv4 this is ARP's function.
In the original Ethernet scheme (via electrial or logical bus), all nodes on the network physically received the L2 frame but everyone except for the destination dropped it. Today, switches use the L2 destination address for direct packet switching for efficiency. Effectively, they do the hubs' (or repeater's) job while omitting the extraneous copies.
Skipping ARP and subsequently L2 addressing would require the NIC to understand IP in order to drop unwanted packets (or the OS stack to deal with all that). Additionally, switches would also need to understand IP to work efficiently - IP is much more complex than Ethernet (which is why routers used to be a lot more expensive and less performant than switches), so your network concentrators would be more costly. (This probably isn't true today but it was for a very long time.)
Note that when Ethernet was invented, IPv4 didn't even exist (outside the labs) and IPv6 was in the far future. It was up to IP to make use of the existing infrastructure and technology.