When a layer has a chunk of data larger than the underlying layer can process, it needs to break it down. The major mechanisms in practice are segmentation by the TCP transport-layer protocol (L4) and (where that doesn't work) fragmentation by the IP network-layer protocol (L3).
OSI is a theoretical model. In practice, IP (in the network layer) can fragment packets that are too large for the link layer below. The largest possible packet size that can be transported by the current link layer is called the maximum transfer unit (MTU). 1500 bytes is the MTU for IP over standard Ethernet, but IPv4 can run over any network with a minimum MTU of 576 bytes (IPv6 has a minimum of 1280 bytes).
IPv4 can run into situations where the MTU decreases along a path, requiring fragmentation by the gateway. IPv6 generally uses Path MTU Discovery for the full connection path; fragmentation can only happen in the source host.
Preferably, the transport layer (on top of the network layer) manages data chunking. Most prominently, TCP segments data according to the network layer's MTU size (using the maximum segment size, directly derived from the MTU), and so on. Therefore, TCP won't try to send a segment that won't fit into an L2 frame.
Other transport-layer protocols do things differently and some protocols are unable to handle chunking (e.g. UDP), so they rely on proper handling on the application layer, or on IP fragmentation if need be.