Remember that the exchange happens using layer-2 addressing, not layer-3 addressing. The frame is addressed to the host MAC address, so the host will accept the frame. A problem may arise at layer-3 for an unconfigured IPv4 process. Some, generally older, clients cannot process a unicast layer-3 packet until IPv4 has been configured, but there is a workaround for that. The client sets the broadcast flag so that it receives the offer as a broadcast.
It is all explained in _RFC 2131, Dynamic Host Configuration Protocol:
In the case of a client using DHCP for initial configuration (before
the client's TCP/IP software has been completely configured), DHCP
requires creative use of the client's TCP/IP software and liberal
interpretation of RFC 1122. The TCP/IP software SHOULD accept and
forward to the IP layer any IP packets delivered to the client's
hardware address before the IP address is configured; DHCP servers and
BOOTP relay agents may not be able to deliver DHCP messages to clients
that cannot accept hardware unicast datagrams before the TCP/IP
software is configured.
To work around some clients that cannot accept IP unicast datagrams
before the TCP/IP software is configured as discussed in the previous
paragraph, DHCP uses the 'flags' field [21]. The leftmost bit is
defined as the BROADCAST (B) flag. The semantics of this flag are
discussed in section 4.1 of this document. The remaining bits of the
flags field are reserved for future use. They MUST be set to zero by
clients and ignored by servers and relay agents. Figure 2 gives the
format of the 'flags' field.
1 1 1 1 1 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|B| MBZ |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
B: BROADCAST flag
MBZ: MUST BE ZERO (reserved for future use)
Figure 2: Format of the 'flags' field