Entries in an ARP table on modern OSes will time out. This is not an official part of RFC 826, An Ethernet Address Resolution Protocol, although the RFC does discuss it at the end:
Related issue:
It may be desirable to have table aging and/or timeouts. The
implementation of these is outside the scope of this protocol. Here is
a more detailed description (thanks to MOON@SCRC@MIT-MC).
If a host moves, any connections initiated by that host will work,
assuming its own address resolution table is cleared when it moves.
However, connections initiated to it by other hosts will have no
particular reason to know to discard their old address. However,
48.bit Ethernet addresses are supposed to be unique and fixed for all time, so they shouldn't change. A host could "move" if a host name
(and address in some other protocol) were reassigned to a different
physical piece of hardware. Also, as we know from experience, there
is always the danger of incorrect routing information accidentally
getting transmitted through hardware or software error; it should not
be allowed to persist forever. Perhaps failure to initiate a
connection should inform the Address Resolution module to delete the
information on the basis that the host is not reachable, possibly
because it is down or the old translation is no longer valid. Or
perhaps receiving of a packet from a host should reset a timeout in
the address resolution entry used for transmitting packets to that
host; if no packets are received from a host for a suitable length of
time, the address resolution entry is forgotten. This may cause extra
overhead to scan the table for each incoming packet. Perhaps a hash
or index can make this faster.
The suggested algorithm for receiving address resolution packets tries
to lessen the time it takes for recovery if a host does move. Recall
that if the <protocol type, sender protocol address> is already in the
translation table, then the sender hardware address supersedes the
existing entry. Therefore, on a perfect Ethernet where a broadcast
REQUEST reaches all stations on the cable, each station will be get
the new hardware address.
Another alternative is to have a daemon perform the timeouts. After a
suitable time, the daemon considers removing an entry. It first sends
(with a small number of retransmissions if needed) an address
resolution packet with opcode REQUEST directly to the Ethernet address
in the table. If a REPLY is not seen in a short amount of time, the
entry is deleted. The request is sent directly so as not to bother
every station on the Ethernet. Just forgetting entries will likely
cause useful information to be forgotten, which must be regained.
Since hosts don't transmit information about anyone other than
themselves, rebooting a host will cause its address mapping table to
be up to date. Bad information can't persist forever by being passed
around from machine to machine; the only bad information that can
exist is in a machine that doesn't know that some other machine has
changed its 48.bit Ethernet address. Perhaps manually resetting (or
clearing) the address mapping table will suffice.
This issue clearly needs more thought if it is believed to be
important. It is caused by any address resolution-like protocol.
Edit based on your comments:
The protocol theory does not cover your situation, and it leaves it up to the OS-specific implementation. Unfortunately, we cannot answer what a particular host will do, as that is up to the host OS, which is off-topic here. Different OSes, and even different versions of the same OS, could do things differently. There is no one answer to what happens.
I understand that this is not a very satisfactory answer (someone even voted it down), but it is reality. You may discover that a particular host does something, but you cannot depend on any other hosts (especially with different OSes) doing the same thing. As Zac67, points out, one host OS may send a gratuitous ARP when changing its address, but that does not guarantee other hosts will do the same thing.
ARP was something added to IPv4, and it is not really a part of IPv4, but it is a separate process. The problem was taken into account when IPv6 was created, and IPv6 has built into it NDP (Neighbor Discovery Protocol) that, among other things, replaces ARP, and it solves the very problem you have pointed out with ARP.