the rfc6864 - Updated Specification of the IPv4 ID Field precisely address this issue.
Extract from this rfc:
- Introduction
In IPv4, the Identification (ID) field is a 16-bit value that is
unique for every datagram for a given source address, destination
address, and protocol, such that it does not repeat within the
maximum datagram lifetime (MDL) [RFC791] [RFC1122]. As currently
specified, all datagrams between a source and destination of a given
protocol must have unique IPv4 ID values over a period of this MDL,
which is typically interpreted as two minutes and is related to the
recommended reassembly timeout [RFC1122]. This uniqueness is
currently specified as for all datagrams, regardless of fragmentation
settings.
Uniqueness of the IPv4 ID is commonly violated by high-speed
devices; if strictly enforced, it would limit the speed of a single
protocol between two IP endpoints to 6.4 Mbps for typical MTUs of
1500 bytes (assuming a 2-minute MDL, using the analysis presented
in [RFC4963]). It is common for a single connection to operate far
in excess of these rates, which strongly indicates that the
uniqueness of the IPv4 ID as specified is already moot. Further,
some sources have been generating non-varying IPv4 IDs for many
years (e.g., cellphones), which resulted in support for such in
RObust Header Compression (ROHC) [RFC5225].
and
4.1. IPv4 ID Used Only for Fragmentation
Although RFC 1122 suggests that the IPv4 ID field has other uses,
including datagram de-duplication, such uses are already not
interoperable with known implementations of sources that do not vary
their ID. This document thus defines this field's value only for
fragmentation and reassembly:
>> The IPv4 ID field MUST NOT be used for purposes other than
fragmentation and reassembly.
Datagram de-duplication can still be accomplished using hash-based
duplicate detection for cases where the ID field is absent (IPv6
unfragmented datagrams), which can also be applied to IPv4 atomic
datagrams without utilizing the ID field [RFC6621].
(emphasis mine)