There are issues, when multiple IP subnets are used on a single layer-2 segment (i.e. VLAN).
Lack of support
Some devices, notably the Cisco Adaptive Security Appliance (ASA) and Firepower Threat Defense (FTD) firewalls, do not support configuring multiple IP addresses on an interface. You wouldn't be able to use these devices as a router/default gateway for all subnets.
Assigning IP addresses to hosts via DHCP becomes challenging as the DHCP server can only infer the layer-2 segment on which a client performed request:
- When the DHCP server is directly attached to the VLAN, it will determine the subnet/IP address pool from the interface on which the DHCP request was received.
- When a DHCP relay is used to forward DHCP requests to a central DHCP server, the DHCP relay (typically, but not necessarily, the router) will put its primary IP address on the layer-2 segment, where the DHCP request broadcast was received, in
giaddr field of the DHCP packet (and usually but not necessarily also as IP source address) and forward it to the DHCP server. The DHCP server will then use the
giaddr to pick the subnet/IP address pool.
If multiple subnets are used per layer-2 segment, the DHCP server must be explicitly told which subnets belong together. This feature is often called shared networks and not all DHCP servers support it. The Kea DHCP server documentation describes this feature quite well.
Direct communication between IP subnets without a router involved becomes possible when sharing a layer-2 segment. This may be a security risk.
IP broadcasts, irregardless of whether they are sent to
255.255.255.255 or a network broadcast address such as e.g.
192.0.2.0/24, will be mapped to layer-2 broadcast (with Ethernet a frame with destination MAC address
ff:ff:ff:ff:ff:ff), that is flooded to every port attached to the layer-2 segment and will therefore need to be processed by every host irregardless of the subnet.
If a appropriate routes are configured on end hosts and end hosts implement the weak host model, end hosts will even be able to have IP unicast communication with each other, by requesting and answering cross-subnet ARP requests.
sysctl settings of the Linux kernel or the Microsoft Windows
weakhostreceive interface settings.
Due to the above issues, I would advise against using multiple subnets per layer-2 segments. Still there are a few use cases (aside from the deceptive simplicity of only configuring a single VLAN).
When migrating hosts from one subnet to another without disruption, it is necessary to temporarily configure multiple subnets.
Address exhaustion/Address space fragmentation
If all/most IP addresses of a given subnet are used and a simple broadening of the subnet mask is not possible, because the adjacent subnet is already used somewhere else and renumbering is not feasible in the short term, shared networks may serve as temporary kludge.
This is only a real use case for IPv4 networks, as in IPv6 networks, the size of a subnet is (almost) always /64, which is virtually impossible to exhaust.
In access networks, such as DOCSIS cable networks, customers may use their own IP router, that is distinct from the provider's modem. The provider may want to manage the modem via IP though, through a different subnet from the customer subnet. Technically this could be accomplished via layer-2 segmentation (e.g. VLANs) on WAN side of the modem too, but I don't known if this supported or done in practice.