MTU mismatch scenario, two hosts handle in a different way ingoing packets bigger than local interface MTU. How does MTU really work?

Question

I’ve been struggling with the MTU topic for a while and I still can’t get how it really works. The majority of books that I read and the courses that I attended all seem to treat this subject in a superficial way.

I tried sending pings between a Mac Os host and a Windows host by simulating different mtu scenarios. Both hosts are in the same Lan and are connected to a Wifi router.

Scenario A

Mac Os host , mtu 1500

Windows host, mtu 1300

I sent a 1500 bytes ip packet with icmp echo request out of the Mac Os interface to the Windows 1300 mtu interface. The Windows machine accepts the packet normally and then sends back the echo reply by fragmenting the ip packet into two parts .

Scenario B

Mac Os host , mtu 1300

Windows host, mtu 1500

This scenario is the opposite of the first one. Hower when I send a 1500 bytes ip packet with ping request to the Mac Os host I don’t get any answer ( shouldn’t the Mac Os host have sent icmp error fragmentation needed ? ).

This behaviour it’s a little confusing. Is the mtu concerned with ingoing, outgoing or both directions traffic ?

How could be explained the different behaviour of the Mac Os and Windows hosts ?

A lot of books and resources talk about “next hop mtu” when referring to icmp error fragmentation needed. This statement seems appropriate if we have a scenario like this :

In this case when we send a packet bigger than 1492 destined to host B the router is going to send an ICMP error fragmentation needed ( provided that it hasn’t a firewall that prevents that from occurring ). This is because the “mtu interface to the next hop” is only 1492 bytes.

However consider this other scenario :

Let’s say I send a 1500 bytes ip packet with icmp echo request to R1 ( address 10.0.0.2 ). What is R1 going to do in this case ? Accept the packet as the Windows host did ? Simply drop the packet as the Mac Os host did ? Drop the packet and send an icmp error fragmentation needed ?

Is there some mtu wizard who could explain how things really work when it comes to mtu and try to make some order by explaining and putting together the scenarios I just showed ? What is this mtu at its core ? Is it a hardware limitation ( NIC )? Is it a software limitation ?

Big thank you to whoever is going to clarify this topic !

Answer 1

Simply put, all nodes within a network (=L2 segment=L3 subnet) need to use the same MTU. If a node receives a packet larger than its interface supports the packet is simply dropped.

Multi-homed nodes may use a different MTU on each interface, depending on the network they're connected to.

Routers/gateways are multi-homed nodes. For IPv4, they may need to fragment IP packets that are larger than the network allows that they are forwarded to. For IPv6, packets are not fragmented within the path. Rather, the sender host is obligated to perform path MTU discovery and intermediate routers simply return an ICMPv6 Packet too big error to the source.

In your scenario, if a host uses a different MTU than its gateway then that's never going to work.

For completeness: the maximum transfer unit MTU for an interface might be different from the maximum receive unit MRU. Of course, an incoming packet is only dropped when exceeding the MRU for that interface. Most often, "MTU" is used to refer to both properties though.

Answer 2

Statement #1 - for successful fragmentation IP packet must not have DF-bit (do not fragment) installed in 1. Statement #2 - oversized packets are dropped by the networking devices and hosts if they are bigger that configured MTU size, i.e. host cannot receive the packet bigger than configure on interface. What may happen in scenario A (big MTU on the left, small MTU on the right, direction is left -> right)? As on host and router's interface MTU is same 1500, router receives the ping packet. Ideally, this packet must be fragmented, sent over lower MTU link (1300), and re-assembled on the end host. Important to note, that MTU value pairs "host X <-> router interface X" must be equivalent. Example: left host 1500 - left router interface 1500; right host 1300 - right router interface 1300. This may not happen if for packet DF-bit is installed into "1". If we change MTU on hosts only, without router interfaces, and start generate traffic with higher MTU, the packets will be dropped on the router level. In other words, you can empty bucket of water into the bucket-size receiver, and then fill several bottles on other size and deliver it to the end consumer. But if you try to empty bucket of water into bottle-size receiver on other size, water will spill to the floor...