How exactly does Path-MTU Blackhole Detection work?

Question

I'm currently trying to understand how the Path-MTU Blackhole Detection really works.

From what I understand: You send rather big packets, starting at like 1460 Bytes (TCP example) and wait for responses. You most likely will get an ICMP message saying Destination Unreachable Fragmentation Needed. By lowering the packetsize you can, after some back and forth, determine the Path MTU. But how do you know when theres a black hole router in between? My guess would be that in that case the sent packet is never acknowledged and since there is never an ICMP message one can conclude that there has to be a black hole router in between the source and destination. Am I correct?

Answer 1

As you wrote, PMTUD works by starting with the default (maximum) MTU. When an in-path router finds that it exceeds the MTU to the next hop, it returns an Fragmentation Needed ICMP message that includes the possible MTU. Accordingly, only one such message is required for each MTU bottleneck.

A black hole connection is caused by an ICMP filter in the return path dropping that ICMP message (usually for "security" reasons - bad practice in this case).

The resolution is for TCP to discover this situation (a segment/packet larger than any of its predecessors is lost). TCP can then actively probe the MTU for the path as described in RFC 2923.

Answer 2

RFC 2923 describes a general method without giving specific details. So it's up to the vendor to decide exactly how to do it.

How to detect

This shows up as a TCP connection which hangs (fails to make progress) until closed by timeout (this often manifests itself as a connection that connects and starts to transfer, then eventually terminates after 15 minutes with zero bytes transfered). This is particularly annoying with an application like ftp, which will work perfectly while it uses small packets for control information, and then fail on bulk transfers.

A series of ICMP echo packets will show that the two end hosts are still capable of passing packets, a series of MTU-sized ICMP echo packets will show some fragmentation, and a series of MTU-sized ICMP echo packets with DF set will fail. This can be confusing for network engineers trying to diagnose the problem. There are several traceroute implementations that do PMTUD, and can demonstrate the problem.

How to fix

TCP should notice that the connection is timing out. After several timeouts, TCP should attempt to send smaller packets, perhaps turning off the DF flag for each packet. If this succeeds, it should continue to turn off PMTUD for the connection for some reasonable period of time, after which it should probe again to try to determine if the path has changed.