MTU the most Confusing Networking term. Lot of questions

Question

I have ton of questions and i feel this is most confusing term for me. The more i read the confusion deepens. I am not able to connect dots. please help.

1. Do we need MTU adjustment with EoMPLS/dot1q tunneling? I had scenario where vpn worked but traffic was not traversing and my boss said interface has non- configure able MTU

2. Most argument says that if you dont adjust MTU then fragmentation will occur! So if we dont adjust MTU everything will work fine and fragmentation will occur any other. But at least things will work? Isn't it?

3. when we use 4470,9000 MTU? Cant we use 9000 by default if our internal LAN supports it? Can we randomly set a MTU size?

4. What is protocol MTU?

5. If MTU mismatch in ospf then what is the problem IP can handle fragments? I know mtu ignore but whats the point in DBD packets?

6. Can BGP neighborship affected by MTU mismatch? May be BGP through SDH?

7. When it is time to increase MTU size of an interface considering the protocol we wanna run isn't working there is no MTU mismatch and for time being we're considering we have best ASICs ever created on planet so fragmentation is not a big deal! I wanna know what are the scenarios we need to increase MTU as last resort and what were the limitation because of which our desired functionality was not working.

Answer 1

Let me address question 2:

Most argument says that if you dont adjust MTU then fragmentation will occur! So if we dont adjust MTU everything will work fine and fragmentation will occur any other. But at least things will work? Isn't it?

There can be a big problem it there is an MTU mismatch between two connected devices. If routers A and B are physically connected together, and A's interface MTU is 2000, but B's interface MTU is only 1500, then A can create a frame that is 2000 bytes long. But because B's interface MTU is only 1500, it will consider that frame a "giant," and drop the frame.

If devices are connected together at layer 3, then their common interfaces and all the (layer 2) devices between them must have the same MTU size.

EDITED to include question 7:

When it is time to increase MTU size of an interface considering the protocol we wanna run isn't working there is no MTU mismatch and for time being we're considering we have best ASICs ever created on planet so fragmentation is not a big deal! I wanna know what are the scenarios we need to increase MTU as last resort and what were the limitation because of which our desired functionality was not working.

It's common to forget this important fact: Routers are responsible for fragmenting packets, but the receiving host is responsible for putting the fragments back together.

You could, with specially designed routers, fragment packets all day long with no throughput impact to the router, but you put a tremendous load on the receiving host who has to reassemble the fragments back together. You want the MTU of the path from sender to receiver to be the same so no device has to fragment, and therefore the receiver doesn't have to put them back together. That is the purpose of path MTU discovery -- to insure that fragmentation is not necessary.

Note that in IPv6, fragmentation is not allowed. Path MTU discovery is essential for IPv6 to work.

Answer 2

Direct answers to your questions are either not possible without more information or might not really help you. I'll try to explain:

The maximum transfer unit is the largest IP packet size that can be sent on a given interface. With Ethernet, the MTU is generally 1500 bytes.

Ethernet's maximum frame size is 1518 bytes for untagged standard frames - 18 bytes is the overhead for the smallest Ethernet header + FCS, 1518 - 18 = 1500 bytes payload. If you tag 802.1Q VLANs the header increases to 22 bytes, with a maximum frame size of 1522 bytes. QinQ adds another 4 bytes and so on.

When tunneling, you encapsulate either an Ethernet frame (L2 tunneling) or an IP packet (L3) into an outer IP packet. If you need to use the standard Ethernet MTU even inside the tunnel, the outer packet and frame need to grow accordingly.

In reverse, if the outer packet/frame cannot increase (e.g. across the Internet) the inner packet/frame cannot use the normal maximum size.

L3 tunneling requires an additional IP header, so the MTU needs to be 20 byte less (with IPv4) than the normal 1500 bytes (assuming the path supports that). L2 tunneling over IP requires both an Ethernet and an IP header, so it eats away 38 bytes from the standard MTU.

Sending a (standard) maximum-sized packet over a tunnel with decreased MTU requires fragmentation either by the router, or (preferrably) by the source using path MTU discovery (PMTUD).

Fragmentation increases processing overhead (in routers) and wastes bandwidth (for the additional fragments' headers), so it's good to avoid.

In some scenarios it might even be reasonable to reduce the MTU/frame size across the whole network, e.g. for a remote location that uses a VPN tunnel for everything anyway, removing the need for PMTUD or router fragmentation.

So, when examining which MTU is the best in your case, you need to find out the maximum packet/frame size for each tunnel you use. In most cases there is no strict standard but you have to consult your ISP.

Jumbo frames - Ethernet frames with a payload larger than 1500 bytes - can be useful when accommodating tunnel overhead in order to avoid fragmentation ("small" or "baby jumbos". However, frames sizes are not negotiated and need to be configured alike across all nodes in a given segment. Nodes configured with a smaller maximum frame size than others (=MTU mismatch) will drop 'oversized' frames. You should restrict jumbo usage for special purpose links and not use them lightly. So yes, OSPF and BGP can both fail if frames cannot be received properly.

Large jumbo frames - usually with 9000 bytes MTU - are only used on high-performance networks to decrease overhead. Most NICs support offloading features that - more or less - make messing with jumbos unnecessary.