0

Background

My workplace has 2 office areas which is only 200 meters away from each other. They have their own local networks, their own Internet connections, with routers configured to fallback to the other if their own Internet connection is down. The connection between 2 routers is stable with multiple redundant links.

There are multiple Wireless Access Points between these 2 areas, with wired connection to either one. Users often move between these 2 areas a lot, and I want to provide them a seamless AP transition instead of setting up 2 separate SSIDs. To achieve it, all Access Points (or SSIDs with the same name) must be on the same Layer-2 network. 1 of the 2 routers will be used as the default gateway, with the other still being used as fallback.

The past 2 weeks was a lot of fun (and headache) to me after I learned a lot about wireless operation modes and how to encapsulate 802.1q frames inside 802.11 packets using GRE tunnel. Putting a 1522 byte packet inside a packet which allows 2304 bytes of payload is straightforward. So far so good.

The problem arises when I try to do the same over a wired interface with 1500 byte MTU. The GRE itself takes 4 bytes, and the outer IPv4 header takes 20 bytes, or 40 bytes in case of IPv6, the MTU of the GRE should be reduced to 1476 and 1456 respectively. On both sides of the tunnel, I bridge the GRE interface to a physical network, to merge the 2 Layer-2 networks on both ends of the tunnel together.

enter image description here

Here comes the issue: all devices and interfaces on the 2 Layer-2 networks use default MTU of 1500 bytes. From the technical perspective, all Ethernet frames with payload exceeding the MTU won't go through the tunnel -- they will be dropped instead. From the end-user perspective, the problem is obvious and takes effect immediately as network services on the other side of the tunnel are reported as online but are not responding to requests.

Question

Note that there is no Layer-3 involved, so no MSS clamping nor Path MTU Discovery here. Manually reducing the MTU of all hosts on the 2 networks is not an option. Is there any elegant solution for this?

P.S.

I thought of setting the GRE tunnel MTU to 1500 bytes regardless of the physical interface MTU. The tunnel traffic over the physical interface is routed at Layer-3 (not switched at Layer-2), meaning it can be fragmented, but I do not know if it will work and at what kind of performance. I am also not sure about what will happen if the outer IPv4 packet header exceeds the default 20 bytes, or the outer IPv6 header exceeds the default 40 bytes.

Improve this question
9
  • Quoting from your question: "Note that there is no Layer-3 involved, so no MSS clamping nor Path MTU Discovery here. Manually reducing the MTU of all hosts on the 2 networks is not an option. Is there any elegant solution for this?" Manually reducing the MTU on end-stations is the elegant solution. It's unclear why you're working so hard to build an overlay with GRE. Please elaborate on: 1) what you need to accomplish with the GRE overlay 2) why are you building this? 3) a network topology map with make / model of your network gear 4) Whether the end-stations are macOS, Windows, or linux
    – Mike Pennington
    Commented Dec 6, 2022 at 11:19
  • @MikePennington I did not describe my network because I was afraid of going against the rules -- NE is not for questions related to network design, although I really appreciate recommendations. I edited my question to clarify what I am trying to achieve. My servers are a mix of Windows and Linux, while clients are Windows, Android, and iOS.
    – Livy
    Commented Dec 7, 2022 at 1:03
  • Why do you use GRE for just 200 m? That looks like a simple VLAN job.
    – Zac67 ♦
    Commented Dec 7, 2022 at 4:16
  • @Zac67 My real network is more complicated than that. It has multiple routers with dynamic routing between them, so that if one link is down, they can still connect to each other. The users frequently move between areas, and currently we have multiple SSIDs for each area. I want to use a single SSID while retaining fault-tolerant when a link is down, by routing the GRE tunnels between the routers, to simulate a single L2 network across all areas. Yeah I could have created a VLAN and enabled STP, but for now I am trying to do it using GRE.
    – Livy
    Commented Dec 7, 2022 at 4:32
  • It is probably a bad idea, but I need to actually do it to know how bad it is, before coming up with a different plan.
    – Livy
    Commented Dec 7, 2022 at 4:49

2 Answers 2

Reset to default
1

Is there any elegant solution for this?

Not really. An MTU mismatch within a bridged L2 segment creates all sorts of problems and there's no solution on L2. As you've stated you could try to use 'hidden' fragmentation (fragment outer packet for full inner MTU) but that seriously increases overhead and may greatly impact performance.

The key is to not use bridging but routing. Routing makes PMTUD work correctly and enables IPv4 in-path fragmentation (not desirable but better than an on-link MTU mismatch). PMTUD (mostly) relies on ICMP packet too big (v6) or fragmentation required (v4) and that can only be issued by a layer-3 router, not a layer-2 switch.

MSS clamping works on the transport layer (TCP only) and is usually only possible on a router as well.

[EDIT after question updates/comments] Your goal is to provide seamless roaming for wireless users, requiring a bridge between the WAPs. Since you seem to be controlling the intermediate link between the buildings you should consider running a VLAN over that link, creating a direct bridge. This would be a very simple design.

With the much more complicated approach using GRE you could use baby giants on the link: grow frame size & MTU, so that you enable GRE to carry full-sized frames with 1500 bytes payload. Depending on the exact frame size and the IP version in use, you'd need an increase of at least the GRE overhead (IPv4 20 + GRE 4 bytes) plus 18 bytes for Ethernet.

PS: Some wondering about the GRE tunnel bridging bit: GRE transports network-layer packets by standard. Pushing Ethernet frames through the tunnel requires some proprietary extension or yet another encapsulation.

Improve this answer
1
  • I edited my question to describe my real network situation, simplified a little bit to describe what I am doing. It is sad to know there is no solution for MTU on Layer 2. But it there any solution to do what I am trying to achieve?
    – Livy
    Commented Dec 7, 2022 at 1:13
2

Moving to using jumbo frames on the 'WAN' link between the networks (where you are using GRE for some unexplained reason) would also work since there would no longer be an issue with the frame size exceeding 1500 bytes.

But I agree that you need some very specific reason to do things the way you are trying to do them and even then it is probably not the right way to do it. A routed connection would be much better and if you need something tunneled, just do a VPN. If you need a single layer 2 network then look for VXLAN, another overlay network technology or something similar. But it would probably be easier to just live with segmented networks with VPN or routed links between them.

Improve this answer
8
  • I edited my question to describe my real network situation, simplified a little bit to describe what I am doing. Please take a look and see if there is solution. In the meantime, I would look into VXLAN and see if it can help what I am trying to achieve.
    – Livy
    Commented Dec 7, 2022 at 1:17
  • With the updated information, I think you are back to a fundamental issue that you have designed the connection between the 2 main networks incorrectly, most likely. You're trying to find an elegant solution for a mistake in the underlying network design because the existing setup doesn't address your needs for the network. Aside from that, I don't see why you can't just have the same SSID on all your APs and let people roam as they wish. 200 m is enough for them to lose signal at a certain point and move to the new network, regardless of the layer 2 design underneath it all.
    – FrameHowitzer
    Commented Dec 8, 2022 at 21:37
  • 200m is the distance between routers (or areas). There are multiple Access Points in each area, and their signal covers the whole factory. The routers use IS-IS dynamic routing as a fault-tolerant feature. What I wanted was to reuse that feature to keep the Wi-Fi network up all the time. That was why I thought of simulating a single L2 network over multiple L3 networks. But as you and other people point out, it turns out to be a quite bad idea.
    – Livy
    Commented Dec 9, 2022 at 2:52
  • I guess I am still confused, your wifi network should be up all the time without any extra work or configuration. Just setup the access points with connections to the networks in use and they should be pretty reliable. If you already have wired connectivity that solves your needs, adding basic wifi APs that connect to those networks should be all you need.
    – FrameHowitzer
    Commented Dec 14, 2022 at 5:45
  • My users frequently move around the factory. I want to have a single Wi-Fi network (SSID) spanning across the whole factory instead of multiple SSIDs in each building. To achieve it, I need a single L2 network spanning the whole factory, to bridge multiple AP into it. Currently I also have IS-IS dynamic routing between routers (buildings) to provide fault tolerant. That was why I came up with the idea of simulating a single L2 network using GRE tunnel running on the routers, to make the Wi-Fi network up if a link is broken. I hope you get my (bad) idea.
    – Livy
    Commented Dec 16, 2022 at 7:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.