1

What are the possibilities of connection/link failover in the following configuration: 2 geographic locations connected by 2 independent radios (a primary and a backup, the backup is a mikrotik device). The idea is to use the primary with higher bandwidth and if there is a malfunction, switch to the second (mikrotik) one until the primary becomes available. The network is equipped with Ciscos 2960s, 3650 and a router on the remote location site. The network heavily relies on VLANs which are in many cases shared between the locations. I have been suggested 3 options:

  1. Complete network separation and full routing
    • It has the disadvantage of having to separate the shared vlans and in my view, complicating the structure of the network.
    • I dislike this solution as it seems to be a travel back in time neglecting the advantages of vlans etc.
  2. Spanning tree
    • Some network engineers dislike this and they claim it would clog the backup connection (nonsense as we have been running with ST on the planned backup for many years).
  3. LACP link aggregation
    • There is a doubt how Mikrotik as an L3 device fits into this schema.
    • The backup device management would not be available when the primary is active and vice versa.

Can you please comment on the above suggestions or provide any more suggestions that would work so that I could decide? Although I am not a cisco certified, but I know a thing or two about networks and for me option #2 is the most appealing, #3 has the problem of availability of the other device pair, otherwise I also like it but I wonder what you think.

  • Can you add a diagram of your topology in question. – John K. Jan 30 '17 at 14:30
  • Did any answer help you? If so, you should accept the answer so that the question doesn't keep popping up forever, looking for an answer. Alternatively, you could provide and accept your own answer. – Ron Maupin Aug 16 '17 at 21:36
1

OK - so there's an old saying I learned when I started doing networking: LAN's are bridged and WAN's are routed.

What this means is that networks that host lots of end stations are going to be well-served by physically proximate switches with effectively infinite, cheap, and readily available bandwidth. Lots of broadcasts and multicasts to find services? No biggie. Random bursty transfers? It's all localized, so who cares. Someone blew something up and the LAN is fried? Painful but it's only effecting people within that site and within that network. In larger enterprises this tends to mean a few dozen to a few hundred people in part of the floor of an office building. If something breaks it's a critical problem but in such situations the workers can often just take the elevator to another floor while someone fixes the problem.

The problem with what you're describing (old-style bridging) is that it exactly fails at all of the above. A problem that saturates a given LAN now also saturates your most limited resource: inter-site bandwidth. That saturation means that an entire site is now potentially offline. The cause of such an issue could be a bad NIC, misbehaving app, a user who decided to run a huge backup job in the middle of the day, a misconfiguration of that STP you've been running for years. Heck, something as innocuous as an end user in a conference room accidentally plugging in two ports from the wall into a single dumb switch can cause a loop that ends up taking down both of your wireless links (and thus all communications) while someone tries to find a laptop and console cable to attempt to isolate the problem - assuming, of course, they're in the right site to help.

Seriously - everything is going to run better if you run L3 over those links. You have the potential to implement traffic shaping / QoS in an intelligent way to prioritize your voice traffic before it hits the radio, you can actually engineer some traffic to normally use the backup link while sending the remainder over the primary. As Ron correctly points out, failover is measured in milliseconds rather than minutes. Want to add a third site? Easy. Try that with STP and it starts to get ugly. Add a third link? Maybe come up with a solution where you put an Internet connection in the second site and use a site-to-site VPN to carry certain traffic? No problem.

The thing is that while - ideally - each site ends up with its own subnets (the model shown to scale and function reliably and the recommendation of literally every significant network vendor on the market) you could add in some kind of overlay to map L2 over the safe/sane L3 backbone. Set up L2TP or OTV via some routers and have the subnets show up wherever you want. Heck, consider some of the possibilities with VXLAN-EVPN where a modern switch can make sure any subnet shows up anywhere while still locally routing traffic over that L3 backbone.

Relying on spanning tree across remote sites for redundancy is a bad idea even with two sites. It was bad in 1995 and it's no better in 2017. It gets progressively more horrible as any amount of size or complexity is added. Seriously - to this day I still periodically see horrific STP problems in major data centers operated by highly qualified dedicated crews under fairly controlled conditions. Moving this into the realm of potentially flaky long-haul wireless and workgroup-scale hardware isn't going to improve things.

0

This type of thing is handled in routing all the time. Your MikroTik is off-topic here, but your Cisco router is on-topic.

In your Cisco router, you can set two static, default routes with different administrative distances. The link with the lower AD will be preferred. When that link goes down, the route is removed from the routing table, and the other default route now takes over. When the preferred link comes back up, its route is reinstalled in the routing table. This is called a floating, static route.

  • Yes, but that is the first option I wrote about and this requires removal of shared vlans between the sites. I see this as a step backward toward a more complex topology. – leosenko Jan 30 '17 at 17:01
  • 1
    It is actually a step forward into the present. You really don't want layer-2 crossing a WAN. All layer-2 broadcasts must cross the WAN. This can really exacerbate any layer-2 problems. Suddenly, both sites are down in a broadcast storm. We live in a layer-3 world, and there is no real reason that everything needs to be on the same layer-2 domain. – Ron Maupin Jan 30 '17 at 17:03
  • 2
    Actually running a network with appropriate L3 segmentation is much simpler than trying to bridge dozens of VLAN's over redundant paths. One of the big steps forward in the history of networking was moving -away- from the topology you're describing. If I had to do L2 extension in the manner you're describing I'd probably look at an overlay topology - so build a redundant L3 network over the radios and then use something like L2TP or OTV (or VXLAN if you can get some different hardware) to encapsulate and bridge the packets. – rnxrx Jan 30 '17 at 17:41
  • 3
    @leosenko, that is simply not true. We run phones over thousands of different sites, each site with its own layer-3 networks. The old way of doing things is to have a single network, but it is very limiting. That is why we moved away from one LAN to multiple LANs connected by layer-3. It simplifies things, and adds a lot of flexibility. You are artificially limiting yourself by not moving to current practices. A business should be prepared to grow 10x, and you simply cannot do that with layer-2. – Ron Maupin Jan 30 '17 at 18:06
  • 1
    Not at all. Remember what LAN stands for: Local Area Network. Routing protocols automate the routing aspect, including your failover scenario. You only add rules where you need to, for example, between VLANs. A single LAN doesn't have that flexibility, and you don't add rules that you don't need. A layer-2 failure can take up to 50 seconds to fail over, but routing protocols can fail over in milliseconds. Layer-2 load balancing can be problematic, but simple in layer-3. – Ron Maupin Jan 30 '17 at 18:13
0

You seem to prefer a link layer mechanism so as to maintain layer 2 adjacency between the sites. A possible gotcha LACP is that the links must be the same speed, layer 2, and the links need to attach to the same switch.

If you really need to have layer2 adjacency you can either forego the backup link or figure out a way to tunnel/bridge over IP. In other words stand up a GRE tunnel between your Cisco routers, then configure bridging between your LAN side interface and that tunnel interface. Layer 3 all the wireless links. That would be a clever config, may be pushing the limits of the OS, would enable redundant link usability, maintain layer 2 adjacency, and would be fun to attempt :-) See IRB

  • I do not need L2 solution, I just consider it more robust, considering that L3 solution which has been suggested to me seems to be very complex. I prefer logical network grouping of devices, as this creates a significant simplification to security rules (routing, nat etc) and thus it is less prone to error (forgetting a particular rule when changes are made leads to opened hole). But maybe I am wrong, maybe there is a way to do it in L3. I just have not heard about it yet. – leosenko Jan 30 '17 at 18:05
  • 1
    Layer 3 partitioning will provide you greatest and most predictable control. – Ron Royston Jan 30 '17 at 19:24
  • 1
    There is no universe where a given L2 solution is more robust than L3. More convenient in the short run? Sure. But robust? One episode of unknown unicast flooding or broadcast/multicast storms will put that to rest.. – rnxrx Jan 31 '17 at 0:29
  • I understand, yet Ethernet won, ...like 20 years ago. There is certainly much FUD around Wide-Area Ethernet, yet, it's inevitable. – Ron Royston Jan 31 '17 at 1:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.