1

Today we are managing two big departments of our University and after a political decision, both departments are joining forces on the IT services. Due to this we are trying to unify our Core Network, but this wasn't easy as it should appear to be.

Both departments runs a huge routed infrastructure with public IPv4 address and only one of them have NAT for clients. But the issue isn't here. The issue is about our WAN links. They are "broken TCP network" in it's glory :)

To illustrate the issue. The WAN static addresses are:

  • Dept#1: 192.0.2.195/26
  • Dept#2: 192.0.2.196/26
  • Common Gateway: 192.0.2.193/26

So as you can see both WAN links are on the same subnet, and this is an issue because on Dept#1 we have seven /24 routed networks and on Dept#2 we have three /24 routed networks pointing to the respective IP's as next hops on the routing tables.

Since TCP/IP networking does not allow two addresses of the same subnet in a single piece of hardware we cannot use those addresses on the same equipment. Basically we need to put some Layer 3 device to process the routes and after it merge the networks in a unified core.

The problem here is how to do this. Today we have two pfSense firewalls to do this, but we would prefer to merge those firewalls to put them in HA with both networks, but we're unable to do so.

I have some ideas using VRF on our Nexus 3048 Core Switches but I don't know if this a good idea or even if VRF should be used this way.

Here are the routing tables of both equipment:

Dept #1:

% interfaces
WAN (wan)       -> bce0        -> v4: 192.0.2.195/26
LAN (lan)       -> bce1        -> v4: 100.64.36.1/24

% netstat -nr
Routing tables

Internet:
Destination        Gateway            Flags      Netif Expire
default            192.0.2.193        UGS        bce0
10.3.12.0/24       100.64.36.36       UGS        bce1
127.0.0.1          link#5             UH          lo0
192.0.2.192/26     link#1             U          bce0
192.0.2.195        link#1             UHS         lo0
100.64.36.0/24     link#2             U          bce1
100.64.36.1        link#2             UHS         lo0
100.64.37.0/24     100.64.36.36       UGS        bce1
100.64.40.0/26     100.64.36.36       UGS        bce1
100.64.40.64/26    100.64.36.36       UGS        bce1
100.64.40.128/26   100.64.36.36       UGS        bce1
100.64.40.192/26   100.64.36.36       UGS        bce1
100.64.136.0/22    100.64.36.36       UGS        bce1
198.51.100.0/24    100.64.36.35       UGS        bce1

Dept #2:

% interfaces
WAN (wan)       -> bce0        -> v4: 192.0.2.196/26
LAN (lan)       -> em0         -> v4: 172.16.0.1/21
MGMT (opt1)     -> em1         -> v4: 10.7.0.1/24
SRV (opt2)      -> em1_vlan29  -> v4: 100.64.29.1/24
VPN (opt3)      -> em1_vlan11  -> v4: 192.168.172.254/24
LIG (opt4)      -> em0_vlan666 -> v4: 172.26.66.30/27

% netstat -nr
Routing tables

Internet:
Destination        Gateway            Flags      Netif Expire
default            192.0.2.193        UGS        bce0
10.7.0.0/24        link#3             U           em1
10.7.0.1           link#3             UHS         lo0
10.172.16.0/26     link#2             U           em0
10.172.16.1        link#2             UHS         lo0
127.0.0.1          link#6             UH          lo0
192.0.2.192/26     link#1             U          bce0
192.0.2.196        link#1             UHS         lo0
100.64.29.0/24     link#8             U      em1_vlan
100.64.29.1        link#8             UHS         lo0
100.64.30.0/26     link#2             U           em0
100.64.30.1        link#2             UHS         lo0
100.64.30.64/26    link#2             U           em0
100.64.30.65       link#2             UHS         lo0
100.64.30.128/26   link#2             U           em0
100.64.30.129      link#2             UHS         lo0
100.64.30.192/26   link#2             U           em0
100.64.30.193      link#2             UHS         lo0
172.16.0.0/21      link#2             U           em0
172.16.0.1         link#2             UHS         lo0
172.26.66.0/27     link#10            U      em0_vlan
172.26.66.30       link#10            UHS         lo0
192.168.172.0/24   link#9             U      em1_vlan
192.168.172.254    link#9             UHS         lo0

So as you can see we've a lot of networks. I changed the public IPv4's for the 100.64/10 range and the WAN links to the 192.0.2/24 range just to keep it generic.

If we could just process the WAN links in some device (in the Core Switches for instance) and them use the LAN addresses on a unified firewall the problem would be solved.

And the last thing, I can speculate how the WAN routing table looks like. It should be something like this:

Routing tables

Internet:
Destination        Gateway            Flags      Netif Expire
192.0.2.192/26     link#1             ?          switchport24
192.0.2.193        link#1             ?          loopback0
100.64.29.0/24     192.0.2.196        ?          switchport6
100.64.30.0/24     192.0.2.196        ?          switchport6
100.64.146.0/24    192.0.2.196        ?          switchport6
100.64.36.0/23     192.0.2.195        ?          switchport5
100.64.40.0/24     192.0.2.195        ?          switchport5
100.64.136.0/22    192.0.2.195        ?          switchport5

And this table cannot be changed since we don't have access to it.

Thanks in advance.

PS1: Talking with the guys responsible for the "WAN" architecture would be impossible right now. So we must find out a way without changing anything on the WAN address space.

PS2: Who needs routing protocols when everything is built with static routes... :(

EDIT: Drawning and long story short.

Today:

+----------------+       +------------------+       
|                | ----> | Firewall Dept #1 | -----------\
|                |       | 192.0.2.195/26   |            |
| Switch         |       +------------------+       +---------------+
| Layer 2 (WAN)  |                                  | Internal Core |
| 192.0.2.193/26 |       +------------------+       +---------------+
|                |       | Firewall Dept #2 |            |
|                | ----> | 192.0.2.196/26   | -----------/
+----------------+       +------------------+

What we want:

+----------------+       + -----------------+
| Layer 2 (WAN)  |       | Unified Firewall |       +---------------+
| 192.0.2.193/26 | ----> | 192.0.2.195/26   | ----- | Internal Core |
|                | ----> | 192.0.2.196/26   |       +---------------+
+----------------+       +------------------+

Routing Table - They are actually real public IPv4 network. I changed to 100.64/10 network to keep it generic.

100.64.29.0/24  gw 192.0.2.196
100.64.30.0/24  gw 192.0.2.196
100.64.36.0/23  gw 192.0.2.195
100.64.40.0/24  gw 192.0.2.195
100.64.136.0/22 gw 192.0.2.195
100.64.146.0/24 gw 192.0.2.196

Issues:

  • I don't have management of the Layer 2 Switch.
  • Our management start on the firewalls.
  • I can't use one cable from the Layer 2 Switch to bind two IP's on the Firewall due to security policies on the Layer 2 Switch. So both cables must be connected with distinct networks.
  • I can find use some addresses from our IPv4 pool on the Firewall, but to do this I must preroute before the firewall. I can use the "Internal Core" for this, but I don't know how.
  • Separate routers solves this.
  • We do have a Cisco Nexus 3048 Switch, so we can do some wizardry on this guy. That's why I mentioned VRF on the first place.
Improve this question
5
  • 1
    Companies which merge often have this problem. This is normally handled with NAT as a temporary measure until one or both of the original companies can readdress the network.
    – Ron Maupin ♦
    Commented Jul 17, 2016 at 19:29
  • Hello Ron, thanks for the reply. But I don't know if this is the case. We don't have NAT, there's only a special case which can be disabled. Everything here is real public routed IPv4 address. The issue is resumed on bad WAN addresses. They should be distinct /30 or /31 address and not being on the same /26 subnet. Hope I was clear now. I think this issue is really specific, and that's way I'm lost. I changed our addresses to the 100.64/10 range just to keep it generic. All the addresses are real IPv4, even the phones on wireless get real IPv4 addresses.
    – Vinícius Ferrão
    Commented Jul 17, 2016 at 19:52
  • What I'm referring to is that each department with overlapping address space must have its own router, and those can connect and NAT to to a common router. If each department is using public, not private, address space, and it is the same address space, then you have a real problem. It is not easy to readdress and entire department or company, but you do what you must. Companies which merge run into this problem all the time.
    – Ron Maupin ♦
    Commented Jul 17, 2016 at 20:12
  • If you look at the tables, there are no Overlapping addresses :) Can we talk on the #chat? I'll try to explain better the issue.
    – Vinícius Ferrão
    Commented Jul 17, 2016 at 20:13
  • Did any answer help you? if so, you should accept the answer so that the question doesn't keep popping up forever, looking for an answer. Alternatively, you could provide and accept your own answer.
    – Ron Maupin ♦
    Commented Aug 12, 2017 at 5:52

2 Answers 2

Reset to default
1

Fair question, but it's way too wordy. Basically, change the IP addresses so that they don't overlap. That's the answer. Just do it. There is no sense in some bandaid NAT hack to allow it to continue. And change your subnet masks to /24's. Keep it simple.

Improve this answer
0

Based on the conversation we had in chat, I really don't see the problem. There are many cases where you will have multiple routers with interfaces in the same network. You just connect them via layer-2:

enter image description here

R1 and R2 are the routers for the two departments, and R3 is the router in common. S1 is a layer-2 switch which connects all three routers. You will need to have some way to let each router know about the routes behind the other routers, either statically configured (doesn't scale) or with a common routing protocol.

This is basic networking, it is perfectly acceptable, and it is not at all uncommon.

Improve this answer
9
  • Ron I've done the drawing. In your scenario it would be OK, and I agree, but it's not the case. See if my drawning is good enough now. Many thanks for spending your time with our issue.
    – Vinícius Ferrão
    Commented Jul 17, 2016 at 21:35
  • What you say want is not what you want. Pick one address or the other for the firewall, and bite the bullet and readdress what needs to be readdressed. If you have control over the network, that should be doable, if not easy. Otherwise, you can do it as I have outlined, and R3 can be a transparent firewall instead of a router, if you need that.
    – Ron Maupin ♦
    Commented Jul 17, 2016 at 21:44
  • We're converging. Some networks are only reachable over 192.0.2.195 and others over 192.0.2.196. Different addresses. That's why we need both IP's working. What I was thinking is to put something in front of the firewall to do the prerouting.
    – Vinícius Ferrão
    Commented Jul 17, 2016 at 21:46
  • @ViníciusFerrão, no. You will be able to reach all the networks from behind one address or the other, or you can use a transparent firewall to R1 and R2 to keep reaching the different networks from different addresses. I really think that you are overthinking this. Why, for instance, can't you reach all the networks from one address? That's the idea behind routing. A router knows how to get to all the addresses behind it, and other routers should know to go to it for them.
    – Ron Maupin ♦
    Commented Jul 17, 2016 at 21:46
  • @ViníciusFerrão, I just don't see why you can't have one address or the other for your firewall in the drawing you added. It shouldn't matter which one. It will advertise to the WAN what addresses are behind it.
    – Ron Maupin ♦
    Commented Jul 17, 2016 at 21:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.