Help with bringing DR site in to general network

Question

I hope I can get some help as I've been wracking my brain to figure out the best way to get a new MPLS connection set up between our main site and our DR site. I'll first give a description of what I have today and what I need to achieve. I'm also throwing in our CE interface configs to show what we have. I changed our PE ip address’ to protect the innocent. (I tried to get formatting correct for the interface info below but it's not playing nice)

Right now we have a DR site/rack in a co-location facility. The DR site has an internal gateway set exactly as the main gateway at our main site. The reason for the "duplicate" internal gateway address being the same is if we have a major meltdown at the main site (fire, etc), we can easily bring up replicated VM's at the DR site, switch over DNS at our DNS provider for our domain names and our websites can be running and available within a short amount of time (DNS replication withstanding).

Right now these 2 sites connect over the internet with a site-to-site VPN through Cisco ASA's and each side is hidden behind NATs through the site-to-site so these 2 duplicate gateways can exist and we can replicate data between sites by sending traffic to the NAT'd address' and then on to the servers behind the firewall at DR.

We just had an MPLS network installed by an ISP directly connecting each site and that's where I'm stumped.

I really need to have the easy access that we have today of all servers sitting at DR running on that duplicated gateway address because like I mentioned, in case of emergency I can just fire up VM's on a Quorum OnQ device and have a dup of my infrastructure up and running in less than an hour. But I obviously can't have the duplicated gateways running there AND here without a conflict.

What would someone suggest to look at to have these 2 sites up and running across MPLS?

(everything is /24 for sub netting)

Main site and DR site internal gateway:

192.168.10.10

Main site 2951 interface configs:

interface GigabitEthernet0/0
 no ip address
 duplex full
 speed 100

interface GigabitEthernet0/0.1
 encapsulation dot1Q 1801
 ip address 21.21.21.21 255.255.255.252

interface GigabitEthernet0/0.2
 encapsulation dot1Q 1800
 ip address 24.24.24.24 255.255.255.252

interface GigabitEthernet0/2
 ip address 192.168.10.251 255.255.255.0
 duplex auto
 speed auto

router bgp 7000
 no synchronization
 bgp log-neighbor-changes
 redistribute connected
 redistribute static
 neighbor 24.24.24.23 remote-as 2500
 no auto-summary

ip default-gateway 21.21.21.20
ip forward-protocol nd

DR site 2951 interface configs:

interface GigabitEthernet0/2
 ip address 192.168.10.250 255.255.255.0
 duplex auto
 speed auto

interface GigabitEthernet0/0/0
 no ip address
 duplex full
 speed 100
 media-type sfp

interface GigabitEthernet0/0/0.1
 encapsulation dot1Q 1901
 ip address 22.22.22.22 255.255.255.252

interface GigabitEthernet0/0/0.2
 encapsulation dot1Q 1900
 ip address 23.23.23.23 255.255.255.252

router bgp 7000
 bgp log-neighbor-changes
 redistribute connected
 redistribute static
 neighbor 23.23.23.22 remote-as 2500

ip default-gateway 22.22.22.21
ip forward-protocol nd

When we first set up the DR site, I posted a question last year on here about concerns I had with the site-to site VPN and the NATs, and it turns out because of NAT I can have the 2 gateways running simultaneously. That was great. Now, things are changing and i'm not sure of the best way to get this to work moving forward.

Answer 1

You could have your MPLS provider configure a L2VPN for you across the MPLS network. This allows you to have the same VLANs at both ends and actually communicate on a frame-by-frame rather than packet-by-packet basis. Once you have that up and running, you can configure a First Hop Redundancy Protocol (FHRP) like VRRP. This would leave your interface IP addresses at your primary site until the site went down (or the link between the two went down) at which point the router at your DR site would start answering ARP requests for those IP addresses. The problem you want to make sure you avoid at that point is split brain where the link between the sites goes down and then both sites think they're active.

Just one possible fix. You might also want to look at load balancing (active/active sites) or using DNS for failover.

Answer 2

You either have to give up your MPLS connection, or give up having the same IP addresses in both places -- you can't have both without some nasty NAT translations.

Fundamentally, you have to change the way you set up your DR servers. They can't have the same addresses as your production servers. There are other ways to accomplish what you want -- everyone has the same problem you do. But I confess my VMWare knowledge is a bit thin.