Handling VLAN based MPLS circuits with site specific internet access

Question

I'm having difficulty wrapping my head around how to set this up and the MPLS vendor is being no help so I figured I would ask here.

I have a 2 node MPLS each site having internet access on the same circuit the MPLS rides in on. These circuits replace dedicated internet access at each site with an IPSEC tunnel between the sites. We want to leave our existing firewalls in place as they provide content filtering and VPN services. I am trying to configure a layer 3 switch (a cisco SG300-10P) at each site to set up this scenario.

The relevant info (ip addresses changed to protect my idiocy)

Site A

1. Local Lan: 172.18.0.0/16
2. Existing Firewall (internal): 172.18.0.254
3. MPLS Gateway to Site B: 172.18.0.1
4. Internet IP Range 192.77.1.144/28
5. Carrier Gateway to internet 192.77.1.145

Items 3 and 5 are on a single peice of copper coming from an adtran netvana (Carrier equip I have no access)

Site B

1. Local Lan: 192.168.2.0/23
2. Existing Firewall (internal): 192.168.2.1
3. MPLS Gateway to Site A: 192.168.2.2
4. Internet IP Range 216.60.1.16/28
5. Carrier Gateway to internet 216.60.1.16

Items 3 and 5 are on a single peice of copper coming from an adtran 908e (Carrier equip I have no access)

So given the above what I want to do at each site is set up these cisco switches so that:

Port 1 = Carrier Connection Port 2 = Interal Lan Port 3 = Firewall

Where the local lan is not exposed to the Internet IP range (ie if some yahoo sets their machine up on a provided internet ip with the carriers gateway it doesn't work) Or put differently from port 1 all traffic in the internet subnet can only exit on port 3 and from port 1 all traffic on the local lan subnet can only exit port 2.

Every attempt I have made so far results in no access between the ports at all or basic dumb swith behavior (any host on any port can get across all of the IP ranges).

First question here so please be kind. :) If you need more information I would be happy to provide it.

Answer 1

Depending on how the service is delivered by the SP will dictate how you can separate the services on your end.

Typical methods are either a port per service or a VLAN tag per service.

If the SP is tagging the traffic you can just set up your switch to trunk to the SP and then separate the traffic into two access ports (one to FW and one to LAN).

If it's a port per service then just create two VLANs with the services in different VLANs for isolation.

Answer 2

Assuming the Adtran is not using VLAN, I would establish a transport network between the Adtran Router and the Firewall (maybe using the one that is already existing on the Adtran interface).

Having done that, you only need to add routes on the Firewall to cover all your communication needs (default gateway pointing to the Adtran).

Afterwards you can connect everything else behind your firwall so that it is protecting your networks.