I have a Firewall that I need to use to join two different VRFs on service provider PEs. The EGP of choice is BGP. I have done a number of tests with different techniques, I have found two solutions that work but I havn't settled on a solution as I don't consider either perfect. So I am putting this out there as a question with my findings to see if anyone has a better way of doing this.
vrf: Cust-A vrf: Cust-B on PE1 \ / on PE1 \_____ Firewall_____/ / \ vrf: Cust-A / \ vrf: Cust-B on PE2 on PE2
In the diagram above the PEs are the same PEs. eg, Customer A and Customer B each have a 'vrf' on the two Service Provider PEs (PE1, and PE2).
Unfortunately the Firewall is not MPLS capable. So now comes the problem of get BGP to distribute routes between the two MPLS VPNs which share the same AS Number.
BGP set local-as
My first attempt was using a local-as on each of the VRFs and the Firewalls. (So that's three unique AS Numbers) Local-AS actually results in both the real and the local-as number being prepended to the AS-Path. So BGP loop prevention sees the native AS number and won't install the routes. Juniper (and I presume Cisco) provide an option to not prepend the native-as number and just prepend the local-as number. However it turns out that that local-as number, (even though it's assigned to just the VRF), is "global" to the BGP process and the Cust-B vrfs will not import routes from Cust-A vrfs.
To verify this I found local-as works if Cust-A and Cust-B are on physically separate devices and native-as pre pending is disabled, (tested with Juniper logical instances), however there might be downstream implications within the Service Provider PE cloud that I am unable to test.
The next tool of choice was AS-Override and surprisingly the Firewall supports AS-Override which I hadn't expected. By overriding the AS-Path learned from the PEs the routes are successfully passed across the VRFs. This has a side-effect of creating potential loops, because routes learned from Cust-B-PE1 are readvertised, (with the AS-PATH replaced), to Cust-B-PE2 (and likewise for Cust-A). In my lab environment this doesn't end up in the routing table only by "luck" that the AS-Path happens to be longer because AS-Override prepends the Native-AS after replacing all instances of the remote-as.
Again I can see that this will have downstream problems in the Customer MPLS cloud when a customer is using AS-Path length to determine the route.
Finally the only option which seems available is Loops-In. This also works, but suffers from the same problem as AS-Override. Loops-In has the advantage that the AS-Path is left intact.
Strip Private AS
Another gotcha is the use of private-as numbers by Customer CEs. If Cust-A and Cust-B have overlapping private AS numbers, neither AS-Override or Local-AS will stop the CEs from rejecting the routes and reconfiguring CE devices for loops-in isn't practical. So this requires the use of the strip-private-as feature on either the Firewall or the VRF instances.
At the moment I have settled on using AS-Override with strip-private-as on the PEs.
To stop the potential route looping I am adding a community at the Firewall on the learned routes and ensuring that I don't readvertise this community to any customer VRFs it was learned from.
A Better Way
I am interested in the best way to do this?
It should be noted that the VRFs have a combination of private IPv4s, public IPv4s and IPv6 addresses. Any private IPv4s are not redistributed to the Firewalls by the PEs.
NB: Static routing to the Firewall is not an option.