4

CORE: Dell N4032 (6.3.0.18) Access: Dell N2048 (6.3.1.13)

We are in the process of setting up a serviced office that will serve multiple businesses. Each "customer" will naturally be on their own VLAN, and each VLAN will require access to the Internet and some other VLANs.

There will be some open VLANs like a "Public Print" VLAN and so forth. There will also be "Server" VLANs which potentially all other VLANs will need to access (or some may not, i.e. filtered)

We have core L3 switches, 20 L2 Access Switches and two HA firewalls.

The question is whether we do the routing for the VLANs on the L3 switches themselves, or via the Firewall.

My thoughts - if we route via the firewall (i.e. Router on a Stick), all inter-VLAN traffic is going to have to go via our Firewall device, which will naturally put on a load on it's overall throughput (especially with DPI enabled). The pro being it's a hell of a lot easier to configure the firewalling between the VLANs this way.

If we route via the L3 switch, we benefit from the full throughput of the firewall for the WAN (which will be a 1Gbps connection), but the con being having to use "Policy Based Routing" on the 2 Core Switches which is generally a pain to administer.

Is there a best practice in a situation like this? What should I consider to make a decision?

UPDATE

The issue using the ACLs on the switch is as follows.

  • There is a maximum of 100 ACLs
  • We will have "Customer VLANs" that will require access to the Internet, to a VOIP VLAN, a Server VLAN and Print VLAN.
  • They however should not have access to the other "Customer VLANs".

Problem is all "Customers" will be within one /16 - i.e.

[Parent subnet being 10.100.0.0/16]
Customer 1 - 10.100.0.0/24
Customer 2 - 10.100.1.0/24
Customer 3 - 10.100.2.0/24

I need to apply the ACLs on the VLAN so when traffic hits that VLAN it is blocked from accessing other customer VLANs.

Currently I have something like this:

1000 deny 10.100.0.0 0.0.255.255 192.168.50.0 0.255.255.255 
1010 deny 10.100.0.0 0.0.255.255 192.168.60.0 0.255.255.255  
1020 permit any any 

This is then applied to all Customer VLAN interfaces. This solves the issue of blocking access to Server VLANs and other private VLANs, however, this doesn't prevent these VLANs from then accessing other customer VLANs (within the 10.100.0.0/16).

Problem is we could end up having 50 plus customers, which would mean an inconceivable number of ACLs (given each ACL can only have 10 entries, and each ACL would need an ACE for every other customer). This hits the max number of ACLs the switch supports almost instantly.

I just can't see another way of doing this at this stage without using our firewall as a "Router on a Stick". Any advice is greatly appreciated.

Update #2

Strange a I attempted this today with an ACL on a Customer VLAN as below (applied "IN" on the interface)

1000 deny 10.100.0.0 0.0.255.255 192.168.50.0 0.255.255.255 
1010 deny 10.100.0.0 0.0.255.255 192.168.60.0 0.255.255.255 
1015 deny 10.100.0.0 0.0.255.255 10.100.0.0 0.0.255.255 
1020 permit any any 

UPDATE #3

When applying suggested ACL below - it prevents access between devices on the same VLAN. The ACL is L3:

ip access-list Block_Customers
 1000 deny ip any 10.100.0.0 0.0.255.255
 1020 permit any any
!

However, then, two devices on this VLAN, with IPs that can speak to each other i.e. 10.100.1.0/24, can no longer ping one another. If I remove the ACL, they once again can ping each other.

This is wrong right, my traffic on one VLAN is L2, why is this L3 ACL applying?

UPDATE #4

If I apply the ACL as suggested and...

  • Two devices on the same VLAN on different switches, they cannot ping in each other (without the ACL they can ping so the trunks must be working). They can ping the Internet.

  • Two devices on the same VLAN, on the same switch, they can ping each other, but can no longer ping the SVI IP Address on the core switch on that VLAN. They can however ping the Internet, so traffic can reach that IP.

  • Why PBR on the layer-3 switch? Many layer-3 switches support ACLs that can block unwanted traffic without PBR. – Ron Maupin Apr 12 '17 at 17:44
  • I am no longer using PBRs - see my update and the associated issue with using ACLs. – PnP Apr 12 '17 at 19:25
  • What is the layer-3 switch model, and why would you need large ACLs? Simply deny inbound anything to the 10.100.0.0/16, and permit everything else. That would prevent communication from any customer to any other customer. That one ACL could be used on all the customer networks. Granted, the question is on topic here, but it may be better suited to Network Engineering, depending on the switch model. – Ron Maupin Apr 12 '17 at 19:29
  • Denying inbound everything to 10.100.0.0/16 also prevents communication between PCs contained within the same VLAN (that are spanned across switches) – PnP Apr 12 '17 at 19:31
  • Not at all. Traffic within a VLAN doesn't cross the layer-3 interface, so it would not be evaluated against the ACL. Layer-2 (VLAN) traffic doesn't know or care about layer-3. You can run many different layer-3 protocols (e.g. IPv4, PIX, IPv6, AppleTalk, etc.) on your layer-2 VLAN, and the layer-2 switching doesn't care or know which layer-3 protocols are in use. – Ron Maupin Apr 12 '17 at 19:34
2

My suggestion is to keep it simple. You seem to be trying to add more complexity to a scenario that could be simplified. This is so far what you are doing, please correct me if I'm wrong:

  • Isolating customers by using VLANs.
  • Removing that isolation by enabling routing between VLANs.
  • Asking what will be the best way to isolate them based on your available resources.

My point is to not route traffic between VLANs by default.

Suggested solutions:

  • Use VRFs on the L3 cores: A VRF per VLAN. This way you could keep the isolation, but provide access to common resources. You might even want to consider doing NAT between VLANs to enable (controlled) communications between them. Why NAT? Because you might also want to use the same IP addressing space on all customer's VLANs - making new deployments easier for you (they're repeatable, standardized)
  • Instead of VRFs, a physical router per VLAN and repeat the above.
  • I do not believe The N4032's support VRFs? But yes essentially I will have lots of VLANs that shouldnt access each other, but some that should, and some that generally shouldn't access any other VLANs except for a few addresses in one given VLAN. – PnP Apr 15 '17 at 0:05
  • Do you have any advice for how I would go about setting the VRFs up to allow what I want to achieve? – PnP Apr 15 '17 at 15:10
  • I would use one VRF per VLAN and only route what's needed. Maybe it makes sense to group some VLANs in a single VRF, but as I guess you'll be seeing that as you advance the project. – Pedro Perez Apr 16 '17 at 3:21
1

L3 and even some L2 services (such as DHCP) are not just a pain to administrate on any given switch directly; they're typically dangerously clumsy.

I can't recommend administrating the vast majority of switches directly beyond very small implementations. Instead, you might consider abstracting their interfaces away via an SDN solution such as OpenDaylight (there are many others - this is just an example). An SDN system will use "drivers" to render your standard and friendly configurations into configurations that the given switch will understand. This can vastly help you to standardize your control plane in a safe and highly effective manner.

In addition to that, CM platforms like SaltStack or Puppet have switch support that may work well for you. This won't give you as complete an experience that an SDN platform would, but it's a step in the right direction.

A router on a stick is to be avoided, though. In small stacks it's fine, but yours sounds too big for those shoes.

  • Thanks - unforunately I have what I have to work with - so I need to make the best of a bad situation in this instance - so its "router on a stick" or L3 Policy Based Switch Routing. I just need some help deciding.. – PnP Apr 11 '17 at 21:44
  • If that's the case, use policy based routing on the switches where applicable. It may not be the most pleasant to manage, but it's hands down the better architecture strategy. – SmallLoanOf1M Apr 11 '17 at 22:16
  • Plus, you can always implement CM, something like Rancid, or SDN later on without needing to significantly refactor your configuration if you take this route. – SmallLoanOf1M Apr 11 '17 at 22:25
  • Please see my update. – PnP Apr 12 '17 at 15:30
1

It appears that you are using some type of Cisco layer-3 switch. It is extremely easy to deny traffic from a network to all the other networks contained within a supernet. If all the customer network are in 10.100.0.0/16, you can have a single ACL that will block one customer from the other customers' networks. Simply apply it inbound to the layer-3 switch on the layer-3 interfaces (SVIs). For example:

ip access-list extended Block_Customers
 deny ip any 10.100.0.0 0.0.255.255
 permit any any
!
interface Vlan10
 ip address 10.100.10.1 255.255.255.0
 ip access-group Block_Customers in
!
interface Vlan11
 ip address 10.100.11.1 255.255.255.0
 ip access-group Block_Customers in
!
interface Vlan12
 ip address 10.100.12.1 255.255.255.0
 ip access-group Block_Customers in
!
interface Vlan13
 ip address 10.100.13.1 255.255.255.0
 ip access-group Block_Customers in
!

and so on.

This will prevent any traffic destined for the 10.100.0.0/16 network from entering the router (layer-3, not layer-2) part of the switch from the customer networks. This only affects layer-3, and it has absolutely no affect on any traffic within the network, even if the network extends across multiple switches (a bad practice, by the way).

  • Many thanks, i will confirm tomorrow and mark as the answer. – PnP Apr 12 '17 at 20:56
  • Would this also affect the ability to NAT traffic from our firewall to a device on that VLAN? – PnP Apr 13 '17 at 8:07
  • This blocks traffic flowing between devices on the same VLAN after testing today. – PnP Apr 13 '17 at 11:00
  • It will also block return Internet traffic returning to that 10.23.0.0 subnet because of the ANY statement. – PnP Apr 13 '17 at 12:04
  • It should not, unless the Dell is weird. The direction, in, means it is only applied inbound to the router from the interface to which it is applied, and it should not be outbound. The any is the source, so it must be any host on the VLAN, inbound to the router, destined for the 10.100.0.0/16 network. – Ron Maupin Apr 13 '17 at 14:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.