11

I'm in the process of over-hauling our network, the issue I keep coming back to is: trying to bring layer 3 to the core, while still having a centralised firewall.

The main issue I have here is that the "mini core" switches I've been looking at always have low in-hardware ACL limits, which even at our current size we could quickly hit. I'm currently about to (hopefully) purchase a pair of EX4300-32Fs for the core, but I've looked at other models, and other options from Juniper and Brocade's ICX range. They all seem to have the same low ACL limits.

This makes perfect sense as the core switches need to be able to maintain the wire-speed routing, so don't want to sacrifice too much through ACL processing. So I can't do all my firewalling on the core switches themselves.

However, we do mostly fully managed servers, and having a centralised (stateful) firewall helps a lot with that management - as we can't have customers talking directly to each other. I'd like to keep it that way if we can, but I feel as though most ISP networks would not do this kind of thing, hence why in most cases it would be straight-forward to do the routing in the core.

For reference, here is the topology I would ideally like to do (but not sure where to fit the FW obviously).

network

Curent Solution

Right now, we have a router-on-a-stick configuration. This enables us to do NAT, stateful firewalling and VLAN routing all in the one spot. Very simple.

I could continue on with (roughly) the same solution by extending L2 all the way to the "top" of our network - the border routers. But then I lose all the benefits of the wire-speed routing that the core can offer me.

IIRC the core switches can do 464 Gbps of routing while my border routers will be able to offer maybe 10 or 20 Gbps if I'm lucky. This isn't technically a problem right now, but more an issue of growth. I feel as though if we don't design the architecture to leverage core routing capacity now, it's going to be painful to redo everything when we're larger and need to leverage that capacity. I'd rather get it right the first time.

Possible Solutions

Layer 3 to Access

I thought that perhaps I could extend L3 to the access switches, and thus break up the firewall rules into smaller segments that would then fit on the hardware limits of the access switch ACLs. But:

  • As far as I know, these would not be stateful ACLs
  • L3 to Access, to me, seems much more inflexible. Server moves or VM migrations to other cabinets would be more painful.
  • If I'm going to be managing a firewall at the top of each rack (only six of them), I probably want automation anyway. So at that point it's not much of a leap to automate management of firewalls at the host level. Thus avoiding the entire issue.

Bridged/transparent firewalls on each uplink between access/core

This would have to involve multiple firewall boxes, and significantly increase the required hardware. And could end up more expensive than buying larger core routers, even using plain old Linux boxes as firewalls.

Giant core routers

Could buy a larger device that can do the firewalling I need and has a much larger routing capacity. But really don't have the budget for it, and if I'm trying to make a device do something it's not designed for, I'll probably have to go to a much higher spec. than I would otherwise.

No Centralised firewall

Since I'm jumping through hoops, maybe this isn't worth the effort. It's always been a nice thing to have, and at times a selling point for customers who want a "hardware" firewall.

But it seems that having a centralised firewall for your "entire" network is infeasible. I'm wondering, then, how larger ISPs can offer hardware firewall solutions to customers with dedicated servers, when they have hundreds or even thousands of hosts?

Can anyone think of a way of solving this issue? Either something I've missed entirely, or a variation on one of the ideas above?

UPDATE 2014-06-16:

Based on @Ron's suggestion, I stumbled on this article which explains pretty well the issue I'm facing, and a good way to solve the problem.

Unless there's other suggestions, I would say this now is a product recommendation type of problem, so I suppose that's the end of it.

http://it20.info/2011/03/the-93-000-firewall-rules-problem-and-why-cloud-is-not-just-orchestration/

11
  • Are you using VRF-lite or MPLS in the network? What brand are the core switches?
    – Daniel Dib
    Jun 15, 2014 at 5:54
  • @DanielDib Not yet using VRFs or MPLS, but I am planning to deploy it between this site and another site. Brand is not definitive yet (still figuring out purchase list)... But right now looking mostly at Juniper EX4300-32F or Brocade ICX 6610-48-PE
    – Geekman
    Jun 15, 2014 at 6:31
  • 1
    I have voted to close; The reason is that the question you are asking delves into very specific details for your solution such as which vendor make/model to choose and budgetary constraints etc, how this will change your product offering for your customers...All that is not suitable here, those are business decisions. You can ask what are the pro's and con's of each topology, but no one can really tell you what is best for you.
    – Baldrick
    Jun 15, 2014 at 11:00
  • 1
    My two pence on your situation is; Have you considered firewalls that support contexts like Cisco ASAs, or just having virtual firewalls even? Have some VM hosts that you can spin firewalls on for each customer with two interfaces, one that you drop into the customer VLAN as the default gateway and one that you plunk into a public facing VLAN towards your edge routers. Just a thought (I prefer virtual firewalls).
    – Baldrick
    Jun 15, 2014 at 11:02
  • 2
    I would seriously look at virtualized firewalls such as Cisco ASA 1000V or Catbird (catbird.com). That way, you can place a firewall on every vserver. Keep your access lists off your core router.
    – Ron Trunk
    Jun 16, 2014 at 2:57

2 Answers 2

5

I'd pick one of two choices:

Individual per-tenant Virtual Firewalls

Pros:

  • Horizontally scalable
  • Spin-up and spin-down
  • Relatively immune to future topology/design changes
  • Complete customer separation/isolation

Cons:

  • Unless you enforce a standard template, you now have n individual firewalls to manage
  • You now have n individual firewalls to monitor
  • You now have n individual firewalls to patch

Large firewall chassis/cluster with a routing-instance/context per tenant

Deploy a large central firewall (cluster) hanging off to the side of your core, and use an internal and an external routing-instance to route traffic over and back to it (e.g.: default gateway on Internal instance is the firewall, default gateway on the firewall is your external instance on the core and the default for the external instance is your border(s).)

Pros:

  • Single box to manage and configure
  • Single box to monitor
  • Single box to patch
  • Customer separation

Cons:

  • Day one cost will be higher
  • No scaling down
  • Depending on configuration inter-customer traffic may start routing across your border routers
0

What core switches are you running? Policies are usually done at the distribution layer, if you are going with a collapsed Core design, then the core should be able to handle your requirements. Also, are you liking for statefull inspection or just acls. If you have any compliance you need to follow, acls may not be enough.

Personally, I would go with a firewall, perhaps look for one that can be clustered so you can cluster each together and maintain centrally managed rule base, such as a sourcefire firewall.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.