Mechanisn for performing bandwidth accounting on Cisco routers

Question

We're about to purchase a pair of Cisco 7201s for our edge routers, and some Juniper EX4200s for core. And I'm looking for implement bandwidth accounting for our hosted servers on this hardware (most likely the 7201s).

I understand that RADIUS is typically used for bandwidth accounting for tails, but what's typically used to track data use of customer servers, in terms of GB / month? Currently each customer is assigned their own VLAN for all servers, so for billing purposes per VLAN counters would suffice. Though, for informational purposes, would probably want per-IP counters so that we can show BW per server.

I'm coming from an environment where we currently use a Linux router on a stick. Right now, I've configured IPTables to store counters based on src/dst IP addresses per customer, and put together a perl script to collect those counters every 5 minutes. The data are then parsed and then stored in a database. This works well, and my tests have shown it to be accurate.

Currently our edge transit is < 50 Mbps, but I'm looking to scale to around 300 Mbps, so I'd want to ensure I can implement this without overloading the routers (as we'll using them for a fair bit - LNS, MPLS, Static NAT, 10~ Mbps AES IPSec).

Should I be looking at NetFlow to accomplish, or perhaps port mirroring, or something else entirely?

Answer 1

Should I be looking at NetFlow to accomplish, or perhaps port mirroring, or something else entirely?

NetFlow is certainly the most common for this type of bandwidth accounting. That’s exactly what it was designed to do. It will give you an accurate view on what an interface is doing, which is why it’s most ideal at the boundary.

Port-mirroring isn’t something I would use for customer traffic accounting. I implemented something with port-mirroring a while back because we removed our InfiniStreams and wanted to retain that functionality, and I found that it’s only good if you want to do full packet capture (FPC) just to have it in case you’re having issues external to your site. If you intend on scaling to 300Mbps, you’ll need a pretty elegant setup that’s capable of sustaining writes up to 300Mbps, sufficient storage, and enough CPU/RAM that can make sense of all of that data.

If you absolutely must go with a SPAN style solution, I would, at a minimum, just log the first few bytes of each packet to keep server utilization down. If you don't need FPC, then don't implement FPC.

My setup has been turned into a ‘Forensics Tractability Server’, or more realistically, a traffic dump location in case someone with malicious intent decides to break in. If the Feds come asking "who/what/where/when/how/why", I’ll be in a position to answer (...sort of).

As for analytics, just get hooked up with some open-source tools that can visualize NetFlow data. Ntop/ntopng¹ is the first thing that comes to mind as an all-in-one solution, but there are tons out there. Wikipedia has a small listing of some of the bigger names (most free/open-source).

I'm coming from an environment where we currently use a Linux router on a stick. Right now, I've configured IPTables to store counters based on src/dst IP addresses per customer, and put together a perl script to collect those counters every 5 minutes. The data are then parsed and then stored in a database.

Your Linux-foo is commendable. Unfortunately, this isn’t the most ideal solution, as you can probably tell, and pigeon-holes you into continuously developing your own applications. You’re obviously on the right track; you’ve come here to figure out a more scalable solution.

---

¹_{Although ntop is still available in every Linux repo, development has ceased and continued with ntopng.}