5

We're about to purchase a pair of Cisco 7201s for our edge routers, and some Juniper EX4200s for core. And I'm looking for implement bandwidth accounting for our hosted servers on this hardware (most likely the 7201s).

I understand that RADIUS is typically used for bandwidth accounting for tails, but what's typically used to track data use of customer servers, in terms of GB / month? Currently each customer is assigned their own VLAN for all servers, so for billing purposes per VLAN counters would suffice. Though, for informational purposes, would probably want per-IP counters so that we can show BW per server.

I'm coming from an environment where we currently use a Linux router on a stick. Right now, I've configured IPTables to store counters based on src/dst IP addresses per customer, and put together a perl script to collect those counters every 5 minutes. The data are then parsed and then stored in a database. This works well, and my tests have shown it to be accurate.

Currently our edge transit is < 50 Mbps, but I'm looking to scale to around 300 Mbps, so I'd want to ensure I can implement this without overloading the routers (as we'll using them for a fair bit - LNS, MPLS, Static NAT, 10~ Mbps AES IPSec).

Should I be looking at NetFlow to accomplish, or perhaps port mirroring, or something else entirely?

  • Hi @Geekman, Questions like "what is the best way to..." are off topic here, because they are just based on people's opinion. The moderators here like cold, hard facts ;-). But IMO, Netflow is your best bet, as it can give you lots of detail for your reporting, and most collectors have robust reporting features. – Ron Trunk Mar 4 '14 at 1:06
  • Geekman, this looks like a reasonable question; as long as we stay away from asking for specific product recommendations we'll be fine. If you have a traffic accounting system that works well, and is already integrated with your billing systems I would be quite scared to change that. Your best bet may well be a span port and continue using the "old way" – Mike Pennington Mar 4 '14 at 1:16
  • @Ron I guess I can kind of see where you're coming from, as it's not implemented yet, it's in some senses an abstract question, and is susceptible to a lot of "it depends" kind of answers. That being said, I feel like I've specified enough parameters to give a reasonable answer, and would be happy to give more. After all - we DO have a solution implemented - just trying to implement that in new HW. – Geekman Mar 4 '14 at 1:31
  • @Mike Yes, I definitely see where you're coming from here, though to be honest, it's not a critical system as of yet (we don't bill for overages, but it's nice to keep an eye on). My main concern with our current method is that, I know it's not likely to scale once we start pushing more traffic, and I figure while we're upgrading everything else, now is the time to look at other options that might scale better. – Geekman Mar 4 '14 at 1:33
  • "Currently each customer is assigned their own VLAN for all servers, so for billing purposes per VLAN counters would suffice". The way to scale your current system is to dedicate a specific accounting server to a set of customer Vlans. The challenge with something new is that you're now hoping that you don't run into integration issues, accuracy issues, or other bugs. If you're ready to dive into the unknown, that is a useful data point. What flavor of NPE are you going to use on the 7201s? What IOS (feature set & version)? – Mike Pennington Mar 4 '14 at 1:37
5

Should I be looking at NetFlow to accomplish, or perhaps port mirroring, or something else entirely?

NetFlow is certainly the most common for this type of bandwidth accounting. That’s exactly what it was designed to do. It will give you an accurate view on what an interface is doing, which is why it’s most ideal at the boundary.

Port-mirroring isn’t something I would use for customer traffic accounting. I implemented something with port-mirroring a while back because we removed our InfiniStreams and wanted to retain that functionality, and I found that it’s only good if you want to do full packet capture (FPC) just to have it in case you’re having issues external to your site. If you intend on scaling to 300Mbps, you’ll need a pretty elegant setup that’s capable of sustaining writes up to 300Mbps, sufficient storage, and enough CPU/RAM that can make sense of all of that data.

If you absolutely must go with a SPAN style solution, I would, at a minimum, just log the first few bytes of each packet to keep server utilization down. If you don't need FPC, then don't implement FPC.

My setup has been turned into a ‘Forensics Tractability Server’, or more realistically, a traffic dump location in case someone with malicious intent decides to break in. If the Feds come asking "who/what/where/when/how/why", I’ll be in a position to answer (...sort of).

As for analytics, just get hooked up with some open-source tools that can visualize NetFlow data. Ntop/ntopng1 is the first thing that comes to mind as an all-in-one solution, but there are tons out there. Wikipedia has a small listing of some of the bigger names (most free/open-source).

I'm coming from an environment where we currently use a Linux router on a stick. Right now, I've configured IPTables to store counters based on src/dst IP addresses per customer, and put together a perl script to collect those counters every 5 minutes. The data are then parsed and then stored in a database.

Your Linux-foo is commendable. Unfortunately, this isn’t the most ideal solution, as you can probably tell, and pigeon-holes you into continuously developing your own applications. You’re obviously on the right track; you’ve come here to figure out a more scalable solution.


1Although ntop is still available in every Linux repo, development has ceased and continued with ntopng.

| improve this answer | |
  • Thanks! I really can't up-vote this enough. You've answered my question, and more. Thanks for the footnote about ntop as well - it's something I used years ago - and I'd like to think I'd do decent research before just defaulting to ntop, but it's good to have the pointer. – Geekman Mar 4 '14 at 9:51
  • @Geekman I do like what the developers have done with ntopng, but they have added additional complexity into the setup; which is why I think there is still such a big ntop presence out there. – Ryan Foley Mar 4 '14 at 10:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.