How can one perform quality of service based on layer 7 traffic? How would you classify the traffic and actively monitor and respond (dynamically prioritizing traffic)? Are there any solutions which provide a kind of weather map?
I think this depends heavily on the platform on which you want to accomplish this. For example, IOS uses NBAR to classify transit traffic, primarily for QoS and security mechanisms. But as far as I know, NBAR QoS configurations are entirely static.
For centralized monitoring, NetFlow is probably your best bet.
There are various tools dependant on platform. For example:
- Cisco - NBAR
- SonicWALL - AppFlow
- Juniper - No specific option in JunOS, though can roughly be achieved using class maps.
For monitoring of traffic Netflow & NFSEN is your friend. The bundle simplifies deployment of the NFDUMP capture, dump and analysis tools and gives you a nice frontend to view it all in and make graphs for management.
You need a proper application aware appliance. Exinda, riverbed, etc. Cisco and Juniper et al (yes I include WAAS) is a crowbar solution. NBAR is nowhere as accurate esp. with the trickier protcols that use random/dynamic ports. For enterprise use NBAR is fine as you can control the environment but when dealing with the internet you are going to be dealing with a lot of clever tricky protocols and varied situations.
For example the app aware stuff can be tied into dynamic white/blacklists and offer far more granular behaviour e.g. for http, allow burst for first 100M, if the rate is above X (set it above HD youtube lol) then start rate limiting after the first 100M if priority traffic is getting contended. This kind of behaviour gives your users excellent experience and appears transparent to them whilst hitting people downloading large files via http/https like filelockers for example. And critically they deal with a lot more protocols, try getting nbar to spot chinese p2p video streaming like pplive or pps for example.
For basic enterprise use yes you'd rely on nbar/static port definitions then use qos classifications to queue appropriately. Works fine in that scenario.
If you run linux you could try l7-filter, which is a iptables/netfilter-module. You can then use the usual iptables-magic to do some QOS. Weathermaps could be created with collectd (writing to RRDs) and then reading from these with http://www.network-weathermap.com or http://weathermap4rrd.tropicalex.net/
I have no tool ready that allows monitoring and dynamic prioritisation — you might have to invest in some soft of tailored linux-distribution or a hardware-firewall of some sort.
NBAR is the Cisco feature that is capable of classifying traffic at layer-7.
This feature allows the use of the 'match protocol ...' command within your class-map so that you can then take action, such as marking the DSCP value or policing, the matching traffic.
NBAR uses something called a PDLM (Protocol Description Language Module) which is basically the logic for determining is traffic is a match. For custom applications you need to write your own PDLM. I have never done this so I cannot comment on how easy or hard this is. I have personally found the IP addresses and/or ports have worked well for classifying my traffic into my supported traffic categories.
As far as monitoring goes I would agree with Jeremy that NetFlow is the best feature to use. There are various free and paid tools that can collect and report this data (the router pushes this data to your management station). Cacti (free) may have support for the "Weather Map" report that you are looking for.
Also for monitoring you may want to look into a reporting tool that uses SNMP to gather the usage and drops from your traffic classes. It would be advisable that you configure your device with 'snmp mib persist cbqos' if you go that route (this keeps your ifIndex values static across reboots). Again there are many tool options and Cacti is a good place to start.
This is way too broad, but here is an effort to answer what I think you are asking. What follows is more information concerning Jeremy's answer above.
I assume you are asking about prioritizing traffic based on specific aspects of the application layer control protocols (btw, I use TCP/IP terminology rather than OSI ones when discussing TCP/IP because IMO the OSI model does not map to TCP/IP as well as one might think --- for example try to understand how H.323 fits on the model, both as originally designed and when over TCP/IP).
As a simple example, maybe you want to prioritize TFTP requests for PXE booting over other TFTP requests, or perhaps you want to prioritize H.323 signalling of some types over others.
To do this you need either routers or the like which can understand the protocols you wish to shape traffic based on. These routers then need to be able to inspect the packets deeply at least when required, and then shape the traffic based on what they find. Needless to say different application protocols will have different requirements and possibilities.
For this reason, it depends on the platform, the protocol, and what you want to accomplish. As others have noted various vendors have toolkits for this, but to really answer what is involved, we need a lot more information about what traffic you have, what problem you are solving, and what tools you have available to you already.
If you have a Cisco device use a combination of NBAR and QoS.
NBAR will help classify traffic based on your custom requirements. You can create custom NBAR PDLMs for the protocol/application you need to track. The command is: ip nbar custom name [offset [format value]] [variable field-name field-length] [source | destination] [tcp | udp ] [range start end | port-number ]
Once this is done create a QoS class for the custom NBAR app and then apply your policies.