Network Engineering Stack Exchange is a question and answer site for network engineers. It only takes a minute to sign up.
Sign up to join this community
i have configured netflow in my router but my router is overheaded. So i decided to configure sampled netflow and sampling rate of 1 out of 100 packets may reduce the export if Netflow data by as musch as 50 percent.So you know its not suitable for security monitoring.
I want to know whats background in "multiply by the sampling rate to calculate actual traffic stats"
Whats your recommendation ? is there any solution on collector?
Typically, the approach is indeed just "multiply the observed octet/packet sample counts by 100" (or 1000, or whatever) and that does pretty well for getting overall traffic volumes roughly correct. It gets packet count almost exactly right, because it's sampling every 100 packets. It gets octet count more or less right because ordinarily the bytes-per-packet ratio is pretty steady.
For individual sessions, however, accuracy varies a lot: long-running high volume sessions like streaming video or large downloads? Does pretty well for the same reasons as above. Very short sessions like ICMP or DNS? It can overestimate them by quite a lot, and many of them are missed entirely.
The main issues that have been identified with sampled flow are:
Of those, the only thing that really affects DDoS detection in most environments is the last one, since it can incur a delay. Sampled flow is just fine for determining "am I under attack" when that attack is a large portion of your traffic because it does get the overall volume roughly right even if individual flows are lost entirely. If the attack on a given host is not a large portion of your overall traffic, though, you'll run into both delay and accuracy issues. And if you're looking to identify sources, you're going to have a still harder time of it. (But since so many of these attacks use spoofed sources, I doubt that's an issue for you.) Also, if you're using this NetFlow stream for anything else, you'll need to be aware that the accuracy of the "civilian" traffic counts will be reduced (appear to be smaller than it is) during an attack, as a side effect of all the one-packet flows.
The other thing that sampling affects is the session count itself, which is nearly always undercounted, and can affect the ability to detect low-volume/high-session-count attacks on the protocol. I can't answer for SiLK, since I don't know it as well and can't find it in the documentation, but I do know that FlowTraq (with which I'm far more familiar) uses some a priori knowledge of the effect of sampling on session count to reconstruct.
Most collectors will do the statistical reconstruction, with the caveat that your router has to actually provide the sampling rate (make sure your router follows the spec for reporting sampling rate and method). Again, I'm more familiar with FlowTraq than with SiLK, but I know SiLK is at least aware of sampling rate. What I don't know is how SiLK handles it, and skimming their documentation didn't enlighten me. I would recommend doing a large file transfer of known size and looking at the flow records SiLK stores: if it has the order of magnitude right, then it's reconstructing. If it's not reconstructing (or your router isn't providing the rate), then in a pinch, you can always divide your detection thresholds by 100 or 1000.
(If you're interested in getting into the mathematics of sampled NetFlow, by the way, Hamed Haddadi has a couple papers on the topic)
Most of the DDOS appliance can get input of sampling rate and multiply the values and should be able to detect volume attacks easily. So you don't need to worry about sampling. Most of the ISPs/DDOS protection providers uses sampled netflow for DDOS detection purpose. 1:100 is ok to have on DDOS perspective. It depends on what kind of collector you use.
