2

i have configured netflow in my router but my router is overheaded. So i decided to configure sampled netflow and sampling rate of 1 out of 100 packets may reduce the export if Netflow data by as musch as 50 percent.So you know its not suitable for security monitoring.

I want to know whats background in "multiply by the sampling rate to calculate actual traffic stats"

Whats your recommendation ? is there any solution on collector?

3
  • 1
    You should add information on what you consider to be 'security monitoring'. What information do you need from netflow? For most purposes, sampling rates of 1:100 or even 1:1000 are workable, but it all depends on what you're trying to achieve.
    – Teun Vink
    Aug 22, 2016 at 7:36
  • I need to detect DDOS attack. But when we sampling, its not possible/hard to decect,right ?
    – user30276
    Aug 22, 2016 at 7:42
  • Did any answer help you? if so, you should accept the answer so that the question doesn't keep popping up forever, looking for an answer. Alternatively, you could provide and accept your own answer.
    – Ron Maupin
    Aug 11, 2017 at 0:21

2 Answers 2

4

Typically, the approach is indeed just "multiply the observed octet/packet sample counts by 100" (or 1000, or whatever) and that does pretty well for getting overall traffic volumes roughly correct. It gets packet count almost exactly right, because it's sampling every 100 packets. It gets octet count more or less right because ordinarily the bytes-per-packet ratio is pretty steady.

For individual sessions, however, accuracy varies a lot: long-running high volume sessions like streaming video or large downloads? Does pretty well for the same reasons as above. Very short sessions like ICMP or DNS? It can overestimate them by quite a lot, and many of them are missed entirely.

The main issues that have been identified with sampled flow are:

  • Short flows get individually lost
  • Large flows can no longer be accurately ranked by volume
  • Flows get created less often

Of those, the only thing that really affects DDoS detection in most environments is the last one, since it can incur a delay. Sampled flow is just fine for determining "am I under attack" when that attack is a large portion of your traffic because it does get the overall volume roughly right even if individual flows are lost entirely. If the attack on a given host is not a large portion of your overall traffic, though, you'll run into both delay and accuracy issues. And if you're looking to identify sources, you're going to have a still harder time of it. (But since so many of these attacks use spoofed sources, I doubt that's an issue for you.) Also, if you're using this NetFlow stream for anything else, you'll need to be aware that the accuracy of the "civilian" traffic counts will be reduced (appear to be smaller than it is) during an attack, as a side effect of all the one-packet flows.

The other thing that sampling affects is the session count itself, which is nearly always undercounted, and can affect the ability to detect low-volume/high-session-count attacks on the protocol. I can't answer for SiLK, since I don't know it as well and can't find it in the documentation, but I do know that FlowTraq (with which I'm far more familiar) uses some a priori knowledge of the effect of sampling on session count to reconstruct.

Most collectors will do the statistical reconstruction, with the caveat that your router has to actually provide the sampling rate (make sure your router follows the spec for reporting sampling rate and method). Again, I'm more familiar with FlowTraq than with SiLK, but I know SiLK is at least aware of sampling rate. What I don't know is how SiLK handles it, and skimming their documentation didn't enlighten me. I would recommend doing a large file transfer of known size and looking at the flow records SiLK stores: if it has the order of magnitude right, then it's reconstructing. If it's not reconstructing (or your router isn't providing the rate), then in a pinch, you can always divide your detection thresholds by 100 or 1000.

(If you're interested in getting into the mathematics of sampled NetFlow, by the way, Hamed Haddadi has a couple papers on the topic)

2

Most of the DDOS appliance can get input of sampling rate and multiply the values and should be able to detect volume attacks easily. So you don't need to worry about sampling. Most of the ISPs/DDOS protection providers uses sampled netflow for DDOS detection purpose. 1:100 is ok to have on DDOS perspective. It depends on what kind of collector you use.

3
  • I am not sure about the sample rate setting in SiLk readily. I can check and get back
    – user88975
    Aug 22, 2016 at 8:09
  • and also I want to know whats background in "multiply by the sampling rate to calculate actual traffic stats"
    – user30276
    Aug 22, 2016 at 8:10
  • In the sensor.conf file there is way to specify the parsing of sampling rate atleast for V9 option templates. So it might be possible to multiply it to the sampling rate configured. As said by John, there will be some accuracy issues, but still it works 99% of the time to detect volumetric attacks.
    – user88975
    Aug 23, 2016 at 16:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.