I am currently working on a DDoS detection problem from Netflow data from an ISP’s perspective. Very large datasets are available to me for analysis. Though I have done considerable amount of work, still I am not yet convinced about the selection of the right set of features that will eventually help us in identifying the DDoS attack(s). Any suggestions in this regard will be very helpful.
It all depends what's your desired level of complexity. I wrote a tool like that some time ago and I ended up simplifying it a lot at the end - initial idea would result in a lot of coding. So what I ended up with:
- list of protocol & port combinations that I consider frequent offenders (ie. udp/123, udp/80, udp/19, udp/161, udp/53 etc)
- assumption that genuine use of above protocols should "never" cause more then xyz packet-per-second towards single destination host
Now you need something [*] that will "listen" to netflow and collect data into some sort of buckets. Since I use sampled netflow I made a rule that threshold for given IP need to be exceeded twice within 30s - this helps ruling out false positives.
Above is obviously good for amplification attacks, I applied similar logic to tcp/80 & tcp/443 with SYN flag set, also works relatively well.
[*] My "something" is: pmacct (www.pmacct.net) + RabbitMQ + my own bucket holding daemon written in Python. When alert is triggered, operator can instruct tool to take action (blackhole, flowspec or off-ramp to commercial scrubber), which is fed back to BGP using ExaBGP.
Hope this helps.