Popularity of Anomaly based IDSes

Question

I've been reading quite some time about anomaly/behavior based IDSes for Web servers.

I understand the downsides: possible high false positives, learning phase, constant training.

My question is how popular are these systems for Web servers? It looks like most of the IDSes are signature based (most popular one that comes to my mind is mod_security).

I know this is a very broad question, I guess I am more interested in if people use them or if these kind of IDSes are still at more theoretical/academic level?

Answer 1

We use an anomaly based detection engine. There is definitely a high false positive rate and the learning phase can take up a lot of time. In my experience, an IDS that is OS and application aware is still a better option. While we have found some interesting things, it is not useful unless you have the staff to maintain it and sift through all of the false positives. I would say they are coming along but are not for everyone.

Answer 2

The problem will usually be defining "normal" traffic against which you can detect the anomalies. From past research, ensemble classifiers seem to be most efficient in lowering your false positives. I myself worked researching a lot of these algorithms and it's still under heavy research. Another area is Artificial Immune Systems in a combination of IDS and Incident Response technology.

Check out multiple classifiers in use: http://roberto.perdisci.com/projects/mcpad Here's a paper on Artifical Immune Systems: http://www.cs.unm.edu/~forrest/publications/hofmeyr_forrest.pdf

How good is the technology behind the fraud detection tools that the financial sector deploy in terms of anomaly detection?

Answer 3

The financial sector has started to adopt the use of web application firewalls (WAF). For this type of environment you will find the use of more commercially footed products such as Imperva (see http://www.imperva.com/products/wsc_web-application-firewall.html) Baracuda (see http://www.barracudanetworks.com/ns/products/web-site-firewall-overview.php?gclid=CJTV3_yszqUCFcxO4Qod9TzUkA) and even Cisco (http://www.cisco.com/en/US/products/ps9586/index.html). My experience is with Imperva, so I can not speak about other products.

However I found that while it is easy to sell the funding of a project to senior management to purchase nice and cool new security devices that are going to totally protect your applications (debate!), the problem comes when that funding has run out before the learning phase has been completed. This is even more of a problem when the applications the WAF should be protecting are based on hugely complex business logic.

When this happens the result is a WAF that is left in monitor mode only, without the real time protection that the device was supposed to be providing.

On a positive note (and from experience!!) even WAF's in monitor mode can be vital during an incident response - even if this is being a source of logging that captures the entire attack in csv format. ;-)