I have been digging the ndpi codebase and trying to understand how its actually detecting the HTTP protocol.

What I found so far is that they are searching the request type in the payload and if not found they perform some checks to categorize it as a HTTP response, but is this enough to categorize a packet? Someone can obviously tangle with the request data and bypass those string checks.

Same thing I saw in the SSDP protocol (just string checks).

What is the high level view of categorizing the packet as HTTP (or any "x" protocol)?


What is the high level view of categorizing the packet as HTTP (or any "x" protocol)?

Standard port numbers are as close as you get to an an official classification of application-layer protocols. But in practice, services often don't adhere to their default ports and there is simply no universal way to detect which layer-7 protocol is being used. Therefore, deep packet inspection relies in large part on regular expression heuristics or just basic string comparison (as in the referenced code from nDPI).

In most cases there is just no better way of detecting a protocol than by looking for typical patterns in the (preferably initial) packets - like a GET request line with subsequent headers to identify an HTTP connection. Consequently, you're right that it's easy to evade these checks by obscuring the protocol (often done by BitTorrent applications). The discontinued Linux Layer 7 filter project has a FAQ that addresses these concerns:

Q: Isn't this just another pointless step in an arms race in which protocols will continually adapt to evade classification?

A: You can look at it like that, but essentially no. Internet standards like HTTP will not do this. Only P2P programs and the like will. This means that, at worst, what you will need to do is to shape the "unknown" classification down, leaving the most bandwidth for known protocols like HTTP, SMTP, and so on.

Also have a look at the l7protocols wiki which documents various methods of protocol identification, most of them regex-based.


To add to the answer from Arminius:
There a different use cases for DPI solutions:

  • One use case is just to quickly classify traffic to optimize network latency for important traffic, like prefer VoIP traffic and slow down BitTorrent. In this use case it is important to use fast heuristics which don't need to be that strict because in the worst case the wrong traffic gets faster or slowed down.
  • Another use case is security. Here it is important to reliably detect the protocol because one needs to decide if the traffic should be passed through without further analysis, should be blocked or inspected in more detail. Although heuristics like in nDPI are used in these cases too they can often be bypassed. A better but usually much slower alternative would be a more detailed protocol analysis and protocol enforcement instead, as done for example within HTTP proxies.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.