An IDS detects suspicious activities via patterns in the analyzed traffic. Most of the traffic I deal with is encrypted (either HTTPS or proprietary, like SMB). It cannot be decrypted for analysis for legal reasons.

I am looking for examples of environments where a network-based IDS can still be useful when most of the traffic is encrypted?


First, IDS technology is not necessarily reduced to simple pattern matching but IDS can often also parse HTTP traffic including decompression of compressed payloads, extract attachments from mails etc and look for pattern there. Also, IDS like Suricata can be extended beyond simple string matching with scripting languages like LUA.

And some modern IDS are also able to analyze the TLS handshake which contains valuable information, like the SNI extension which contains the target hostname or the certificate returned from the server. Both of these information are not encrypted. And sometimes malware can be detected because they use TLS ciphers or TLS extensions in a special way different from other clients. See Hiding in Plain Sight: Malware’s Use of TLS and Encryption for some research in this area.
Apart from that IDS might be used together with an interception proxy or similar which feeds the decrypted traffic to the IDS for deeper analysis.

  • Would a host-based IDS be useful in this case because the traffic is only encrypted until it reaches the server where it is then decrypted, allowing for a HIDS to see the traffic decrypted? – Ryan Kelso Mar 13 '17 at 17:40
  • 1
    @RyanKelso: a HIDS is usually a different technology than a NIDS. While the last one watches the network traffic a HIDS monitors the local system, i.e. file operations etc. Insofar a HIDS can see things a NIDS can't but also a NIDS sees things a HIDS does not. – Steffen Ullrich Mar 13 '17 at 17:57

Regarding SSL: SSL Termination may happen before the IDS looks at the traffic. Just because a webserver is capable of handling SSL/TLS doesn't mean that it has to be done at the webserver. There might be a machine at your network border which encrypts and decrypts SSL traffic.

Apart from that, an IDS isn't limited to individual network streams. It might record all traffic and build statistics about what is considered normal. Then if traffic flows suddenly change, for example if the distribution of source adresses changes significantly, or traffic volume massively increases, or there is suddenly traffic between machines that never communicated before, or any other pattern you can think of changes, it might alert humans. This can all be done without looking at the traffic payloads; looking at the TCP/IP header and the traffic volume and timing is enough for this kind of analysis.

  • As for SSL: as I mentioned, decryption and reencryption is not an option. The other possibilities (stats) can be handled by a firewall and/or a web proxy. – WoJ Mar 13 '17 at 18:13
  • Usually firewalls and proxies don't handle these kinds of analytics. An NIDS that does this also wouldn't just look at web traffic. But an IDS that did this kind of statistical network traffic analysis would need to have at least a monitor satellite placed at a traffic concentration point such as a firewall. Just one sampling point might not be enough, though, depending on the size and structure of the network in question. – Out of Band Mar 13 '17 at 18:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.