Applications of probability/statistics in IT security?

Question

What are some areas of computer security that utilizes statistical modeling? I ask b/c I'm a studying statistics at school and have become interested after reading a few papers.. some of the areas I'm aware of are anomaly in network traffic, IDS's, spam filtering (though not strictly security)... one of the neatest ones I've found is of inferring keystrokes in SSH traffic by building a time series model of typing. What else is out there?

Answer 1

Several areas use statistical means and probability theory.

• Intrusion Detection was mentioned already
• You mention SSH fingerprinting, which concerns the very large field of traffic analysis. There was a paper at Oakland Security & Privacy this year where they use this to decrypt encrypted VoIP.
• Another aspect is side-channel analysis, where you probe a system and measure timing etc to infer secrets based on statistical patterns. You can attack smartcards, remote authentication services or the encryption running on a VM next to yours in the cloud, inferring the secret key from cache timing.
• Probability theory is also used in reputation systems, e.g. p2p sharing systems or when designing a reputation-based routing protocol for sensor networks. You assign trust as a probability/estimate that another device will behave as expected and modify that value based on different classes of input, like recommendations or observed actions.
• There is also the (minor) field of developing metrics for security. The metricon conference papers can give you an impression on that area. They measure the security procedures or specific programs using statistical analysis and probability theory to estimate future results.

Addendum: If you are asking about potential future fields of work in security, but have no background in security, I would recommend the field of trust management and reputation systems. It is easy to get into this rather active topic and many existing works are rather adhoc. Regarding the other options...traffic analysis is very old. Some great attacks are possible but there is not much room for substantial research. It is similar for side-channel attacks, except in the area of hardware attacks. You can find research positions and industry jobs there, but you also need strong skills in electrical engineering. For Crypto, you need very good math skills and there are few positions outside of research. IDS research is mostly dead, but the IDS industry is thriving (like antivirus). Not sure if they need dedicated developers for their statistical evaluations, though, I think they'd rather take people with strong experience in network and system security.

Answer 2

Intrusion Detection Systems indeed are a natural fit for statistics.

An interesting example I've come across:

At host based IDSes, track and statistically analyze the system calls of an application. An interesting approach is to group the system calls in pairs, triples etc. and then observe their behavior when the application encounters an attack.

Have a look at this:

P. Astithas, V. Pappas, B. Maglaris, "Detecting Intrusions by Monitoring System Processes", in Proceedings of the 8th HPOVUA Plenary Workshop on Network and Systems Management, Berlin, Germany, June 2001. (it's a postscript file)

You may also find interesting analysis work done with high speed networks where specialized network processors capture enormous amounts of traffic. Theoretically such analysis can reveal ongoing DDoS attacks and be used to deploy dynamic filtering.

I would also suggest to take a look for interesting ideas at the proceedings of RAID, the International Symposium on Recent Advances in Intrusion Detection. From what I have seen much of it is available online with free access

Answer 3

In addition to the things others have mentioned, one other area of some importance to security (in terms of statistics) is general probabilities. For instance, calculating the number of possible passwords of a certain length to create a 'key-size' that would need to be iterated through in a brute-force attack. You can also model the likelihood of people to choose certain passwords over others and thus make your 'brute forcing' method more efficient than simple iteration. I.E. the likelihood that people use a password that is a proper noun (or just english word in general) and then possibly followed by some numbers, over a truly 'random' password.

Answer 4

Crypthography is in need of lot of statistics. From midlle-age (and before?) cipher has been broken by statistics means.

Now, their is needs in pseudo-random number generator that should fit statistical distribution. hashing function that need to comply to some rules.

Also the steganography analysis uses statistical tools to built oracle to determine if some content holds secret information or not.

Well anormal behaviour detection is very nice topic also, not only for Intrusion dectection, but also for theft detection, terrorism mitigation and so on.

Answer 5

Check out Statistical Techniques for Network Security: Modern Statistically-Based Intrusion Detection and Protection
http://www.amazon.com/gp/product/159904708X/ref=wms_ohs_product

Anomaly detection is a hard problem given that algorithms are often combined in different ways to detect different types of anomalies. Massive streams of comingled and fragmented data (e.g. xflows, IDS alerts, HIDS alerts, modsec alerts) means huge numbers of permutations and huge opportunity.

You may think about turning pro with machine learning skills:

From wikipedia:

"Machine learning, a branch of artificial intelligence, is a scientific discipline concerned with the design and development of algorithms that allow computers to evolve behaviors based on empirical data"

Lots of security companies attempt to mix and match machine learning techniques to better their products.

Doing an Amazon search reveals additional books, for example:
Machine Learning and Data Mining for Computer Security: Methods and Applications
http://www.amazon.com/Machine-Learning-Mining-Computer-Security/dp/184628029X

In addition there are literally hundreds of research papers available on statistics/probability/machine learning as applied to security.

Answer 6

A few have already been mentioned:

• IDS
• Cryptanalysis
• Cryptography (2 different angles)
• Password strength and other metrics

Some that haven't been mentioned yet:

• SIEM/SOC - analyzing and corrolating logs and alerts
• WAF (web application firewall) - since configuring these systems are a real beast, most modern commercial WAFs have incorporated automatic learning mode, based on behavioral analysis
• Behavioral biometrics - e.g. your gait, your speech patterns, your motions, how you type... and, to a lesser extent, other "regular" biometrics, too.
• "risk-based" authentication systems, such as e.g. this one.
• Fraud analysis - this is a big one.