A company allows its developers access to snapshots of their entire codebase through a local network share. In order to perform their job, the developers compile code daily, where each compile reads a set of files from the codebase, transferring the data within from the network share to their local development machine. For convenience purposes, the company allows its developers administrator access to their local development machines, and USB port access is unrestricted.

In this environment, a malicious developer could read every file from the network share, and store those files on an external drive locally, which is (obviously) a security risk. If the company wanted to be notified of suspicious data access patterns on the network share, using only tools on the server hosting the share, how would they go about doing this?

Ideally, this system would eventually be extendable to any large set of files hosted on a network share. In one example, the server would recognize that the .h and .cpp files for a particular part of the code are frequently accessed together, and no red flags would be thrown if they're both accessed in a short interval. However, if a large number of unrelated files are accessed in a short period of time, that should throw a red flag.

  • 1
    Tool recommendations are off-topic here, I'm afraid. It sounds a bit like solving the wrong problem though - why aren't the code bases using some kind of distributed versioning system, so that permissions can be applied at that level? Tools for DVCS management are mature, and can easily ensure that developers can only access specific content, and monitor which content is accessed by which developer. Your proposed system also struggles with a long term attack, siphoning off files over a number of days/weeks/months.
    – Matthew
    Commented Jun 8, 2016 at 20:23
  • I've removed the request for tools and literature. Perhaps my examples aren't highlighting the purpose of my question properly, but the intent is to find a way to detect abnormal file access patterns on a network share, within a short time interval (24-48 hours). Its not meant to be just for code. Commented Jun 8, 2016 at 20:31
  • 1
    As a different example, and assuming I understand you correctly, you are asking for a monitoring strategy to detext broad data downloads such as the one Edward Snowden did at the NSA. That right? Commented Jun 8, 2016 at 23:23
  • Yes, thats a perfect example of a use case for this strategy. When I get home, I'll update the post to include that, thanks. Along the same line of thinking, it could also be used to detect people who disseminate journal articles from sites like JSTOR and Elsevier's archives. Commented Jun 8, 2016 at 23:24
  • The "short term" 24-48 hour limit, in hindsight, isn't particularly important. The core idea I'm attempting to flesh out is just detection of abnormal file access patterns, for some definition of abnormal. I've been thinking about it, and I believe I could accomplish this in some way by using a graph network that coalesces frequently pairwise-accessed files together into a single node, and weights edges over time based on normal file access activity. If a user hits too many infrequently pairwise-accessed nodes, a red flag is thrown. I'm going to think on it a bit more. Commented Jun 9, 2016 at 0:30

3 Answers 3


It looks like you want to consider Data Loss Prevention tools (https://en.wikipedia.org/wiki/Data_loss_prevention_software).

However before you go straight to the technology solution, you have to understand what "normal" file access looks like. If you do not know what that looks like it becomes impossible to know what abnormal looks like. This is similar to teaching IDS/IPS what the network looks like.


There are quite a few commercial eDLP products that can try to do this using machine-learning algorithms, or you could probably write something that wraps inotify and alerts on specific trends of read events (pynotify -> elasticsearch -> percolate perhaps), but given your working environment it seems to me like you're concerned with fixing a drafty window when you don't even have a roof.

Local admin and unrestricted USB access are your first two problems. You can have the most sophisticated abnormal file access pattern detector ever conceived but that doesn't change the fact that you've lost all visibility of what happens to acceptably-accessed data the second it goes onto someone's USB stick.

All developers having access to the entire codebase is your third problem. Presumably you're not dealing with PII so simply accessing files, regardless of velocity, is not inherently suspicious. IME I've noticed people tend to exfil in one of three ways--

  • "I know I'm going to quit/be fired, so I'm going to take EVERYTHING"
  • "I am steadily employed but not paid enough so I am going to exfiltrate and sell (data asset) piece by piece over time."
  • "I am steadily employed and an idiot, so I'm going to take EVERYTHING and resell it"

The second case is the bigger pain. This person will collect data little by little over time before busting out (#1).

The real threat to you is that you have no way of knowing what they're doing with these files after they've accessed them, abnormally or not. It doesn't sound like there's anything stopping them from doing a git push of everything they're already authorized to have. I suggest you address that first.

In the meantime, depending on the size of your shop you will probably experience a quicker return on investment correlating high numbers of reads following notification of employee resignations. We see that all the time.


Yes, I think this is entirely possible. In order to define what normal or typical file access behaviour looks like you would need to spend some time recording the activity of each individual.

An analysis of these data would allow you to define normal access parameters. This would be things like the number of file requests, the duration between requests (to differentiate between manual and automated access requests), the previous pairwise frequency with which files are accessed. You could, as you suggested, build a network or graph to represent groups of files that are commonly accessed together. Various parameters that characterise the network such as the shortest path distances between accessed files, or some sort of `cost' function defined by summing over weighted edges, could then be computed for individual actions.

In short, you would need to establish typical usage profiles in terms the above parameters. Individual actions can then be characterised by the same parameters and compared to the typical profiles in order to detect aberrant behaviour or "outliers".

The challenges are to design a system that will collect file access behaviour, establish parameters or statistical patterns that can characterise normal usage, and then implement a real-time monitoring system that will generate an alert when atypical access patterns are detected. Someone copying whole swathes of files onto an external drive should be quite easy to catch.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .