Methods for determining if a file is encypted?

Question

I am working on a anti-ransomware project. It simply monitors the filesystem, and watches for files being created, and deleted. (also monitors for known ransomware file extensions.)

Basic Operation: If (File X in Directory Y is created), and soon-after (File Z in Directory Y is deleted.) Scan File X and try to determine if it is an encrypted file.

I am stuck because I have no idea how to determine if a file is encrypted or not. I have read through quite a few threads on SO, but I've still turned up empty.

My question is basically: What methods can be used to determine if a file is encrypted?

Answer 1

You can use the file command if available in your environment (a standard part of BSD/OSX/Linux). For example, once encrypted content is available:

$ gpg --encrypt test-encrypted -r sample@sample.com

Enter the user ID.  End with an empty line: 
$ ls                       
test-encrypted      test-encrypted.gpg
$ mv test-encrypted.gpg testfile.txt

file can identify it as GPG Encrypted content:

$ file testfile.txt 
testfile.txt: GPG encrypted data

However, this only shows information based on the headers and trailers. A while back, someone wrote a Powershell script to find Cryptolocker files (link to PS script) which may or may not help you. The issue you will run into is: "HOW was this encrypted?" For example, if an attacker just creates a complex password for a "password protected zip" file, that isn't encryption per-se, so your script will be detected as a zip file, not an encrypted file.

Also noteworthy is, many of these ransomware files change on every iteration. E.g., locky appends a *.locky to each file, so while some may be easy to detect, imagine if all your files were renamed to random checksums. Now you cannot identify files: E.g.:

MD5 ("salesfile") = 64d11ab29c2d78b325d8a944119d1150.doc
MD5 ("payrollfile") = d9087b158cd38e844999456d17611f1c.doc
MD5 ("engineeringfile") = 14e8e9011a4d3343df39e35fc7f2cd29.doc
MD5 ("researchfile") = 26779202429523339305a90e6ec74146.doc
MD5 ("managementfile") = 419765bc586cdd1bf741afe771951bec.doc

Now you have a bigger issue. Which file is more mission critical to spend time "cracking/decrypting/etc." For the most part, the file command on Unix based systems, and or the Powershell script should give you a starting point. There is also YARA if you REALLY want to be technical about it. Create a YARA signature, and use that to search.

Answer 2

You could run ent to see how much entropy a file has, a file with high entropy is likely either compressed or encrypted (or both). A problem is that JPEG, XLSX and ZIP are compressed, so actually compressed files are very common.

If you suspect that X is an encrypted version of file Z then you could check if they have similar sizes, plus small delta for encryption headers.

Answer 3

If you have a clean system, you can use PowerShell to generate hashes for the valid files (i.e., you may have valid encrypted, compressed, binary and regular files); and each sweep you generate new hashes and compare if any hash has changed.

Based on type, etc you can determine if this is normal behavior.

For instance, I look use PowerShell Get-Process in Windows to alert if there are changes to processes, etc.

Here is demonstration of idea that can be expanded on.

PS C:\windows\system32> ## Demonstration
#  Show how you can fingerprint a server and find out if any new 
#  processes are started since boot or last baseline
#   
Get-Process * | Export-Clixml 'D:\Digital Forensics\__\baseline-local-GPROC-000.xml'
$gpbaseline000 = Import-Clixml 'D:\Digital Forensics\__\baseline-local-GPROC-000.xml'

Start-Process notepad.exe

Get-Process * | Export-Clixml 'D:\Digital Forensics\__\baseline-local-GPROC-001.xml'
$gpbaseline001 = Import-Clixml 'D:\Digital Forensics\__\baseline-local-GPROC-001.xml'

Compare-Object $gpbaseline000 $gpbaseline001 -Property processname

See image for sample output.