I have been working on testing whether same encrypted files are same without decrypting it. I have a test file provided to me one with normal data and another encrypted version of it. I need to check whether that test file(without encryption) which goes through an api and encrypts it will be the same as encrypted file that I got. Here, is the python script for checking these files:

def compare_binaries(self,f1, f2):
    Using ittertools zip_longest to check each lines of both files
    f1 - binary file of source file
    f2- binary file of target file

    Returns True if all lines are same else False if any line is not same
    for line1, line2 in zip_longest(f1, f2, fillvalue=None):
        if line1 == line2:
            return False
    return True 

Is this process okay for checking the encrypted files are same or are there better alternatives for it?

  • 2
    Well this question is more suitable for programing than for security, did you check in stackoverflow? by the way you should check the hashes of the files.
    – camp0
    Mar 15, 2021 at 11:59
  • 9
    How the files are encrypted? What is the encryption method? What is your aim?
    – kelalaka
    Mar 15, 2021 at 17:30
  • 18
    Now wait, you're talking about "binaries" but then reading the files line by line? That's typically not the best way to do it.
    – TooTea
    Mar 15, 2021 at 20:21
  • 3
    @TooTea: It's definitely not the best way to do it in Python. It will normalize '\n', '\r', and '\r\n' to just '\n'. Depending on your threat model, this may be an exploitable loophole.
    – MSalters
    Mar 16, 2021 at 10:34
  • 3
    If it doesn't have to be Python, unix cmp compares binary files for exact byte-for-byte equality without any shenanigans like "lines". Mar 16, 2021 at 12:17

6 Answers 6


It depends on how the API encrypts the file, and there are a LOT of possible variants here: symmetric (block cipher and stream cipher), or asymmetric, and different implementations of each one.

Some encryption modes employ something called random IV, and in practice it means that encrypting the same file multiple times with the same key will output different files each time. So even if the API is rightfully encrypting the file, the result from the API and the file you just encrypted will be different.

You will have to check what encryption mode the API is using. AES with a deterministic IV would work (like using the truncated hash of the file as the IV, for example). The vital part is to never encrypt two different files with the same IV, or it could leak information on the key.

Only if the API returns the same output every time you submit the same file, then it is possible to compare the hashes of the files.

  • 1
    Yes the API is returning same output every time I checked opening file in binary format. The issue is this will take a long time for large files so I am thinking are there other ways for it. Seems like hash checks should be fine for my case but I am having problems on it. Mar 16, 2021 at 8:41
  • I looked through code for hash checks below and it seems to work for my case Mar 16, 2021 at 8:53
  • 2
    If an encryption system generates the same ciphertext for the same plaintext then it is far from secure.
    – OrangeDog
    Mar 16, 2021 at 9:46
  • "never encrypt two different files with the same IV". This sounds like dangerous advice. One might think that it is a good idea to store used IV's to avoid IV collisions, but storing IV's is a much bigger security risk. In practice, the IV is long enough that a random IV is almost certainly unique.
    – MSalters
    Mar 16, 2021 at 10:39
  • @OrangeDog It’s more correct to state that it’s far from what most security minded individuals would call secure. The only goal of the encryption may be to discourage casual attackers, in which case reproducibility is probably not a major issue. Mar 16, 2021 at 11:48

"I have been working on testing whether same encrypted files are same without decrypting it"

In general, it will not work. The more secure encryption schemes explicitly use a "non-deterministic" (actually they endeavour to be "truly random", not just use a crypto-level PRNG, by collecting entropy through various means) approach, and will never encrypt the same data in the same way twice.

It is sometimes possible to add a (either signed or unprotected) "hash signature" that represents the unencrypted content of the file.

This is usually not done because determining that an encrypted document is identical to a known plaintext one effectively allows "decrypting" the document, or being sure of its content, defeating the cryptography.

The more robust cryptography, therefore, is designed to prevent exactly what you seek to accomplish.


If you are dealing with a deterministic encryption, and you're (e.g.) writing a test case to verify that the encryption is working, then this is doable in principle.

For safety, I would not check the files line by line (depending on the platform this might cause unexpected behaviour, or even false positives).

Verify that the size is the same. If not, they files are obviously different.

Otherwise, compare them byte by byte, or use a library function to extract a reliable enough hash (sha256, but in practice, for testing purposes, MD5 is usually enough - chances of false positives are negligible). Then you can compare the two hash strings.

  • 4
    If you're writing a test case, then you presumably control both the plaintext and the key. So decrypt it yourself and do the comparison in plaintext instead.
    – Kevin
    Mar 15, 2021 at 21:24
  • Yes I am writing test case for whether the encrypted files are same or not. My approach above does say results are same but as u said it might be false positives. I am thinking of MD5 hash checks too Mar 16, 2021 at 8:53

You did not mention the type of encryption is g. There are various ways to achieve comparison on the encrypted data and here some of them;


To have equality you may need to use ECB mode of operation with a block cipher like AES. This is insecure and there are better and more secure ways than this.


Hash can be used to check equality, too. However, there are two issues here

  1. The hash must be calculated with the plaintext.
  2. hashing is free therefore one can calculate hashes of all files found on the internet and may decide some of your files.


HMAC ( hash-based message authentication code) requires a key, and we can consider it as a keyed hash. This doesn't have the second problem of the hashes since the attackers don't have the key. Use HMAC-SHA512 to reduce the collision change to negligible. As in hash, the HMAC-SHA512 must be applied to plaintext. Note that, once calculated, the comparison is very fast since it has a 512-bit output.


If you use the same IV and key for each file with AES-GCM ( internally uses CTR and this valid for any stream cipher) then you can have the equality on the ciphertext, too. Again this is dangerous and can remove the confidentiality and even more, the attacker can forge messages too ( this is catastrophic). This is called nonce misuse, that is the (key, nonce) pair uses more than once. To mitigate this there is Synthetic IV mode (SIV). SIV mode is nonce misuse-resistant authenticated encryption. It simple has 3 properties;

  1. The IV is modified with the message itself ( requires double pass on the data)
  2. The ciphertexts are equal if the messages are the same.
  3. If the messages are different, with a great probability that they are different, too.

Therefore with a fixed nonce and key, you can test the equality.


Although slow, one can use Fully-Homomorphic-Encryption for the equality of the ciphertext. This will be 100% true, there are no false positives. Microsoft uses this to securely check that you are pwned or not.

  • I do not know which encryption is used in API but it does seems like it outputs same binary format for same file Mar 16, 2021 at 8:42
  • You should have been asked to the API owner in the first hand. That is crucial since even they may not aware that they created en insecure API.
    – kelalaka
    Mar 16, 2021 at 8:53
  • It was for regression testing and they like to test whether encrypted files output were matching, so I was looking for options to check it. I think I will go with hash checks for now. Thank you Mar 16, 2021 at 8:55
  • 1
    You should tell every detail from the beginning so that you get proper answers to your exact question. For example, I wrote this what kind of methods must be applied first hand so that one can compare on the encrypted data, not for a ready system to go.
    – kelalaka
    Mar 16, 2021 at 9:16

The question is probably more simplified by saying "How can I check if 2 binary file are the same?"

md5sum f1 f2

If you get the same value, they are the same. It will even help you detect duplicates in a large set of files.

  • Is it viable for large files though, I think it would take long time to check in binary format Mar 16, 2021 at 8:43
  • 1
    @ManishShrestha To compare two binary files, you need to read both files from start to end. You then either compute some form of hash which represents each and you compare those, or you compare byte by byte, but in both cases you read the whole file. In the latter case it's nearly pure I/O. In the former case there's a bit of CPU overhead depending on the hash. Using a hash will only be quicker if you compare repeatedly against the same file (so you compute the hash of the reference once, and then only need to read the file to be compared, not both).
    – jcaron
    Mar 16, 2021 at 10:39
  • 1
    @ManishShrestha Also, how big are your files? shasum -a 256 takes about 5 seconds to hash 1 GB file from an SSD on my Mac. Depending on your use case this may be a lot or this may be negligible...
    – jcaron
    Mar 16, 2021 at 10:56

Comparing the hashes of the files, as @camp0 suggested, is considered to be best practice. This script should point you in the right direction:

import hashlib

def sha256file(fname):
    BLOCKSIZE = 65536
    hasher = hashlib.sha256()
    with open(fname, 'rb') as afile:
        while True:
            buf = afile.read(BLOCKSIZE)
            if not buf: break
    return hasher.hexdigest()

print(file1, file1hash)
print(file2, file2hash)

    print('files are the same')
    print('files are not the same')
  • 2
    It won't work depending on the encryption mode. If you use a random IV, you get a different output every time you encrypt the file, even with the same key.
    – ThoriumBR
    Mar 15, 2021 at 12:26
  • @ThoriumBR, thanks for this clarification. I agree - encrypting the same file twice will likely produce different encrypted files, due to different IV's, or different salts if the key is derived from a password, or a number of other variables. But, I'm not sure if this is what OP is asking, as he says, 'Is this process okay for checking the encrypted files are same'? The above script will check if two encrypted files are the same. OP, you should consider what ThoriumBR is saying, if you haven't already.
    – mti2935
    Mar 15, 2021 at 12:34
  • Yes @mti2935 thank you for this I did test it out and seems like it is working out from my end. Thank you! I dont know whats the encryption mode on the api end but for now it seems to work, needs further testing Mar 16, 2021 at 8:48

Lots of people pointing out the general flaws in the idea of "encrypt X, see if that matches Y". However, there's a much bigger problem with your specific implementation of the checking code:

Unless the encryption that the API uses works line-by-line, instead of on the whole file, the idea of comparing line lengths is meaningless!

In a normal text file, a line ends when the program parsing the file finds a newline character (\n), or sometimes a combination carriage-return+newline (\r\n). In an encrypted file, those characters have been encrypted and will no longer be recognizable. Sure, in an encrypted file (or any other binary blob of effectively random noise) approximately 1 in every 256 bytes will be a newline character, but it won't in any way be correlated with where newlines occurred in the original file.

Furthermore, checking for matching lengths (of lines or even the whole file) is unlikely to work. Encryption can, and often does, increase the length. Block ciphers usually need padding, which will extend each encrypted message out to a multiple of the "block size" (typically 16 bytes for modern ciphers, some older ones use 8 bytes). Stream ciphers don't have this property, but both block and stream ciphers can (and if even half-competently written, do) have a random "Initialization Vector" (IV) or "NONCE" (Number used ONCE) that is used to ensure each encrypted message (ciphertext) is unique even if the plaintext was the same. The IV/NONCE is not technically part of the ciphertext, but is usually prepended or appended to the ciphertext because the decrypting party needs to know it. Additionally, good cryptography uses an integrity check (something to ensure the ciphertext wasn't messed with before decrypting), such as an HMAC, digital (asymmetric) signature, or the authentication tag of an authenticated encryption cipher. In any case, this requires sending additional data (typically appended) with each ciphertext.

If the API does, in fact, encrypt line by line... stop worrying about whether you've matched the file and start worrying about why you're using a site whose encryption scheme makes WEP look secure.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .