Take the 2-minute tour ×
Information Security Stack Exchange is a question and answer site for Information security professionals. It's 100% free, no registration required.

I have two files which were encrypted using OpenSSL. I know that that first one was encrypted using bf-cbc, and I also know the password to decrypt it. The second file is likely to use the same password, but when I try to decrypt it I get the following error message:

bad decrypt
29034:error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt:evp_enc.c:330:

If I try and decrypt it using a different cipher (say aes-128-cbc as it's the the first in the list) I get the exact same error message.

So is the error I'm getting due to a bad password or the wrong cipher? Is it actually possible to tell? I figure I could try every cipher in my OpenSSL version and see if any of them work. Is there a shortcut to try this?.

share|improve this question

2 Answers 2

If your cipher is using padding (i.e. is not in a streamed mode such as CFB, OFB, CTR or CTS), then it may be able to tell just by looking at the encrypted data's size. For instance if it is 24 bytes (192 bits), then it could be Blowfish (because it has a block size of 64 bits, and 192 is divisible by 64), and it cannot be AES because 192 cannot be divided by 128 (AES's block size). But since most block sizes are multiples of one another, this will probably not be very helpful, but it can be used to narrow the choice down sometimes.

If the above method failed, then it is computationally infeasible to distinguish data decrypted with the wrong key with the correct cipher and data decrypted with the correct key with the wrong cipher. They both produce random data. (this can be formalized assuming the cipher is a random permutation). This is unless the cipher has big weaknesses, of course, which is probably not the case if it is included in OpenSSL (except the old export-safe ones like 40-bit rc4).

So the only way is to try all the possible ciphers the data could have been encrypted with (there are an infinite number of potential ciphers, but only a few are actually used in the world). You could always whip up a script to automate it.

Of course, you can store the cipher used to encrypt the file next to it (or within it), like a byte that takes a different value for each cipher. If the key is kept secret, this presents no security risk (modern ciphers are designed to rely completely on the key - see Kerckhoffs's Principle).

share|improve this answer
    
OpenSSL encrypted files are actually recognizable: they start with Salted__ followed by a random salt. I don't think that this header depends on the cipher, however. –  Gilles Apr 21 '12 at 17:11

So is the error I'm getting due to a bad password or the wrong cipher?

Possibly either or both. I believe the EVP_DecryptFinal_ex() function checks that the padding value and size after decryption are consistent with the block size used by the encryption process and size of the input message if you employ padding(the default). The error returned to you seems to indicate that the check failed and the choice of algo and/or key is wrong.

Is it actually possible to tell?

I do not think so. Experts may want to weigh in here.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.