1

I'm working on an app that will download files from external servers over https and store them on disk in a relatively secure way. Specifically, using a cryptographically unique IV, an AES256 cipher block, CTR block mode, and an HMAC hash of the encrypted data and IV.

One of the requirements of this project is that no duplicate files should be stored. Given the need for a unique IV to securely store the files, I can't simply use a hash of the encrypted file to see if the download is a duplicate. A sha256 or sha512 of the plaintext content isn't sufficient either because the client may download and store single character files making their contents easily guessed, potentially opening the system up to known plaintext attack vectors.

So I was wondering if it would be considered reasonably secure to store HMAC hashes of the plaintext, using an Argon2ID derived key as the HMAC signing key?

These hashes would be stored in a key/value database, and used as a lookup key to see if a downloaded file is one that we already have. The Argon2ID key will never be stored.

  • Can you better explain why plain hashes aren’t sufficientm and why you think an HMAC might be? Typically an HMAC is used when you’re concerned an attacker can both alter the encrypted data and update the hash to match, but you don’t mention any such concern. – Swashbuckler Jun 16 '18 at 21:27
  • 1
    My concern, was that some of the files that will be downloaded and stored encrypted on disk, could have a small number of characters, like 1 or 2. If I used a regular sha256, and the encrypted files and the lookup database are compromised, it would be trivial for an attacker to just hash every known character at a length of 1 or 2 characters and guess the plaintext content of these short files. An HMAC requires a secret key to use which an attacker would not know and would change the hash. Although now that you mention it I suppose it would also work to just prepend a secret key to the sha256. – Raz Varren Jun 16 '18 at 23:34
  • Re: "These hashes would be stored in a key/value database, and used as a lookup key to see if a downloaded file is one that we already have": Are the external servers fully trusted? If not, then a malicious or compromised external server could potentially detect whether you already had a given file. – ruakh Mar 21 at 0:17
1

Yes. You can do this. It is secure. One application is ssn encryption. If you simply hash an ssn, it’s trivial to loop through all ssn, hash them all, and create a map. But if you use hmac with a well protected secret, then you can hmac(secret,query) and use that to look up data.

You do need to protect the secret.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.