Symmetric encryption for data protection

Question

My team is working on a solution to store sensitive information on a remote storage solution. I am proposing that we use symmetric encryption of the data where there is a unique key per document. The keys for this will be stored such that it is accessible to where the data will be used but separate from the encrypted data. Encrypted data will be written to and read from the storage. While this is possibly overkill, I think it will ease some of the concerns that are hampering the adoption of this technology. The idea is that even if something goes wrong and someone gains access to the document storage, the data will not be useful without access to the keys. In the event someone has access to these keys, these documents will be the least of our concerns.

In a nutshell, the concern here is that someone can access this data from basically anywhere and retrieve documents through interactions with that storage provider alone. The proposed solution is intended to make it such that even if an attacker were to gain full, authenticated access to the document store, they would not have enough information to get the content of the documents. The would need to access to information that is not available at that storage provider. This (in theory) makes the security of this data basically no worse than the current situation. This might be a questionable goal but the problem I have is largely political.

I should also add that this is not meant to be the only security for these documents. This would be auxiliary to out-of-the-box protections. A side question would be whether there is anything about this that could weaken those standard approaches.

After doing some reading, I think using 256 AES should be adequate. Is this the case and for how long should we expect this to be good enough?

I also understand that using a MAC is standard to ensure that the data has not been corrupted. After reading the wiki page here and then delving into more detail and some debate, I'm struggling to make sense of this in this use case. Data corruption is not a primary concern here but I don't think it hurts. But if all I want to do is verify the message has not been corrupted, is there any reason a hash (e.g. SHA-256) stored with my key wouldn't meet my requirement? I'm probably missing something here.

Does the overall approach and algorithm seem legit? Any help with the MAC/hash stuff is appreciated.

Answer 1

After doing some reading, I think using 256 AES should be adequate. Is this the case and for how long should we expect this to be good enough?

Yes! AES-256 is a widely used and standardized algorithm. The one cavaet is AES-256 is not in itself, the entire algorithm. The algorithm requires you to specify a block-cipher mode which is the mechanism in which it distorts the blocks of data before encryption to avoid patterns emerging. I recommend a look at this image. The most common block cipher mode is CBC and it is ideal for encrypting data that is more than one block in length (more than 128 bits which from my understanding, your data will be).

Properly configured AES-256 should be secure well into the future of computing. Symmetric encryption is currently believed to be secure even in a post-quantum world (unlike asymmetric cryptography such as RSA which will most likely crumble) so you should sleep well at night knowing your data is secure well into the future.

But if all I want to do is verify the message has not been corrupted, is there any reason a hash (e.g. SHA-256) stored with my key wouldn't meet my requirement? I'm probably missing something here.

The key difference between traditional cryptographic (collision-resistant) hash functions such as SHA-256 and HMAC functions is the HMAC hash functions are "keyed" meaning they require a secret key. HMAC-SHA256 (as an example) is often used in cases where there is reason to believe an attacker may try to manipulate data and re-calculate the hash to make the system appear to be un-altered.

Let's say I have a text file that contains instructions on when to launch an attack and a second file to contain the SHA-256 hash of the first. I write to the file "attack at dawn" and hash it with SHA-256 and logout. The attacker now logs in (through means out of scope to discuss how here, this is a simple example) and the attacker edits the text file to say "Do not attack!". But now the SHA-256 hash won't match, so the attacker re-calculates the SHA-256 hash of the file he edited and replaces my old hash. Now I send this file and its hash to my generals. They hash the file and see it matches (and therefore trust it) and see "Do not attack!".

In walks the HMAC family. In the same example if I used HMAC-SHA256 to hash my file, it would require my secret key to complete the operation. Therefore, when the attacker edited my text file he does not know my secret key and fails to generate a valid HMAC-SHA256 of the edited file. When it is sent to the generals (they too have my secret key, either through pre-distribution or PKI) they generate the hash with HMAC-SHA256 and they do not match. They know the message has been tampered with.

Long story short: SHA-256 is a great way to validate non-malicious data corruption did not occur as it is collision resistant but anybody can calculate a SHA-256 hash. If your threat model includes the possibility of malicious tampering by an attacking party, using HMAC-SHA256 hash with a secret key can ensure the attacker cannot manipulate the data and re-create the hash to make it appear un-altered.

Answer 2

I am proposing that we use symmetric encryption of the data where there is a unique key per document. The keys for this will be stored such that it is accessible to where the data will be used but separate from the encrypted data. Encrypted data will be written to and read from the storage. While this is possibly overkill, I think it will ease some of the concerns that are hampering the adoption of this technology. The idea is that even if something goes wrong and someone gains access to the document storage, the data will not be useful without access to the keys. In the event someone has access to these keys, these documents will be the least of our concerns.

This is a home-grown approximation to some standard approaches to key management. Your per-document keys are a version of what's standardly called a data encryption key (DEK). Why are they called by this seemingly redundant name? Because there's also a concept called a key encryption key (KEK), a set of keys (often called "master keys") that are used to encrypt the DEKs. This approach has some significant advantages:

• Encrypted DEKs can be stored alongside the documents they encrypt. Only the KEKs need to be stored separately, and the volume is much smaller.
• It's possible to architect the system so that front-line applications never see the KEKs. The KEK can reside on a dedicated service/hardware security module that client applications must send them the encrypted DEKs and the service will respond with the decrypted DEK (over an encrypted connection, of course).

After doing some reading, I think using 256 AES should be adequate. Is this the case and for how long should we expect this to be good enough?

There's a saying that goes something like this: "If you're typing the letters A-E-S into your code you're doing it wrong." Cryptography is much more complicated than just using AES. For example, you talk about using a MAC to protect data—an endeavor that people get routinely wrong.

I think you should bring in external help to do all this. It doesn't sound like your team has the knowledge and experience to do it reliably. If not an external consultant, I've had good experiences using things like Amazon Web Services' key management and encrypted storage features. For example, AWS client-side S3 encryption with AWS KMS-managed keys is a very good solution that's reasonably easy to use. Some informational links:

• AWS Key Management Service
• Protecting Data Using Client-Side Encryption (AWS S3)

I don't know what comparable features other cloud providers may provide.