File encryption for a storage API

Question

I'm implementing a Dropbox-like API, and one of my requirements is to have the files stored encrypted on the server. It is not required that the files are made unreadable by the server, only that they aren't stored clear on disk. I'm using Python with Flask for the API part and PyCrypto for the encryption.

I know absolutely nothing about crypto and don't want to make anything (too) stupid. This is what I have implemented, please tell me how [un]reasonable it sounds:

• When the user PUTs a file, the content is encrypted as it is read from the network using AES-256 MODE_CTR (I wanted to avoid the block size restriction of MODE_ECB and MODE_CBC)
• The encryption key is the user hashed password: this avoids using a single 'server key' for every stored file, and makes the files unreadable if the user happens to change password
• The counter generation uses Crypto.Util.Counter with a random initial_value parameter which is specific to each file and stored in database (just like a password salt)
• I'm considering extending the API to allow the user to provide his own encryption key along with the PUT request. He would then have to provide that same key in his GET to retrieve the deciphered content.

Of course, communication with the API is done over HTTPS.

Answer 1

Very unreasonable.

The first question I think you need to ask is if file encryption is necessary. Based on your question it sounds like the answer is yes. To me that means that you A) should implement file encryption and B) do it well. What you describe is A but not B which is arguably worse than nothing at all because it may give your users a false sense of security.

Here are my concerns (as an engineer and potential user of your product):

"I know absolutely nothing about crypto" -- Crypto is easy to get wrong and hard to get right, why would you want it implemented by someone who self-admittedly doesn't know anything about crypto? Think about the products you rely on as a customer, if their security was implemented by someone who didn't know anything about security, would that give you confidence?

"The encryption key is the user hashed password" -- Generating a random, unique symmetric key per file is easy. Each time you need to store a file, create a new key and encrypt that key with something only the file-owner knows (password may not be a bad choice). Why would you want all the files to become unrecoverable when they change their password?

"allow the user to provide his own encryption key" -- This is actually an excellent idea and something that Amazon S3 just introduced. Allow users to provide encrypted files or plaintext files along with an encryption key. Your servers take on the work of encryption but the user gets control. Just make sure you purge the encryption key after you use it, don't let it sit around in memory or in log files.

Answer 2

A couple of pointers

• When you use symmetric encryption purely, you will have trouble with concept such as multiple key encryption, escrow, etc. You could consider using PGP instead.

• Ideally, every session/file would have its own random symmetric key, when you use the hashed user password, you're missing out on this. PGP would fix this for you. When you do decide to use a user password, make sure you use a strong hashing algorithm such as PBKDF2 with a MAC to generate the encryption key from the supplied password.

• Make sure to use a secure PRNG to initialize the cipher. As far as I know CTR mode is considered alright (I'm no cryptographer, though)

• Sharing the secret is far less secure than using asymmetric keys, in which case the user can freely share his encryption(public) key, while keeping the secret key actually secret. Again, this is easily done with PGP.