How to store private data both securely and publicly accessible?

Question

The task may not be solvable in as broad a formulation as in the title, but this is my scenario (which is somewhat similar e.g. to newsletter subscriptions and therefore certainly has already been thought about)

I want to implement a database on a web server where private data about users is stored. The users are not supposed to have a fancy login name, instead they are to be "email validated", i.e., access to a mailbox should be sufficient to authenticate.

Users should be able to create, update, delete only their own data. Central administration should be able to export the data from the webserver to a secured host.

The attack scenario is that the web server together with all data on it (including web page scripts and the database) gets compromised. The requirement is that private data stored there before the compromise remains protected (assuming no further user access is attempted).

Here are my ideas so far: To authenticate I have only the users email.

• A new enters his emailinto a form, which causes hash(email) + publicencrypt(email) + random nonce + timestamp to be stored representing the user and the user is sent a mail containing an URL with parameters random nonce and email. (As there is only (protected) email and not (unprotected) username plus (protected) password, things like salt cannot be used and I have to resort to hash(email), don't I?).
• The same happens with an existent user, where only the hash(email) will be the same as in any previous attempt. publicencrypt(email), random nonce, and timestampwill differ.
• Anyone accessing an URL with parameters email and nonce as generated above can perform operations on the corresponding database record after checking: Does there exist a record with matching hash(email) and matching nonce and not too old timestamp? If yes, the user can proceed. Note that during the web session of this user, the cleartext email is available.
• The export to the secured central administration host happens by exporting the data encrypted as it is. Decryption happens with the private key matching the public key used to encrypt (the latter is considered public because it resides on the compromisable server).

Adding a new entry is completed at this point, by adding a flag isValid to the record. Deleting the record is of course easily done. Displaying non-confidential data (e.g., when was the entry created or last modified) is also a trivial task.

My problem is: How to handle additional private data securely? I might just store it in encrypted form (like the email), but in that case the content cannot even be shown to the authenticated user. He may change it, but cannot decrypt it in a later web session. This is of course sub-obtimal. My only idea is to encrypt the additional data with a different key: Instead of the public key of an asymmetric key pair use a symmetric key that is reproducibly produced from email, i.e., store additional data as symmetricencrypt(data,key_generated_from(email)). As the cleartext emailis available to the user himself during his web session, as well as in the decrypted export on the secured host, data can be reconstructed.

I see the following problems:

• Given the restricted format of email addresses (e.g., some domains may occur very often), could a key generated from the email as described above be secure enough at all?
• Actually, given the restricted format of email addresses, is the hash(email) I used in the basic concept that secures only the email good enough at all? (Is there enough entropy in email addresses, so to speak?)
• Is there a better approach?

Answer 1

I think you might be looking for a token based authentication system. They submit email address, the server produces a token and sends it out to them, they click on the link which must post the token back to your website in some form. The token is random and valid only for a limited amount of time ( say 15 minutes - think memcache ^.^) and can be refreshed based on user action. As for encryption, you could derive something from the email hash as a key and never keep the plain version of the email address. And refuse to reuse hashes in the new user phase..

As a side note, it sounds very scary using just the email to gain access. Mind you that most apps outhere that users will touch nowadays the first thing they will look at is the contacts. Therefore there's a very high chance someone already has a list of users and access.

Answer 2

A new enters his emailinto a form, which causes hash(email) + publicencrypt(email) + random nonce + timestamp to be stored representing the user and the user is sent a mail containing an URL with parameters random nonce and email. (As there is only (protected) email and not (unprotected) username plus (protected) password, things like salt cannot be used and I have to resort to hash(email), don't I?).

You can add an extra column with an extra random data (generated on subscription) that is used only as salt for hash computation (one random salt for user). That would give you the same effect as using the (non existing) username. The salt can be stored in cleartext in the same row of the email, no problem with that at all. Just use one salt per user and make it random and you are fine.

Given the restricted format of email addresses (e.g., some domains may occur very often), could a key generated from the email as described above be secure enough at all?

Use PBKDF2 to extra security. This would give you as much security as is possible when the only "secret" the user can provide is his email address.

Actually, given the restricted format of email addresses, is the hash(email) I used in the basic concept that secures only the email good enough at all? (Is there enough entropy in email addresses, so to speak?)

Already asnwered before, use a salt value stored in the database.

Is there a better approach?

Probably just by not puting all your trust on having the email address as password. But if that is really a requirement, you cant´do much better. You could have extra info on the server to use on the key derivation (derive key from email + server info), but this would not improve security on the attack scenario that the user gets all data offline. And of course you could use some encryption hardware (HSM) and use the derived key as PIN to the key in the HSM. That way, the attacker must not only dump the data from server and identify the email associated with each account, but also controll the server to use the HSM to decrypt data.