Keeping user data private in a cloud environment like Google App Engine

Question

I am writing an open-source Java application for Google App Engine (GAE). The application will let users create content that is intended to be private. I want to provide reasonable assurances that no one (including me, as the site administrator) will be able to read private content that belongs to someone else. What is the best way to accomplish this?

The site will be served using https. Users will log into my application using OpenId, so I am not in control of their login credentials (I don't have their password).

Based on the research I have done so far, I think one solution might be to encrypt data written to the GAE datastore using a password-based encryption key. The user would choose a separate password to be used only for encrypting their private data. To generate the encryption key, I would hash the password somehow (maybe bcrypt?). Data written to the GAE datastore would then be encrypted using the encryption key plus a salt, and the salt would be stored alongside the encrypted blob. I would never permanently store the password or the encryption key, but I'd probably need to keep the encryption key on the session while the user is logged in.

Is this a good solution? Are there other solutions I should consider instead?

I would also appreciate any pointers to existing open source applications that do this kind of thing properly.

Edit (20 Dec 2011):

I'll try to clarify what I'm looking for.

Google App Engine supports two roles: users and administrators. In this case, a user is anyone who chooses to log into the site via OpenId. An administrator is a special type of user which also gets access to the application's administrative console. Among other things, administrators can view logs and directly manipulate the back-end datastore (read and write). Some administrators -- let's call them administrator/developers -- also have the ability to deploy new code.

Users can be expected to create both public and private content. I want users to feel confident that content they mark as "private" really can't be viewed by anyone else.

I recognize that a malicious administrator/developer could upload code with a back door to circumvent the privacy design. However, that problem will have to be solved via auditing. Because the code is open-source, users who are concerned about the software implementation have the opportunity to review the code and convince themselves of its integrity.

I also recognize that a large part of getting this right is simply good application design. For instance, the back-end should require authentication and should not let authenticated users view data that they don't own. There should also be test coverage sufficient to confirm that this functionality works as intended. This part of the application design is already in place.

When I asked this question, I was hoping to find a solution that offered a higher level of assurance. I am looking for a design that protects -- as much as is practical -- against malicious users, malicious administrators, and unintentional software bugs in systems that I control. If at the same time this solution also offers some protection against malicious or incompentent system providers (i.e. Google personnel looking at things they shouldn't, or App Engine bugs that inadvertently disclose my data), then that's even better.

I realize that I can't protect against every attack vector in an application like this, and certain solutions are going to be too ugly -- or too annoying from a user perspective -- to be practical. There may be no solution which is "better" than what I already have implemented. In that case, I'd like to document the reasoning behind that conclusion. However, if there is a reasonable solution, I would like to find it.

Answer 1

You can't. There's basically no technical mechanism that will prevent you from reading the user's content, if you wanted to and were malicious.

Here, let's look at some obvious attempts, and understand why they don't actually work:

• You could encrypt the user's data under a key that is hardcode in your program, and store in the database in encrypted form, to prevent you from peeking at the user's data. But then your code will have the decryption key, so you could always peek at the decryption key, and then you'll have the ability to see user data. Doesn't work.

  (Having the program generate the key for you doesn't work, either, because where is it going to store the key for future use? Anywhere it can store the key, you can read.)

• You could ask the user to provide a passphrase, derive an encryption key from the user's passphrase, encrypt the user's data using this key, and then store it in the database in encrypted form. But then your code sees the decryption key. You could modify your code to collect and log users' passwords, which would then let you read the users' data. The users would have no way of detecting this. Doesn't work.

• You could send Javascript code to the user's browser. The Javascript code could prompt the user for a passphrase, derive an encryption key from the user's passphrase, and encrypt the data (all executing on the browser, so the user's passphrase never leaves their browser). The Javascript could then send the user's data, in encrypted form, to your server to store it. This sounds attractive. However, there is a serious problem with it. If you were malicious or snoopy, you could easily modify your server code to send new Javascript to users, which collects their passwords and ships a copy to your server. That would let you read the users' data. While you might say "Gee, I would never do that", the point is that users would have no way of detecting this, so they have no way of verifying whether you are being honest or not, and there's no technical mechanism that prevents you from being dishonest. Doesn't work.

So, as you can see, there's no technical solution to this problem that really prevents you from reading user data. If you really wanted to read user data, you could, and you could even do it in a way that users wouldn't notice. What assurances do users have that you won't do this? Well, they have to trust you to be an honest sort. Technical mechanisms can't solve the problem.

Now, there's still a good reason to encrypt. If you are honest, and want to protect your users not from a dishonest version of you, but rather from inadvertent mistakes you might make or possible security breaches of the contents of the server database, well, now, that's a problem that can be at least partly solved using encryption.

But the particular problem you mentioned is not one that can really be solved through cryptography alone, in practice.

Answer 2

If you want users to feel confident, then it is a matter of adding a few pictures of padlocks throughout the interface. This is psychology, not security. Take example from 19th century banks, which always built their offices in a neo-classic architecture, with big pillars of stone and marble floors: this was so that customers, unconsciously, would mentally link the bank with notions of solidity and longevity -- a crucial notion in the USA before the creation of the FDIC in 1933.

If you want to convince technically savvy third parties that you are not interfering with so-called "private user data", well... you are mostly out of luck. Adding encryption and hashes will not help; being the sysadmin/developer, you are still in control and can technically do everything you want. Piling layers of unwarranted cryptography will just make you look slightly incompetent in security matters (at least in the eyes of IT security professionals), which will in turn decrease confidence, not increase it.

The best we can have right now about avoiding bugs and holes and backdoors is to do a lot of auditing. Have the server source code dutifully inspected. Edit written procedures for every administrative action to be taken on the servers, and make sure that they are followed; namely, hire an external auditor who will physically witness all operations. Enforce dual control (e.g. the administrator password is long, and people know only half of it, so it takes two operators to type the password). Require security clearance (including debt situation) for all developers and sysadmins. There is a framework for that called Webtrust. Two important warnings:

1. It will be expensive, long, mindbogglingly complex, and will permanently shatter your soul.
2. It is not good. It is just the best we have.