How to make a secure key distribution scheme/protocol?

Question

We are building a data processing platform on Amazon AWS. We are going to store sensitive data in the S3 (encrypted) and then the Amazon EC2 node is going to be utilized to process this data. We are planning to use hybrid encryption for the S3 data i.e. using symmetric Data Encryption Keys (DEK) & asymmetric Key Encryption Keys (KEK) & it will be done on client side before uploading the data to S3. The encrypted DEK will be stored with S3 object in metadata The are going to be group of datasets & the keys will be different for these datasets.

We are planning to build a small KMS system which can distribute keys to the EC2 nodes when they are going to process the data to decrypt it. The KMS will be hosted on our premises and the transfer of the keys will be on VPN.

The question we are having is regarding the most suitable Key Distribution Protocol to build to distribute the keys to EC2 nodes. We are considering building a webservice using REST/SSL to distribute the keys however since there are no WS-Security spec available for REST, we feel only REST/SSL will not suffice.

1. What can be additional implementation to tighten security?
2. What are ways to authenticate/provision the EC2 node when it calls the REST webservice that it is spawned by us?
3. What is most suitable - passing encrypted DEK to webservice & getting decrypted DEK vs simply getting KEK from the webservice & using it to decrypt DEK to decrypt the data?
4. Any additional points to consider?

Answer 1

First, you cannot securely move data over an untrusted network between two untrusted endpoints. To securely distribute anything, whether it's keys or 100 GB Blu-Ray images, you're going to need one of two things set up on each endpoint to establish initial mutual trust between the two:

• A shared secret (i.e. a password or a shared key)
• One half of an asymmetric keypair (i.e. a trusted certificate)

Second, you say you're using a VPN. The whole point of a VPN is to transform an untrusted network into a trusted network between two now-trusted endpoints. If your hosts automatically authenticate and join the VPN, they already have either a shared secret or a trusted certificate.

Assuming your VPN is encrypted, your data will already be encrypted in transit between your network and the EC2 network. If your EC2 hosts can join the VPN on startup, they already have enough information to identify themselves as one of your hosts. If they can't, you can establish trust when the hosts are manually connected to the VPN by an administrator.

Once the endpoints are authenticated to join your trusted network, your EC2 hosts can exchange certificates, establishing a trusted link all the way through. This is (sorta) how Kerberos works.

Note that this won't protect your data against:

• A malicious insider - say, one of your own system or network admins
• Programs that retain key material on the EC2 hosts longer than they need it

Third, at an absolute minimum, for your own protection, your company will want to be able to audit all access to key materials in a way that's secured against an attacker with root access. You may also need to audit all access to encrypted materials in the database. Think hard about what it would take to implement that.

Fourth, protecting your keys at rest (against an attacker who wants to gain root on your KMS or any of your EC2 hosts) is an entirely different problem, and it's even harder to solve than protecting keys in transit between two mutually trusted hosts. I've been involved in implementing a "PCI-DSS network island" in the past, and it's a giant pain.

I strongly recommend that your company hire a security consultant with experience setting up distributed key management systems. The problem here isn't only the known risks - the things you know you need to defend yourself against - but what Rumsfeld famously called the "unknown unknowns" - risks you don't even know you have.

As a developer myself, I wouldn't trust my abilities to design and implement a full crypto-system without outside advice from an expert.