How to give a web application a key in order to decrypt data in a database

Question

My situation is that I need to store sensitive data in a database for a web application. The sensitive data includes things like refresh tokens for Oauth2 systems, secret keys for other APIs etc.

I'm trying to build this assuming that an attacker already has access to the server and can see the binaries that are running the application (no source code on the server), the environment variables and the databases etc.

So basically the thing I'm trying to protect against here is an attacker being able to read sensitive information from the database.

To do that I assume I need to encrypt the data before storing it in the database, but this has left me with the problem of how to provide the web application with the keys to encrypt and decrypt the data in the first place.

If an attacker already has access to the server then environment variables, config files and even other servers that the application can make HTTP requests to seem useless because the attacker could just do the same things the application does to request the keys anyway.

So from what I can figure out implementing a workflow where the application requests the keys isn't going to work. Instead I need a workflow where the application is given the keys instead.

Initially I thought that I could just enter the keys manually on application startup and keep them stored in memory, but the problem with that is then I would need to manually intervene to start up each instance of the application. Ideally I want startup to be automated as much as possible which would mean giving the keys to some kind of script or something, in which case I'm back at square one.

So far the only way I've come up with of providing the application with keys somewhat automatically is to implement an endpoint of some kind that can be POSTed to with the keys.

The idea is that I start up the application on the server, but the application rejects all requests because it doesn't have the keys yet. The keys are stored on a separate machine somewhere; this machine could be in the office or another server etc.

The machine with the keys then POSTs to the application endpoint to give it the keys over an HTTPS connection and the application stores the keys in memory until it's restarted, at which point it will need the keys POSTed to it again.

Once the keys have been provided the application will accept requests like normal and shut down the endpoint that accepted the keys.

This does mean that an attacker could find the endpoint and try POSTing keys to it themselves, but it's highly unlikely that they would provide the correct keys anyway and even if the application accepted the keys in the small window of time where the endpoint would be available the decrypted data would be incomprehensible anyway.

So now my questions are, am I overthinking this, and is there a better, industry standard way of providing keys to a web application for the purpose of encrypting/decrypting data in a database?

If there isn't an agreed upon method for doing this then would my idea be feasible and what kinds of things am I overlooking that could actually weaken security instead of strengthen it?

Answer 1

Initially I thought that I could just enter the keys manually on application startup and keep them stored in memory, but the problem with that is then I would need to manually intervene to start up each instance of the application.

Whilst memory only is in practice notably more secure than on disk if they have enough access to the underlying machine running the application they can just take a core dump and lift the keys out of there.

So now my questions are, am I overthinking this, and is there a better, industry standard way of providing keys to a web application for the purpose of encrypting/decrypting data in a database?

The best approach I have seen is to have them stored on the application server in a config file which is not readable by the user which runs the application. They are passed in either as arguments or by IPC when the application is started.

This means privilege escalation on the application server is required to obtain the keys. If the DB server is compromised the keys aren't there.

Answer 2

ASP.net core has some built in supported for the scenario you describe which you could use as an example of implementing in your framework of choice.

This article documents an example setup of the Azure KeyVault Configuration Provider.

It essentially uses Azure KeyVault to store your secrets, secured using Role Based Access Control.

It then has a configuration provider which can load application settings from the keyvault. The configuration provider is authorized to use the keyvault via a "client secret" or certificate which would typically be set during deployment of your application (and not appear in source control).

Azure has in beta the ability for a "managed service identity" which essentially means you don't need to worry about distributing the client secret or certificate yourself.

This might be overkill for your use-case. You may wish to simply have an encrypted configuration file (protecting it whilst on the file system) in which the machine can decrypt on the fly.