48

In a nutshell, let's consider a web application which stores some information in a database as encrypted data. While I'm purposely trying to keep this some what generic, here are some assumptions:

  • The encrypted data is only stored in the database (not elsewhere). Some fields in a given record are encrypted, some are not.
  • The web app must write the encrypted data to the database.
  • The web app must be able to read and display encrypted data. Data is only displayed in an authentication and authorized managed section of the app.
  • The data is sensitive information that needs to be protected in case the data is exposed/accessed (without the keys*).

Questions:

  • Where/how do you store the key(s) used for encryption? Firstly, in a system where the web and database servers are the same, how do you manage the key? Secondly, in systems where the web server and DB servers are separate, how do you manage the key?
  • Are keys just permission-restricted files? Stored in some separate tool/software? I've seen mention that sometimes the encryption keys are also encrypted, but not clear on where that may help.

I know this is a fairly basic security question and so if there are resources that perhaps I'm missing that would be also very, very helpful. I've spent a lot of time (over the years) trying to grasp this information based on information about web app security (OWASP, PCI DSS, etc.), but many times this information is very generic (e.g., "protect the data", "encrypt the data") or very specific ("to encrypt something with AES do this...").

* I understand that this is not a complete security solution in it's own right. I understand that there are several vectors to get at this information and to decode it. I acknowledge that this may be a poor question in that light :)

Edited to add more information about the assumptions to this question

3
  • 1
    This question is poorly posed. Why do you want to use encryption? What threats are you trying to defend against? It is not clear that encryption is of any use whatsoever here, given that the web app has the keys to both encrypt and decrypt all the data. I suggest you're more likely to get useful answers if you explain what problem you are trying to solve and what the threat model and constraints are, rather than pre-supposing that a particular mechanism (e.g., encryption) is the answer.
    – D.W.
    Jun 26, 2011 at 4:31
  • 3
    Sometimes it is difficult to bridge the gap between theory and implementation. Most of those resources are assuming you are familiar with both the theory and the practice, and are able to translate "protect the data" into encryption, key (creation, storage, use, and retirement), hashing, user account management, access controls, loggin, auditing, indentification, authentication, authorization, ... Well you get the idea. Security is complicated.
    – this.josh
    Jun 26, 2011 at 6:31
  • 1
    @D.W.: thank you for the suggestions. I have tried to add some more information above, but also I am trying to keep this generic. Not so much because I don't want to discuss details but because I'm interested in this specific aspect of encryption and web applications. You are correct that it may not be the best or even a suitable solution. Hopefully soon I can get some more detail oriented questions out here, but I'm looking to learn more about this aspect of encryption as I don't feel that it's well discussed in resources I've read (meaning I'm probably missing stuff!). Thanks again!
    – Rob
    Jun 27, 2011 at 14:59

2 Answers 2

11

Where/how do you store the key(s) used for encryption?

Standard approach if you are using built in encryption on something like SQL or Oracle database, the database will generate an encryption key and encrypt this with another key protection key or Master key. This can be a pass-phrase or a longer key stored in e.g. .pem file.

This is usually stored in a restricted directory that only root/Administrator and the database service account can access. On database initiation the key is read and loaded into memory. This is then used to decrypt the encryption keys.

Firstly, in a system where the web and database servers are the same, how do you manage the key?

Key is stored in a directory on the server. To renew or revoke is usually through the database program.

Secondly, in systems where the web server and DB servers are separate, how do you manage the key?

The key is stored locally on just the database server. A more secure option is to use a Hardware Security Module (HSM) in either scenario. This is a hardware device that is attached to the server and used to store the key rather than a restricted folder on the server.

Are keys just permission-restricted files? Stored in some separate tool/software? I've seen mention that sometimes the encryption keys are also encrypted, but not clear on where that may help.

Restricting the folder is acceptable as long as you manage the administrator access and monitor for things like unauthorized logins, logins as the service account, new accounts or groups added with access to the folder. As mentioned above a HSM is the more secure option if the data your are protecting and the threats that you are facing need it.

Generally the database will create instance or table keys and then encrypt this with the master key. There is minimal benefit in further encrypting the master key.

5
  • Thank you for this great answer. The points about monitoring the users/groups that have access to the key is very interesting and I'll take that into consideration. An HSM and database-provided security are two things to look at for future applications as well. Currently I have been using the programming side of the application to manage the encryption.
    – Rob
    Jun 24, 2011 at 18:35
  • 2
    @Rakkhi- storing the master key in a file that's only readable by root, the db, and the webapp may well be a standard approach (I don't know), but it provides very little security. It is equivalent to using no encryption whatsoever and just using file permissions to restrict who can access the database to only root, the webapp, and the db. This smells like a misuse of cryptography, one where cryptography is being treated as magic fairy dust and is providing a false sense of security. Can anyone justify this practice?
    – D.W.
    Jun 27, 2011 at 17:06
  • @d-w sure. The better way is the use HSM but this may not be justified by the threat model. You could encrypt the key master key on disk but they what do you do with this key?
    – Rakkhi
    Jun 28, 2011 at 8:17
  • 1
    There is some good information here, but also some of it is wrong (or incomplete). Instead of writing a series of long-winded comments, I wrote a seperate answer, long winded though it may be....
    – AviD
    Jul 4, 2011 at 11:02
  • PCI DSS states that encryption keys should not be stored on the same server as the data. Aug 26, 2016 at 16:35
31

While some good information has been provided in previous answer, there is a critical design flaw, which is not really noticed - but comes from some of the implicit assumptions in the question.

The difference should not be between:

  • Web app and DB on same server
  • Web app and DB on different servers
  • application vs. db encryption

The design should start from:

  • Who needs access to the encryption / decryption key(s) ?

Or, actually even more correct:

  • Who is responsible for protecting the confidentiality of the data ?

Which, in turn, comes from:

  • What threats are you trying to protect against?
    Or
  • Who are you trying to prevent from reading your data?

Only based on these questions, can you design a properly engineered cryptosystem. (And prevent the crypto-fairy dust that @D.W. referred to).

So, some example scenarios:

  • The user should be responsible for encryption, preventing even the app admins from seeing the data
  • The client application should be responsible, without having to involve the user - but this still needs to be done client-side (e.g. a backup program).
  • The appserver applicatively and specifically encrypts selected data, as part of the business logic (even the dba cannot see the data)
  • The database server should automatically and transparently encrypt stored data. (This protects only against theft of the db file / physical disk, with caveats...).

For simplicity, let's assume that encryption should be done server side, responsibility lies on the server, and the end-user (and administrator) has no influence on the encryption.

The main question, then, is the encryption performed by the application server, or the database server? (Actually not so simple, there are other options...)
This, as most security issues do, comes back to a tradeoff:

  • Will you manage your own keys, or do you prefer the infrastructure abstract this automatically? (the original question... more info below...)
  • Are you worried about malicious (or incompetent inexperienced) programmers?
  • Are you worried about malicious admins?
  • Are you worried about unauthorized application usage?
  • Are you worried about unauthorized SQL access?
  • Are you worried about unauthorized direct file access?
  • Are you worried about physical disk / server theft?

And so on. (Note that some of the above questions are irrelevant, and cannot be solved by server encryption).


Now, let's assume you made your decision, based on the above, between appserver encryption and database encryption.
Note that whether the web server and the db are on the same server or not, has almost no effect here - if the app is encrypting, then it's just sending "special" data to the db, otherwise it's moot. However, this does affect the above threats discussion - some issues are obsoleted by the design, some are enhanced...

  • Web / Application server encryption
    • The encryption key (EK) you use to encrypt the data, should be itself encrypted by a key encrypting key (KEK).
    • The encrypted EK should be stored in a protected place - this can be a file, a registry key, whatever - the important thing is restricting access to it, to the appserver's user only (and, perhaps the admin - see below).
    • The KEK also needs protection - this is a bit trickier.
      If you're on Windows, you have easy access to DPAPI - this provides OS-level key management, so you can have the OS encrypt your KEK and transparently manage the keys for you. (Behind the scenes, DPAPI eventually relies on knowing the user's password (in USER_MODE), so that is the final, weak point of an actual known secret - but noone else should know the server's password in the first place, right?)
      Another option is external device / HSM, discussed further below.
    • The KEK also needs access control, of course - restrictive ACLs and all that.
    • There are several reasons you want to separate the EK from the KEK: you don't want to encrypt huge amounts of data with a single key; if you need to replace the key, it becomes a lot easier (just re-encrypt the EK with the new KEK); you can use stronger, slower, more complex encryption/storage on the KEK; if you need to be PCI compliant, you can implement split-knowledge on the KEK; and more.
    • Both the EK and the KEK should not be stored long-term unencrypted - depending on your platform/language, this means e.g. not storing it in a String object in .NET or Java. Instead, store it in mutable byte arrays, and always ensure you dispose of it as soon as possible.
    • Some frameworks even go a step further, and provide specific protected objects. E.g. in .NET you can use the System.Security.Cryptography.ProtectedMemory and/or System.Security.Cryptography.ProtectedData classes to keep the keys protected, even in memory.
  • Database encryption
    • If you have your database server perform the encryption, this becomes a lot easier: the tables / fields are configured to automatically encrypt any data stored in it, key management is completely transparent (from the PoV of the application, not the DBA), and so on.
    • For example, Oracle and MS SQL Server both support automatic built-in encryption. For MSSQL, the key management is complex, yet similar to what I described above. Simplified: There is an EK, stored in the database, that the dba can configure to use to encrypt/decrypt tables or fields. This is in a protected table, encrypted by a KEK, in turn encrypted by a server level KEK, in turn encrypted using DPAPI on the MSSQL's service account.
    • Alternatively, MSSQL also supports using asynchronous encryption, e.g. storing public/private key pairs - this can be useful in specific scenarios.
  • Third option: external encryption storage, or HSM (hardware security module). This can be a shared hardware appliance, server, or a hardware card or dongle... etc.
    • These devices are designed to very securely protect small bits of data - e.g., your KEK.
      Again, another reason why to separate the EK from the KEK - this will enable you to efficiently encrypt and decrypt the masses of data with EK, while still securely managing the KEK.
    • The HSM is usually accessed via a special driver or API, it may be possible to plug these into the database encryption too (depending on product, etc).
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .