Storage of 'secrets', keystores, HSMs and the rest

Question

I've been digging for the past few days into storage of sentitive data. If I store this data in a database, I believe the general accepted practice is to store sensitive data encrypted, for various reasons out of the scope of this question.

Encrypting data, however, means I have to manage the 'secrets' to this data. I've seen 2 main paths for this (in the java world), either KeyStores or HSMs. My first question is, are there any other valid alternatives (in the java world)?

There are multiple keystore formats that I can use in java, to name a few:

• jks
• jceks
• pkcs12
• bks
• uber

I've seen very little information on the security of a keystore. I've been thinking that, had I access to a keystore, and when I look at the java API since all keystores are protected by a password, I could simply brute force the keystore, however secure it is. The second question then becomes, is it generally accepted practice that since keystores are on different systems than the encrypted data and since they are password protected, then this is 'secure enough' for real life conditions? What about the storage of the password to the keystore? Are there any accepted practices there? For example, storing it in yet another system? Having it manually entered on application startup? Having it in an os-protected file where only the application has read access to it?

The next question is in regards to HSMs. Since I can brute force keystores, what's to prevent me to do the exact same thing to a HSM? Time? Having to be logged in to the remote system since there is no file for me to copy and work with remotely?

The final question is does anyone have any idea of what is considered secure enough in terms of secret management? I would assume it varies depending on the type of information encrypted with the secret. From what I've read elsewhere, keystores seem good enough for most typical scenarios and HSMs are necessary in very niche scenarios, for CAs, governments and the like.

Any insights appreciated.

Answer 1

Broadly speaking, key stores are just different 'formats' for storing keys and metadata. To my knowledge there aren't any major security differences between them. Think about it as something similar to encoding an image as Jpeg or PNG or GIF. Subtle differences, but ultimately they store images.

You rightly picked on parts of the problem of key management in general, and hope you start to appreciate why it might get complicated. So you have data encrypted with a key, the key stored inside the keystore, protected with a passphrase, the passphrase stored... you see where this is going? At the end of this chain, there's some relatively small piece of information that if you obtain it - you can pretty much break the whole chain. This will likely give you access to all the data. Similarly, even if you solved this particular problem, the key for encryption/decryption/signature needs to be used. i.e. loaded into a program that uses it to manage the data. When it's loaded into memory, it's in some ways accessible to the CPU, and potentially other programs running on the same systems. If one of them is compromised, your key can be extracted from memory. Even if it is stored in encrypted form.

This is more-or-less where HSMs come in. Without HSM you are limited in your capability to store keys securely, and also limited in being able to perform cryptographic operations in-memory without exposing the key. HSMs 'hide' the key in a (usually physically protected) storage, and perform crypto operations inside this protected area. They operate in such a way to prevent access from other programs or the operating system. All those see is the input/output from this HSM. HSMs also vary in terms of the level of assurance they provide. FIPS-140 is one of the standards that cover different levels of assurance/resilience. Some HSMs also offer some degree of key management / authorization for you. For example, they can 'grant' encryption-only to certain users, but other users will also be able to decrypt data. They can limit the number of crypto operations per user/group, and they can be configured to lock-out after several failed attempts. This is something no keystore is able to do.

So whilst you are correct to note that keystores (and in-memory crypto operations) are usually 'good-enough'. In some circumstances, it might be preferable to use something even more secure. It all depends on the value of the data, the perceived level of current protection, the cost of additional protection using software-only, using HSM or other tools, and the threat profile to this data (who wants it, how capable they are, how much are they going to invest to do it etc).