Best practices for Private Key management

Question

How should I go about handling my RSA Private Key for my high-security Rails web application?

This may seem like a "soft" question, not related to cryptography per-se, but is nevertheless very important. What management procedures and general development security tenets should I be following? What are the definitive resources detailing how to prevent not only the technological side of hacking, but also the inevitable failures in human resources and plain old theft?

Up until now I have been generating RSA keypairs on my local MAC machine, placing them in an applications' /secret folder, committing them to private git repos and then deploying to server. While no incidents (yet) have occurred, I can see that there are two main security concerns - access to Git repos either via hacking my account or Git backend or theft/access to my machine. In regards to these two, I can only increase the strength of my Git and machine passwords, increasing the physical security of the offices is another possibility.

Thinking back to first principles of security, I have been considering generating keypairs for an application on the secured application server (access on a custom port using public keys only) and never committing them to a repository of any kind, however there are cases when the private key is needed even in development, so I would scp the key from the server onto my local machine, exposing my application once again to physical security risks.

Am I overthinking this? What am I missing?

Answer 1

key management for (highly sensitive) private keys is usually achieved by storing them within hardware crypto modules (also referred to as HSMs). there are varying levels of hardware crypto modules, which are usually rated in line with the FIPS 140-2 standard

The hardware resides either on the machine (e.g. PCI card), or on the network (also known as a network-based HSM).

Where it starts to get tricky is if the application is virtualised, or where you need to start to share the key between many hosts (e.g. 16 web servers that all use the same certificate).

For HTTPS scenarios: As a scale-out option i've seen a lot of people go to load-balancers, and use them to terminate the initial connection from client's browsers. Some load balancers have an option of storing the key onboard in hardware. From there, you have an option of initiating another connection to your web server (that sits behind the load balancer), or going cleartext, since the traffic is now on a semi-protected(?) network.