Our organization uses Amazon Web Services (AWS), and we have multiple EC2 instances running in different subnets (VPCs) for different clients. Our application is in development and we have need to remote (SSH or RDP) into these instances.

EC2 instances require a key-pair in order to gain access (SSH in the case of Linux, and administrator password in case of Windows RDP). The private key is in the form of a physical file which can only be generated once and downloaded once. Loss of this file equates to loss of the ability to connect to the instances.

Is there a widely accepted, industry-standard best practice for managing the private keys?

Currently I have a physical back-up (encrypted) on a portable hard drive, but that does not seem to me to be the best solution. What methods are employed by large companies to ensure that they are able to survive a disaster?

  • 1
    I had a similar query on ServerFault and a few tools were suggested, which our team has not explored yet. However, that wasn't for the private keys. It sounds like you keep a copy of the private key for each user? Do you password protect the pem?
    – Jedi
    Jul 9, 2016 at 17:33
  • Thanks for sharing the link to that thread. Our application is still in development so we have shared keys. For production we will likely not SSH/RDP into our sessions, so looking at the same scenario, some common root/admin access key to protect. Right now I have keys in encrypted files (AES-256) and distribute on as-needed basis. My main question is, what the industry best practice/accepted way to manage, protect these keys is, while still having a solution that is resilient to disaster.
    – Lemonseed
    Jul 10, 2016 at 10:08
  • Do all users log in to the same account on your instances or do you have a PEM and unique login per individual? If no, why not?
    – Jedi
    Jul 10, 2016 at 13:08
  • 1
    Our instances are automatically created from an existing image. In production there would be very little need to log into a running instance, except maybe to retrieve logs or perform some sort of emergency fix. Any user account would have to be pre-built into the image, so all users (who are strictly admins) would use the same account.
    – Lemonseed
    Jul 10, 2016 at 18:22
  • 1
    Private keys should be private to the user. Users should have their own keys and accounts both for personalization and auditability. serverfault.com/questions/471753/… serverfault.com/questions/697708/…
    – Jedi
    Jul 11, 2016 at 3:47

2 Answers 2


Regarding the general managment of private/public keys, there are already other answered questions here on SE: What is the best practice: separate ssh-key per host and user VS one ssh-key for all hosts? and What's the common pragmatic strategy for managing key pairs?

Regarding AWS specific details: you can and should create your own key pairs outside of AWS and upload the public keys to Amazon. You can find the documentation for creating your own keys for both Linux and Windows systems in the Amazon EC2 documentation. Just make sure that you use the correct format for the key (the text you paste into the key upload form should start with ssh-rsa).

If you have added additional keys, you can select them when launching a new instance. You can also add them manually lateron (for Linux systems: add the public key to the ~/.ssh/authorized_keys file).

Edited: You should follow the advice given by @Jedi in the comments to your question:

Private keys should be private to the user. Users should have their own keys and accounts both for personalization and auditability.

Not only for personalization and accountability, but also with regard to key revocation when somedody leaves the team / a key pair is compromised.


I wrote up a bunch of best practices with respect to EC2 and ssh. Here's the link. You may find this useful. In summary:

  1. Have one SSH key per person
  2. Guard your SSH key - never share the private key, encrypt if possible
  3. Never use AWS generated keys
  4. Use individual login (user) IDs
  5. Don't give sudo access to everyone
  6. Hide non-public instances
  7. Use SSH Port Forwarding and SSH Agent forwarding
  • Wow, why the -1 on this one? Apr 11, 2019 at 1:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .