How should I store and distribute deployment configurations?

Question

When developing web applications that require access to a database, or any other service, they will somehow have to possess a secret like a password or connection string. This secret has to be configured in the database and web application. If it's only one secret, it can probably be done manually in configuration files after deployment, but adding more secrets to the infrastructure, at some point automation will be required.

Additionally, in the age of DevOps, automatic building, continuous integration and continuous deployment you sometimes have to be able to deploy your application regularly in test, stage and production environments. Usually, multiple developers will require the ability to deploy, at least on the test and staging environments.

I can think of numerous ways of how to manage secrets and read quite some documentation and blog posts about this, but I'd like to gather more insight and maybe improve my process.

I'd, therefore, like to know, what you think of the following approaches:

1. Deploy secrets only manually after deployment. Store them in encrypted files outside of version control.
2. Deploy secrets automatically after deployment. Store them in encrypted files outside of version control and only on the configuration management server.
3. Deploy secrets automatically after deployment. Store them in encrypted files in version control for configuration management (i.e. ansible vault, chef secret data bag, puppet secrets).

I am currently in favor of option 3, as it reconciles security with usability, but I've also just read this question and some of its answers, which generally advise refraining from storing (even encrypted) secrets in version control.

Maybe I even have better options?

Answer 1

The full answer depends on budget, compliance & paranoia, or lack thereof, but here are some risks with your approaches.

1. Manual deployment leaks the secret to the humans, is error-prone, and does not scale.
2. Encrypted files only on the configuration management server make that server an attractive target.
3. Using multiple keys for secrets in version control becomes cumbersome quickly, although Ansible's vault-id's and vaulted variables help managing larger projects.

If you want to continue with approach 3, there is room for improvement. My answer is based on Ansible, but might be applicable to others.

• De-coupling the access to the required secrets from the access to the configuration management logic is key. In Ansible that means secrets in separate inventory repositories (one for dev, one for test, one for prod, etc.), and playbooks and such in a project repository each with a different vault-id and password.
• The developers can manage the project and dev/test, the information asset owners can manage the secrets for production.
• ANSIBLE_VAULT_PASSWORD_FILE can be an executable that decrypts the vault passwords at runtime. Could be with GPG, or a query.

There are alternatives dedicated to secrets management, and command-line configuration management tools can integrate with them. A fully integrated tool is AWX/Ansible Automation Platform, which can store secrets/credentials encrypted in postgres, yet pulls the projects and inventories from version control. It works with ansible-vault, Hashicorp Vault, CyberArk, to name a few.

Answer 2

The answer by @bbaassssiiee is already a good fit for many.

Nonetheless, I would like to add an option 4) which is especially interesting in larger organisations with many projects. You keep a central vault somewhere (There are products), then each product/service/application/build system fetches it's required secrets from there.

With this approach you do not even need to bake in the secrets into config files.

Given your application has the appropriate access to this vault, it can fetch the secrets just as it needs. You could even change the secrets on-the-fly without a new deployment or config change.

Plus, it allows very fine-grained access control and access logs to your company's secrets.