1

There are recommendations on this website and in the internet suggesting to never store credentials (e.g. login/password to the database that certain web application is using, or S3 access key on non-AWS instance, etc.) in plain text on the filesystem, apparently due to possibility to recover password from the disk.

The alternative suggest transmitting secrets from a some sort of remote secret manager over the network, I believe this model expects application that makes use of corresponding secret to store it in memory. Variations of this scheme also support storing secrets in encrypted files on the target host's filesystem and transmit encryption key to the application in some secure way over the network.

Given that I'd still like to store credentials unencrypted on the local filesystem - let's say due to framework limitation that expects unencrypted password in some local file - would it be possible to still store secrets in plain text in certain file and mitigate the risk by:

  1. Using encrypted filesystem - so malicious actor who steals the disk still won't be able to read secrets on the encrypted partition.
  2. Using ram FS (let's forget about necessity to transmit secrets first for now).
  3. In addition to either 1 or 2 limit access to file to a certain user which is used to run the application.

P.S. there are similar questions about this matter, e.g. Storing database password in plain text? (this one is perhaps too vague and therefore it did not receiving a clear answer) and Is it okay for API secret to be stored in plain text or decrypt-able? (more broad in scope; the answers though didn't address particular aspect I'm asking about).

In addition, I want to clarify that (1) password IS NOT stored in the application configuration, let's just say it is distributed via some secure mechanism to the host during application's deployment; (2) end user who uses web application I'm referring to does not have access to the host.

My primary question is whether there is something inherently insecure in filesystems that essentially leaves me with options that expect me to never store credentials (for accessing other services from my applications) unencrypted on any filesystem. In order to narrow down the scope, let's limit this question to modern Linux OS.

  • Why do you think the second example is too broad in scope compared to your question? Looks to me quite similar and the answer is contained within and the related articles such as security.stackexchange.com/questions/38566/… – LTPCGO Jan 9 at 23:34
  • @LTPCGO it didn't make a lot of sense to me, to be honest - it is obvious that one dealing with real world software can't avoid storing, at least temporarily, sensitive data in unencrypted form (think of AWS/GCP/Azure access keys, TLS certificates that you need to feed to, say, nginx, etc.). So, I guess saying "don't store any kind of sensitive information unencrypted anywhere for any period of time" is almost equivalent to "stop writing software" as it is just can't be achieved for a broad spectrum of libraries/frameworks, albeit can be mitigated with things like instance principals. – Alex Jan 10 at 0:26
  • I will write an answer that addresses your questions, the issues or not with that, and best practices – LTPCGO Jan 10 at 0:31
  • @Alex: TLS certificates? Certificates are intended to be public. Your server is expected to show its certificate to any client that asks that. That's why it makes no sense to encrypt certificates. May you be you mean private key for SSL/TLS? – mentallurg Jan 10 at 0:54
  • @mentallurg ah, yes, my bad, I meant private keys of course. – Alex Jan 11 at 18:19
1

With any question like this it's all about risk mitigation and usability.

The first thing to accept is that if a third party has unrestricted access to hardware you won't be able to stop them. The same largely goes for having root access. At some point a secret will have to exist unencrypted in memory, and at that point it is possible to retrieve it. So let's deal with the case where we have a user who has access to a server and is able to run commands, but who isn't an admin. How best do you store your secrets?

Ideally, you don't store them on the server at all. You can utilise a secrets management system and pull in a secret that is regularly rotated that way and not have to worry about storing it at all. This is widely used and deployed on the cloud in things like AWS Secrets Manager or Azure Key Vault. In this scenario it doesn't matter about the security of your web host to a degree since the security is handled off-host.

Let's say for whatever reason it has to exist on the same server. Well you made sure it's not hard-coded into the code, good. Next thing is that the web application needs to read it. That doesn't mean though that any other user should be able to. The sensitive files should be have 400 permissions ideally in a separate folder outside of the web root. If they must sit under the web root, use something like .htaccess to make sure they can't be viewed.

What if someone steals your server? Well, yes it's good that everything is encrypted, but given enough computing power and time that could be broken. What if you have restored an offsite backup in the meantime and then our person has all the keys? That's the last and most important bit - regular rotation of unique secrets so that even if they have been compromised the amount of time that compromise is useful is low.

The final thing to do might be to just eliminate secrets altogether - connect to your database via presentation of a certificate rather than a password, restrict access to internal services to only come from the local machine, etc.

A filesystem is a way of cataloguing and organising a bunch of binary on a hard disk into something useful for an end user. There are good and bad filesystems depending on what you are looking for - speed, redundancy, or security. The principles are largely the same for all, and in your scenario where the user only has access to the web application and not the host then the security of the filesystem should not come into it as long as that application has been designed securely.

|improve this answer|||||
  • So, as a conclusion - there is no definitive negative answer as to whether FS can be used for scenario described in my question, it is more a trade off that someone may or may not be willing to consider. Certain use cases might warrant the use of filesystem for storing sensitive data in clear text. – Alex Jan 11 at 18:18
  • 1
    I think your idea in 'point 1' works but I guess my answer can be summarised thusly: there is no complete solution that is infallible, and like anything the best practice is to maintain security in depth. So use ACLs for reading the secret, encryption of the file system, and regularly change the secret. On their own none of these things are a complete solution. – LTPCGO Jan 11 at 20:16
3

OWASP comments on this in Cleartext Passwords in CATALINA_HOME in the context of managing Tomcat's web server database credentials. Their view can be summarized as

best practices is to never to store clear text passwords, but it is very difficult to avoid

Their primary recommendation is to use file system security to secure the credentials file and limit the database user permissions to the minimal required.

If you have heard the expression, "it's turtles all the way down", it nicely summarizes the OWASP view that hiding the password in another place isn't really resolving the key issue and may provide a misplaced trust that the issue has been solved.

Enterprise security teams are sometimes humorless and likely lacking in appreciation for philosophical arguments involving turtles and the problem of infinite regression. Therefore, there are several password vault solutions with integration for web servers including Tomcat Vault, OracleWallet, CyberArk, etc.

|improve this answer|||||
  • Yeah, I'm not sure I'm convinced by such statements, most of these articles merely say "don't do this" and oftentimes lack suggesting usable alternatives or even explain what consequences such storage may entail in certain cases. Storing that cleartext password on the unencrypted filesystem is one thing, what if I store it in /dev/shm/? Or what if I use encrypted FS? These alternatives is what I'm curious about. – Alex Jan 10 at 0:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.