I have an linux application that needs to read a secret to decrypt some data. The application also allows the users to change the encryption password during runtime. The application will run under a separate user account and the secret is currently in a file owned by that user. I am trying to think of ways to improve this situation since the password is in plaintext in a file.

The application could be started as root and then drop privileges to the application user. This way the secret file can be owned by root but then the password change scenario breaks unless I am running another daemon as root.

I am basically looking to prevent the secret from being in plaintext on the disk, and without a human in the loop to bootstrap the encryption process, the best that can be done is to make it harder to get to the secret even if someone gets code execution.

Can anyone share any suggestions on how I can improve the current situation? Is keyctl something that is applicable here or is that normally just used to store kernel secrets?

Any suggestions will be appreciated!

  • Do users have physical access to the Linux machine on which the application runs, or is it a web service? What is the potential for an attacker to have physical access? Commented Nov 9, 2016 at 21:58
  • 1
    It is a network accessible service. Physical access is not expected to this server.
    – johne
    Commented Nov 9, 2016 at 21:59

4 Answers 4


Your service should not run as root generally. Using a separate user to store the secret is a wise precaution, but storing files with root ownership is no better for secrecy than using another dedicated user for this. As long as your secret file has permissions 600 or less it will be safe from other non-root users. Anything running root has full access anyway. (unless you are running some complex SELinux setup)

It may be helpful to use two users for this single application to isolate secret management from normal operation. Either have two daemons running with real-time communications, or have utility commands to operate on the secret, gaining the necessary privilege (user or group-based) via sudo.

You should consider potential attack vectors.

  • SQLi if properly configured does not grant filesystem access, but Path Traversal attacks could. There are other types of attacks on your application that you should secure against.

  • Exploits in C libraries (i.e. OpenSSL running your HTTPS encryption, image processing, etc.) might be possible. Using a separated user on network-exposed processes can help limit the scope or add difficulty to an attack.

  • Break-ins from other attack vectors could be an issue. (SSH, other services on the machine) Make sure anything with the potential for root access is well secured.

All of these considerations should help you to secure the secret itself. Now the question is, does the secret really need to be stored in plain?

At the most basic level, yes, the secret must be stored somewhere for it to be usable. It at least has to be stored in Memory, but also permanent storage of some kind is necessary.

Beyond of user separation we've discussed, it could be helpful to off-load secrets to a separate Hardware Security Module, or even a separate mini-computer. (i.e. Raspberry Pi) There is some discussion of physical separation here.

There are very few cases where the secret can be encrypted itself in permanent storage.

If your concern is hard drive theft, (leaving the rest of the machine behind) then you could have a boot-up script that queries some of the other hardware for unique identifiers, and derives an encryption key using a good Password-Based Key Derivation Function; to decrypt the secret.

If the client can provide a portion of the secret, that would be the helpful. I've outline a way to derive decryption keys from user passwords and session tokens here. This is an elaborate yet effective system which focuses on limiting the amount of time a secret is stored, even in Memory. (assuming you do not require an automated password reset)

  • 2
    SElinux? blog.siphos.be/2015/07/restricting-even-root-access-to-a-folder
    – symcbean
    Commented Nov 9, 2016 at 22:33
  • 1
    Good point on the user storing the secrets not needing to be root. My understanding was FDE takes care of the hard drive theft issue.
    – johne
    Commented Nov 9, 2016 at 22:40
  • Also I had not considered file traversal (was mainly worried about code execution), but that is even more likely than RCE. To prevent that particular vulnerability a second user makes a lot of sense. Does creating a new binary, setuid as the secrets owner, but can be executed by the application make sense? That should take care of everything but RCE in theory.
    – johne
    Commented Nov 9, 2016 at 22:48
  • I find sudo easier to setup than setuid, but both would work. Commented Nov 10, 2016 at 13:35

If the secret is stored on the server in any way which is readable by the application without human intervention, then its readable by at least root on the machine.

It does not matter that the file is in plain text; it can only be protected by permissions and root can bypass those permissions. Encrypting the file (or the filesystem it sits in) is only going to help if the encryption key is only made available to the process to read the file, and that requires external intervention (and verification); you are just moving the problem of access to the secret somewhere else.

Even if you retain control of the all accounts on the system - building it as an appliance, but that still leaves the problem of physical access to the system. If you are happy that physical integrity can be assured, then you have the problem of maintaining the system as an appliance - how are will patches be installed?

I am far from convinced that a hsm, even in hardware provides a completely secure solution, rather than making normal access overly elaborate (while also complicating, but not preventing unauthorized access).

Given that any solution would be a compromise on your expectations, you would need to be more specific about the constraints (development costs, unit costs, expected volume) along with more detail on the thret model.

  • I am not really worried about root reading the secrets. My only intention behind storing it as root was that when(if) the service is compromised, the service user which the attacker is running code as cannot read the secrets. I feel according to the comment by George, there is some utility in controlling/complicating access to the secret (except remote code execution, in which case it is game over). About the threat model, did you want to know anything in particular? I am not worried about physical level threats.
    – johne
    Commented Nov 9, 2016 at 22:57
  • Then make the file 0600 owned by any user which does not run a service on the machine, and make a temporary copy available when root starts the daemon, delete this file in your daemon and add a watchdog to check the file is deleted in the root task which starts the daemon. But you also need toharden your application so that it cannot be subverted and its memory read.
    – symcbean
    Commented Nov 9, 2016 at 23:01
  • I considered this approach, but the password change scenario will not work. The service will need to read/write the file again if someone requests to change the secret. Otherwise the service will have to restart (to be started as root again) and the network connections will be disrupted.
    – johne
    Commented Nov 9, 2016 at 23:09

What you have is the traditional way of doing things. That's how Tomcat, for instance, recommends to do it, with OWASP concurring. So, what you have isn't bad.

However, there are other ways to do it - Hardware Security Modules are limited access storage designed for key storage. Hashicorp's Vault, KeyWhiz and similar technologies are secret management systems, basically attempting to be software HSMs. Of course, all of those technologies need to be able to authenticate the process - their advantage is that they have a variety of mechanisms for that. You could use root to gain access to a key which is then provided in memory only to the web user - it could use that key to access the HSM/Vault/KeyWhiz to be able to access and change the actual key. There are other mechanisms, too - AWS specific ones using EC2 keys, a bunch of others. You may be able to find something that meshes well.

My personal opinion is that the tried and true restricted user (used ONLY for this purpose) and restricted files to only that user is not a bad way to go. It doesn't scale that well, though, where other options might.

  • 2
    Thanks for that OWASP link! My understanding was that kernel keyring/keyctl was such a virtual HSM. Is that wrong?
    – johne
    Commented Nov 9, 2016 at 22:26

keyctl is applicable here. It can store any type of secrets. It has lots of capabilities:

  1. Thread Keyring: This keyring is associated with a specific thread. It is used to store keys that are valid for the duration of the thread.
  2. Process Keyring: This keyring is associated with a specific process. It is used to store keys that are valid for the duration of the process.
  3. Session Keyring: This keyring is created when a new session is initiated. It is used to store keys that are valid for the duration of the session.
  4. User Keyring: This keyring is created for every new process. It is the default keyring where keys are placed.

Ref: https://man7.org/linux/man-pages/man1/keyctl.1.html

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .