9

As far as I understand, a problem with the idea of a pepper is that, if it's stored as part of your code, then the hacker can read it if they can access your code.

So I was wondering, would it not be better to store the pepper in memory? I'm thinking of running the server in such a way that the pepper is made available to the server's memory, without it being visible in the environment or in the process list or in the shell history. To obtain it, the hacker would need to run a memory debugger as the user that's running the server, or as root. Maybe even run the process under something like RamCrypt, to encrypt its memory at runtime.

The scenario is that of running a server on a cloud Linux machine.

In that case, is there any better way of storing a pepper? Or is there something wrong with the in-memory idea?

2
  • 2
    A pepper is commonly used to protect database data. So, the attacker has your database data AND your application code? There's not much left to protect... Aug 19 '20 at 17:08
  • @NeilMcGuigan, I was thinking that if they have the database and the code, then they'd still be missing the pepper, which is not stored anywhere. So they'd need to extract it from the server's memory. Until they do, the code is useless and the data is still protected by the pepper.
    – rid
    Aug 20 '20 at 10:10
12

First: you are definitely correct, it can't be stored alongside your code, no secret should.

It obviously needs to be held in memory by the server since it needs to be used. However, it can't be in-memory only. As long as a single password hashed with the pepper remains, it must be available. If it's only held in-memory, you'll have a different pepper for every server, and for every restart.

The real question becomes: where to persist it.

At this point, this is the same as storing any sensitive cryptographic material (eg: a private key). You have a lot of options, some of which are:

  • encrypt/store the pepper using a cloud KMS/Vault (eg: Google/AWS KMS, Azure Vault, etc...). Either directly or encrypt the pepper using a cloud KMS and store the encrypted pepper (the encrypted pepper can be stored somewhere less sensitive, but still with restricted permissions). But the key stays inside the KMS.
  • same as above with HSM.
  • roll your own secrets storage system. Things like keywhiz and Vault can be very powerful, but these are not simple systems to deploy and manage.
  • deploy the secret to the server at setup or startup time. This moves the responsibility for fetching the pepper but the ultimate persistent storage recommendations still apply (it just gives you more options).

The general idea behind all of those is that to have access to the pepper, you should show that you have control of the server. This does nothing to stop attackers with sufficient privileges on the server but that's not what peppers are for. If they have that level of access, they can just replace your binary with one that logs the plaintext passwords.

13
  • 2
    Storing a pepper in code, while obviously not ideal, still provides protection in the event of a database only breach. Also, neither a KMS not an HDMI store anything, so you can't store your pepper in either. Aug 4 '20 at 15:32
  • 1
    We should never say, "it's never okay to do this" :p I do agree with your general point though. Storing a pepper in code is definitely not a good choice, but a pepper is also a tertiary line of defense. If that's the only place someone can find to store it, then it is better than nothing. Then again a pepper is a tertiary line of defense, so if you are trying to setup peppers but still have secrets stored in your code base, then your priorities are wrong... Aug 4 '20 at 15:51
  • 2
    @rid what you are suggesting certainly makes sense, but in a way just moves the problem one more "deep". Where do you store the pepper so that it can be readily available when a server starts? If you are talking about an auto scaling environment or the like, then it must always be available. Therefore you would need something like a secret manager that can supply the pepper to the new server as it starts. Aug 4 '20 at 19:18
  • 3
    @rid I think you need to consider your threat model. Every security feature has trade offs, so you want to find a good balance between "more secure" and "worth the effort". It's just as possible to over do security as it is to under do it. For instance, if an attacker has just a database dump then a pepper in the code base will stop them from reversing hashes. Aug 4 '20 at 19:57
  • 3
    If however an attacker has server access, then hiding the pepper won't help you because the attacker can just modify the application to have it send off plain passwords as users login. Therefore, jumping through hoops to protect the password in the event that an attacker has completely compromised your system is of limited value, and you're probably best off sticking to simple, tried-and-true, methods. "Typical" secret management will serve you well here. Aug 4 '20 at 19:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.