Is it okay for API secret to be stored in plain text or decrypt-able?

Question

Aren't API keys considered usernames and API secrets considered passwords? Why is it that API servers like Amazon Web Services allow you to view your API secret in plain text? It makes me think they store it in plain text or at least in a decrypt-able format.

Isn't it better if API secrets were treated as passwords that you should type-in to create then hashed in the database instead of being handed to you in plain text? If for some reason their API secret database were compromised it will easily open the flood gates for many applications that are using their API. However if it was hashed in a non decrypt-able manner then all is not easily lost.

Answer 1

There seems to be a bit of confusion here and other places, so I'm adding this answer in hopes that it will clarify the situation for other users.

There are two types of API secrets that might be used.

The use case that most of the answers on this page are discussing is a secret key that is used to sign and verify messages using an algorithm such as HMAC. In this case, both the client and API need the actual value of the key to create a signature of the message. The server then compares the signature that it generated to the signature that the client sent and is able to verify the message. As mentioned above, if the original key is not reproducible on the server this cannot work. After initially sharing the key with the client (over an encrypted channel) the key does not need to be transmitted between the client and server ever again.

The other common API client secret is merely a secret credential (as you might see in a typical OAuth2 access token request). This credential is transmitted on each request, and therefore should only be transmitted over an encrypted channel (such as HTTPS). In this case the server only needs to retain enough information to verify that the secret value supplied by the client is correct, so the secret can and should be hashed. In this case, the secret is effectively acting as a password, and so the same precautions should be taken.

Disclaimer: I am not a security researcher, and you should absolutely do more research on this and any subject before blindly following any of the advice on this StackExchange.

tl;dr The most important takeaway is that different APIs may use the secret in different ways. It is important to understand how your API is using the client secret in order to evaluate best practices for storing it.

Answer 2

No. The API keys need to be stored in cleartext.

They are not passwords: they are cryptographic keys. These keys are used for things like authenticating requests (using SHA1-HMAC). The server needs to know the crypto key to apply the cryptographic algorithms.

Therefore, the API key needs to be stored in cleartext on the server. If the server stored only a hash of the API key, it could not verify the authenticity of messages from the client or authenticate messages sent to the client.

Answer 3

Hashing is not storage; it irreversibly destroys data. We can get away with calling password hashing as "password storage" because when we actually need the password, we have a handy human operator to type it in. Indeed, when we hash the password we do not store the password, but only a token sufficient to verify the typed-in password.

An API key must be stored in such a way that it can be used again in an unattended fashion. The server must really store it, instead of just remembering a ghost of it, because it needs the genuine key to access the API.

Answer 4

Whilst it would be more secure to hash the API tokens, it presents a usability problem: the tokens are long and random, and would have to be told to the user only once before hashing. This makes it a little difficult to secure them with a hash.

My suggestion for a secure scenario would be the following:

1. Generate a random salt value and store it in the database. This salt must not be equal to the user's password salt.
2. Take the user's plaintext password and compute KDF(password, salt), where KDF is a key-derivation function such as bcrypt. Call the resulting value k.
3. Generate an API key.
4. Encrypt the API key value with AES, using k as the key, and store the ciphertext in the database.
5. Discard the plaintext API key and k.

When the user logs in, the webapp knows their password and uses it to compute k, which is then used to decrypt the API key and display it to them.

When the user wishes to use the API, they must send k as well as the plaintext API key, over SSL. The webapp can then decrypt the stored API key value using k and compare it to the given API key. If they match, it's valid. This allows API key use without the app needing to store the user's password.

If an attacker breaches the database, they must crack the user's password in order to compute k.

Answer 5

Putting in my own two cents because of one issue that has not yet been mentioned. In particular, I agree with what @NickSloan has said, but with one more caveat.

There is one more important difference between an API key and a password. API Keys are unique to your site. The part that make password security so critical is password sharing. If someone gets a hold of the password that a user saved on your site, it isn't just your site that is compromised: it is every other site that the user used that password (and email/username) for. Many people reuse passwords across critical sites: banks, social media, email, etc, which means that good password security isn't so much about protecting access to your site as it is about protecting your users on other sites. We can't force users to do security better, so we have to assume the worst and take extra steps for them to protect their own privacy.

An API Key is a completely different scenario because it is unique to your site. As a result, if it is stolen, the attacker gains access to your site, but gains nothing that they can leverage against that person's bank/email/etc. This means that the risk level for API keys is actually lower than that for passwords. They aren't quite in the same ball park: API keys are more akin to session identifiers than passwords.

Understanding that these are more akin to session identifiers than passwords gives some practical insights into how to properly protect such keys. Note that the following discussion is applicable to API Keys for things like OAuth credentials, and not encryption credentials (as discussed by @NickSloan in his answer).

The biggest threats to passwords are phishing attacks and hacked databases, which is why we store them hashed in our databases. The biggest threats to API keys are the same as for session keys: weaknesses in front-end applications such as XSS vulnerabilities, MIM, and the like. It's important to note that the front-end application needs the password in unencrypted form, so hashing the API key doesn't really gain you anything when the person who is most likely to have it stolen needs it in plain text.

Instead, you protect a session identifier (or OAuth token, or client-specific API key) primarily by keeping track of the client. If a session identifier get stolen via XSS or other means from the client, a typical result is for that identifier to be used by the attacker on a new device. That means that all of a sudden you will see an old key coming up with a new user agent. Such a change is easy to detect and fix: immediately invalidate all API keys. Especially in the case of OAuth requests, this is very painless for the user: they just log back in and immediately get a new one, while someone who only stole the key (and doesn't have the password) is back to the drawing board.

This is a common enough defense measure that a smarter hacker will also try to record the user agent associated with the stolen key and use it in all future requests, but you can still easily see something fishy going on when the same user has requests coming up from multiple IP Addresses. Like everything else in the world, this can be a bit of a cat-and-mouse game, but there are some important take-aways:

1. An API Key/OAuth Token/Session ID is really just a way of remembering a user on a particular device, so they don't have to put their password on every page. As a result, when in doubt, you can just kill all authorized keys and force the user to log back in.
2. In all three cases, since losing Key/Token/ID leads to an account being compromised, it is important to have some active security around these things and detect/respond if anything fishy happens. If you've ever gotten notices from facebook/google/etc about someone trying to login to your account from Mexico, it's because a security guy somewhere is doing their job right.
3. The primary attack vectors for Keys/Tokens/Ids are different than those of passwords. You may still want to hash them in your database (again, we're not talking about encryption credentials which need to be in plain text), but you can't just hash them in your database and call yourself secure. You need to make sure and secure them against a whole new range of attack vectors. If you think of them as passwords only, you'll miss some important parts of the big picture.

Answer 6

One of the main reasons passwords are hashed before storing them, is to protect against an attacker gaining a read-only access to the database. In this way, if an attacker gets hold of the login credentials list, they still won't be able use them directly because the passwords are hashed/salted.

And this is not just a theoretical problem we're trying to solve here, these type of attacks really happen (including to major players in the industry), and proper hashing does help mitigate the impact of an attack.

If you're lettings users authenticate using an API secret then the API secret has effectively became a password therefore, from a security standpoint, they should have the same protection mechanisms as regular passwords have.

Amazon itself now doesn't allow retrieving the API secret. If you forget it, you have to ask for a new one.

Answer 7

No it's not okay for some kind of services.

Implementation

1. Upon user connection with login password, generate a key derived from his/her password to encrypt data for this user and keep it in a session on the server.

That way, each user will have a unique key. This derived key must not be reversible, this is the main point. Using hash_mac or other hashing method with salt and/or master key will allows this.

2. Use this key to encrypt the generated API Secret using AES 256 with a random iv for each user.

Or

2.bis Use this key to generate the real encryption key at the very last moment and encrypt the API method using the same way as in 2. to avoid in memory floating passwords.

3. Store the iv in the data base along with the user id and app id in a dedicated table

4. Store the encrypted API Secret in the database in the appropriate table.

Usage

Now when the user connect to his control panel you can recreate the key, fetch the iv, decipher the API Secret for this user and display it in clear text. You also display the key and ask for both when making API Call so that you can verify the App Secret.

If you want to avoid to display the key and not have to request it on API call you can also for authentication use the same method as for login. Meaning you can create another table to store a random key/salt for each client app and store a hash_mac version of the API Secret. Thus only the API ID and API Secret are required to authenticate API Request. but it's more work.

Note

This is practically the same as @Polynomial answer.

This way it's very very difficult for someone hacking your server and database to steal your users credentials and make api call on their behalf. You can now assess that if a request has been accepted from a API client it's the right client or it's their leakage.

If the user loses his password then you regenerate a new API Secret and Key from his new password.