How is storing an API secret key in plaintext (in a database) secure?

Question

The answers to this question Is it okay for API secret to be stored in plain text or decrypt-able? are somewhat unsettling to me. I'm trying to wrap my mind around how storing a secret key in plaintext is in any way secure.

This is how I'm imagining an API access key / secret key based system to work:

1. Client takes text of request and applies HMAC function with secretKey, resulting in signature
2. Client adds signature and accessKey to request (effectively as authentication headers) and sends to server
3. Server extracts accessKey from request and looks up corresponding serverSecretKey
4. Server takes text of request and applies HMAC function with server's version of serverSecretKey, resulting in serverSignature
5. Server compares signature to serverSignature; if they match, great, we're good - we know that the client has both a valid access key and a valid secret key

So before I go on, if the above is incorrect, then the rest of this question will more or less make no sense - so please let me know if the above understanding of API access/secret keys is valid or not :-)

Assuming my understanding is valid - my question comes back to how the secret key is stored - it has to be stored in plaintext for the above system to work.

So what happens if an attacker gains access to my database that has my all of my clients' API access keys and secret keys? Wouldn't they easily be able to use any access/secret key combination they like, and happily impersonate any client?

I'm looking at this from the perspective of bcrypt-ed passwords. If an attacker gains access to a database with a whole bunch of bcrypt-ed passwords, they have a whole lot of work ahead of them to make that database worthwhile to them, since brute forcing a set of bcrypt-ed passwords is computationally expensive. That feels a whole lot more secure than storing secret API keys in plaintext in a database somewhere.

It seems like an API access key/secret key combination really only provides protection against tampering with a message (since the digital signature computed during steps #1 and #2 above is tied to the secret key) and doesn't really provide any assurance that the client is who they say they are. I don't see how it provides a level of security comparable to bcrypt-ed passwords.

What am I missing here?

Answer 1

Your thinking on this is too black and white: safe or unsafe. There's no safe here, just a spectrum of risk.

Companies that provide these APIs are making a judgement call and trying to balance risk with performance. Not all of these API calls need to happen over SSL. Think about uploading an image to S3, does it need to be encrypted? Probably not. The key requirement is that the calls are authenticated and authorized. Think about a million distinct calls to SDB or SQS, how much overhead on the client and server side will SSL create? Basically SHA2 is fast and SSL expensive, now multiply by 1,000,000.

Is there risk involved with keeping the API keys in plaintext on the server side? Certainly, but these companies are making a bet that they can put other mitigations around those key stores that lower risk to an acceptable level. (Ultimately if I get access to your key store, hashed or plaintext, there are several malicious things I can do like swap in my secret key for yours.)

About your statement: "doesn't really provide any assurance that the client is who they say they are"

That's true about almost all authentication schemes. If someone logs into this site with my username and password it doesn't prove that they are me, just that they possess my credentials. If someone uses my thumbprint to unlock a biometric lock, it indicates that they have some way of providing my thumbprint in a way the lock will accept it. What changes here is how difficult that is. Stealing credentials may not be that hard, impersonating thumbprints is probably harder but not impossible.

Answer 2

I recently added an answer to the other question that I think is also relevant here. I've edited my answer slightly though to more directly address your question:

There is one more important difference between an API key and a password. API Keys are unique to your site. The part that make password security so critical is password sharing. If someone gets a hold of the password that a user saved on your site, it isn't just your site that is compromised: it is every other site that the user used that password (and email/username) for.

An API Key is a completely different scenario because it is unique to your site. As a result, if it is stolen, the attacker gains access to your site, but gains nothing that they can leverage against that person's bank/email/etc. This means API keys are more akin to session identifiers than to passwords.

From a security standpoint you will look at the problem more clearly if you think of API keys not as passwords but as session ids, especially in cases where API keys can automatically be assigned (OAuth and the like). Yes, API keys can get stolen which will result in your account being compromised, but the ways in which keys are vulnerable to being stolen are very different than the ways in which a password can be stolen. In particular, a password is (theoretically) only stored in two places: the user's head and your database. As a result your database is the biggest threat to password security (if we ignore phishing attacks), and password re-use makes the actual danger to end-users very high in the case that your database is stolen with plain text passwords in it.

In the case of a session identifier or API key though, none of that analysis is true. API keys are stored in your database, but they are also stored in whatever client is using the password. This might be in the memory of a front-end javascript application (which is vulnerable to XSS attacks), a configuration file on a server that is integrating with the API (which is vulnerable to a completely different set of attacks), or who knows where else. As a result, the database is no longer the primary attack vector from which to gain access to API keys. This definitely changes the security priorities.

To make one more point, session identifiers is the proper perspective to view API keys from. Any website that relies on sessions to store user data typically has a session id that is also stored in the database and stored in a cookie with the user's browser. As a result, if your database is hacked and downloaded, your user's sessions are completely compromised. A malicious user could grab a session id, edit their cookie for your website, and be logged in as any user with almost no effort. Hashing session ids before storing them in the database would stop this particular attack, but it would do nothing to stop the XSS attacks that are more commonly used to steal sessions.

Answer 3

In short it's not, however the secret key does not necessarily need to be stored in plaintext on the server. It could itself be encrypted (but not hashed as others have explained) with information that is part of a client request.

Similarly, if you login to a service that displays your secret keys, the keys need not have been stored in plaintext as they could be encrypted with a one way derivative of the password that was used to login, and which is kept associated with the session. While the secret key could be discovered given access to the session data, only a subset of client secret keys would be at risk at any one time.