Generate API keys with JWT and regenerate the same key when needed

Question

I recently found my self in need of developing a public API for my application. I developed my application with node.js and MongoDB. After some research, I decided to use jwt for generating API keys for users and authentication. For authenticating jwt tokens, they come with the benefit of not needing to store them in a database, as they can be decoded and verified without knowing the exact generated token.

But I see that lots of applications show the users their API keys in the application dashboard, so I need to store the token to show them to the users later on. I know that storing tokens is a bad idea and in case of a database breach, it could let the hackers impersonate others with the API keys.

So long story short I am trying to find a way to not to store the exact tokens, but store only the payload in the database and every time users request their API keys I just generate the same one with a SECRET and pass it to them. I currently find that if on the signing token step, I pass the same payload with the same iat (issued at) every time, the generated token will be the same every time. So by saving the iat with the payload data in the database, I can generate the exact token every time.

---

Now my questions are:

• Does this approach is good or is there a better way to achieve this?
• Is there any good practice for generating API keys without storing them?
• Does this even necessary (considering if there ever be any database beach, all of the data is already stolen)?
• Is there any method other than using jwt to achieve this?

Answer 1

tl/dr: A JWT is simply not the correct solution for a server-to-server API token/key. API keys need to be revokable in the case the user needs to change it (for whatever reason), and JWTs by their nature cannot be revoked. You need a completely different solution for server-to-server authentication.

I'm going to answer with a frame-challenge.

You are actually talking about two completely different kinds of authorization tokens with two different purposes. You don't want to figure out how to regenerate the JWT and show it on the user dashboard because a JWT is not what you want on the user dashboard anyway.

Understanding JWTs and their use cases

JWT's have two properties that distinguish them from the other kind of access token that we'll talk about: they can't be revoked and they are typically short-lived (which is a necessary consequence of them being non-revokable).

The fact that they can't be revoked is a natural consequence of the fact that you can verify them without checking the database. Everything you need to check the JWT is built into the JWT, and the only way they could be revoked is if you marked them as such in a database and checked the database everytime you verified the JWT (which would defeat its purpose). Therefore, since they are effectively irrevokable, it is best for them to have short life-times to minimize the damage done in the event that an attacker steals one. As a result JWTs typically also come with a refresh token, which is longer-lived and which the user can use to generate a new JWT when the first one expires.

This combination of properties gives JWTs a particular use-case: authenticating a user who first logs in with a username and password. The idea is that when the user logs in the application gives them a JWT and refresh token, which the app then uses to keep the user logged in for as long as they need (aka the phone app or website use the refresh token to automatically get a new JWT as needed, and send up the latest JWT with all requests).

Note, if your JWTs don't have a short lifetime, or don't expire at all, then you are leaving a potentially severe security vulnerability in your system, since an attacker who gets one effectively gains permanent access to the account even if a password is reset. Since JWTs go to the actual client (aka a mobile app or a web application), they are much more vulnerable to theft by outside actors, which is why short lifetimes are necessary.

API access tokens

Of course there are a wide variety of ways to manage access tokens for APIs, but you mentioned systems which show the API token on the dashboard. These typically have a different set of properties. In particular, they are long-lived and revokable. They are long-lived because they will be used for server-to-server communication. This means that they are often sitting in a configuration file (or similar) and so you don't want to force your end users to manage the refresh process themselves or update the token without good reason.

As a result, revokability is very important. If the user decides they want a new access token (for whatever reason), they need to have an easy way to generate a new one. In more advanced systems they also come with their own privilege system, and the admin can generate as many access tokens as needed and assign privileges to them. This gives the users the tools they need to manage the security of their own accounts.

Bringing it all together

In summary, these things are two different kinds of access codes for two completely different reasons. A JWT is a short-lived access code because it goes down to an end-client where it is at higher risk of being stolen. A new one is assigned everytime the user logs in, expires shortly thereafter, and is automatically refreshed by the client.

A server-to-server API token is long-lived and only stops working when explicitly revoked by the customer. They can copy it into the configuration information of their server and only change it if necessary. It never expires and doesn't have to be refreshed.

These are two completely separate authentication needs - the way you authenticate a front-end client is not the same way you authenticate server-to-server communication, because the two communication channels have different security needs. You're trying to fulfill both needs with one "kind" of authentication, and the reality is that trying to adjust your JWTs to work for server-to-server authentication as well is just a bad idea. If your user's need to interact with your system in two different ways, then you need to provide two different kinds of authentication. In particular, it sounds like you're not planning on allowing your JWT tokens to expire (so that they can be used for server-to-server authentication), and that's a very bad idea.

The solution

So how should you generate the API keys for the user? That just depends on your needs, but since I don't know what those are I'll give you a suggestion that may be overkill in some cases, but sufficient in all cases.

The most secure solution is to allow the end-user to have multiple keys. When they request a new key generate a public/private key pair, let the user download the private key, and then throw away the private key on the server side. It is from then on the user's responsibility to use/store/copy it as needed. The server keeps the public key and requests are signed and verified using standard asymmetric cryptography methods. This also allows the user to generate multiple keys if needed, potentially with different access levels.

There are of course simpler ways to do things, but I would say that it's a bit out of scope to try to choose the "best" authentication method for you. Everyone has different needs and needs to balance security/usability/cost for themselves.

Answer 2

Does this approach is good or is there a better way to achieve this?

I believe there is a better way. When a user signs up for an API key, display it to them once via the GUI and have them copy it. Then one-way hash it into a database. The only way to retrieve it at that point is to create a new one. Think of what you might need to do to secure any data at rest.

Obviously the user will need to create the claims on their side and then encode it with base64. Your API Endpoint should be using SSL which handles the security of the data in transit. Don't store any of the claims in your database. How would you expect to track accesses to your API if the iat (Issued At) is the same every time?

I WOULD definitely recommend using an expiry on all tokens generated by your endpoint for security.

I'm actually going to steal a link from another SO answer that does a pretty good job of explaining a safe storage option using HMAC and both an API key and a Secret key: https://stackoverflow.com/a/37953361/9199874

Is there any good practice for generating API keys without storing them?

I got nothin'

Does this even necessary (considering if there ever be any database beach, all of the data is already stolen)?

My answer to number 1 could reduce risk if this does happen. Risk is impossible to eliminate completely.

Is there any method other than using jwt to achieve this?

I've used JWT as an API developer and I like it. You may also find some values in custom claims if you're looking to include different scopes within your API calls.

Answer 3

Short answer: Decode the token locally. This is most likely what those other applications / dashboards you mention are doing.

Do ask the question of why you would want to show the token to the user. Maybe just display when the session will expire, and/or when they last authenticated (exp and iat fields in the token)

Unless you specifically need the token to be encrypted, don't. Just sign it and let the "dashboard" decode it to display the payload to the user.

If you did want to store something, store the payload without the signature.

Whether storing this is a security risk depends on what is included in the token. But that information is probably in any case available in the database already. The general practice of not overloading your tokens should come to mind - store the user id (sub field) but don't store the password or username needed to log in.

Secondly storing JWTs isn't a bad practice per se - You would however need to apply the same risk mitigation as you would when storing passwords if you did. Storing tokens comes with a big gotcha - it defeats many of the benefits of having JWTs - one of the biggest benefits is that you don't need to store them.

Note: Users / applications should not request an "older" token, and systems should not lie about when a token was issues. iat should be "now" and the token should be fresh if it is requested anew. So every time the user authenticates (or refreshes their session token) issue a new one, complete with new iat and exp.

If your autentication server - the entity that issues tokens, are separate from the service or system that consumes the tokens, do make sure that you use a public/private key scheme for signing the keys, and never copy or transfer private keys from one system to another.

Encrypt the token only if it includes sensitive information (such as the user's password) but seriously question why you want to do that. You would probably be better off using a basic authentication scheme over a secure transport layer.

Note that a token is usable without being decrypted by the client, so encrypting it doesn't mean that a malicious third party cannot use it - if they can get a hold of it, they can use it.

Keep your tokens lean.

Answer 4

As you said, with API key you mean secure code similar to what Google Map and Facebook use for access to their API.

Then I'd suggest you to consider following.

1. Don't embed API key into JWT. Embedding contradicts to the whole idea of JWT. The idea of JWT is that a token is public and can be read by anyone. It should not contain any secret information like API key (in your definition of API key). Instead of embedding consider following approach. In the login request user sends his API key (secret code). You validate it and create a JWT with content like "this token allows access to the service X till 10:35 AM today", but you don't include the API key into the JWT. When service receives this token, it does not need to know why access is allowed. The service should only validate if the toke is valid (not outdated, signature is correct, ...), and then execute the request.

2. Don't store API key on server. This best practice is similar to handling password on the server side. When you generated an API key to a new user, dont store it on the server as is. Instead, transform it using key stretching algorithms like Argon2, scrypt, PBKDF2, depending on what your platform and tools.

3. Use client to display the API key. You say you want to show the users their API keys. A) Send API key only when needed. One solution would be to ask client code to send the API key only for particular request, e.g. only when user is opening the dashboard. You would render dashboard on the server without necessity to store the API key in the database. B) Use client side communication. The client application that calls your API is cannot pass any data to your dashboard part of DOM because of cross origin restrictions. But you can use HTML messaging. The client application can send an HTML message to your part of the application (dashboard) and pass the API key. Your dashboard receives this message (on the client side, within a browser) and displays the API key. The implementation is simple, less than 10 lines of JavaScript. It has the advantage that the API is not sent to the server (except login).