So this question is delving into security and encryption and the problem potentially hasn't been encountered by many. Answers may be theoretical. Let me outline the scenario...

  1. A website frontend is driven via a backend API. The backend has an endpoint handling a generic registration form with username and password. It's using SSL.
  2. The backend API handles registration via an async job queue. The queue does not return responses to the API server. It's a set and forget operation to queue up the registration.
  3. Queued jobs are picked up by workers. The workers take care of creating the user account. These workers need access to the plaintext user password so that they can trigger a third-party API registration call with the password.

So the real crux of the problem is the syncing of the password to the third party API while not revealing it to prying eyes. The queue poses the problem of not having direct access to the plaintext password from global POST data anymore, meaning it needs to be stored in some fashion in the queue.

The queue can easily store the hashed password and copy it directly to the users table. This solution does not allow for syncing of the password with the third party API, however, as it's already encrypted. I toyed with two-way encryption, but am whole-heartedly concerned with leaving the password prone to decryption by an attacker.

Can anybody think of a secure way to handle this scenario of password syncing?

The queue is a requirement and it's assumed that this is readable by anyone with access to the server. The passwords don't necessarily have to be synced; the password for the third-party API could be a derivation of the original so long as there's a secure means to decrypt via the logged in user without supplying their password. This is essentially to simulate Single Sign-On with a third party API that does not support SSO.

  • This question is a better fit for Security.SE. I recommend that you click 'flag' and ask the moderators to move it over there.
    – D.W.
    Mar 13 '13 at 9:08
  • Would you please elaborate on point 3 as it seems to be the crucial point of your question. Why would the third-party API require clear-text password in this theoretical situation?
    – Adi
    Mar 13 '13 at 14:10
  • @Adnan: Because it is third party. He has no control over it. Most third party APIs take plaintext passwords as input. Mar 13 '13 at 15:08
  • @Adnan I am performing account creation on the third-party website via their API and I also need to support password syncing between our user accounts and the third party user account. Mar 18 '13 at 13:35
  • Does the original password really need to be sent to the third-party API? Why?
    – titanous
    Mar 18 '13 at 13:53

This appears to break down to simply;

  • I have a job queue containing plaintext passwords.
  • I would like to prevent other server users from accessing those passwords.

This seems like it should be quite easy, assuming 'other server users' does not mean a superuser account.

What you need to do is engineer it so that your frontend <-> jobqueue link acts as a diode, jobs are able to go in, but no data can egress from the jobqueue in the direction of the front end.

Some simple examples;

  • The job queue exists in a database to which the front-end does not have access. The data is encrypted under a key generated at run-time.
  • The job queue exists only in RAM in a process marked as untracable (in Linux you will need AppArmor and I believe this is possible by removing the capability CAP_SYS_PTRACE)
  • The job queue does not exist - make the back-end calls synchronous.

That said, doing anything other than immediately hashing plaintext passwords is antipattern imho.

  • The job queue uses Redis as a backend. A superuser is exactly what I'm fearful of (somebody gained remote SU access). In most cases, a strongly hashed password is unencryptable, but this is obviously not my case. Mar 18 '13 at 13:36
  • @cballou, Protections against superusers will be incomplete. Once the solution is implemented, the superuser could adjust the implementation to copy off the plain-text to a file. That doesn't mean you should give up on implementing this security feature, but what I suggest is that you prevent the use of superuser. You could ask a separate question like "How can I give users limited superuser permission for only permitted tasks?" Mar 18 '13 at 14:16

You should use bcrypt to hash the password and discard the plaintext as soon as it hits your frontend's API server.

For the third-party API password, use a cryptographically secure pseudo-random number generator to make a long random password that you store in the user record. Depending on the sensitivity of this API, you may want to symmetrically encrypt the API password before storing it in the database. I'd suggest using Keyczar to manage the encryption if you want to encrypt the API passwords.


What I don't get is why you're propagating the same password to the third party API. That's not Single Sign On. That's Single Password Stupidity.

Instead hash their password and store it. Then generate a random password for the user, one per third party sign on. And sign on using that. That way, only the individual site passwords go into the queue.

Looking at your actual question: This is impossible. I assume the job queue and the backend API are on the same server. If you don't trust the queue's security, you can't really trust the backend security either. And you have to store the password somehow, somewhere in plaintext, or in a format from which your server can extract the plaintext password.

  • It's not SSO; it's mimicking SSO functionality. Some providers we use have true SSO while others we're forced to work around to provide what appears to be SSO. Mar 18 '13 at 13:37
  • @cballou: No, it's only superficially similar to SSO. The main reason folks use SSO is so that they don't have to use the same password on all sites (a major security vulnerability). Your program does the same thing, just that the user doesn't have to enter the password again and again. Instead, have your program generate separate passwords per site. Mar 18 '13 at 14:04

This answer is different from the earlier questions that suggest you should hash the password, store it in a database, etc. Those solutions are needlessly complex in my mind.

If you're front-ending the 3rd party API everywhere, then you can/should seed and hash the password yourself and simply send that hashed blob to the backend. One approach could look like this:

  1. Hash the password with BCrypt / Scrypt with a salt
  2. Send that hash to the back end as the user's password, and update it when the user changes their password.

  3. Use the same hash solution for your authenticating webservers

In other words, don't send the 3rd party API a clear text password, send them a hash you create. Just be sure to re-create this hash format on each and every validating webserver or webservice.

Then when you proxy the authentication requests to the 3rd party API, take the HTTPS POSTed cleartext PW, hash/seed it, then send it off to the API for further validation.

This approach will prevent the casual observer of the queue from impersonating a user, and the salt will prevent a rainbow table from being created.

  • It's not a good idea to make up new password hashing systems. Use something that's designed for the task and includes salting. bcrypt, scrypt, and PBKDF2 are your best options here. Also, passing a hash of a user's password to a third-party API is a very bad idea.
    – titanous
    Mar 15 '13 at 19:36
  • @titanous I agree one shouldn't invent their own crypto. But why is it a bad idea to hash a password before sending it to a 3rd party API? As long as it meets length and complexity limitations I think thats' a fine thing to do. bcrypt/scrypt/pbkdf2 are meant for the offline database file. They create LONG passwords that may be truncated or refused at the 3rd party. RIPEMD creates a small hash that should fit. Mar 15 '13 at 21:38
  • The problem with the scheme you originally proposed is that was unsalted, so it's vulnerable to rainbow table attacks. You want to use a key derivation algorithm to slow down brute force attacks (GPUs can run hundreds of millions to billions of SHA-1 hashes per second). The most secure solution is to generate an entirely new random password for the third-party API. Odds are that there is a database in play somewhere, so there's no real complexity to this approach.
    – titanous
    Mar 16 '13 at 0:48
  • @titanous You're very correct. I used the words "seed and hash" in every revision of my answer, but illustrated an algorithm that didn't do that. I remember changing my mind 1/2 way though thinking "it's not that important for a temporary queue to have seeding and hashing" mainly thinking that the number of bits created by that would be too long. But this is a question (how do I truncate or shorten a hashed pw) for Thomas Pornin. Mar 16 '13 at 1:27

Wait, unless you have you kernel configured to allow root to view any memory in the system, why not just use pipe()? It's stored only in volitile memory, it is secure within the system, and supports both synchronous and asynchronous data transfer.

If the queue is just for connecting workers directly to the API, it seems like there would be no practical way to exploit the system.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.