How Facebook hashes passwords

Question

I found this presentation how Facebook stores customer passwords and how they do authentication.

This slide shows how they hash passwords:

Is it a good idea to do such operations with a raw password? Isn't it "security-by-obscurity"? Does it make sense to do such operations with PBKDF2 as we're going to use?

Answer 1

In one of the comments you wrote:

I'm asking if something like it makes sense now in 2018. We're going to use PBKDF2 HMAC-SHA-256 with 8 bytes salt, we're discussing what hashing schema use.

The answer to that is no. The only part that should be replicated from the FaceBook solution is using a memory-hard function and the cryptoservice peppering. The MD5 and HMAC parts don't add to security. (They are fairly harmless though.)

You should not be using PBKDF2. In order you should prefer:

1. Using a binding to the official native optimized implementation of Argon2
   • Refer to the Argon2 spec's "Recommended parameters" section for how to choose parameters.
   • The choice between Argon2d, Argon2i, and Argon2id may be up for debate. The current official recommendation is Argon2i for people that don't know which one to choose.
2. Update your software so you can choose 1
3. Protecting encrypted data using a password and can't use Argon2? Use a binding to a native optimized scrypt implementation. Set the single cost parameter high enough to use a lot of RAM and time. If it seems safe to reduce the time scrypt runs then reconsider. That also reduces the RAM-based cost.
4. Protecting a few passwords, can't use Argon2, and can tolerate slow responses? Use scrypt with the same cost parameters as 4. If you're trying to protect admin accounts then prioritize using very strong passwords and encryption first.
5. Protecting many users' account passwords and can't use Argon2? Use a library that uses a binding to native optimized bcrypt. Make sure the library correctly works around all of the original bcrypt algorithm's quirks. (The password will get truncated if it's too long or contains a null character.)
6. Use a binding to native optimized PBKDF2. Don't use a derived key length greater than the underlying hash function's output length.
7. Don't implement your own password hashing function. And especially don't run that implementation in an interpreter where it will be much slower. Update!

As for whether the FaceBook method is security by obscurity. No. But the HMAC parts and probably the MD5 part are redundant. The cryptoservice part probably uses an HSM. This should allow password hashes to be encrypted/decrypted without necessarily exposing the encryption key to the type of hackers that can steal a password-hash-database. Using mainstream encryption algorithms with a secret key does not count as obscurity. The password still needs to be hashed in case the key is ever leaked. That's what scrypt is for, but now Argon2 should be preferred.

Edit: See comments. As presented in the slide, the password does not get encrypted. HMAC is one way, just like hashing. The alternative of encrypting after hashing is useful because it allows the key to be replaced. Users need to enter their password if you change the key with this FaceBook setup, in addition to the salt or scrypt parameters. If you encrypt the hash then you can decrypt and re-encrypt it without requiring the user to enter their password. (If your key is ever leaked... But you can't rehash a password without the original.)

Also use 20 bytes of salt. Or at least 16.

Answer 2

This "Onion" construct can be dangerous and should only be done if you have enough reviewers, it does however add multiple favorable properties compared to simple hashing:

The first unsalted MD5 can be done by clients or frontend servers. It normalizes charset and length and obfuscates passwords early on without being too CPU intensive.

The salt is read from the user database so it is not present on the client, this is was the first HMAC adds.

The crypto service (or better HSM) defends against brute force (as the secret key is not known). In fact this predates NIST recommendations to do exactly this. This peppering is not often done as it requires complicated system setup (HSM or process isolation) - something Facebook can afford.

Since the pepper step could be compromised by insiders, the data is then passed to scrypt, a function designed to make brute force testing of passwords more expensive.

The final step might not be needed, or maybe it is used to add some diversification (SHA2 instead of SHA1).

It looks a bit like snake oil overdo but the steps can be reasonable explained. If you compare it with the (much weaker) glibc-SHA2 Voodoo it is really clean.

BTW1: memorable Authenticator are generally a bad thing, such schemes can only reduce the risk of handling them.

BTW2: collision weakness of SHA1 does not apply to HMAC constructs and the short MD5 hash size does not hurt much here (as passwords almost always have even less entropy). It would be more risky for key derivation, but that’s not the task at hand.

Answer 3

The reason for the many layers is that over time requirements have changed.

Based on a talk I was present at, the following additional info may assist you in understanding the whole process:

Plain MD5 hashing was the original way of storing the password for the service. This was known to be weak, but short of getting everyone to log in and change their password or to have two (or more) different password formats in the database, the only way to improve security was to wrap the previous hash in another step. As it is layered it is called 'the onion'

You will notice that they did this by adding a properly salted hash as was recommended at the time.

The cryptoservice is used to detect patterns of attacks - either from one or more endpoints/servers/services or for a specific user. This allows proactive detection of attacks and mitigation.

The scrypt step provides a way to make hashing harder over time. As CPUs get faster, you can increase the hardness and make the total hashing time take around the same time.

The last HMAC step is mostly to just make the resulting hashes more uniform for database storage.

Is this a good idea for new deployments? No, but it was done only due to the scale and unique problem that Facebook has.