Is PBKDF2-based System.Cryptology.RFC2898DeriveBytes() "better" for Unicode Password hashing than traditional methods?

Question

When is it appropriate to use RFC2898DeriveBytes versus a typical hash?

Update

I now understand that a KDF is typically used to create a symmetric key for possible use in encrypting a stream. I also now understand that PBKDF2 obsoletes PasswordDeriveBytes. The intent of this question is to explore the possibility of re-purposing the KDF as a strong way to store a hashed password. The content I intend to encrypt is a Unicode string that a human can remember and type into a computer.

The reason I'm interested in the KDF is because it is

1. Slow, and therefore computationally expensive to create a rainbow table
2. The base library in .NET is FIPS approved and it's possible to use this KDF with NIST recommendations if we use SHA-256

Question

What are the arguments for/against using a KDF in this manner as opposed to the normal hashing method? (Disclaimer: What is the normal hashing method?)

Answer 1

PasswordDeriveBytes implements the PBKDF1 key-derivation function. A KDF is a function which transform a piece of secret data (here, a "password", i.e. the kind of data which fits in a human brain and can be typed with human fingers) into a sequence of bits adequate for algorithms which need a symmetric key (e.g. symmetric encryption). A KDF is not meant for anything else, in particular password storage.

A hash function can be used as a KDF, provided that the symmetric key you need is no longer than the hash function output size. However, such a KDF would be very crude. One feature which good KDF should provide is to be adequately slow. This is because "passwords" are, by nature, vulnerable to exhaustive search (also known as "dictionary attack": users tend to choose as passwords rather simple words or combinations which can be guessed with only a few millions or billions tries). In a given system, one can usually tolerate a relatively slow KDF: a user trying to authenticate will not see the difference between a 1µs and a 1ms delay for the key-derivation function; but a 1000x slowdown is deadly for the attacker: it converts a one-day breaking effort into a three-years breaking effort.

PBKDF1 includes an "iteration count" which is a parameter meant exactly for that: to make the key derivation adequately slow, in a configurable way. A simple hash function is just too fast for that. Usage as a KDF is precisely where you would prefer PBKDF1 over a hash function. Actually, PBKDF1 is not recommended; PBKDF2, from the same standard, is supposed to be more robust.

Hash functions are much more generic objects than KDF, and have many other usages, which KDF do not fulfill.

What you want to do is unclear: you use the term "signature", which normally means "asymmetric digital signature" as with RSA or ECDSA; there are some people who tend to use the term "signature" to designate a symmetric integrity check, such as a MAC (calling it a "signature" is improper, but widespread). However, this entails some piece of secret at some point, a key, and a hash function is key-less.

Answer 2

The NIST approves of PBKDF2 when hashing and storing passwords, however that is not its original intended purpose. Notably, StackExchange also uses PBKDF2 for the same purpose. Source code is available here.

See this answer for a comparison between BCrypt and PBKDF2. BCrypt is the the more conventional method of storing passwords.

I'm considering PBKDF2 since it's built into .NET and is already FIPS compliant.

Answer 3

Unless I've missed something, PasswordDeriveBytes - and other PBKDF implementations - are not intended for storing passwords, nor are they to be used instead of a "typical" hash.

What it is intended for, is to create an encryption key, for symmetric encryption, based on a user-provided password.

To clarify, consider the following situation:
You have an application, that requires encrypting a file. This is done at the user's discretion, and can be decrypted at his choice.
Oh, let's say MS Word has a feature to encrypt it's contents. Or an encrypted ZIP file.
You want to give the user the control to access it, but don't want to rely on the user to generate a strong, random, exactly-256-bits-key, which would make it difficult to remember, etc.
So, you allow the user to set any password he wants - and derive the encryption key from that.

At no point are you storing the password, or using a simple hash to store it. It is only intended for creating key material.

Answer 4

PBKDF2 is better than PBKDF1, which was deprecated in RFC 2898 10 years ago. It is implemented for .NET in Rfc2898DeriveBytes Class (System.Security.Cryptography)

I'm glad to also see that as of Java 6 there is an implementation of PBKDF2: PBKDF2WithHmacSHA1 in SunJCE.

For earlier versions of Java, as discussed at BlackBerry App Security - Stack Overflow, an implementation of PBKDF2 is at A free Java implementation of RFC 2898 / PKCS#5 PBKDF2

I wonder, though, if using SHA-256 would be better than SHA-1, which is now deprecated.

Answer 5

Using RFC2898DeriveBytes with a non trivial iteration count should be better than using a straight hash function for authentication purposes.

First it requires you to put in a salt, and while it's still possible for the developer to do something stupid like hard code the salt, it's at least one step closer than where developers normally messup which is not salt and all. Just having a salt makes its less likely a rainbow table already out there is going to work on someones trivial password and if you salt randomly for each user it stops leakage of information when multiple users have the same password and makes those easy passwords very hard to get because you'd have to generate table for each salt.

2nd while RFC2898DeriveBytes unfortunately doesn't let you choose the hash function and it uses HMAC-SHA1 and it's not really a problem as the nice thing about it versus PBKDF1 is then size of your key can be as large as you practically could want. So that helps with making it difficult to store and disseminate rainbow tables vs any standard hash algorithm.

Finally, the iteration counts are key! Have a really high count slow that hash calculation down, standard hash algorithms are designed to be fast, data storage is growing exponentially you can't just count on just the size of your key.

Answer 6

The OP's question and the goal of said question are somewhat contradictory. As everyone is pointing out, PBKDF2 is not primarily used for password enc/storage, but key generation. Most commonly, it is used in WPA2 PMK and other WPA key generation schemes, using such example components as the password + a salt of the SSID name and SSID name length, which is then run through a 4096 iteration SHA1 hashing.

When encrypting passes, the most important things to require for security sake are length and a salt that is dynamic and not transparent to the user (ie: the exploiter cannot determine what the salt is). If you use an AES encrypted store with a minimum 20 character password, a random salt, and 4K+ iterations of SHA-256 (PBKDF2 recommends at least 1K iterations of SHA1 for its purposes), a hacker is going to have a very difficult time breaking that password (see: order of magnitude in years or decades if not longer).

Password encryption is also highly dependent on end use (speed, necessity of security level, etc), so without knowing what potential end use is it's more difficult to recommend an ideal solution.