Would a Password Using All Possible Unicode Code Points Cause Problems in Traditional Hashing Algorithms, such as Bcrypt?

Question

I've been toying around with this idea, but hypothetically, if you had a password manager that would use any possible renderable (e.g. not control characters such as BEL, NUL, DEL, etc., or surrogate characters) Unicode code points, and a site backend and hashing algorithm that would accept them, how many bits of entropy would a sixteen character password provide?

I'm not 100% sure I'm calculating this correctly, but according to this StackOverflow answer, there are 1,111,998 possible valid code points for this purpose, so each character should provide log2(1111998) ≈ 20 bits of entropy per character, correct?

Would this break any common password hashing algorithms such as bcrypt? Are there any unforseen problems in using unassigned unicode code points in passwords, save for some sites restricting the character set available to use?

Answer 1

Not really. Hashing algorithms and Key derivation functions (side note: bcrypt is a KDF, not hash) works on bytes, not characters. This means instead of working on Unicode, it sees instead the utf-8 encoded bytes (or whatever encoding you use). So the hash/KDF itself wouldn't have any problems processing Unicode passwords.

However, Unicode had many different representations for the same characters. This is due to combining characters. Different OS and different text entry methods may produce different code points for the same characters. Application developers need to ensure that Unicode characters are normalized before passing it to bcrypt.

Another potential complication is that there are a different number of possible characters between unnormalized and normalized Unicode string. To produce a uniformly random Unicode password generation method, you need to generate uniformly over the normalized characters otherwise your password generation method may bias towards the characters that have multiple representations.

Answer 2

BCrypt won't work with a NUL byte because it is reserved for its own internal use (I think to keep track of the end of the password or something).

Other than that what BCrypt really does is hash an array of bytes, not characters. From its point of view, its just blending up numbers and whether those numbers correlate to a presentable string in one character set or another never plays a factor in the algorithm.

This is also true for all other major hashing algorithms (as far as I'm aware).