I am currently designing a login for a web service. I will use a PBKDF2 implementation for hashing the passwords.

However, I intend to allow unicode for passwords, as I will have international users, which might want to use, for example, cyrillic characters. To avoid any issues with unicode ambiguity, I thought of using the NFC unicode normalization before encoding the password as utf8 and passing it on to the hash.

The question now is: Is that safe or does it introduce any unwanted ambiguity into the password validation? It is clear that "a\u0308" (a + combining diaresis) and "ä" should be the same, but does NFC fold any more differences which users could be relying on?


I found that there is a stringprep (RFC 3454) profile called SASLprep (RFC 4013) which is appearantly used for passwords and usernames in some protocols. It specifies to use a KD normalization, which I consider a bad idea. It will fold differences like ² and 2, which are two characters commonly on keybords in the western world at least, which could be used to enrich the password entropy. Unfortunately, no rationale is given for that.


If you treat 2 and ² as the same character, you're essentially removing one character from the character set. That isn't really so bad if it increases usability, especially if that encourages longer passwords.

Say you take a 8-character password, with a drawn randomly from a set of 2000 characters. That gives log₂(2000⁸) ≈ 88 bits of entropy. If you had a 9-character password, drawn from 1000 characters (half as many!), that's log₂(1000⁹) ≈ 90 bits of entropy. In fact:

|     |    character set size     |
| len |  500 | 1000 | 2000 | 4000 |
| 6   |  54  |  60  |  66  |  72  |
| 7   |  63  |  70  |  77  |  84  |
| 8   |  72  |  80  |  88  |  96  |
| 9   |  81  |  90  |  99  |  108 |
| 10  |  90  |  100 |  110 |  120 |
| 11  |  99  |  110 |  121 |  132 |

As you can see, in the normal range of password lengths and Unicode character set sizes, the exact size of the character set isn't that important.

  • So you are encouraging that users with a password like E=mc² are also allowed to log in using E=mc2? – Jonas Schäfer Feb 26 '14 at 9:34
  • 2
    @JonasWielicki well, apparently that RFC does. The benefits to doing so are fairly obvious (e.g., can the user manage to type ² on a smartphone?) And it turns out the downside is fairly minor. – derobert Feb 27 '14 at 15:33
  • Sorry for coming back that late. I see your point with respect to the character set size, but this does in my opinion not cover the risk from accepting multiple distinct passwords for one account. From my understanding, it would follow that the entropy introduced by two characters is not neccessarily equal. In any case, applying these semantics requires to inform your users about this, so that they are aware of the subtleties. One could do that, e.g., when setting the password, by comparing the normalized version against the non-normalized and showing a warning in case of mismatch. – Jonas Schäfer May 6 '14 at 11:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.