Why do some passwords not allow certain types of characters?

Question

Is there any programing related reason why some password cannot have certain characters? If the idea in storing a password is to store it's hash, since a hashing function can take any input (ok at least I guess any reasonable hashing function can take punctuation) why would a program not allow some passwords? Does this mean a hashing function is not used, then what is?

An example is LAN Manager

Passwords are not case sensitive. All passwords are converted into uppercase before generating the hash value.

When I worked for IBM we had to fill out the hours we worked by entering them into mainframe running z/OS. The password for this had to be a very particular format, something like 3 digits followed by 3 letters. I asked if the passwords were stored in plain text and the answer was "no but the system is ancient".

Answer 1

Any restriction of any shape or form on the passwords will decrease their entropy, making them weaker. Fewer limitations = more possible combinations. There's no technical reason why passwords should be limited in anyway*. It's not a programming related issue, it's a chimpanzee-related issue.

That is especially the case if you're properly and securely hashing your passwords.

---

* In some very old and legacy systems, there might be some technical limitations related to encoding, acceptable characters by the operating system itself, and so on. Of course, that doesn't exist anymore, and there's no reason for us to imitate the behaviour of those old systems.

Answer 2

In the LM hash, passwords are case-insensitive because it was to be used with a operating system (Windows) which is supposed to be case-insensitive throughout, and Windows users are trained to disregard case. This was a rather logical move, if misguided in the case of passwords. They (partially) learned from their mistakes and switched to MD4 over UTF-16, thus supporting much longer passwords and arbitrary characters.

Having non-ASCII characters in a password is not necessarily a good idea. It implies some issues when it comes to typing these passwords on foreign keyboards or smartphones. There are also Unicode decomposition issues: an "é" character can be, at Unicode level, one or two code points, so being able to type an "é" on a keyboard does not mean that you get the right one. Some system designers and administrators restrict possible characters on the basis that they should protect overzealous users against themselves.

Answer 3

Either storing in plaintext or encrypted (but not hashed), wanting to avoid having to properly sanitize data input, or for convenience or compatibility. Those who store their passwords in plaintext will often put length and/or character limits in place because of restrictions on their database; the same applies to people who use reversible encryption. If they want to avoid having to properly sanitize data input (which isn't much of a problem for websites, but other mediums it can be), then they'll put a strict limit on the characters allowed so that nothing can be used for injection attacks. I won't go into any detail on convenience or compatibility, since the other answers already cover that.

Answer 4

As @Adnan aptly point out, there is no good reason to restrict characters, but the practice isn't going to go away soon.

One of the most egregious examples of such restrictions is what is done in the x-cart shopping cart system that many websites use. It silently truncates passwords are <. So if you give it a password like lwB<Ln#q5iDVnW!K&ZQ0u(zD, it will treat your password as lwB. Steve Thomas has described this, but I can't find the precise source, so I will just credit him in general.)

It is clear from errors like this what the original intent was. Someone threw in a password parsing rule to prevent XSS. Obviously there are much much better ways to doing this, but this shows that some of these policies are an attempt to sanitize user provided data before doing any further processing of it.

US-ASCII only

There are good reasons to restrict passwords to US-ASCII. A user may have something like ü in their password, but sometimes they will be providing that as UTF8 and other times as Latin1 (or any other set of encodings). The user may be unaware of such distinctions.

Allow whitespace?

There is some debate about whether white space should be allowed in passwords. I'm in favor of allowing spaces as it can be useful for creating stronger, more memorable, and easier to type passphrases. But there are two reasons to be wary of spaces in passwords.

1. Spaces are very audible when people are typing.

   On most keyboards it is easy to hear when a space versus any other key is typed. Thus someone who hears you type in your password a few times will be able to learn in which positions there are spaces. This can make cracking much easier.

2. Stripping trailing and leading whitespace.

   We may wish to strip trailing and leading whitespace from entered passwords, as people may not know that those are there. (Possible copy/paste sloppiness, etc). So we just add confusion if we say that white space is allowed as long as it isn't trailing or leading.

Despite these problems, I still like the idea of allowing spaces. But a lot of smart people disagree with me.