Why do the large majority of big organizations have "known bad" password policies?

Question

I saw a recent password question on security.stackexchange "Is the NHS wrong about passwords?" The question is very specific about one organization but from what I've seen the same "known bad" password policies are used in most big organizations.

When I say "known bad password policies", I mean to say most big organizations focus on relatively short (8 to 14 characters) passwords with a focus on requiring minimum number of special characters (upper/lower case, symbols, numbers, etc).

When I say "big organizations", I am not just referring to commercial but non-commercial (i.e. NIST's for example. i'm sure others can come up with other such examples). Now that I think of it ... most linux distros have some sort of default password policy that also is very similar (relatively short passwords but requiring special characters).

Why do the large majority of big organizations have "known bad" password policies?

Obligatory XKCD reference about the same subject:

p.s. another very popular question: XKCD #936: Short complex password, or long dictionary passphrase?

p.p.s. Normally I wouldn't ask such a question but I feel strongly this question needs to be asked and answered. Ever since the XKCD comic came out in 2011) I see people asking about this same subject again and again and again... and all the questions are asking legitimate questions... but they all dance around the "why" part without directly addressing it.

Answer 1

[ "Why does someone else do something I consider dumb?" is always a very open-ended question subject to speculation (and is not a great fit for Stack Exchange.) But because this problem is so prevalent among password policies, I believe it's worth exploring. ]

First, there is a lot of misinformation, superstition, confusing math, and technobabble around passwords.

• Some security experts believe that a combination of special characters, numbers, and mixed case letters automatically make passwords unguessable, but they don't know or disagree what the optimal mix is. The guidance also should be changed depending on the native language of the users, but many site administrators are unaware of the differences.
• Some don't understand the massively parallel computing resources available to modern password attackers. Hard drives containing rainbow tables of every possible 8 character string hashed with MD5 are sold online. Cloud based computing clusters, purpose-built arrays of graphics cards, and thousands of botnet controlled computers are all being employed by people to crack passwords. Databases of previously cracked passwords, dictionaries from every language, song lyrics, and movie quotes are all readily available to the enterprising hacker when it comes time to testing guesses.
• People remember hearing things like "don't write passwords down", which was advice from 20 years ago when you could count on the secretary to write her boss's password on a post-it note stuck to her monitor, and the biggest threat was someone could walk by her desk when she was at lunch. I have yet to disassemble a keystroke logger that is able to read a sheet of handwritten paper kept in my wallet. But some people still repeat this advice today.
• And technical people don't help when they toss around terms like "entropy" and numbers like "7 to the 95th power" that mean nothing to an average person; at the same time they ignore the psychology of the average user who doesn't want to enter 12 cryptographically random characters, let alone be asked to remember them for each of a dozen web sites.

Next, good passwords are notoriously difficult to study. If asked to enter a work password into a web site so they could study its complexity, most people would say "no". Many studies involving people entering a made-up password don't have test conditions that reliably mimic memorization or usage in the real world. So our primary sources of data on user-chosen passwords are the passwords and hashes published after (the increasingly frequent) data breaches. But these are often from sites like "long-eared-rabbit-fanciers-forum.com", where people know they have little of value to lose from using a weak password. They seem to be less often published from sites where the users are protecting their own credit cards, account balances, or other forms of digital valuables. I'm unaware of any case where a bank's cleartext password database was published online. And in many cases, the published databases are hashed and to analyze them means the researchers have to brute force attack them with a hashing algorithm before analysis can begin.

There is also education. Web site administrators don't take classes in "choosing a good password complexity scheme," nor do they take classes in "securely storing passwords". And a huge number of computer courses use an example of "create a sign-on screen" as an introductory program; but none of the many examples I've seen have ever demonstrated using pbkdf2() to hash the passwords before placing them in the database. And with all that, if you do come up with effective rules on choosing a good password, you still have to convince users to pick secure more passwords than 'P@ssword1'.

All that said, there are many researchers who have done a lot of work on passwords. You'll find hundreds of curated scholarly papers on the subject at http://www.passwordresearch.com. There are presentations on passwords at many of the premier security conferences. There is an annual International Conference on Passwords. So there are bodies of knowledge being built up, but they haven't converged on a standardized set of conclusions or recommendations yet. And this information has yet to be disseminated to many of the architects and engineers who build web sites. There is simply no straight pipeline of solid information from the researchers into practice. And that may be the biggest hurdle of all.

Answer 2

I strongly suspect the answer is a web of corporate CYA and laziness. Alice from Legal needs to make sure that everyone is "secure", so she asks Bob from IT what best practice is. Bob googles "password length best practice" and finds something like this as the second result:

https://technet.microsoft.com/en-us/library/hh994560(v=ws.11).aspx

Since it's concise, he likes it, and gives it back to Alice, saying "Microsoft says 14 character passwords are best, but minimum 8 is acceptable, and it should probably have complexity requirements."

Alice figures if Microsoft says it's good enough, then she's done her due diligence, and Bob knows he won't get sacked for following Microsoft's best practices, so everybody is happy with this answer.

If Bob had sent Alice to http://www.passwordresearch.com/papers/pubindex.html (pick an article, any article), odds are reasonably good she wouldn't be able to validate the trustworthiness of the source or make sense of the content.

Answer 3

One other possible reason that other answers haven't touched on is legacy technology. Large organisations are often dependent on outdated technology, that predate our modern understanding of password security.

I remember working on a contract at a major bank, and being surprised that version of AIX they were using ignored all but the first eight characters of a password.

Similarly, Microsoft Windows before Vista stored passwords in two ways by default, one of which was laughably easy to crack (14 characters, case insensitive, and split into two 7 character parts that could be cracked independently). I suspect many organisations are still using this outdated password storage, either because they haven't migrated beyond Vista, or because they're using a newer version but have enabled the old mechanism for compatibility with a legacy system.