Is checking against password lists a good way to enforce a secure password policy and possibly detect dictionary attacks on a web-based app?

Question

I have written a script to check a given password against a ~65 million long list of passwords collated from various lists released after hacking attempts.

I am thinking about incorporating it into the password checking process when a user chooses a password - notifying them that a password is on a list used by hackers.

Plus I wondered if it could be used to detect a dictionary/ brute force attack when passwords in the list are ysed in failed login attempts.

Any thoughts on the usefulness/ effectiveness of this approach appreciated.

Answer 1

As an extra precaution in addition to minimum length, I would say it is useful to check this when people sign up or change their password. Nowadays, at least one party, the National Institute of Standards and Technology (part of the US Government), agrees. Published in 2017, their NIST Special Publication 800-63 says the following (emphasis mine):

When processing requests to establish and change memorized secrets, verifiers SHALL compare the prospective secrets against a list that contains values known to be commonly-used, expected, or compromised. For example, the list MAY include, but is not limited to:

• Passwords obtained from previous breach corpuses.
• Dictionary words.
• Repetitive or sequential characters (e.g. ‘aaaaaa’, ‘1234abcd’).
• Context-specific words, such as the name of the service, the username, and derivatives thereof.

If the chosen secret is found in the list, the CSP or verifier SHALL advise the subscriber that they need to select a different secret, SHALL provide the reason for rejection, and SHALL require the subscriber to choose a different value.

There are probably many ways to do this in practice, but an arguably popular one is the "Pwned Passwords" API by security researcher Troy Hunt's "Have I Been Pwned". Here's an article explaining the idea and how to use it.

Answer 2

If you are concerned about them using weak passwords, I would just focus on the complexity requirements. Force them to use a 12+ character password with special characters. A list of 65 million sounds to be relatively small, are these mostly less than 8 characters?

In consideration of a brute force attack, it doesn't matter if there is great entropy or distribution among the characterset if the length is low (8 or less characters).

You get the same bang for you buck if you just enforce client side length and complexity or "password strength checker". Probably better to put your effort into securing the storage of the password on the backend (salt, pepper, hsm, etc.) so it can't be compromised.

From the brute force perspective. If the attacker steals your DB and does an offline attack, clearly your check if not going to be of any consquence. If they are trying to do an online brute force, you can just lock the IP out after X number of failed attempts or to lock out the account if its being targeted with multiple attempts in a distributed matter and let the user reset when they want to legitimately get on. It sounds like it would be much more complex and require more resources to check against the list on each attempt to logon - possibly could lead to a denial of service on a modest web server by over taxing the system.

While it can have some value, I am not sure if users would get it. While its nice to want to try to protect them, if they want to use short passwords and you let them, they will - maybe they take their five character password and add a number - if this isn't on your list, its done nothing to help them since it's a still only six characters. So your list will detect some common passwords, but it doesn't address the core issue which is length and complexity. Trying 65 million passwords in an offline attack is no effort for an attacker with a GPU cracking rig.

Answer 3

A good security strategy is generally simple and elegant, just like Mathematics. There are established methods for checking the safety of a password. Checking a password against millions of possible words seems a bit intensive and overly complex. Plus, you also may be in risk of a new brute-force attack against your script via resource depletion, since it requires quite some CPU cycles to check a password against millions of entries.