When implementing a login / password checking mechanism, should one check if a given password is already in use (I'm aware that knowing that a given password is already in use presents a risk of brute force attempts ) or should we perhaps warn the administrator that a particular weak password is used quit often by users ?
When implementing a login / password checking mechanism, should one check if a given password is already in use
No. Actually you should be following a slightly different rule: when implementing a login / password checking mechanism, one should not be able to recover passwords, only verify them.
To achieve this, here is what you need to do:
- Pick a slow password hashing function. Argon2 would be ideal, If you cannot use this (or wish to make a more conservative choice) then choose one of bcrypt or PBKDF2.
- Generate a random, unique salt per user (it should be unique per user, this is important).
- Apply the hash function you have chosen to the password+salt, as described in its documentation. Usually this will simply be H(salt||password) or H(password||salt), but follow the documentation for the function you choose.
- Store the result and the salt in your database, alongside the user identifier.
Pepper has just been mentioned in comments. You can also generate a global load of random junk for your site and hash your passwords like this: H(pepper||salt||password). Store both the pepper and the salt. Pepper buys you further resistance against any kind of rainbow table attack and further resistance against any kind of brute force attack. See Password Hashing add salt + pepper or is salt enough?
To verify any password, you simply repeat this process with the submitted password. At the end, the result should be the same, if it is not, fail the login.
What this buys you:
- If your database is stolen, rainbow tables will not work (thanks to the salt).
- If your database is stolen, bruteforcing will take a very long time (thanks to the slow hash function).
- If your database is stolen, there is no easier way than brute force, thanks to the fact you are using a hash function and not secret-key encryption. You should assume that if your database can be stolen, so can your secret key for decrypting your passwords.
For improving password quality, you could reject certain passwords. For example:
- Set a minimum password length of of a decent amount. Some will say 8 characters; personally I would err on the side of caution and say 12 characters.
- Do not set an upper limit, or set a reasonable one such as 63 characters. Users who use 32 character passwords of randomly generated printable ascii are helping themselves massively and you should not arbitrarily stop them or get in the way of password manager usage.
- Require some reasonable combinations of letters, numbers and punctuation. When setting any complexity requirements, first read this answer.
- Do not set an regular password expiry. This will encourage all manner of bad behaviour in users, for very little benefit.
- Encourage better passwords with a "password quality" meter.
To improve the overall security of your system, consider implementing two-factor authentication for accounts. Yubikeys (U2F) are inexpensive and straightforward for users of Google Chrome (and maybe soon Firefox); TOTP is also good. SMS authentication is considered dubious due to the vulnerability of mobile networks to tampering with and intercepting SMS delivery.