How can one test if a password generator is cryptographically secured?

Question

Given the plethora of random password generators (RPG) available, I'd like to do some black box testing on some.

Let's take https://passwordsgenerator.net/ for example. Assuming the whole generator is a black box with 0 information about how the passwords are generated (can't even view the .js stuff), and all we have is a "Generate password" button that somehow outputs a seemingly random password each time it's clicked.

We do NOT know:

• Who made or hosts the RPG
• What algorithms are used to generate the password
• How they get the randomness (Atomic decay? Lava lamps? Monkeys on type writers? People trying to exit Vim?)
• Source code.

What we know:

• If you click "Generate password", you get a seemingly random password

We can get thousands or millions of passwords as testing data. Given just those passwords, can we analyze them to figure out (even just estimate) how cryptographically secured the RPG is?

(I'm not looking for processes enough to generate whole reports and research papers. I'm just thinking of a way the common people can do a "quick" and rough test on the RPGs, maybe to choose between the common RPGs)

Answer 1

It is very hard to evaluate the quality of a random password generator without knowing its underlying algorithms. The quality (the entropy) depends essentially on 2 elements:

• the number of possible combinations
• the equivalence of probability of different combinations

The problem is that a random generator that would iterate over 10 millions random password phrases will be a very poor password generator if the attacker could know the used passphrases. But if you only use several thousands of generated passwords it will look very good.

If you really know nothing, you can do little more than testing a rather large sample for collisions: if at least one password occurs more than once, chances are that the generator has a low entropy. But unless you can test for very large samples (the magnitude order of the expected entropy) no collision is not a sufficient proof of quality.

So my advice is: if you do not know enough from the generator, just do not use it.

Answer 2

On 64-bit machines the maximum integer value is 0x7fffffffffffffff (9,223,372,036,854,775,807), and in a random number generator which is not cryptographically secure the seed value will typically be an integer in this range starting from 0, so it should be clear given this number of examples in a very simple generator that patterns could be spotted. You could also perform an attack which would require the server to be reset, potentially resetting the seed and comparing values generated. Essentially though techniques for randomness are a fairly well defined branch of statistics and a good example can be found e.g. here: https://csrc.nist.gov/publications/detail/sp/800-22/rev-1a/final

Answer 3

First off, no password generator on a website "secure" in any sense - unless you read the source code and host it yourself.

Second, the only requirement for a password is that it is sufficiently random.

You could perform statistical tests for randomness, but that would require some knowledge about the composition rules to be meaningful (e.g. does it always have a special character? Is it diceware with English words?).

And even if your test finds the result to be statistically random, it will not mean that it is secure, as you will not know the seed (or state) of the generator at the time when you generate the "real" password.

To sum it up: There is no test that laypeople could use to "vet" a password generator.

The correct way to generate passwords is to use a password manager that you trust. Not to create passwords on websites or download and run random software from the internet.

If you think you can still live without a password manager in this age and day: Use diceware. It is demonstrably secure and truly random.