# Does entropy apply to passwords generation when some passwords are more likely? [duplicate]

Entropy is a standard measure of the complexity of a password generation. The standard way to calculate entropy is to take the probability p of a password being generated and take the sum of p*log2(p) for every password in the set of possible passwords. This entropy measures complexity based on how the password is generated or a worst case attack (e.g. we're not trying to calculate the entropy of rolling 6,2,6,3 on a loaded die, but the entropy of the distribution that led to those numbers getting rolled). Entropy here is not measuring how the password would stand up to brute-force, etc.

However, for passwords with patterns (non-uniform distribution) Shannon entropy's not an accurate measure of guessability, since two random numbers can have the same entropy but one can act much more predictably if they have different distributions. When the entropies are similar, the distribution with the least uniform distribution (the least random-looking) is always less secure.

For example, consider the following schemes with the same Shannon entropy but different security levels: flip a coin repeatedly and count the number of tails you get, stopping when heads comes up first. Looking at the probability distribution, the numbers generated here has a Shannon entropy of 2 bits. Great! That means this scheme is as secure as two independent coin flips, right? Not quite. If you think about it 50% of the numbers generated in the first scheme are 0 (heads on the first flip), and you have a total 1/3 chance of two users getting the same result using the scheme. Two independent coin flips, on the other hand, gives a 1/4 chance of producing each number, for a 25% chance of two users getting the same result. (The math for calculating the probability of such a collision is not complicated but requires you to know the frequency of each possibility, as explained here). Entropy gives a misleading figure for the security of the first scheme.

What happens when a password is generated using a random process that does not use a uniform distribution, or a distribution that's not easy to describe with or map to equally likely outcomes, like say:

• A scheme to pick long phrases taken from a large English corpus
• A scheme to generate a mnemonic-based password
• Rolling a loaded die 15 times

How do you compare the strength of random but non-uniformly generated passwords like these to passwords generated from uniform distributions? Does entropy apply? Is anything else a widely-accepted substitute? In my experiments entropy isn't completely accurate but still gives a rough estimate of password strength.

Note: This question was closed as a duplicate, although some of this question isn't accurately addressed by the linked answers (e.g. how do you calculate the strength of a 4-word passphrase where the phrase is taken from the COCA corpus?) I've edited the question to address these points. If you think this question should be reopened please vote to reopen it, and if you think this question would be better if made more specific please say so.

• Possible duplicate of Calculating password entropy? and Confused about (password) entropy. – Steffen Ullrich Jul 19 '17 at 5:47
• @SteffenUllrich The first question you have marked seems to only address user-chosen password and uniformly distributed password. No mention is made of non-uniform distributions for passwords. The second question you have marked as a possible duplicate does not address my question at all. I never implied that I was treating passwords as strings of characters and stated clearly that I am asking about calculating entropy based on distributions used to generate the password. – Cody P Jul 19 '17 at 16:37
• Perhaps my question before I edited it was unclear, but I don't see how the question linked even answers my question, let alone meets the criteria for duplication listed at meta.stackexchange.com/questions/213976/… Can anyone please explain by pointing to a paragraph in the "duplicate" answers that answers my question about non-uniform distributions? – Cody P Jul 19 '17 at 16:43
• The focus of your question has shifted from "what happens" with the entropy to "how to compare" the entropy in non-uniform distributions. The first question was actually answered in the ones I've referenced in that you cannot naively apply simple entropy calculation any longer in non-uniform distributions. And the second question actually has an example of how to compute the entropy in such cases and how not. Essentially it boils down to mapping the non-uniform distribution to a uniform one, i.e. don't use the characters but the words as a base in case of a dictionary. – Steffen Ullrich Jul 19 '17 at 17:48
• I'm still confused about how that makes this a duplicate. None of the questions include the point you've explained about mapping a non-uniform distribution to a uniform one, and not all my examples can even be correctly mapped to a uniform distribution. If you're referring to the "What dicitionary is your password in," that scheme wouldn't be accurate for say, choosing a 5-word phrase from the COCA corpus. As you can see from these complexities, having one answer in question A that is general enough to answer most of question B does not usually make question B a duplicate of A. – Cody P Jul 19 '17 at 18:19