How strong is a password consisting of words arbitrarily chosen from the dictionary?

Question

Diceware wants us to actually randomly pick n words from a given set of m=6⁵ words.

Assume a user does not actually use a dice. Instead they take a physical dictionary of m words and "randomly" picks n words.

Can it be somehow quantified how big would the drop of randomness be?

For example, I suppose that they would never pick words from the beginning of the dictionary nor from the end of the dictionary; neither would they pick a word from the bottom or top of a dictionary page. Very well, this restricts the wordset to, say, ^m/₄.

I guess it can be ignored here that a diceware restricts us to a set of m=6⁵ words while a physical dictionary can convincingly have m=100k words because diceware can easily be modified to include 100k words as well.

Answer 1

Pretty much all of my answer is based off of my own interpretation of the results from this paper:

https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0041531

(aka, I'm not an expert in how the human brain works and I haven't done tons of research). I'll start with the most important detail and then answer your question

Under realistic circumstances the resulting password will likely be as hard to guess as one generated with dice.

Here are (IMO) the relevant details:

1. People are in fact bad at making random numbers, and are prone to following patterns.
2. However, different people have completely different patterns they follow, and the pattern a particular individual follows cannot be known without first observing a long random sequence that they have generated
3. Therefore, to make it easier to guess a particular person's password, you would first have to ask them to generate many passwords in an attempt to learn about their particular "pattern"
4. Therefore, getting the data necessary to improve the chances of guessing a particular individuals password would involve one heck of a social engineering attack, and is probably much less effective than rubber hose cryptanalysis.

It's difficult to estimate the decrease in entropy without a dedicated study, but the loss of entropy is probably very small.

In the linked study they had people make sequences of random numbers (600 numbers total). Using half of those numbers they tried to make a model to guess numbers in the other half of their sequence. Their ability to predict the first number in the sequence was not appreciably different than random guessing. However, once the "sequence" started, their odds of correctly guessing the next number in the sequence plateaued at about 25%.

For a baseline, they have a 10% chance of guessing the next number in the sequence regardless. If we want to make a ridiculously naive conclusion, this suggests that a good model might improve the chances of success by a factor of ~2. Applying this to your initial numbers means that the entropy has gone from 83 bits of entropy (log2(100000**5)) to 82 bits of entropy (log2(0.5*(10000**5))). Needless to say, the difference is practically negligible.

Of course, our number-guessing situation probably doesn't directly apply to dictionary-word guessing. I personally suspect that a good model for dictionary-word guessing will have a much smaller improvement over randomness than experienced for number guessing. However, the human brain is complicated enough that I'm not confident in anyone's ability to guess at the answer without doing a study specifically for this.

Answer 2

It all boils down on how the users does the "randomly" picking of words.

And the main problem are users not understanding things or doing bad choices.

We don't really need to discard first-last pages nor words on the page. That would be mostly useful IMHO for forbidding extremely bad practices, such as a user that chose as the password the first 4 words from the dictionary.

In fact, I would hypothesize that a user trying to come up with a random word would be more likely to choose a word in the middle of the page than at the top, since the top or bottom one looks "less random" (they should actually be chosen with the same probability). Similarly, the user will probably tend to select little-known words over common ones.

The page will probably come from "opening the book at random", which will add as another factor the wear down of the book and how likely it is to open on a particular page.

The beauty of using a dice is that it gives you a fair initial randomness: we assume that we have an unbiased generator of 1-6 random values as the seed of our password.

Not using a dice but letting the user randomly pick up pages and words we know that the entropy will suffer (which you try to counter by using a larger dictionary) but not really "how much".

There are a few tricks we could try to attempt getting the user to be more neutral, such as having them choose a priori the entry or page they will be choosing, before opening the dictionary and reading the page, but the initial issue persists.

I would recommend you to take a step back and formualte why you want not to use a dice. Perhaps it is problematic to roll it many times? You consider the diceware list unreliable? You actually don't have a dice available?

As an alternative of using a dice, I would recommend the following:

• Write the numbers 0-9 on ten cards
• Mix them and choose one of them. Save the card back. Repeat three times.
• Visit the page of the dictionary whose number is those extracted in the prior step (if the dictionary has less pages than that, discard that page number and start again to extract three values).
• Mix the cards and extract a couple of numbers. Pick the entry in the page with that number (if there are less than that, start again. If the page only has 50 entries, you may pick the first number from 0-4 instead of 0-9)

Here we replace the dice rolling with picking a card, which is also a simple model, as compared with determining how much randomness you lose by opening certain dictionary.