Here is a thorough explanation of the mathematics in this comic:
The little boxes in the comic represent entropy in a logarithmic scale, i.e. "bits". Each box means one extra bit of entropy. Entropy is a measure of the average cost of hitting the right password in a brute force attack. We assume that the attacker knows the exact password generation method, including probability distributions for random choices in the method. An entropy of n bits means that, on average, the attacker will try 2n-1 passwords before finding the right one. When the random choices are equiprobable, you have n bits of entropy when there are 2n possible passwords, which means that the attacker will, on average, try half of them. The definition with the average cost is more generic, in that it captures the cases where random choices taken during the password generation process (the one which usually occurs in the head of the human user) are not uniform. We'll see an example below.
The point of using "bits" is that they add up. If you have two password halves that you generate independently of each other, one with 10 bits of entropy and the other with 12 bits, then the total entropy is 22 bits. If we were to use a non-logarithmic scale, we would have to multiply: 210 uniform choices for the first half and 212 uniform choices for the other half make up for 210·212 = 222 uniform choices. Additions are easier to convey graphically with little boxes, hence our using bits.
That being said, let's see the two methods described in the comic. We'll begin with the second one, which is easier to analyze.
The "correct horse" method
The password generation process for this method is: take a given (public) list of 2048 words (supposedly common words, easy to remember). Choose four random words in this list, uniformly and independently of each other: select one word at random, then select again a word at random (which could be the same as the first word), and so on for a third and then a fourth words. Concatenate all four words together, and voila! you have your password.
Each random word selection is worth 11 bits, because 211 = 2048, and, crucially, each word is selected uniformly (all 2048 words have the same probability of 1/2048 of being selected) and independently of the other words (you don't choose a word so that it matches or non-matches the previous words, and, in particular, you do not reject a word if it happens to be the same choice as a previous word). Since humans are not good at all at doing random choices in their head, we have to assume that the random word selection is done with a physical device (dice, coin flips, computers...).
The total entropy is then 44 bits, matching the 44 boxes in the comic.
The "troubador" method
For this one, the rules are more complex:
- Select a random word in a given big list of meaningful words.
- Decide randomly whether to capitalize the first letter, or not.
- For the letters which are eligible to "traditional substitutions", apply or not apply the substitution (decide randomly for each letter). These traditional substitutions can be, for instance: "o" -> "0", "a" -> "4", "i" -> "!", "e" -> "3", "l" -> "1" (the rules give a publicly known exhaustive list).
- Append a punctuation sign and a digit.
The random word is rated to 16 bits by the comic, meaning uniform selection in a list of 65536 words (or non-uniform in a longer list). There are more words than that in English, apparently about 228000, but some of them are very long or very short, others are so uncommon that people would not remember them at all. "16 bits" seem to be a plausible count.
Uppercasing or not uppercasing the first letter is, nominally, 1 bit of entropy (two choices). If the user makes that choice in his head, then this will be a balance between user's feeling of safety ("uppercase is obviously more secure !") and user's laziness ("lowercase is easier to type"). There again, "1 bit" is plausible.
"Traditional substitutions" are more complex because the number of eligible letters depends on the base word; here, three letters, hence 3 bits of entropy. Other words could have other counts, but it seems plausible that, on average, we'll find about 3 eligible letters. This depends on the list of "traditional substitutions", which are assumed to be a given convention.
For the extra punctuation sign and digit, the comic gives 1 bit for the choice of which comes first (the digit or the punctuation sign), then 4 bits for the sign and 3 bits for the digit. The count for digits deserves an explanation: this is because humans, when asked to choose a random digit, are not at all uniform; the digit "1" will have about 5 to 10 times more chances of being selected than "0". Among psychological factors, "0" has a bad connotation (void, dark, death), while "1" is viewed positively (winner, champion, top). In south China, "8" is very popular because the word for "eight" is pronounced the same way as the word for "luck"; and, similarly, "4" is shunned because of homophony with the word for "death". The attacker will first try passwords where the digit is a "1", allowing him to benefit from the non-uniformity of the user choices.
If the choice of digit is not made by a human brain but by an impartial device, then we get 3.32 bits of entropy, not 3 bits. But that's close enough for illustration purposes (I quite understand that Randall Munroe did not want to draw partial boxes).
Four bits for punctuation are a bit understated; there are 32 punctuation signs in ASCII, all relatively easy to type on a common keyboard. This would mean 5 bits, not 4. There again, if the sign is chosen by a human, then some signs will be more common than others, because humans rarely think of '#' or '|' as "punctuation".
The grand total of 28 bits is then about right, although it depends on the precise details of some random selections, and the list of "traditional substitutions" (which impacts the average number of eligible letters). With a computer-generated password, we may hope for about 30 bits. That's still low with regards to the 44 bits of the "correct horse" method.
The paragraphs above show that the maths in the comic are correct (at least with the precision that can be expected in these conditions -- that's a webcomic, not a research article). It still requires the following conditions:
The "password generation method" is known by the attacker. This is the part which @Jeff does not believe. But it makes sense. In big organizations, security officers publish such guidelines for password generation. Even when they don't, people have Google and colleagues, and will tend to use one of about a dozen or so sets of rules. The comic includes provisions for that: "You can add a few more bits to account for the fact that this is only one of a few common formats".
Bottom-line: even if you keep your method "secret", it won't be that secret because you will more or less consciously follow a "classic" method, and there are not that many of those.
Random choices are random and uniform. This is hard to achieve with human users. You must convince them to use a device for good randomness (a coin, not a brain), and to accept the result. This is the gist of my original answer (reproduced below). If the users alter the choices, if only by generating another password if the one they got "does not please them", then they depart from random uniformity, and the entropy can only be lowered (maximum entropy is achieved with uniform randomness; you cannot get better, but you can get much worse).
The right answer is of course that of @AviD. The maths in the comic are correct, but the important point is that good passwords must be both hard to guess and easy to remember. The main message of the comic is to show that common "password generation rules" fail at both points: they make hard to remember passwords, which are nonetheless not that hard to guess.
It also illustrates the failure of human minds at evaluating security. "Tr0ub4dor&3" looks more randomish than "correcthorsebatterystaple"; and the same minds will give good points to the latter only because of the wrong reason, i.e. the widespread (but misguided) belief that password length makes strength. It does not. A password is not strong because it is long; it is strong because it includes a lot of randomness (all the entropy bits we have been discussing all along). Extra length just allows for more strength, by giving more room for randomness; in particular, by allowing "gentle" randomness that is easy to remember, like the electric horse thing. On the other hand, a very short password is necessarily weak, because there is only so much entropy you can fit in 5 characters.
Note that "hard to guess" and "easy to remember" do not cover all that is to say about password generation; there is also "easy to use", which usually means "easy to type". Long passwords are a problem on smartphones, but passwords with digits and punctuation signs and mixed casing are arguably even worse.
The comic assumes that the selection of a random "common" word yields an entropy of about 11 bits -- which means that there are about 2000 common words. This is a plausible count. The trick, of course, is to have a really random selection. For instance, the following activities:
- select four words randomly, then remember them in the order which makes most sense;
- if the four words look too hard to remember, scrap them and select four others;
- replace one of the words with the name of a footballer (the attacker will never guess that !);
... all reduce the entropy. It is not easy to get your users to actually use true randomness and accept the result.
The same users will probably complain about the hassle of typing a long password (if the typing involves a smartphone, I must say that I quite understand them). An unhappy user is never a good thing, because he will begin to look for countermeasures which will make his life easier, such as keeping the password in a file and "typing" it with a copy&paste. Users can often be surprisingly creative that way. Therefore long passwords have a tendency to backfire, security-wise.