The Diceware FAQ has two entries that address this:
Why are there so many meaningless words like abc, du, rrrr or 456 in the Diceware list?
An important goal of Diceware is to keep passphrases short. Based on the limited survey I did, I concluded that most people simply will not accept a 50 character passphrase that they have to type in several times a day to read, send or sign e-mail. Peter Kwangjun Suk had the clever idea that short non-words like "abc" "456" or "dn" are about as easy to remember as regular words and reduce the average length of a randomly selected password.
Also, I never heard of ncaa, boise or a&p.
The original Diceware word list is slanted somewhat to American English. Alan Beale has compiled an alternative list that replaces most Americanisms and many obscure words with more recognizable alternatives. You can find it at http://world.std.com/~reinhold/beale.wordlist.asc
There are some obscure words in both lists. If you passphrase includes a word you don't know, look it up in a good dictionary. Learning the word's meaning will aid you memory and your vocabulary.
You're not the first person to raise your objection, however, and plenty of people disagree with Rheinhold's criteria. The simplest solution is to use a different word list. For example, the Electronic Frontier Foundation designed and published alternative word lists based on criteria that agree with your objection:
The Diceware list can provide strong security, but offers some challenges to usability. In particular, some of the words on the list can be hard to memorize, hard to spell, or easy to confuse with another word.
- It contains many rare words such as buret, novo, vacuo
- It contains unusual proper names such as della, ervin, eaton, moran
- It contains a few strange letter sequences such as aaaa, ll, nbis
- It contains some words with punctuation such as ain't, don't, he'll
- It contains individual letters and non-word bigrams like tl, wq, zf
- It contains numbers and variants such as 46, 99 and 99th
- It contains many vulgar words
- Diceware passwords need spaces to be correctly decoded, e.g. in and put are in the list as well as input.
Note that several of these problems are exacerbated for users with a soft keyboard or other typing systems that relies on word recognition. Using only valid dictionary words makes this setup much easier.
If you like the EFF word list better, just use that. You're going to be typing quite a bit more, which I think illustrates Rheinhold's motivation:
The words in our list are longer (7.0 characters) on average, than Reinhold's Diceware list (4.3 characters). This is a result of banning words under 3 characters as well as prioritizing familiar words over short but unusual words.
For a 6-word passphrase with spaces between the words, that's 47 vs. 31 characters on average.