Passwords, or more generally something you know, are often relatively weak, because users cannot remember high-entropy secrets. As a result, passwords (or anything you need to memorize) usually ends up being a low-entropy secret, which enables random guessing, offline dictionary search, and other attacks. While it's possible to create and remember a pretty good password, experience shows that users don't -- and that it is probably unreasonable to expect users to do so.
There is a tremendous amount of academic research and practical experience that backs up this statement. Here are some example references:
In addition, any secret you know can potentially be phished (i.e., someone might be able to social-engineer you into revealing it).
Remember the classic statement:
Humans are incapable of securely storing high-quality
cryptographic keys, and they have unacceptably slow speed
and accuracy when performing cryptographic operations.
(They are also large, expensive to maintain, difficult to
manage, and they pollute the environment. It is astonishing
that these devices continue to be manufactured and deployed. But they are sufficiently pervasive that we must
design our protocols around their limitations.)
Charlie Kaufman, Radia Perlman, Mike Speciner, Network Security: Private Communication in a Public World.
At this point you might be wondering: Given the passwords have so many issues, why do we still use them? If so, I recommend you take a look at this question: Why do we even use passwords / passphrases next to biometrics?.