As others have mentioned, by looking at smudge marks, it is often possible to figure out the passcode. For a thorough evaluation of this risk, see the following research paper:
Adam J. Aviv, Katherine Gibson, Evan Mossop, Matt Blaze, and Jonathan M. Smith, Smudge Attacks on Smartphone Touch Screens, WOOT 2010.
They show how by taking a picture with a digital camera held at an angle to the screen, it is possible to recover most of the pattern. Surprisingly, they found that it is possible recover the pattern even if the user has placed the phone in their pocket: you might think this would wipe away the fingerprints, but it doesn't. In fact, they even did experiments where they wiped off the screen of the phone with a cloth after entering their patttern, and found they were still able to recover the password pattern.
(Incidentally, they also give a calculation of the number of possible password patterns: 389,112 patterns. But they show that with a picture of the smudge pattern, this often can be reduced to just 1 or 2 possibilities.)
The paper is a tour-de-force, and well worth reading. Even just looking at the pictures and figures will give you some idea of how surprisingly effective the attack is.