For what uses would it be smart to get a YubiKey?
I bought, and enthusiastically use, my yubikey. The way it stores two OTP profiles is particularly useful: with one set of keydata (short-press-the-button) I kept the secrets close, and use it for sudo (and, one day for straight ssh login) access on my colo'ed box. The other (long-press-the-button) I shared with Clavid, who offer an OpenID authentication service which allowed me to properly tighten my whole OpenID authentication loop .
As others have pointed out, unlike a number of other hardware-authentication-dongle services, you control your own keying material, and don't have to trust any third party to get things right if you don't want to.
One of the biggest wins with it is being able to use untrusted endpoints to authenticate via the 'key, because the password so disclosed is useless as soon as it's used. The biggest problem with that, however, is that many classically-insecure endpoints (eg cybercafe PCs) disable their USB ports, which can make finding somewhere to plug the Yubikey in quite hard.
On the whole, though, I think they're a big win.
 I'm a big fan of OpenID. The thing that originally brought me to server fault, the SE site where I do most of my posting, was its offering OpenID authentication.
Things about Yubikey to consider:
It appears as a keyboard device. (Works almost everywhere they say you can't use Keysticks... there are ways to prevent it on almost all OSs but most places don't lock down Human Interface Devices.)
If someone else gets a hold of it and your password, (and you're using OTP) and they record a few OTPs, they can use that information to connect to a resource up until you use your token again.
It should only be a second factor, never a single factor. (You can't lose your password, but you can lose your keys)
It is (moderately) cryptographically secure, and if you are an organization you can handle the whole cryptosystem without relying on a third party to secure their seeds.
It's a bad idea to rely on it for security by itself. But as an additional factor I like the idea.