As far as I'm aware (and I don't have any concrete facts about this), iphone/android phones do not have any special storage area for secure data. However, if you use an OAUTH type app as a 2nd factor authentication, then this shouldn't normally be a huge concern (this is not the only means of authentication - you need a password too). Of course it depends on the situation.
Comparing to something like yubikey, I would imagine the following pros/cons or considerations to take into account:
- Lost device - People easily notice when their phone is missing. Also they don't stay away from their phones for too long. Not necessarily so with something like yubikey, which you have to use only specifically for authentication (might not be very frequent)
- Yubikey probably has better hardware protection and it's a dedicated device
- With both, you should have good, quick procedure in place to revoke access. If someone lost their phone/key they should notify and the access blocked for this device as soon as possible
- Cost - yubikey will cost extra if you already have a smart phone, but much less for people who don't
- deployment - might be trickier to distribute the shared keys to smart phone users rather than buying a bunch of yubikeys and handing them to people
- education - depends on your situation, people might be happier with an app on their phone, but some might prefer a separate device.
- user-friendliness / mobility - as George Bailey pointed out, it should be easier to use the yubikey as it will automatically type in the key for you. However, if you want to access the app using your mobile phone, then yubikey is probably not usable at all without a USB port.
