Yup, this is a risk. However, there are reasonable ways to mitigate the risk of loss or theft of a smartphone.
One standard method is to wipe your phone once you've noticed it is lost. Usually, if you lose your phone, you'll notice that fact fairly rapidly -- and if it's stolen, you might even know right away. That gives the chance to immediately wipe your phone, which then prevents a loss of your passwords. Many corporate-oriented systems also provide a way to lock the user's corporate accounts and reset their corporate passwords at that point, so that the password to your corporate email service is worthless.
Another standard defense is to require a secret PIN, swipe code, face recognition, or other password to unlock your phone. That also helps protect you against most folks who would recover or steal your phone, since they're probably not going to be able to guess your PIN. Of course, this defense is far from perfect: someone with sufficient sophistication may be able to bypass the unlock screen and get access to your data by reading it directly off the flash storage. However, this requires more sophistication than most thieves are likely to have, so a PIN unlock screen may be good enough to deter common thieves.
So, if you put all of these together -- are they perfect? No, they're not. But they help.
I realize this didn't answer your question of how to enable auto-forget. That will probably be platform-specific. If you don't get a good answer here, you might want to try asking on Ask Different (for iOS) or Android Enthusiasts (for Android), two of our sister sites focused on specific smartphone platforms.
Trivia: On Android, have you looked at Menu > Settings > Location & security > Credentials storage > Use secure credentials? I think this only protects your Wifi passwords, but it's one mechanism to know about.
