(Note: I am not actually planning to do this. This question is strictly theoretical.)
TOTP is commonly used as the second factor in Two Factor Authentication. It is an open standard with available implementations in multiple programming languages and platforms. There is an easy to use client app for both iOS and Android in the form of the Google Authenticator app.
Is there any fundamental security issues with using TOTP as a single factor authentication method in place of passwords? Ignoring of course that single factor authentication really should be discouraged.