You are correct, this is not effective in the case of the device being stolen - you need good local authentication for that such as a long pin, password, passphrase, and potentially biometrics.
However these OTP solutions are not trying to address that threat. They are intended to prevent someone from attempting to access a service by guessing usernames and passwords, or using usernames and passwords stolen from elsewhere (e.g. through keyloggers, database dumps etc).
- XYZ website is hacked and the usernames and passwords cracked and published online
- I get the list and try each of the credentials against the ABC website
- Often users will have used the same usernames and passwords and I will gain access to some accounts
- However if my account on ABC website is linked to a mobile number and uses an SMS to authenticate then knowing my username and password doesn't allow you to compromise my ABC account unless you can also intercept the SMS