Without a CA, you need some type of other secure secret exchange. It could be a side channel exchange of the thumbprint or use of some pre-existing shared secret. Web of trust can also be used, though this is basically another form of side channel.
Using the UID as you suggest however, is likely insecure. The UID of a phone isn't a secret and an attacker could obtain the UID and then impersonate the device.
Now, if you don't actually care about the identity of the phone, it could just submit an anonymous public key and you will know that it is the same phone in the future, but you will know nothing about the identity of that phone. You don't need to involve the UID (and shouldn't) for that though.