You are quick to point out the defects of passwords — and you're right. But no other method has all the advantages of passwords, so any replacement method comes with benefits and costs.
On this topic, I recommend the work by the security group at Cambridge. You may be interested in the authentication tag on their blog. A recent paper on the topic is The quest to replace passwords: a framework for comparative evaluation of Web authentication schemes) summarized in this blog post.
Unfortunately, the paper does not analyze the scheme you propose, with private keys stored on the user's PC and public keys provided by the user at enrollment time as part of their identity. But we can look at the criteria used in the paper and see which ones fail.
- Scalable-for-Users: that depends whether you expect users to have a single identity or one per service.
- Nothing-to-Carry: as long as you accept that the private keys will be stored in a mobile phone, your scheme has the Quasi-Nothing-to-Carry property.
- Easy-Recovery-from-Loss: ok as long as the private key is used for authentication exclusively — changing the private key is as easy as a password reset. (The authors do not consider password resets within the scope of their study.)
- Server-Compatible: no, like just about any other alternative to passwords, servers must be modified. This is admittedly more of a marketing than a technical issue.
- Browser-Compatible: this would depend on how the private key is used on the client (this has implications on security).
- Resilient-to-Theft: passwords are not very resilient, but memorized passwords are resilient to theft, unlike a key stored on a device.
The two major hurdles I see are the risk of device theft and management of multiple identities. Device theft can be countered to some extent by protecting the private key with a password, but the key file is subject to unlimited high-speed brute force search by the attacker, unlike passwords used for online authentication which can be throttled.
Management of multiple identities is a bigger problem: it means the user must decide whether to use the same identity on multiple sites or not. Reusing the same key is convenient but trivially discoverable so it is a major privacy concern. While it is a bad idea for security, users can (and do) reuse passwords with no privacy implication. Using different identities on different sites requires cumbersome management of multiple keys, which is more burden than typical users want. A better scheme would combine the private keys with some repository of secrets, which shares all the difficulties with password managers.