The client is a network concept: the data is transported between two machines, the server and the client. The client is the one who initiates the conversation; the server is the one who is sitting all day, waiting for clients to connect.
The user is the biological entity (presumed human) who controls the client. Authentication is about the server making sure that whatever it receives originates from a specific user. However, the server is not talking to the user (the human being) but to the client (the user's computer). When a server authenticates the user by way of a stored HTTP cookie, it is really authenticating the client: since the cookie value was previously sent to a specific client, only that client may show the exact cookie value. The server then assumes that a given client is under exclusive control of its purported human owner, and then feels content that the human user has been involved.
The distinction between client and user becomes apparent when the user's machine is subverted (e.g. infected by malware). The machine will begin to do things outside of the control of the human user.
To make an analogy, if you have a dog and that dog bites me, then I will sue you, not the dog, because the dog is supposed to be under your control. Even if the dog contracts rabies (explaining its vindicative behaviour), it is still your responsibility. In that sense, authenticating the dog is as good as authenticating you (in the eyes of my lawyers, at least) and it is up to you to make sure that your dog is really under your exclusive control.
With Web applications and "API", the picture can become more complex, because the server can also behave as a client with regards to another server. That secondary server may want to make sure of the identity of either the first server, or the human user who initiated the whole process, or both. The terminology can be source of confusion if the same terms are reused.
When using public/private key pairs, the private key owner (the machine which contains the private key) uses it to perform some computation which links to another computation which involves the public key. For instance, with digital signatures, the private key owner generates a signature over a given message (a sequence of bytes); the signature can be verified by using the public key. Digital signatures are a good tool for authentication: by computing a signature on a challenge sent by a verifier, the private key owner demonstrates to that verifier its mastery of the private key. The verifier thus obtains a guarantee that it is really talking to "whoever controls the private key corresponding to a given public key". The interesting part of the process is that the verifier needs not know the private key for that, so it is possible to verify a signature without having the power to generate signatures.