The following are the "accepted naming", i.e. the terms that I would accept:
A public key is a mathematical object; it is half of the cryptographic key pair. For RSA, a public key is a pair of big integers (modulus and exponent), nothing more.
A private key is a mathematical object; it is the other half of the key pair. For RSA, a private key is a bunch of big integers (namely n, e, d, p, q, d mod p-1, d mod q-1 and q-1 mod p, where n and e are copies of the two corresponding values from the public key).
A key pair is a public key with the corresponding private key.
A certificate is a signed object which some elements and binds them together. That's what X.509 is about. Most certificates contain a public key (not the private key) and an identity (a "name") designating the owner of that key (i.e. the entity who controls the corresponding private key).
That being said, a lot of people use sloppy terminology and you can encounter a lot of jargon abuse. E.g.:
For usual algorithms (everybody uses RSA), the public key is easily recomputed from the private key (with RSA, the public key is a subset of the private key, so this recomputing is trivial). This leads some people to talk about "key pair" when they mean "private key" or vice versa.
Some people (in particular a substantial fraction of Microsoft's documentation) talk about "certificate" when they mean "the certificate and the private key corresponding to the public key which is in the certificate". PKCS#12 archive files (also known as "PFX") are usual containers for a certificate and its private key.
Many people don't grasp the difference between a certificate and a public key, and use the two terms in a seemingly random way.
Confusion is further enhanced when people being to talk about "public certificates" or "private certificates". Part of the problem is that games with key pairs and RSA-like algorithms are called public-key cryptography as opposed to "normal" cryptography, i.e. symmetric algorithms like AES. In symmetric cryptography, keys are secret and no key can be made public. So there is an expression: secret key. This term normally designates a key for a symmetric algorithm, which is a bunch of bits. But some people say "secret key" when they mean "private key", or vice versa.
I have also encountered "digital identity", "private identity", or "digital credentials", meaning a certificate, a private key, both, or something else. Depending on who does the talking.
There is also a notion of secret-key certificate which is either someone who considers "secret" and "private" to be interchangeable, or the specific case of a certificate used in some advanced "blind signature" schemes. This is research-level work, nothing standardized.
Bottom-line is that if you want to be reliably understood, then you must take care to define your terminology properly. Similarly, be prepared, when reading documents from other people, to encounter just about anything. Decent standards begin with a glossary which unambiguously defines the meaning of terms within the standard.