If it is public it cannot be secret...
A key is "a parameter for an algorithm, which concentrates secrecy". This means that the key is not only secret, but, normally, nothing else is (algorithms are known to everybody).
A public key is a paradoxic terminology which was coined when asymmetric cryptography was invented. The "real" key is what we call the private key. It is the secret parameter which allows for performing an operation which only the key owner should be able to do (e.g. decrypting a piece of data, computing a signature...). The magic of asymmetric cryptography is that the reverse operation (encrypting a piece of data, verifying a signature...) can be done by anybody, using a mathematical object which is linked to the private key, and which can be made public without revealing the private key. You can imagine the public key as the face of a human being: it is distinctive and can convey information, but recognizing the face does not tell you what goes inside the brain. So a "public key" is not really a "key" in the strict sense of the term; and the "private key" could be called simply "key" or "secret key" because that's what it is.
However, Tradition dictates that we say "private key" for a key of the asymmetric kind (a key with enough mathematical structure to have a public face, which can be revealed without endangering the secrecy of the private key), while keys which are just bunches of bits, for symmetric encryption and similar algorithms, are called "secret keys" (or "symmetric keys"). That's traditional, not ultimately rational; terminology is always like that.
Asymmetric cryptography is mightily cool, but it is ill-suited for bulk encryption of gigabytes of data. For that, symmetric cryptography is much better, with "secret keys" (i.e. keys which do not have a public face, they are 100% secret). Diffie-Hellman is an asymmetric key exchange algorithm, in which two parties do the mathematics of DH and end up with a shared secret, i.e. a key: they both know it (thus "shared"), but nobody else (thus "secret"). With it, they can do symmetric cryptography, which was the point.