Authentication is related to some notion of identity: a message (encrypted or not) is said to be authenticated if it can be guaranteed in some way that its originator is some specific entity. The definition of that entity, and the extent of the guarantee, depends on the context.
For instance, when two people (traditionally called Alice and Bob) share a common secret key K, they can encrypt data with K and also authenticate it relatively to K with a MAC: from the point of view of Bob, the MAC authenticates the message as originating from someone who knows K; from that, Bob infers that the message comes from Alice, because K is supposedly known only to Alice and Bob, and Bob remembers never having sent that specific message, so it must come from Alice.
However, third parties (Charlie), not knowing K, cannot verify the MAC, and thus cannot consider the message as authenticated. Even if Bob reveals K (e.g. Charlie is a Judge and Bob sues Alice with regards to the message contents which could be a contractual element), Bob cannot convince the Judge is really from Alice since Bob also knows K and could have produce the same message himself. In a nutshell, this is why banks are reluctant to authorize big stock exchange orders from customers through a Web-based interface: the bank can get some guarantee about the connected client's identity (that's authentication) but cannot turn that guarantee into a convincing proof for a third party.
In general, authentication is required whenever there is encryption. Encryption without authentication is robust only against passive-only attackers, who spy on the line but never modify any bit. This attack model turns out to be rarely realistic. Combining encryption and authentication properly is non-trivial. There are some encryption mode which do that.
The main drawback of authentication is that it tends to imply some slight overhead. If done properly, we are talking about a dozen extra bytes per message, and maybe a +20% CPU cost (which will not matter in most cases).