Getting Started
First, before you dive into the world of Crypto-Libraries, you need to understand the abilities of your development team. Do you have developers or do you have power-users with sudo development knowledge?
This is important because it likely determines the ability for a person understand an API's purpose, vs implementing it and crossing your fingers.
Crypto-Modules
If you want the Shortest Answer, find one that is FIPS PUB 140-2 compliant. If you not familiar, its basically the government standard for Crypto-modules. I'd minimally suggest looking at the Wiki and get a general understanding of the standard.
Most Crypto Libraries worth there implementation are at least Level 1 Certificated. A list of them can be found here.
High Level vs Low Level Interfaces
When people say don't role your own Cryptography, they tend to mean, don't reinvent the proverbial wheel. I.E Don't build your own libraries based on what you think the algorithms should be. (Stick with FIPS)Since we established that your not going to do this (right?) than the question is how do you best use these tools.
Some Libraries give you an interface for the low level functions, like Java's Cryptography Extensions (JCE). These libraries have all the algorithms and and Object needed for Cryptography, but will probably require a developer to abstract away those function in most cases.
Some libraries offer a high level interface, such as OpenSSL, which abstracts away a lot of low level work and you just make the calls. Many may argue that you should uses this, however, as many developers experienced by using OpenSSL libraries willy-nilly, if you don't know what your using at the Highest level, you can end up digging to find out if your vulnerable.
There is also Cryptography As A Service. Personally, I don't have much knowledge on them. They usually offer an API (such as REST) for you to send your Cryptographic needs too.