There is no good documentation on OpenSSL save its source code. Sadly. The best you can do is to gain some understanding of the underlying concepts, and then make some trials with OpenSSL to see how it lives up to them. For SSL/TLS, I suggest reading the TLS 1.0 specification, which is quite readable and includes some schematics on how things go in a TLS handshake.
For the elementary cryptographic algorithms, see the indispensable Handbook of Applied Cryptography, which is freely downloadable. Chapter 13 includes some discussion on the concept of public key certificates.
For X.509 certificates, you first must understand what ASN.1 is about; see ASN.1 Complete, by John Larmouth. The book is freely downloadable (after registration) and is good as long as you do not buy the idea that ASN.1 is great: it is an awful incarnation of what could have been a great solution (the Devil is in the details; in the case of ASN.1, the Devil had enough room to hide an entire legion of demons with him). Once you see what ASN.1 is about, you can try to have a look at the X.509 standard, or at least its Internet profile. It is a deep and dark and complex domain, paved with all the Good Intentions in the world, and as nightmarish as could be imagined by a demented scientist who read too much Lovecraft. All hope abandon ye who enter here. Also, have a look at the X.509 Style Guide.
My trusty friend Google came up with this page which links to some sample code and documentation (as two PDF files) explaining how to use OpenSSL (as a library).