How and when should I use OpenSSL for IT Security? How do I understand the output?

Question

I'm discovering OpenSSL, and already used it to diagnose a faulty SMTP TLS connection, but I'd like to get a better / full understanding of the tool and how to use it.

What resources should I use to get started learning this tool?

I'm interested in anything from sample commands, to any guide how to understand and use the output. I have a basic understanding of TLS, PKI, Encryption etc, but given the complexity of this tool, there is so much more to learn!

Answer 1

There is no good documentation on OpenSSL save its source code. Sadly. The best you can do is to gain some understanding of the underlying concepts, and then make some trials with OpenSSL to see how it lives up to them. For SSL/TLS, I suggest reading the TLS 1.0 specification, which is quite readable and includes some schematics on how things go in a TLS handshake.

For the elementary cryptographic algorithms, see the indispensable Handbook of Applied Cryptography, which is freely downloadable. Chapter 13 includes some discussion on the concept of public key certificates.

For X.509 certificates, you first must understand what ASN.1 is about; see ASN.1 Complete, by John Larmouth. The book is freely downloadable (after registration) and is good as long as you do not buy the idea that ASN.1 is great: it is an awful incarnation of what could have been a great solution (the Devil is in the details; in the case of ASN.1, the Devil had enough room to hide an entire legion of demons with him). Once you see what ASN.1 is about, you can try to have a look at the X.509 standard, or at least its Internet profile. It is a deep and dark and complex domain, paved with all the Good Intentions in the world, and as nightmarish as could be imagined by a demented scientist who read too much Lovecraft. All hope abandon ye who enter here. Also, have a look at the X.509 Style Guide.

My trusty friend Google came up with this page which links to some sample code and documentation (as two PDF files) explaining how to use OpenSSL (as a library).

Answer 2

I hate trying to learn things from the point of view of "here's a tool, what can you do with it?".

Instead, I like things from the point of view of "here's a problem, here's how you can solve it with a tool".

This post of verifying the Comodo hacker's key is a good example. It starts with the problem where a hacker claims to have obtained a fraudulent SSL key for "addons.mozilla.org". Is he telling the truth? How can you use the 'openssl' tool to verify this? I then work backwards to show how, using the 'openssl', to show how this is done, from several perspectives.