Though OpenSSL has some quality issues, it would be quite optimistic to believe that the rest of the software that you expose to the Internet fares better. OpenSSL is one of the most attacked pieces of software because it is a high-value target:
The same library is used in many servers of many types (HTTPS, SMTP, IMAP,...) so any vulnerability has a wide potential applicability range.
People think of SSL as "highly sensitive" so the associated bragging rights are huge. Technically, when you use SSL, it is to protect some sensitive data which is the true target of the attacker; an attacker who steals the private key of a server is not after the key for itself, but wants to run a fake server so as to obtain the credit card numbers of the users. SSL and its keys are merely enabling tools. However, private keys are terrific trophies; geekish attackers may gain much fame among their peers by flourishing a purloined RSA private key, bringing them closer to alpha male status.
One consequence is that OpenSSL is actually one of the most secure pieces of software around, because when vulnerabilities are found, they are widely publicized, and since there are so many systems using OpenSSL, risks that attackers apply the exploit to your server before you install the relevant security patch are very low.
Practical software security is a race between the discovery of vulnerabilities and their fixing. Using a library with a very large user base, and an history of efficient reactive patching (which is helped by the opensource status of OpenSSL), grants herd immunity. It works very well !
So don't fret about OpenSSL. In fact, use it. What really matters is that security fixes are promptly applied, which is a sysadmin issue. (On the other hand, using OpenSSL in embedded systems such as home routers, whose software is never really updated, is probably a not-so-good idea, for about the same reasons.)