Your question is very vague and covers a number of areas. An example would be most helpful.
Based on the title of your question, it seems you're interested in how to securely handle credentials that are required by an application. If this is a Microsoft environment one can encrypt sections of the web.config which is used to store sensitive information (SQL logon credentials, etc.). This post offers a good explanation: why-to-use-rsa-for-dpapi-web-farm-encryptionwhy-to-use-rsa-for-dpapi-web-farm-encryption. In short, if you have a single webserver DPAPI is the recommended solution whereas RSA keys are preferred for web farms.
However I believe you're also interested in a enterprise password management solution. SANS instructor Jason Fossen posted a great entry on his Windows Security Blog. He wrote a script that will assign a unique password to local accounts and leverages asymmetric encryption to save the password to a centralized location. In the entry he also mentions a number of commercial products in the Password Vault or Enterprise Password Management space.
HSM is typically only used for critical accounts like domain/enterprise administrators or for certificates used by critical infrastructure. For general or everyday use, these enterprise password management solutions are great.
Quite some time ago I looked into the Lieberman solution and found it to be very cool. There's full auditing to log the user accessing credentials, what credentials were access, and of course when. There's also flexibility to setup temporary accounts (only good for x time) which is useful for contractors, consultants, etc.