This answer attempts to guide you towards the ideal solution. Ideally, if an attacker can read your entire hard drive, it should still be impossible (in our lifetime) for him to get the key and data and decrypt without some additional interception. (for example a password). It is possible that this ideal solution is not feasible for your use case. More details required.
Where do I store this key when not in use?
Encrypt the key, and store it on the hard drive.
How do I securely retrieve this key?
Use a password to create a key that is used to asymmetrically encrypt your symmetric encryption key. This way, as long as the password is not intercepted, the symmetric key is unknown because the key has been encrypted. Further Explanation (The password can either be entered on boot, or it can be entered on login. More details needed.) Make sure that your key derivation routine is slow (100-1000ms) to make brute force sufficiently expensive.
How do I securely use the key?
In the case of an SQL database related operation, perform the encryption in the application. Do not send the key or the unencrypted text into the SQL command, unless you are sure that there is no modify statement logging (binlogs), slow statement logging (slowlogs), likely intercepts using the
show processlist command. Further Reading, Partially Relevant (my information is MySQL oriented)
Also note where you are storing the information. Some intrusions will only grant the attacker access to your SQL database, others will provide only non-root access to the filesystem, which means that UNIX permissions can be used to hide the data. Some attacks will provide read-only access, others will provide read-write, which means that the code could be changed to log the unencrypted information.
What operations do I need to perform once encryption is finished to maintain security?
Perhaps make yourself aware of the heap dump possibility, swap reading. You should also make it a point to look on IT Security Stack Exchange regularly to expand your knowledge of security. (half joking, half serious)
It is possible that my answer is missing some important measures. I would suggest editing your question to provide more information.
You may also wish to examine how you are using the key, once it is made available. By this I mean, the encryption protocol that is used, as well as the algorithms used to create the stream encryption out of the underlying block encryption, may require examination. Further Reading
I trust the experts here will upvote me if I am right, or comment on where I am wrong.