The complete picture is the following: the asymmetric cryptography (RSA, DH...) results in a shared secret called the master secret. From the master secret are derived the symmetric keys used for encryption.
The distinction relates to the difference between a connection and a session. When a client opens a connection to a server, the two may engage in a new full handshake, with asymmetric cryptography, which results in a new master secret. OR they could agree to reuse a master secret from a previous connection. This latter case is called an "abbreviated handshake". Different connections which work over the same master secret are called a session, and is identified by a session ID which client and server send to each other in their
ServerHello messages. See this answer for details on this process.
When a master secret is reused, i.e. for a new connection within a session, new symmetric keys are derived nonetheless, because the derivation works over both the master secret, AND two random values that the client and server send each other in their initial messages. Therefore, symmetric keys are renewed with each connection, but through derivation from a "master secret" which can be remembered for a long time, and reused for many successive connections.
When a connection is established, the normal situation is that keys are not renewed until the connection end. At any time, as per the protocol specification, the client or the server may request a new handshake, within the connection; the new handshake may or may not be "abbreviated" (an abbreviated handshake occurs only if both parties agree to it), but will anyway result in new symmetric keys (but not necessarily a new master secret).
In practice, this does not happen. An extra in-connection handshake will occur if the server wants to force the client to authenticate with a certificate (this is the common case with IIS: first handshake without client authentication, and when the HTTP request is sent and the server learns the actual target path, it triggers an extra handshake). But it is pretty rare for deployed software to really do new handshakes just for the key renewal. The main reason is that key renewal is not needed for cryptographic robustness: AES uses 128-bit blocks, and not 64-bit blocks, precisely so that keys need not be renewed in practical conditions.
So what occurs (especially in a Web context) is the following:
- The symmetric keys for a connection remain unchanged until the end of that connection.
- A connection lives as long as client and server allow it; in a Web context, the server will normally close connections which have gone inactive for more than one or two minutes.
- Sessions, i.e. master secret values, are remembered by clients and servers and are reused when possible. A typical Web browser will remember SSL sessions until it is closed (as a process, i.e. all windows closed). A typical Web server will remember SSL sessions for some given amount of time, which is configurable (typically 10 minutes to 10 hours or so), also subject to available RAM and a configurable limit (each remembered session will use a couple of kilobytes at most).
The symmetric keys for a connection are kept in RAM only. The master secret will normally be kept in RAM only, but when several front-end servers operate behind a load balancer, they may share the master secrets through various methods, which can even be a database. In such configurations, master secret values may thus be stored to a permanent storage medium (which raises a lot of potential issues, of course).
If you shut down the client system (not just put it to sleep or hibernation, but really shut it down), then it is pretty much guaranteed that all symmetric keys, including the master secret, have disappeared for good from the client machine.