What are the benefits and risks of various TLS Handshake protocols?

Question

This question asks if it's possible to separate the rest of a TLS session from the handshake for "security reasons". The question wasn't specific on what that "reason" might be.

This made me research the various TLS handshakes to determine if there was any benefit (DOS attack, or cryptographic oracle) that would be defeated with such a configuration

Some basic research tells me that the following handshakes exist:

• SSL 2.0 unprotected handshake allowing for a downgrade
• Typical TLS 6 step handshake
• TLS 1.1 RFC 4346
• 1-RTT in TLS 1.3
• 0-RTT in a future version of TLS (draft in TLS 1.3)

Question

1. How many TLS handshake protocol variations exist?
2. Is the list above each considered a "handshake" or can they be grouped into primary and sub branches? (e.g. X is a minor variant of Y, but Z is completely different)
3. What is the notable difference between the TLS handshakes?

Answer 1

Depending on how you see it, there is one, two, three, or many more "handshakes".

A first distinction is between "SSL-2.0" and "SSL-3.0 and later". The format of records and handshake messages for SSL-2.0 differs a lot from the records and messages used in subsequent protocol versions. Apart from the format and the implemented algorithms, a notable semantic difference between SSL-2.0 and the later protocol versions is that in SSL-2.0, both client and server send lists of cipher suites, and the client chooses, not the server. In later versions, the client sends a list, and the server chooses.

When doing a SSL-3.0+ handshake, the client sends a ClientHello message, and the server responds with a ServerHello. The ClientHello may contain many extensions, which are (by definition) open-ended, and it is up to you to consider whether extensions "change" the handshake or not. In any case, if the client and server agree to resume a session (either they reuse a session ID and remember the past session, or the client sends a session ticket, then you get the abbreviated handshake:

  Client                                                Server

  ClientHello                   -------->
                                                   ServerHello
                                            [ChangeCipherSpec]
                                <--------             Finished
  [ChangeCipherSpec]
  Finished                      -------->
  Application Data              <------->     Application Data

In all other cases, you get the "normal" handshake, that looks like this:

  Client                                               Server

  ClientHello                  -------->
                                                  ServerHello
                                                 Certificate*
                                           ServerKeyExchange*
                                          CertificateRequest*
                               <--------      ServerHelloDone
  Certificate*
  ClientKeyExchange
  CertificateVerify*
  [ChangeCipherSpec]
  Finished                     -------->
                                           [ChangeCipherSpec]
                               <--------             Finished
  Application Data             <------->     Application Data

The "*" above means that the message is not always there. For instance, the server will send a ServerKeyExchange only if the cipher suite calls for such a message ("ECHDE_RSA" implies such a message, while "RSA" alone does not).

So this leads to a number of possible combinations of messages sent or not sent. Moreover, a number of extra RFC define some extra messages, that do not appear in the diagram above; e.g. RFC 5077 defines a new handshake message NewSessionTicket that may occur just before a ChangeCipherSpec from the server. Similarly, RFC 6066 defines the new message types CertificateURL and CertificateStatus. RFC 4680 also defines SupplementalData that both client and server may send to each other.

---

How you choose to classify all of these features, and whether they induce notable differences, is up to you. Moreover, there cannot be a definitive, exhaustive list of variants, since new extensions that trigger new message types or new handshake flows can always be defined afterwards.