Presumably the SHA is for deriving the AES key from the shared secret. Where else is the hash used?

ECDH just does ECC (no hashing). RSA does masking and padding but this doesn't involve the negotiated hash, does it?

For AES-GCM we don't need to stir the IV or anything between messages, we can just increment, so as far as I can tell key derivation is the only place to use it.

I have TLS and SSH in mind. (I realise the details of the protocols are a bit different. It's just confusing reading the RFCs, where a hash is negotiated directly or with a cipher suite, but it's not obvious what it's used for.)


For SSL, there is SSL 3.0, TLS 1.0, TLS 1.1 and TLS 1.2.

In SSL 3.0, an internal Pseudorandom Function (PRF) is used to extend the shared secret (obtained from the key exchange mechanism) into symmetric keys for the subsequent symmetric encryption. This PRF always uses both MD5 and SHA-1.

In TLS 1.0 and TLS 1.1, the PRF is replaced by a new construction, which still uses both MD5 and SHA-1, in an arguably "better" way (cryptographically speaking).

MD5 and SHA-1 are also used in SSL 3.0, TLS 1.0 and TLS 1.1 to compute the signatures in the handshake (a signature from the client when using certificate-based client authentication, a signature from the server when using a DHE cipher suite); that which is signed will be hashed with either SHA-1 (for DSA/ECDSA) or with both MD5 and SHA-1 (for RSA).

Also, exchanged handshake messages are hashed, again with both MD5 and SHA-1, to compute the Finished messages which conclude the handshake; in the case of SSL 3.0, the two hashes are used "as is", while with TLS 1.0 and 1.1, an extra PRF invocation is involved.

Beyond the handshake, the application data will be encrypted with some symmetric algorithm with an integrity check and that integrity check will normally use HMAC (in the case of SSL 3.0, a HMAC-derivative) which will be based on a hash function specified in the negotiated cipher suite. Cipher suites which end with "MD5" use MD5 at that point, while cipher suites which end with "SHA" use SHA-1.

With TLS 1.2, things change. The PRF and is still there, but uses only one hash function, and that hash function is the one negotiated in the cipher suite. The same hash function will be used to hash the handshake messages for the Finished message (by the way, that's cumbersome for embedded RAM-starved implementations, because implementations must then either buffer the ClientHello and the start of the ServerHello, or start a bunch of hash functions, because the hash function which will be used is not known until the ServerHello is parsed). When the cipher suite ends with "MD5", the hash function is MD5. When it ends with "SHA", that's SHA-1. When it ends with "SHA256", that's SHA-256.

When GCM is used, there is no per-record HMAC; integrity is obtained from the GCM mode itself. So the hash function specified in the cipher suite is used only for the PRF and other handshake-related usages.

Now the tricky point: there is no standard GCM cipher suite for TLS before TLS 1.2. GCM cipher suites for TLS begin with RFC 5288; others are defined in RFC 5289 and some subsequent RFC. All currently defined cipher suites are published in the IANA registry. All the GCM cipher suites are for TLS 1.2+ only, and, moreover, they all use either SHA-256 and SHA-384, not SHA-1. Correspondingly, as of July 2013, there is no standard way to use both SHA-1 and GCM in TLS. If you want GCM (with AES or some other algorithm), you will need TLS 1.2, and the hash function (for handshake purposes) will be SHA-256 or SHA-384. Presumably, cipher suites with SHA-512 may be defined as well, but this has not occurred yet.

Of course, every rule has exceptions. Public keys, in SSL, are exchanged through X.509 certificates, which are complex objects using a lot of cryptography, and you will often find some SHA-1 or even MD5 lurking there. But that's mostly out of scope for TLS, and, in particular, is not impacted by the negotiated cipher suite. Theoretically, client and server may/should negotiate supported hash functions for that, but in practice the server will send the certificate it has obtained from its CA, and has little choice in the matter.

For SSH, the algorithms are negotiated independently of each other; there is no real notion of an all-encompassing cipher suite. See RFC 4253, section 7.1. RFC 5647 defines use of AES/GCM with SSH, but does not care about the hash function which will be used internally; of course, there will be no hash-based MAC for bulk data encryption if GCM is used (that's the point of GCM: it comes with its own MAC), but a hash function is still used internally for, at least, key derivation (similarly to what is called the "PRF" in SSL).

RFC 6239 is more thorough in that it tries to define sets of algorithms for SSH, following the so-called NSA Suite B. This means elliptic-curve Diffie-Hellman ("ECDHE" in SSL terminology: with SSH, the key exchange always uses "ephemeral keys"), AES/GCM... and SHA-256 or SHA-384, not SHA-1. Also, when strictly following RFC 6239, the SSH server public key (the permanent one, used for signatures) should use ECDSA, not RSA.

So you can, at least theoretically, used SSH with ECDHE, RSA signatures, AES/GCM and SHA-1, but when you use GCM you are encouraged to also switch to SHA-256 or SHA-384. What SSH implementations actually do is prone to depend on the exact software version, compilation options and local configuration, and needs testing.

  • 1
    Great answer. That's what I thought, I wasn't expecting such a thorough reply! – Nicholas Wilson Jul 26 '13 at 17:34
  • Are you certain that TLS 1.2 will sometimes use MD5 or SHA-1 in the PRF? Section 5 of the RFC says, "This PRF with the SHA-256 hash function is used for all cipher suites defined in this document and in TLS documents published prior to this document when TLS 1.2 is negotiated." – Matt Nordhoff Aug 22 '14 at 12:07
  • FYI this PRF difference between TLS 1.2 and older versions is the basis for the recent SLOTH attack on TLS. Because TLS 1.2 allows the downgrade of this PRF (HMAC) to MD5, which (surprisingly) is not possible for older TLS versions. – rugk Mar 15 '16 at 20:33
  • @rugk: SLOTH in 1.2 is not due to the PRF change (in section 5), but to a simultaneous and similar but separate change to digitally-signed in section 4.7. – dave_thompson_085 May 26 '17 at 10:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.