How exactly is the used cipher decided when using TLS (mail)

Question

(I hope this is the right place, wasn't quite sure which of the stackexchange sites to use) I have the following question:

When exchanging mail with another company, the mails we are sending are encrypted using TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x0039). However, our Hello message shows: Cipher Suites (49 suites) Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014) Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a) Cipher Suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x0039) Cipher Suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA (0x0038) [...] and their Hello message (when they are sending mails to us) shows: Cipher Suites (49 suites) Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030) Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c) Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028) Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 (0xc024) Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014) Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a) Cipher Suite: TLS_DHE_DSS_WITH_AES_256_GCM_SHA384 (0x00a3) Cipher Suite: TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x009f) Cipher Suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 (0x006b) Cipher Suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 (0x006a) Cipher Suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x0039) [...]

From what I understand, the cipher order the client sends in his Hello should be used to pick the cipher.

Consequently I would expect TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014) to be used.

When we are receiving mails from them they are encrypted using TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014) as expected.

Are there any other factors I am not aware of? The "issue" happens with multiple other companies.

Answer 1

An important point to make is that the machine and software that the other company uses to send emails is not necessarily the same as the machine and software that is used to receive emails. And even when they are the same, the SSL configurations for "client" and "server" roles may be distinct.

If you want to infer the cipher suites used by a given server, you have to talk to it in its server role, with a scanning tool that will connect repeatedly and make handshake attempts to work out how the server reacts; one such tool is TestSSLServer. Two caveats here:

• If the SSL/TLS is done after an explicit STARTTLS command, then the tool will need to send that STARTTLS, and more generally handle the bit of SMTP protocol before the handshake. Not all tools do that (TestSSLServer, right now, does not).

• You should not scan machines other than yours. Be sure to obtain the other company's consent to do that.

In any case, to answer your specific question: in SSL and TLS, the client sends a list of supported cipher suites, ordered by preference. Then the server chooses. How the server chooses is entirely up to it. Some servers are courteous and follow the client's preferences. Other servers enforce their own preference order. Still other servers do a kind of mix between the two, e.g. they will enforce AES over 3DES (regardless of what the client claims to prefer) but would follow the client's choice for key size (AES-128 vs AES-256).

There are other parameters to take into account, too. For instance, an SSL client may announce, along with its support of cipher suites that do ECDHE, a list of elliptic curves that it can handle for ECDHE. The server might also support ECDHE but elect not to use it because it does not support any of the curves implemented on the client side.

Notably (and this seems plausible, since you observe a DHE cipher suite being chosen, not ECDHE), I have observed the two following things, which may match your situation:

• Some clients do not send the supported elliptic curves extension and/or the supported point formats extension.

• Some servers, when faced with a client that does not send either or both of these extensions, will conclude that no elliptic curve cryptography is possible with that client (regardless of the claims in the list of cipher suites), and fallback to non-EC cipher suites, e.g. some with DHE.

Answer 2

... the cipher order the client sends in his Hello should be used to pick the cipher.

The server is free to pick the cipher which suits the server most from the list of ciphers the client offers. Typical strategies are either to pick the first cipher the server support in the preference of the client (i.e. pick based on client preference) or to pick the first cipher the client supports based on the preference of the server (i.e. pick based on server preference). These two ways are often configurable at the server, like ssl_prefer_server_ciphers on with nginx but the server might also employ any other strategy to pick the cipher.

... and their Hello message (when they are sending mails to us) shows:

The ciphers and their order an application uses as a TLS client might be the same but does not need to be the same as the one it uses as a server. This means you cannot defer the cipher order when used as server from the order in the ClientHello . If client and server ciphers and order are the same depends on the specific application and TLS stack.