Does TLS fall back to unauthenticated key exchange if the CA is not known to the client?

Question

My lecturer said that if the server sends a public key certificate (RSA) signed by a Certificate Authority that the client does not know about (i.e. the client doesn't have a master certificate for the root CA in that chain), then the key exchange protocol falls back to some form of unauthenticated Diffie-Hellman (e.g. ECDHE or DHE). Quoting from the slides:

• If the server does not have a certificate signed by an authority that the client has a master certificate for then an alternative has to be used.
• The most common is to use Diffie-Hellman. This is then vulnerable to a man in the middle attack.

It seems strange to me that, if the client doesn't recognize the CA, it just skips any attempt at authentication and goes ahead with the connection anyway.

So I asked him whether this is something that happens automatically, i.e. whether this (ClientHello -> ServerHello -> client doesn't recognize CA, asks server to fall back to DH) happens automatically within a single TCP/TLS connection. He said, essentially, yes, that the client can then decide whether to fall back to DH or to abort the connection, and in the first case, there is some additional dialogue between client and server to make that happen.

Is there any TLS cypher suite or sub-protocol that entails this behaviour?

Or maybe I misunderstood: maybe this does happen, but not at the TLS level but rather at the application level? E.g. a browser tries to access a web page with an authenticated TLS ciphersuite, the connection fails because the client doesn't recognize the CA, so the browser asks the user whether they want to go ahead in "unsafe" mode (i.e. with some unauthenticated TLS cipher suite), and if the user accepts a new connection is established but this time using another TLS ciphersuite.

Answer 1

I am being generous and I assume your lecturer is merely extremely out of date and they meant anonymous TLS cipher suites.

In the 90s, a TLS client might have sent a ClientHello which lists cipher suites in preference order and included something like "TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_DH_anon_WITH_3DES_EDE_CBC_SHA". The TLS server might have looked at it and decided that since it doesn't have an RSA certificate (because it has a DSA certificate or a static DH certificate or no certificate at all) it can't use the first option, so it's going to use the second option, and they would have used unauthenticated TLS.

They may as well also use TLS_NULL_WITH_NULL_NULL (yes, no key exchange, no authentication, no encryption and no MAC).

In the real world, those cipher suites are not allowed. Decent TLS 1.2 implementations simply don't support them. TLS 1.3 doesn't include such things even in theory, AFAIK.

Additionally, there has never existed a mechanism by which a TLS client can look at the certificate chain offered by the server and change its mind and ask for a different cipher suite in the same session. It's too late, the cipher suite has been chosen already.

A TLS client can try some values in ClientHello, look at the response from the server, decide it doesn't like it, close the connection and try again with different values in ClientHello. This might be required to deal with server bugs related to negotiation, in particular the famous TLS version intolerance. SCSV has been invented to prevent downgrade attacks on such clients.

Almost universally, the TLS client does not tell the TLS server which CAs the client trusts. The "TrustedAuthorities" extension exists (and a TLS 1.3 replacement), but I've never seen it used. If it's used, it is used only by small embedded devices (same as Maximum Fragment Length).

In common use, a TLS client tells the server:

1. The supported TLS protocol version.
2. Optionally, the domain name, using SNI (this is often needed to make HTTPS work)
3. Optionally, the next protocol (using ALPN), which is needed for HTTP/2.
4. Optionally, which kind of digital signature system and which kind of hash function (TLS 1.3 version) the client is willing to use to verify signatures, which is an opportunity for the server to choose between RSA and ECDSA certificate, if it has both types for the domain (or at all, if selection by domain name is not used). At a time when SHA1 certificates were not allowed in general but still allowed for clients that do not support SHA2, this was done.

A TLS client might try different values of all that (and more) to see if the server sends different certificate chains (or cipher suites). No TLS client I know of performs the fallback retries conditionally on user input. Either it performs retries or it doesn't, based on policy, and at the end it comes back to the user with the best result it was able to get and either shows "secure" indicator or shows an "insecure" indicator or shows a "full page interstitial" (a full page error with a hidden "disregard and continue" button, like this).

Note that the TLS protocol really is meant to work without the client probing different handshake options. For example, consider a server that has a preference for cipher suites in a particular preference order which is ordered in a silly way. A client that offers "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA" will get CBC instead of GCM, even though the server supports GCM. The client might see the choice of CBC and try a different handshake in which it offers only GCM. If it then gets GCM, it might learn that the server is silly and it should craft handshakes for that server in which it does not offer CBC. The TLS protocol authors would like it if people did not have to do this, and so they would prefer if TLS servers always made sensible choices given any particular ClientHello. GREASE was created to try to fix one reason for such fallback retries.

BTW, TLS 1.3 made significant changes, so much that they thought to call it TLS2.0. You should look at the RFC.