Why do I get the error "dh key too small" when the RSA key is 2048 bits?

Question

When I try to connect to the site https://api-mte.itespp.org with OpenSSL (openssl s_client -connect api-mte.itespp.org:443 -brief), it complains that the DH key is too small. But when I analyze its certificate, it says the RSA key is 2048 which I understand is a standard size for a secure key. Also, when I analyze the site with SSL Labs (https://www.ssllabs.com/ssltest/index.html), it says the server supports weak DH key exchange paramters which, again, I don't understand given the adequate size of the RSA key. Other sites I analyze also have 2048 bit key sizes but give no warnings. Why is this? I ask out of curiosity about how the TLS handshake and specifically how DH works. Thanks!

Answer 1

In general TLS 1.2 and below (TLS 1.3 has changed things, but your server doesn't support TLS 1.3 so it's not relavent) ciphersuites can be divided into two categories. Traditional and Ephemeral.

In traditional ciphersuites the servers long-term key is used directly in the key exchange process. For example in a traditional RSA ciphersuite, the client generates a random secret, encrypts it with the server's public key and sends it to the server.

The downside of this approach is if the server's long term private key is ever compromised, all previous sessions with that server can be decrypted.

In ephemeral ciphersuites, short-lived (ephemeral) keys are used in the key exchange process. The long term key is only used to authenticate the key exchange process. So a compromise of the long term key only enables active attacks on future sessions. It does not enable passive attacks or going back and decrypting past sessions.

RSA is not used* for ephemeral keys because RSA key generation is expensive. So when RSA certificates are used with ephemeral ciphersuites, there are two different asymetric algorithms involved. One for the key exchange itself and one for authenticating the key exchange.

The size of the keys used for key exchange in traditional diffe-hellman ciphersuite, depends on the parameters chosen by the server. In particular the "strength" of diffe-hellman is largely related to the size of the modulus. There is nothing in the protocol that ties the size of the diffe-hellman modulus to the size of the RSA modulus. Sensible server admins would likely use the same size for both but nothing forces them to do so.

The "negotiation" process for ciphersuites consists of the client sending a list of acceptable ciphersuites, and the server picking one. The client lists cipher suites in order of preference, the server may or may not respect the client's preferences.

---

Why do I get the error "dh key too small"

Fundamentally the issue is that there is no way for the client to say "I accept this ciphersuite, but only if the parameters are acceptable".

So your client supports one or more DHE-RSA ciphersuites, and the server has put them at the top of it's preference list, so they get chosen, but the server is using a dh key length that your client finds unacceptable.

1024 bit diffe-hellman, like 1024 bit RSA is now considered to be something that well-funded entities could plausibly crack.

Your server supports a couple of decent modern ciphersuites with sensible parameters, but for whatever reason they are near the bottom of the server's preference list.

Some clients may choose to start again, and make a new connection attempt with a different list of ciphersuites, web browsers are particularly known for doing that sort of thing. s_client doesn't though, it just reports the error to you.

Any particular reason why, for this server, when I specify to use an ECDHE cipher there is no complaint about they key size but when I use a DHE cipher (which OpenSSL appears to attempt by default) it complains about the key size? I think ECDHE uses smaller key size, but I'm not sure.

Generally we rate the strength of cryptographic algorithms by asking the question "given the best known attack, roughly how long would it take to break this". We often measure this in bits, n bits of security means it would take roughly 2ⁿ operations to crack the cryptography.

This is a very rough measure of the cost to the attacker. It ignores the cost of storage, it doesn't define exactly what an "operation" is, and generally when we talk about strength in bits we are talking about algorithms where (at least as far as public literature is concerned) the attacks are still theoretical. Nevertheless it's often the best we have.

Different algorithms need different key lengths to achieve similar levels of security. Symmetric ciphers are generally considered the "gold standard" for cryptographic strength, if a n bit symmetric cipher offers significantly less than n bits of strength it is generally considered broken.

RSA and traditional DH require much longer keys to achieve the same strength. The best known attack on RSA and traditional diffe-hellman uses the same basic algorithm, so RSA and traditional diffe-hellman give a similar "strength" for a given key length. 1024 bit RSA is generally seen as offering roughly 80 bits of security which is now considered within the range of well-funded attackers.

Elliptic curve algorithms on the other hand tend to have a security level of roughly half their key length. So a 128 bit security level "only" requires a 256 bit Elliptic curve key.

* Some of the historic "export" cipher suites did use ephemeral RSA, but these have been obsolete for a long time.

Answer 2

"dh key too small" is about the size of the key in the DH (Diffie Hellman) key exchange. This is unrelated to the size of the RSA key in the certificate. The key in the certificate is only used for authenticating the server, the DH key provided by the server is used for key exchange.

Some older servers have configured a DH key size which is no longer acceptable with today's security requirements. So this is a problem in the server setup, not in the certificate.