Diffie-Hellman protocol CA, confusing

Question

In the Diffie-Hellman protocol, if the participants have their own private key and a generated public key. How can we determine whether or not they have a trusted CA for authentication or just the original protocol wherein participants are not being authenticated?

Do we need to specify it, put a digital signature in the messages or can we assume authentication whenever there is a public key involved?

Answer 1

Diffie-Hellman (DH) is a key agreement algorithm, used to establish shared symmetric key material.

It is sometimes called the "Diffie-Hellman protocol" but that's a bit misleading. For DH certain steps need to be taken in order using specific data elements such as public keys. The DH protocol however doesn't specify exactly when those steps need to be taken nor the bitwise representation of those steps. Even worse, there are different ways of performing DH as well. To get just an idea you could take a look at NIST SP 56 revision 3 schemes.

INTERLUDE: If you read this document you can see that it is actually possible to authenticate using DH itself. For that a trusted public key needs to be available at the side doing the authentication. That means using a static key pair for the entity that is to be authenticated. Generally this could be done using a DH (or ECDH) certificate. Those are however hardly found in real life; most certificates rely on digital signatures rather than DH key establishment.

If and how an entity is trusted relies on the higher level protocol, e.g. a transport protocol such as TLS. In the latest TLS version 1.3 so called ephemeral-ephemeral Diffie-Hellman key agreement is required to be used. In that case the key agreement is separated from the authentication part: the DH parameters are just simply included in the final authentication but the agreement itself does not play a part in it.

This also means that the presence of a trusted CA certificate can only be detected by looking at the X.509v3 based authentication phase that happens after DH key establishment. That is: if DH takes place at all, of course. You would expect it for e.g. server authentication but other authentication methods may be deployed as well.

Short answer after the long explanation: yeah, you need to specify it together with the rest of the higher level protocol you try to implement.

Answer 2

if the participants have their own private key and a generated public key

Diffie-Hellman doesn't have a public and private key. Let's imagine you're in a room with at least three people: Alice, Bob and Eve. Alice and Bob want to talk privately (and going into a different room is too easy), so Alice proposes to use Diffie-Hellman:

Alice: Let's do a DH key exchange. Pick a random number and raise 5 to the power of that number. Divide the result by 23 and tell me the remainder.

Alice picks 16, 5**16=152587890625, and the remainder when divided by 23 is 3.

Bob picks 25, 5**25=298023223876953125, and the remainder when divided by 23 is 10.

Bob: 10.

Alice: I got 3. Now raise my number to the power of that number again, and divide by 23 again, and take the remainder.

Alice raises Bob's 10 to the power 16, 10**16=10000000000000000, and takes the remainder when dividing by 23, 10000000000000000%23=4

Bob raises Alice's 3 to the power 25, 3**25=847288609443, and takes the remainder when dividing by 23, 847288609443%23=4

They got the same result. They have a common number that Eve does not know. Eve only heard that Alice sent 3 to Bob, and that Bob sent 10 to Alice. She cannot compute the number 4.

Alice: Now use this key for AES and let's speak encrypted!

Of course, in the real world, the numbers are all much bigger. Guessing that the password for AES is "4" is not very difficult.

Do we need to specify it, put a digital signature in the messages or can we assume authentication whenever there is a public key involved?

In the above example, Alice and Bob could see each other. If Alice spoke, Bob could see that is was Alice. On the Internet, this is not the case. It's more like communicating by letter: anyone delivering mail could inject fake letters.

So you still need to digitally sign this exchange, to be sure that a (wo)man in the middle does not inject anything into the conversation. Diffie-Hellman cannot do this, so you need an additional algorithm: one that uses public and private keys. The most popular systems to do this are the RSA algorithm and elliptic curve cryptography.

How digital signatures work is explained abundantly elsewhere. Just apply digital signatures to the above conversation (where Alice gives the parameters 5 and 23, where Bob says 10 and where Alice says 3) and you have an authenticated key exchange protocol.

But to exchange public keys, you have the same issue: how do you know that nobody in the middle switched out the real public key for their own public key? That is what Certificate Authorities (CAs) are used for: they are a trusted third party, and they put a digital signature on the public key. Another way of solving this is TOFU: Trust On First Use. This has the disadvantage that, if your first communication was already intercepted, then your communication is compromised. But it has the advantage that there is no CA system needed, and thus there is no CA that can be compromised.