First, I realize trying to come up with your own crypto schemes is not a good idea. I'm a relatively new member of a team of developers utilizing such a custom scheme and I'm trying to come up with arguments to argue against that policy.

To secure communications between a client and a server, we have a standard Diffie-Hellman implementation as described on Wikipedia but modified in such a way that p (modulus), g (base) and b/B (the server's private/public key) are static. p, g, b are constants in the server code, while p, g and B are embedded in the client code.

The client then picks a, computes A, computes the shared secret, uses that as a key for AES256-GCM, then embeds the encrypted payload & A in one message to the server, which can then derive the shared secret and decrypt as in standard DH.

Standard Diffie-Hellman provides no authentication and so is vulnerable to MITM - in this case, since the server's public key is fixed, it seems that this weakness is avoided?

I've been reading about ephemeral vs static DH modes, the article on Wikipedia has a section on the subject. It seems like we're using the ephemeral-static variant. However, the information I found is usually in the context of SSL/TLS ([1], [2]), which incorporates X509 as an authentication mechanism, so for purposes of considering MITM weaknesses, that seems different.

In summary:

  • does our modification of DH (alone) provide sufficient security against man-in-the-middle attacks?
  • does it break anything in Diffie-Hellman otherwise?
  • 2
    Embedding B (the server's public key) at the client essentially amounts to public key pinning, which is an effective measure against MITM attacks. Having said that, you can implement public key pinning with most implementations of TLS, so there may not be any need for you to re-invent the wheel here.
    – mti2935
    Feb 24, 2022 at 17:24
  • 1
    Note that with integer DH, the work needed to crack any given key exchange can mostly be pre-computed if you know the parameters that are hardcoded here. It's still computationally very expensive, and quite large to store, so you might be fine if you use a large enough key. However, if somebody with enough resources decides to crack your key exchanges, you've made it relatively easy for a (large) one-time effort to be enable easily breaking all future keys. That's even more true if b ever leaks; you have no forward secrecy.
    – CBHacking
    Feb 25, 2022 at 11:26
  • 1
    By the way, this might be better suited to crypto.stackexchange.com. Modifications of cryptographic algorithms are really their domain. Security.SE is more for "is modifying a cryptographic algorithm a good idea?" questions. (The answer is "No". It's almost always No, but it's especially No here since I don't see what this modification even gets you vs just using TLS with a pinned certificate.) Even if your key exchange is flawless (aside from not being forward-secret), there's still lots of room for error in the bulk encryption. Just use TLS and don't reinvent it.
    – CBHacking
    Feb 25, 2022 at 11:31
  • I agree 100% with @CBHacking, especially wrt PFS. wrt x509, x509 is simply a format for storing certificates, which in turn contain public keys. You might want to consider using a tried-and-true TLS library, and creating a self-signed certificate (in x509 format) using openssl, then pin this certificate at the client. This will provide authentication (for preventing MITM's), secrecy, integrity, and PFS if you use a modern version of TLS - without re-inventing the wheel.
    – mti2935
    Feb 25, 2022 at 13:32


You must log in to answer this question.

Browse other questions tagged .