Our development team is implementing TLS protocol for a web server. The type of clients are mobile apps and web browsers.

Now there is a concern about bypassing TLS in any way trough MITM attacks and disclosure of the server's private key.

Is there any solution independent of TLS for data-in-transit protection so that developers use it at the application layer in parallel with TLS?

updated section: according to owasp recommendation : If possible, apply a separate layer of encryption to any sensitive data before it is given to the SSL channel. In the event that future vulnerabilities are discovered in the SSL implementation, the encrypted data will provide a secondary defense against confidentiality violation

  • 9
    Just for the avoidance of doubt: the server's private key is of course never sent to clients. It would take a fatal weakness in the TLS protocol or implementation for it to be revealed. – jcaron Dec 1 '20 at 13:42
  • 16
    Never, ever invent cryptography yourself unless either you are playing or you are a educated cryptographer and intend to show it for a review to a great number of other cryptographers, at least some of them reputable. schneier.com/blog/archives/2011/04/schneiers_law.html – fraxinus Dec 1 '20 at 13:53
  • 5
    Designing your own will more likely make your system very insecure, susceptible to side-channel attacks, etc. Also lack or peer review and scrutiny will likely keep it vulnerable indefinitely and you will always be one step behind the bad guys. Use standard approach and libraries with a proven security record. – akostadinov Dec 1 '20 at 22:30
  • 7
    >Now there is a concern about bypassing TLS in any way trough MITM attacks and disclosure of the server's private key. < I think there is a huge misunderstanding. TLS prevents MITM thanks to certificates. TLS is the state of the art. So the OP should not look for replacing TLS – usr-local-ΕΨΗΕΛΩΝ Dec 2 '20 at 9:09
  • 3
    I don't understand your question. No encryption mechanism ever can protect you in the event that the attacker has obtained your private key. That's so trivially obvious that I must be missing something. What? (I mean, you can simply nest encryption as deep as your CPU allows -- but if your private key has been stolen you must assume you have been pwned, right? That all private keys have leaked?) – Peter - Reinstate Monica Dec 3 '20 at 7:25

For web clients, you are out of luck, the only two protocols supported by browsers are HTTP (without any security) and HTTPS (with TLS).

For mobile applications, if you really must use another protocol than TLS 1.3, here is my recommendation:

First, you can use a library like libsodium to encrypt data and handle any cryptographic function. Have the server's public key in the application to authenticate the server. To authenticate the clients, you can derive a key from the user's password.

Second, and that's the most important, transmit this encrypted data in a TLS 1.3 tunnel, over HTTPS. This way, you can tell your management that you are resilient in case TLS is broken, but you still benefit of all the security provided by TLS that you cannot achieve with your custom implementation.

You can try to do something similar for your web application using a JavaScript cryptography library. However, please keep in mind that is is only to ease the management requirements. In practice, this adds zero security against an active eavesdropper. JavaScript cryptography is only useful when the user trusts the server and the connection, the later needing TLS.

Also, instead of naming your protocol TLS, you can name it by the full name of the cipher suite used by TLS, like "ECDHE_RSA_WITH_AES_GCM_SHA256" for example (it's the current suite for security.stackexchange.com).

  • Note that having your mobile application use a more secure protocol provides only minimal protection from a TLS compromise. Attackers can just attack the web client instead. – Brian Nov 30 '20 at 22:05
  • @Brian I wouldn't call having a more secure protocol for an entire subset of users "minimal protection". The value of the protection really depends on the context of the app and the goal of the attacker. – Jon Bentley Dec 1 '20 at 9:53
  • @A.Hersean this library provide additional layer for Encyption so authentication and integrity is base on TLS1.3. OK? – Mahzad Dec 1 '20 at 12:50
  • 1
    @Mahzad This library supports authenticated encryption (ensuring integrity, data authenticity and confidentiality): libsodium.gitbook.io/doc/public-key_cryptography/… Please read its documentation, it is very complete and didactic. Authentication of the user is neither supported by TLS nor libsodium, you need another protocol for this, like OAuth for example. – A. Hersean Dec 1 '20 at 12:57
  • 2
    +1 for being the only answer that actually answers the question. I would add a warning about JS crypto libraries: a MITM attacker could simply replace the crypto library with their own version (or strip it out entirely), making it worse than useless (because it then creates a false sense of security and might do other things that benefit the attacker). – Brian Drake Dec 2 '20 at 7:15

Rejecting TLS out of fear an attacker could have stolen a private key is like rejecting medicine out of fear someone could have poisoned it. The downsides far outweigh the risk of not using it.

Why TLS is secure

There are many many angles to answering this, but one that speaks for itself is that everyone is using it. Government agencies are using it, huge tech companies are using it, banks are using it, hospitals are using it, StackExchange is using it too. If TLS would be insecure, don't you think at least someone would decide that it's a bad idea to use it and switch to someone else? The fact that TLS is nearly ubiquitous and recommended everywhere by everyone should tell you that it's a good idea to use it.

Furthermore, TLS is secure, if configured correctly. Version 1.3 makes this a no-brainer, as of the time of this writing, there is no wrong way to configure TLS 1.3. TLS 1.2 is a bit more difficult, since there are some insecure ciphers in TLS 1.2. SOGIS - which also use TLS - have an extensive guide on which ciphers are recommended, and which are still tolerable for legacy usage.

Finally, compromise of the private key leading to insecure communication isn't a flaw with TLS that some other protocol could alleviate - it's a flaw inherent to network communication. If you assume that an attacker has full control over your server, then no matter what protocol you're using, the attack would be able to decrypt any traffic, manipulate any traffic and forge new traffic.

In other words, TLS is not designed to protect against server compromise, and neither TLS nor an alternative to TLS - self-made or otherwise - would protect against that.

Alternatives to TLS

Since the question asks for alternatives to TLS, there is one: DTLS

DTLS is basically TLS, but for UDP. The reason you might want to use DTLS is because your application uses UDP instead of TCP (e.g. a VoIP program), but you still want your datagrams to be encrypted.

DTLS is not more or less secure than TLS, but instead is designed to work with a different underlying layer.

  • Comments are not for extended discussion; this conversation has been moved to chat. – Rory Alsop Dec 1 '20 at 12:27
  • This does not answer the question as currently written: "Is there any solution independent of TLS for data-in-transit protection so that developers use it at the application layer in parallel with TLS?" (emphasis added) In fairness to the answerer, this was not clear from the question at the time this answer was posted. – Brian Drake Dec 2 '20 at 7:05
  • @BrianDrake This answered the question as it was written at the time. – MechMK1 Dec 2 '20 at 10:31
  • "If TLS would be insecure, don't you think at least someone would decide that it's a bad idea to use it and switch to someone else" <- the OP is doing exactly that – user253751 Dec 2 '20 at 12:35
  • @user253751 I should have clarified that I mean someone who has a proven track record of having extensive knowledge about security - which OP does not have. – MechMK1 Dec 2 '20 at 12:40

This seems like an X-Y problem. The question is "Management are concerned about the potential for TLS related attacks. How do I layer more encryption on TLS?". Other answers have covered how you could layer on additional encryption, and why you shouldn't, but let's look at how you should deal with the real problem: management's concern. Let's turn their concerns into threats we can model.

One threat you mentioned is that the server private keys are compromised. This does not automatically lead to your users being compromised, since the attacker also needs to intercept traffic between your users and your servers. This is not impossible (there are a few attacks that could achieve this, such as DNS or BGP related attacks, or your users using a fake WiFi hotspot, maybe some ARP attacks if they can get onto the same local network as a user), but it does require a second vulnerability.

There are a few mitigations you can put in here. One is to rotate your server keys frequently, so an attacker doesn't have access for very long. Keeping your server keys in a hardware security module also makes this much much harder to exploit. Another is to only support cipher suites with forward secrecy, since this requires attacker to actively intercept, rather than passively eavesdrop on your communication. If your application supports it, you could use client-side certificates, which would mean an attacker would need to obtain both client and server certificates to successfully MITM the connection. Another important mitigation is to put intrusion detection in-place on your servers. You can also potentially monitor for BGP and DNS related anomalies, which would prevent the attack escalating to the point of exploitability.

You also mentioned MITM as a possible threat. This is the flipside to the previously mentioned risk, and again needs an additional break in order to become exploitable. Leaked server certificates are one (as discussed before), but there are others, such as certificate mis-issuance or SSL stripping.

You can mitigate the MITM risk by proactively monitoring for certificate mis-issuance. Check https://crt.sh for new certificates issued for your domain. If you control both sides of the communication, you can eliminate the possibility of certificate mis-issuance by not using or trusting certificates from a public CA at all, but issuing certificates from an internal CA. You should also look at HSTS and potentially HPKP (although the latter has risks of its own). If your service is only offered in a well defined geographical region, GeoIP blocking may also help. Client side certificates also help here. Secure cookies can help with the SSL stripping risk too.

So the remaining question becomes what mitigations can be put in place against the threat of an attacker who has both successfully obtained your server private keys for your domain, and has man-in-the-middled your users? At this point, it's probably worth mentioning that whilst defence-in-depth has value, it also has a cost: complexity. There is a possibility that the added complexity of the new measures add more surface area, and hence more risk, than they mitigate. And if you've got most of the mitigations detailed above, this becomes a very unlikely threat.

Still, there are some potentially viable mitigations here. One is to design the system with data-level encryption, that the server does not have the keys to (only the user does). This way, your system may still be somewhat secure even if your server is totally compromised. Another is simply to limit how much trust is put in the web interface - maybe you can ensure that admin actions can't be done from the public web interface, only from an internal interface that's only available from your internal network. Confirming high-value actions out-of-band may also help (e.g, phone users to confirm when they ask to do a high-value action).

But as others have pointed out, these risks probably aren't the most pressing your application faces. The threats of attackers identifying an off-the-shelf component with a known vulnerability, identifying an SQL injection vulnerability, discovering that one of your APIs doesn't do the authorization checks it's supposed to, finding a SSRF vulnerability that leaks your AWS keys, finding an endpoint with CSRF or XSS weaknesses, discovering user data or account credentials in an unsecured S3 bucket, phoning your support desk claiming to be your system administrator and have forgotten their password, or just launching a DDoS attack, are far more likely than the threats in the question.

  • 2
    I don't know what the rules are for frame challenges here, but this doesn't seem right. It doesn't seem to explain why implementing another layer of security over TLS is a bad idea, nor does it explain how to do so, so I am inclined to flag it "not an answer". – Brian Drake Dec 2 '20 at 7:12
  • One problem with TLS is that clients must either have a means of ensuring that they will always receive new root certificates before old ones expire (which may be difficult if they may go years without direct access to the Internet) or of securely uploading replacement certificates after the old ones expire. Protocols using shared secrets, however, need not have expiration dates. – supercat Dec 2 '20 at 8:32
  • @supercat, you usually create a very secure CA and only sign server certificates with it. So obtaining server cert, you can invalidate it and create a new one without distributing new CA to clients. – akostadinov Dec 2 '20 at 10:00
  • @BrianDrake With crypto, just layering more stuff on isn't entirely unproblematic. Cryptography is like medicine, in the sense that you should only use it as directed, and mixing things can have possibly disastrous consequences. – MechMK1 Dec 3 '20 at 12:55
  • @BrianDrake I've added a bit more preamble explaining why I've answered the question this way. I think the problem is really an X-Y problem - other answers have done a good job at the X (whether and how to layer on more crypto), but I felt the Y needed covering (how to mitigate concerns about possible TLS-related threats) – James_pic Dec 3 '20 at 13:24

Now there is a concern about bypassing TLS in any way trough MITM attacks and disclosure of the server's private key.

Is there any solution beside TLS for data-in-transit protection?

There is no known encryption scheme that can protect against MITM attacks if you assume the server's private key can be disclosed. Any solution will have this same vulnerability.

The best known defense against MITM attacks is the use of certificates. TLS already supports this.

The best known defense against compromise of the server's private key is HSMs. TLS already supports this.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.