What is end-to-end encryption and how to do it (correctly / securely)?

Question

//update 1 - there came some good replies (sadly mostly as comments instead of direct answers) - look further down for additional edit

I know this is a hot topic for the past few years, and although I would say I know quite some stuff about transport encryption (from the client to the server) I didn't yet got what's all about this "end-to-end" stuff. From as far as I could dig up information it's about securing the communication channel between the communication endpoints - maybe over several other connections - and maybe not in real time. Although TLS and S/MIME are topics I'm well familiar with I couldn't get my head around end-to-end.

So, please correct me if I'm wrong - I don't fully understand it yet:

"end - to - end" means securing the channel between two communication end points - this means if I have Alice and Bob (as often used in crypto) Alice and Bob are not necessary only linked by one connection like a client using TLS to connect to a server but maybe by several links like Alice connected to server A - which then is linked to server B to which Bob is connected (pretty much like IRC with multiple servers). In this case e2e means that Alice and Bob communicate over the IRC network without any of the servers being able to read the message unlike the case if Alice and Bob would only use transport encryption from their clients to the server but not e2e.

Ok, so when I stick to IRC: When both are online at the same time they could just do a TLS handshake over IRC (and spam others with their encrypted messages) - and although this shouldn't be possible by other security protections - how could Alice prove to Bob she's the real Alice and how could Bob prove to Alice he's the real Bob (prevent MITM)? Some rogue entity sitting between them at any point could just do a simple MITM. I read about this Alice and Bob sharing fingerprints out of band (so, in the IRC example: maybe by e-mail - if both know the correct e-mail-address of the other). Or maybe meeting in person and exchange fingerprints by scanning QR codes on their phones. But how to do it without out-of-band (if possible at all)?

And how about when both are offline? How would they establish a handshake to share a common secret (and some authentication data to prove the messages)? I'm lost about this topic and although tried to search for information I couldn't find any specific example showing how it's done (in any way - couldn't find any about doing it securely). Any help appreciated.

//edit 1: So, over time I got a better understanding about end-to-end and possible ways to implement it. First of all I learned that E2E is more of a concept rather than some specific protocol or such. Guess that's an important lesson to learn about that topic instead of just TLS wich is a clear defined protocol with known implementations. Next I learned that one has to think a bit out of band when it comes to verify that the one who I want to exchange messages with really is the one and not someone else. If you can't meet in person (one of the main verifications in weboftrust and pgp/gnupg) one (or both) can share their keys (and/or fingerprints) over several ways like random mailing lists, random comments on random blogs. An attacker would need to either controll all of them (unlikely) or successfully MITM the connection of at least one of the participants and by this break TLS' countermeasures to prevent exactly that. A possibility I didn't thought of before as it seems I was (and maybe still am) a bit to single-minded towards "I want some trusted instance like a CA" - if I want to get deeper into e2e I have to let lose that one.

Answer 1

We call communication e2e encrypted if one side encrypts the message and it remains encrypted on the whole way to the other side, so that only the other side can decrypt it.

Fir instance, Alice encrypts an Email, so that only Bob can decrypt it. Bob sends messages ecncrypting them so that only Alice can decrypt it. The email can be relayed via multiple servers and the whole time remains encrypted. The can use pre-shared key and symmetric encryption, or one time pad (it can be hard to implement in real life, but still it is a valid option), or - much more often - they can use PKI, for instance they both can obtain S/MIME certificates issued by some trusted Certification Authority and use public key of each other to encrypt messages. The same is applicable for messengers.

You are asking about offline. It does not matter, if one side is online or offline. When Alice is encrypting a message to Bob, Bob doesn't have to be online. In case they use PKI she has to know the public key of Bob. Alice uses his public key to encrypt a message to Bob and sends it to him. When Bob comes online, he receives the message and decrypts it.

To handshake: We speak about handshake in case of TLS. Here is direct connection between both sides is needed. That's why TLS is normally use only when at leas one of both sides has a fixed IP or fixed domain name. When two people communicate (e.g. via messenger) they are in most cases in the networks of their internet providers and have neither fixed IP nor fixed domain name. Means, Alice does not know how she can reach Bob to establish a connection with him. And Bob doesn't know how he can reach Alice. In such case they just cannot establish a TLS connection to each other. They can do that only if they are in the same network which happens rarely.

In many messengers users establish connection with the server, because the server has a fixed domain name and at any time it is known how to reach the server. Alice and Bob both establish connection to server, and the server forwards messages from Alice to Bob and vice versa. One handshake happens between Alice and server, the other one between Bob and server, but no handshake between Alice and Bob.

If Alice and Bob use their public keys to encrypt messages to each other, then the server cannot decrypt them. But many messengers don't support that and only have encryption between ALice and server and between Bob and server. Server obtains an encrypted message from Alice, decrypts it into plain form, encrypts it for Bob and sends it. This is of course not e2e encryption.

Answer 2

TLS has mechanisms to protect against MITM. In particular for your question, it provides sender authentication and integrity protection at the transport layer. So, it is providing a secure e2e transport channel. Now if applications are riding on top of TLS, say IRC for example, and you want e2e including the application layer as well, then the application itself needs to provide the necessary protections.

For offline cases, then application layer messages would be stored somewhere in transit (it depends on the application), possibly in a server in the middle or at the source or destination, until connectivity is restored. In such cases, they would need to remain encrypted and not decrypted for such storage, in order to not break the end-to-end principle.