I am working on a secure messaging platform and am having trouble figuring out how to protect against MITM attacks.
My current setup can be summed up as follows: The server keeps a private key and makes the public key public which can then be installed into clients. When sending data to the server, it must be ciphered with the public key which will then be decrypted on the server and re-encrypted with the private key before being forwarded to the recipient.
ASCII sketch of normal situation:
Encrypted, can be intercepted but not deciphered-->Server--
Encryped with the private key, if intercepted can be broken since the public key is public--> Recipient
From that information alone we should see that the Sender->Server part is secure and a man in the middle wouldn't do anything (Trying something will corrupt the protocol and the connection will be terminated). However, the Server->Recipient part is not secure against MITM attacks.. at all!
To make matters worse, when data is sent, it is also sent back to the sender to confirm that it was sent, thus compromising both clients to MITM attacks.
So here is my question, upon the initial connection, how can the server prove that it is indeed the real server?
Evil Server must not be able to pretend to be the Real Server Client-->Evil Server-->Real Server
Real Server must be able to prove that he is, in fact, the Real Server. Client-->Real Server
Notes: This is done over TCP. I am looking for a solution that won't require heavy infrastructure but I can allow using a couple of external servers to check (However these too, will need a way of proving their authenticity).