Best encryption mechanism for message storage/exchange in client/server application

Question

So I am building a client/server system that is similar to email idea where clients exchange messages through the server with the following main requirements (note that the clients are pre-registered to the server and they have to authenticate themselves before any message transmission) :

• Each message should be encrypted and no one is able to read it except the sender and receiver.
• The encryption time should not take long
• The message should be stored encrypted on both the client and server sides(message encrypted once created even if client choose to store it and sent it later: Encryption at the application layer).

I proposed three solutions to accomplish that and I am still studying the pros and cons of each solution if you don't mind helping me:

A- Using symmetric key encryption with the server Each client has a shared secret key with the server Sender A encrypts their message with their shared secret key KA > server decrypts the message with KA and re-encrypt it with receiver B shared secret key KB > receiver B decrypts the message with his shared key KB

The challenge in this solution is the server having to decrypt then encrypt the message again. is it secure or not? why?

B- Using hybrid symmetric and asymmetric key encryption with the server

Server stores 2 different types of key for each registered client

1. Shared Key between server and client
2. Public Key of client

• When Sender A wants to encrypt a message to Reciever B

1. A communicate with server and get B public key Bpk using server shared secret Ka-s
2. A generates a shared secret for him and B and encrypts the message with it Ka-b(message)
3. A encrypts the generated shared secret with B public key and appends it to the encrypted message Ka-b(message), Bpk(Ka-b)

• When B receives an encrypted message

1. B gets Ka-b by decrypting it using his private key Bpri(Bpri(Ka-b))
2. B decrypts the message using the shared secret Ka-b

The challenge in this solution is the overhead to encrypt the message where the sender have to generate a key every time he wants to send a message.

C- Using symmetric without server involvement:

Each registered client have a shared key with any other registered client. If A wants to send message to B, he can fetch Ka-b stored shared key and encrypt the message with it then upload it to server.

In this solution the server won't be able to read the message thus less security required on the server.

The challenge here is storing all keys on each client application and managing the keys (synchronization between two clients when renewing an expired shared key )

Is there any better solutions to accomplish both requirements with high security?

Answer 1

Problem with A - Anyone with access to the server can decrypt the message because a copy of all symmetric keys is stored on the server. Plaintext of message may be stored in server RAM temporarily when re-encrypting message

Problem with B - Why not immediately Bpk(Message)? Why use an additional session key Ka-b(message), Bpk(Ka-b). For both methods, B is unable to authenticate A or guarantee the integrity of the message. Anyone else can also request Bpk from the server. Maybe you could do ApriK(sha1(message)), Bpk(message). So when B receives the message, he can use his private key to decrypt the message, request Apk from the server, decrypt the message hash and compare it with the message he received.

Problem with C - key management issue due to large number of keys. Total no. of keys in system = (n)(n-1)/2. Handshake problem. Non repudiation is also a problem when symmetric keys are used. A can claim that he never sent the message to B. B crafted the message himself using the symmetric key which is known to him.

Answer 2

If you can package the client with certificates, I suggest using TLS/SSL for your message encryption. Just package each client with the certificate(s) for your server(s) and you'll be able to verify communications with the correct server and that the messages are encrypted.