Storing data encryption key (client side)

Question

I have moderate knowledge of programming , and currently I am writing a client-server messaging application for windows. My project is written from scratch using sockets , and all it does is data transmission from one client to the server, and then form the server to the rest of the clients in a room (I have implemented chat rooms). At the moment , my data is transmitted as a row series of bytes (using .encode('utf-8') if you are familiar with python). But obviously , I need to create a proper encryption algorithm for it. I am planning on creating my own , so that I am sure of the lack of vulnerabilities (I know it can be the opposite , but I trust myself). However , I am not sure how am I supposed to store the encryption key. If I bluntly store the key in the client , is it considered secure ? Because generating a random key and sending it to the recipient sounds horrible un-secure , since even a toddler with MITM will be able to see it. So what do I do ? How to I generate and send/store keys so that I can be sure that it is at least somewhat secure

Answer 1

First of all, don't ever roll your own crypto for commercial software. It is very, very, very easy to make a mistake.

But if you're just doing this for a personal pet project, or perhaps as a school project, you can do something like this:

1. Client generates random symmetric encryption key
2. Client encrypts message using symmetric key
3. Client encrypt the symmetric key using your server's public key
4. Client sends both encrypted message and encrypted key to server
5. Server can decrypt the symmetric key using its private key, then use the symmetric key to decrypt the message.

This scheme is vulnerable in a number of ways, but it solves the specific problem you are asking about in your question.