Recently I got interested in encryption and the methods behind them. So for a fun / school project I am building a program in python to encrypt data between two clients (chat and files).
I'm just really not sure how to truly make it secure against both passive and active attacks. Right now my idea is to use a server to establish a handshake between the clients in the following manner:
- Once a client starts, it connects to server to let it know its alive.
- Client bob lets the server know it wants to connect to client Alice
- Server sends message to client Alice that bob wants to set up a secure connection
- Alice tells the server she accepts the connection
- The server generates a random number and encrypts it using the client public key The client has private RSA key hard coded (doesn't change per connection / client)
- Server sends the random number to both Alice and Bob
- Alice and bob decrypt the random number and hash it, both send it back to the server
- Server checks if both hashes are the same (none are tempered with)
- Server sends an OK or error to both parties
- Alice and bob hash the number a number of times equal to the random number (so if the server sends 1234 they hash it 1234 times.
- Alice and Bob setup a common key for this connection using Diffie-Hellman key exchange and encrypt the exchange with AES.
- Once a common key is decided, use this key or a hash of this key for AES encryption (this key does change per connection and all the data is sent over this channel)
The only problem I see with this protocol if adversaries can reverse engineer my code to get the private key, and can use active attacks at both Alice and Bob's connection OR if the server is ever compromised.
I will use py2exe to compile the code, that way it should be hard to get the key but I'm not sure just how hard it would be.
Sorry for the long question but it just feels like a really horrible idea to hard code the key into the client but I'm not sure where to go from here.
Thoughts on this idea / improvements / How can I do it better?
Sorry for any typos, English isn't my first language.