I need to communicate over insecure TCP sockets where SSL/TLS is not available. Does the following solution provide a secure means of authentication and encryption?
- Client sends: username
- Server generates a random token (31 bytes).
- Server retrieves a password salt and hash (bcrypt) from the database for the given username.
- Server encrypts the random token by the password hash.
- Client receives: passwordSalt + encryptAes(token, passwordHash)
- Client gets the passwordHash by using bcrypt on the password and the received password salt.
- Client gets the token by decrypting the encrypted token with the password hash.
- Client generates a random 'validation' salt (15 bytes).
- Client generates a hash of the token using the validation salt and encrypts it with the token.
- Client sends: encryptAes(validationSalt + md5(token + validationSalt), token)
- Server gets the hash by decrypting it with the token.
- Server compares if the received hash is equal to md5(token). If they are the same, the client has authenticated.
- Server generates a hash of the password hash using the validation salt, pads it with 15 random bytes, and encrypts it with the token.
- Client receives: encryptAes(randomBytes(15) + md5(passwordHash + validationSalt), token)
- Client gets the hash by decrypting it with the token.
- Client compares the hash with md5(passwordHash + validationSalt). If they are the same, the server has authenticated.
All data will now be encrypted with the token generated by the server.
The server is in Java, the clients are in C++ (UE4).
Public key encryption is currently not available.
Is there any flaw or weakness in this solution?