I need to communicate over insecure TCP sockets where SSL/TLS is not available. Does the following solution provide a secure means of authentication and encryption?


  1. Client sends: username
  2. Server generates a random token (31 bytes).
  3. Server retrieves a password salt and hash (bcrypt) from the database for the given username.
  4. Server encrypts the random token by the password hash.
  5. Client receives: passwordSalt + encryptAes(token, passwordHash)
  6. Client gets the passwordHash by using bcrypt on the password and the received password salt.
  7. Client gets the token by decrypting the encrypted token with the password hash.
  8. Client generates a random 'validation' salt (15 bytes).
  9. Client generates a hash of the token using the validation salt and encrypts it with the token.
  10. Client sends: encryptAes(validationSalt + md5(token + validationSalt), token)
  11. Server gets the hash by decrypting it with the token.
  12. Server compares if the received hash is equal to md5(token). If they are the same, the client has authenticated.
  13. Server generates a hash of the password hash using the validation salt, pads it with 15 random bytes, and encrypts it with the token.
  14. Client receives: encryptAes(randomBytes(15) + md5(passwordHash + validationSalt), token)
  15. Client gets the hash by decrypting it with the token.
  16. Client compares the hash with md5(passwordHash + validationSalt). If they are the same, the server has authenticated.

All data will now be encrypted with the token generated by the server.


The server is in Java, the clients are in C++ (UE4).

Public key encryption is currently not available.


Is there any flaw or weakness in this solution?

  • I understand that TLS may be complex to implement on some devices, but why isn't public key encryption an option? – Neil Smithline Mar 14 '16 at 16:05
  • It simply isn't implemented yet on the client side. I can't use libraries like boost, because then I won't be able to export to HTML5, Android, PS4, etc. I could try to implement RSA for example myself, but I don't know enough about RSA to make my own implementation, and I can't find any examples for C++ that would work together with Java. – Zwarmapapa Mar 14 '16 at 16:10
  • Zwarmapapa - the original formatting was confusing. I tried to fix it without changing the meaning of your question. Please edit your question if you feel I changed the meaning or somehow made your question less clear. – Neil Smithline Mar 14 '16 at 16:10
  • One thing i can see is bruteforce on validationSalt + bcrypt(token, validationSalt) or at bcrypt(passwordHash, validationSalt) – Sravan Mar 14 '16 at 16:13
  • 2
    @Zwarmapapa Have you looked at/tried libraries like OpenSSL? Nobody is expected to implement TLS on their own these days... It's supported on most platforms and for the web you can readily use HTTPS/WSS with some abstraction layer. – billc.cn Mar 14 '16 at 16:23

You basically created a shared secret authentication scheme here.

However, you did not define what your security goals are. Entity authentication, strong entity authentication, confidentiality, anonymity, ...? So it's hard to say "(yes/no), this is (not) secure". If your security goals were none of the above, this is probably secure. But I wouldn't use this thing (hope you don't mind)! :-)

You seem to assume that the authentication key is the password. However, this is not the case as knowing the hash from the database is sufficient to successfully authenticate. Try it out and play the client which only knows the password hash and you will see that it works.

The problem here is that you didn't specify what exactly the secret is. Knowing that you can make the whole process much simpler. Assume the secret is the hash from the database and you want to authenticate the user to the server:

  • Server generates a session key and a random number, encrypts both with the secret and sends it to the client
  • The client generates the shared secret by asking for the password
  • The client decrypts the session key
  • The client adds 1 to the random number, encrypts it with the session key and sends it back to the server

As you can see the security still relies on knowing the hash from the database, but setting up a shared secret and authenticating the client has become much simpler. Please note that this is still far from secure! I only wanted to show that things are often not what they seem. Fiddling around with salts doesn't make things necessarily better.

The point is: Complicated does not mean secure, and - as pointed out before - do not trust you self-made crypto ;-)

Edit: Just for the record, think about oracles, chosen protocol attacks, reflection attacks, ... get yourself used to these things and try find attacks against your own protocol. Makes fun and you will learn a lot.

  • Won't your given example be weak against replay attacks from the server for example? How do you know the server is the actual server? Anyway, I like the tips and help, but saying "it's probably not secure" doesn't really help me that much :P In order to know what I'm doing wrong I'd need something a bit more specific, like "x is wrong because x can do x". Anyway, the hash from the database is indeed the secret, and both sides being able to trust each other is the goal. Also, didn't know this had an official name, I'll try looking up some examples of shared secret authentication schemes. – Zwarmapapa Mar 14 '16 at 17:54
  • +1 for the pass the hash notification. – Neil Smithline Mar 14 '16 at 18:05
  • @Zwarmapapa "being able to trust each other" would indicate you want mutual authentication. Please excuse me not writing things down more specifically but analyzing your construct is just beyond scope. Check out e.g. "Protocols for Authentication and Key Establishment" which is available via google books online (partly) to get an understanding of the basic principles first. – fr00tyl00p Mar 15 '16 at 9:53

Your solution could be affected by the following concepts:

Do-it-yourself Cryptography

DIY crypto is an anti-pattern of secure software design. Most (>99%) attempts will fail. Nevertheless I respect your try! :-) For example: Don't reveal the "passwordSalt", if you don't need to. It helps the attacker to crack the challenge. After one request an attacker knows the salt and has a challenge which could be cracked. The security relies only on a strong password. The attacker can crack the password offline. This should not be the case for online services. I rate this as highly insecure.

Perfect Forward Secrecy

The key for the encryption (serverside generated token) should be calculated on both sides (client and server) using Diffie-Hellman Key Exchange or similar. If someone recorded the traffic and gets the user password, he is able to decrypt the token and then decrypt the whole traffic, afterwards.

Steps 13-16 not needed

If the client can decrypt "passwordSalt + encryptAes(token, passwordHash)" successfully, the server had to know "passwordHash", therefore the user password.

This list is not complete, as several other weaknesses could be present.

  • 1
    Salts are not designed to be secret, and exposing them is not equivalent to storing unsalted password hashes. – Xander Mar 14 '16 at 16:16
  • You are right, but it makes it easier. – honze Mar 14 '16 at 16:21
  • While your statement about salt is incorrect, your PFS statement is insightful. – Neil Smithline Mar 14 '16 at 16:21
  • I will try to fix that. – honze Mar 14 '16 at 16:22
  • @honze - the whole point of salt is to provide protection once the password hash (with the salt) is stolen. If salt doesn't provide protection here, then the password hash was not complex enough (or the password was a very common one, in which case an online attack has a chance of success). – Neil Smithline Mar 14 '16 at 16:23

Your trying to recreate a wheel, and unless your background is in cryptography it's also likely to not be a great as your intent.

Your goal is to send encrypted data from a Server to a client with the limitation that a PKI cannot exist yet. The short is we are limited to Symmetric Key's right?

Their are two rough approaches that you can consider:

Pre-Shared key:

Pre-Sharred keys are generated and set in the client and Server and used as a common key for encryption and decryption. Your encryption of data can be secured quite well this way, but you deal with the maintenance of a PSK.

Key Derivation Function:* Key Derivation Functions work under a similar principal as PSK's but they are generated on the client and server at the start of a session. The data is shared data between the two systems to generate a secret value which can be used to encrypt and decry pt systems.

Both Java and C,C++ have libraries which can build these functions.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.