I'm writing an authentication system and would like some suggestions on improving its security. Currently, I use RSA to communicate data between the client and the server. Here's the process:

  1. The client and server each have their public and private keys.
  2. The client encrypts data using its private key and sends it to the server.
  3. The server decrypts the data using the client's public key to read the message.

Now, I'm wondering if there's a more secure way to handle this. Specifically, I'd like to know if it's possible to avoid hard-coding keys in the client and server and instead establish them securely without the need for direct key transfer.

Feel free to share any insights or ideas for improving the authentication system.

  • You might be re-inventing the wheel here. It seems like you could simply use tried and true TLS to provide a secure channel between your client and your server, instead of rolling your own protocol.
    – mti2935
    Aug 4, 2023 at 19:32
  • Hello @mti2935, and thanks for your response. Although yes I may be, I like to always have an extra layer of security just in case. Can never be too safe these days. Aug 4, 2023 at 19:51
  • 3
    If TLS is secure enough, you don't need your own crypto. If TLS isn't enough, neither your own crypto will be.
    – ThoriumBR
    Aug 4, 2023 at 20:00
  • 2
    @DanielRayburn I applaud your spirit. A the least, designing your own secure protocol to use 'on top' of TLS will be a learning experience - especially if you endeavor to provide secrecy, integrity, authenticity, perfect forward secrecy, defend against replay attacks, etc. (all of which are implemented in modern versions of TLS).. By the end, I think you'll have a profound appreciation for TLS.
    – mti2935
    Aug 4, 2023 at 20:54

2 Answers 2


I think you have your asymmetric encryption backwards. The client's public key is, well, public. So in theory anyone can decrypt data encrypted with the private key.

In server/client asymmetric cryptography the client generally encrypts data with the server public key and the server decrypts it with their private key. Only the server knows it's private key so the data is safe barring the exposure of the key.

That last sentence is important and exactly why comments have suggested to use a scheme like TLS. The scheme in your post does nothing to ensure forward secrecy. If an attack like this is not on your radar (with the rest of crypto minutia), it's best to use an established protocol like TLS, noise or similar.

As far as the establishment of keys without hard coding, this is generally accomplished through certificate chaining to a widely recognized root CA. I usually hear "Lets encrypt" talked about in the case of small projects. I'm sure there's a wealth of information around the internet on how to configure and use this service correctly.

  • It depends on whether the goal is confidentiality or authenticity. Your proposed "fix" adds confidentiality at the expense of authenticity.
    – Ben Voigt
    Sep 4, 2023 at 20:41
  • 1
    If authenticity is desired there is mutual TLS, noise KK/KX or some other method of authenticating as well as encrypting a message. I'll admit I read "encrypt" rather than sign and assumed the goal was secrecy rather than authenticity despite other uses of the word authenticat(e/ion).
    – foreverska
    Sep 5, 2023 at 16:40

Securing data as it travels from a client to a server is a top priority in authentication systems. You're on the right track using RSA encryption, but the key to strong security lies in how you handle your encryption keys.

To make it more secure and avoid hard-coding keys in your client and server, consider working with a Certificate Authority. This trusted third party specializes in generating and managing digital certificates. The server possesses a certificate issued by the CA, which it shows to the client during connection. The client, in turn, checks the server's certificate and uses the public key for encrypting data. This way, you can improve security without dealing with direct key transfers.

Another option to steer clear of hard-coded keys is by using a Key Management System. The KMS securely stores and manages cryptographic keys. When a client connects to a server, it can request a key from the KMS. The KMS then creates a key and securely sends it to the client. The client can use this key for data encryption. This approach simplifies key management while enhancing security.

To make your authentication system even more robust, remember to employ strong passwords or passphrases, consider implementing Two-Factor Authentication for added security, ensure your system uses secure session management like HTTPS, keep a close watch for any suspicious activities in your system, and keep your software up-to-date to guard against potential vulnerabilities.

  • A small nitpick - it depends, but usually only a fingerprint is needed. Oct 23, 2023 at 20:24
  • 1
    This misses the core problem: how do you authenticate the client? This is so generic, that it sounds a little like an AI answer. I'm not sure the author understood the problem being posed.
    – schroeder
    Oct 23, 2023 at 21:15
  • To authenticate the client, the server can verify a username and password, or a client certificate. Once authenticated, the server can establish a secure communication channel using TLS. Oct 25, 2023 at 14:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .