Is following communication scheme secure?

To my mind, it uses RSA modulus and public key exponent as a cryptographic secret, which is strange usage of RSA, but can it be easily broken because of that?

Description of communication provided by the vendor:

  1. Client generates a new pair of 1024-bit RSA keys on his machine every 10 hours and uploads public key to the Server. Client is authenticated using password and Server authenticates itself with SSL/TLS certificate.
  2. If the Server wants the Client to provide the data, the Server generates AES key and sends it to the Client encrypting the message with Client's public key using OpenSSL library.
  3. Client sends the data encrypted with AES to the Server.

Potential problem: is the Server authenticated against the Client in the 2nd part of communication?

  • Is the key upload actually encrypted or does the server only authenticate with the certificate?
    – marstato
    Dec 28, 2015 at 14:14
  • RSA public key upload is encrypted using SSL/TLS (HTTPS POST with BasicAuth). Dec 28, 2015 at 14:19
  • 4
    If there is the availability of TLS already, then I see no real benefit in using this additional crypto. Dec 28, 2015 at 15:45
  • 5
    @random_crane If I were asked to evaluate this, I would recommend not using it. While there aren't any obvious flaws, it's unnecessarily complicated which creates lots of opportunities for things to go wrong. Additionally, there are general operational issues that might arise that aren't security related per se such as is one party fails to manage the exchange on the defined schedule.
    – JimmyJames
    Dec 28, 2015 at 17:46
  • 2
    @random_crane: Can you name the vendor and give a link to that claim? Dec 28, 2015 at 18:01

3 Answers 3


As far as I can see, they should have just used TLS to get better encryption with less work.

Using RSA to exchange ephemeral AES session key is one of the most common way TLS are configured.

Several issues with this custom scheme:

  1. 2048-bit RSA is considered the minimum standard for RSA nowadays. The 1024-bit RSA is nowadays considered to be brute forceable by highly advanced attacker in the foreseeable future.
  2. AES encryption does not authenticate the sender, and adding authentication to an encryption is quite fiddly to do correctly. I suggest changing step #3 to use Authenticated Encryption, recent TLS libraries supports AEAD, for example, with GCM
  3. In step #2, the session key is sent over the network. TLS would use Diffie-Hellman to have the client and server derive a symmetric session key over untrusted communication channel.

Potential problem: is the Server authenticated against the Client in the 2nd part of communication?

If the public key is truly publicly distributed, then yes, that would be a pretty flawed way to do secure transmission. It also seems pretty weak to use a password is step 1.

  • The case is that the public key is not publicly distributed (no matter whether it is called "public" or not). By design it is available only for the Client who generated it and the Server. Dec 28, 2015 at 14:59
  • 3
    Then it should be fine but I don't see any advantage over a symmetrical key. As I noted in my answer, the use of passwords in step 1 seems pretty weak. Using the last provided key pair would be stronger. The whole thing seems a little convoluted to me, honestly. I think you are right to question it.
    – JimmyJames
    Dec 28, 2015 at 15:06
  • No advantage over symmetrical key: I agree. The fact that some piece of cryptographic key can be publicly available is not used at all. But can it be treated as symmetrical cipher with key being 1024-bit RSA modulus? (is it not worse than symmetrical cipher, taking into account that public key is a secret?) Usage of passwords is weak: You're right, it would be better to use the previous keypair. In this case they are complex enough (8+ characters, 3 of 4 groups from uppercase/lowercase/digits/symbols). Dec 28, 2015 at 15:57
  • @random_crane Whether the encryption level is strong enough in this scheme is really not something I can answer for you but you are already using asymmetric keys in this schema elsewhere so I see no additional risk. asymetric encryption is more costly so that would be the main disadvantage that I see since it is not really necessary.
    – JimmyJames
    Dec 28, 2015 at 17:40

This setup is basically secure.

It doesn't follow normal best practices, but I cannot see a direct way to attack the protocol. The RSA public key is kept private, so someone external cannot readily do step 2.

There's still a risk of implementation flaws, including:

  • Buffer overruns
  • Injection flaws
  • Password handling flaws (e.g. no lockouts)
  • Cryptographic weaknesses (e.g. insufficient randomness)

In particular, I'd be interested to see if at step 2 you could send all zeros to the server. It may be that whatever the RSA key is, this will result in an AES key of all zeros being used. This won't work if they're using padding though.

A better implementation would be to abolish step 1 entirely, and in step 2 have the server authenticate with a client certificate. The data would then be sent directly over this TLS connection without additional encryption.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .