The idea is to send an encrypted message in such a way that the receiver can verify the sender. Here is the proposal:

  • I generate an RSA public-private key pair, and I give the private key to Barry.

  • I keep my public key secret (!) and I use it to encrypt messages.

  • When I send an encrypted message to Barry, if he can decrypt it using the private key I gave him, then he knows the message came from me.

Is this secure? Without access to my public key, but perhaps an eye on the messages, can anyone else fool Barry? (And does it matter if other people see the private key?)

If not, what kind of approach should we be using to sign encrypted messages, without having to perform a multiple-trip hand-shake? (Preferably still using RSA.)


It is less safe than using a private key:

However, these are small limitations that may be acceptable in your case.

  • Thank you. Based on this I am planning to swap the public and private keys around for extra security. But it is also good to know it's not too dangerous if we don't. (We plan to keep both keys secret for this particular system.) – joeytwiddle Jan 20 '17 at 2:54

The common way to sign and encrypt a message is to... both encrypt and sign it!

That means that you give Barry your public key, and he gives you his public key. Then to authenticate a message, you encrypt it with your private key, and to encrypt it so that only Barry can read it you encrypt it with his public key. If you want both, you use both passes one after the other in a consistent order.

In fact that is more or less what MIME mailers implement, except that the encryption uses a unique symetric key, and only that key is encrypted with the public key of the recipient, and for authentication, only a hash of the message is encrypted with the private key of the sender. Because it is much simpler and as secure...

Alternatively if both parts must be kept secret, do not use public key and just share a secret that is used for a symetric encryption.


Public key encryption can be used for authentication and is also used in practice. But your proposal is not public key cryptography. In your proposal the "public" key is kept private and your "private" key is made public, i.e. exactly the wrong way.

When using public key encryption for authentication usually party A creates some random challenge and sends it to B, B signs it using B's private key and A can verify the signature using B's public key and thus can be sure that A has access to the private key which is equal to a successful authentication since only A should have the private key. The step with the random challenge created by A and signed by B instead of some data which is both created and signed by B is essential because otherwise somebody could capture the traffic and replay it later to claim to be B.

  • Thank you for pointing out the limitation with the single-step approach. We will mitigate that by having the client include a timestamp (encrypted), but that still leaves the possibility of a duplicate request being made by a snooper within the time limit threshold we specify. (A temporary cache of not-to-be-repeated requests on the server could be used to prevent that.) – joeytwiddle Jan 20 '17 at 2:49
  • @joeytwiddle: Timestamp is not random and failure to use proper random where needed is a classic in failed cryptography. Don't invent your own way of doing things but instead use existing techniques. Even experienced cryptographers sometimes get things wrong. Chances are high that you mess it up with your level of experience and that others will exploit the vulnerabilities in your authentication scheme and laugh at you. Do you really want this? – Steffen Ullrich Jan 20 '17 at 2:57
  • I don't want that! But I have been tasked to implement a single-step mechanism. The timestamp will be used to reject any traffic which is replayed again later. (And the cache to reject any which is replayed immediately.) So I don't think it needs to be random? I will of course ensure that the product owner is made aware of the concerns raised here, and move to a random challenge if that is deemed necessary. (We do have other security mechanisms planned, such as whitelisting client IPs, which are not expected to change too frequently.) – joeytwiddle Jan 20 '17 at 3:08
  • @joeytwiddle: I don't consider myself knowledgeable enough to decide if your challenge is really safe enough for what you do. But I read enough examples on failed cryptography so I know that I would not invent my own authentication with my current knowledge but instead use established schemes. – Steffen Ullrich Jan 20 '17 at 3:19

Not the answer you're looking for? Browse other questions tagged or ask your own question.