Using RSA with 3DES instead of plain 3DES. Does it make sense?

Question

I have a large legacy code. Currently it is using 3DES encryption for a network based application. It encrypts data packets with a main key. This key is used to encrypt and decrypt (symmetrical). And for a double security, we are encrypting this main key again with another key (this time it is using a key which is hardcoded in the code itself).

Excuse me if I don't use sentences that make any sense for you, as I am new to this sector. I just have this legacy code and the way of using 3DES doesn't make any sense for me. It's not secure, we have a hardcoded key in our code and I didn't like the way 3DES is used here. Because the whole thing is based on symmetrical encryption.

Therefore I'm said that we should use RSA and they hand out me a design plan how to implement RSA along with 3DES. They came up with this design (they said they invented this design and it's the only way to do this):

At client:

• Create a random 3DES key every time
• Encrpyt the data with this key
• Encrypt the key used for 3DES with RSA and prepend it to the data block

At server:

• Decrypt the prepended data with client's RSA public key to get 3DES key
• Decrypt the remaining datablock with 3DES key

I've asked for the meaning of using two different encryption algorithms together (RSA and 3DES). I asked why we didn't just use RSA for the whole thing instead of using two different algorithms and combining them.

They said encryption of the whole data would cost more and is expensive (in term of performance). However if we just encrypt the key itself and prepend it to the data, then it will not using much performance because this time the data itself is small(before we encrypt the whole data, but this time we only encrypt the key).

Now, is the new design approach the correct way doing this? Or is this nonsense crap and the best way is to implement it RSA-only. What is the best way to implement a secure data communication? Or any way i could fix the previous way to make it more secure?

Answer 1

They said encryption of the whole data would cost more and is expensive (in term of performance).

This is correct. RSA, an asymmetric key algorithm needs a lot more computing power/time to encrypt/decrypt data compared to a symmetric key algorithm like DES/3DES.

Asymmetric key algorithms are usually used as a means to communicate a key for use in another symmetric key algorithm for data transfer.

For more information, see: http://en.wikipedia.org/wiki/Hybrid_cryptosystem

Basic example taken from wikipedia:

To encrypt a message addressed to Alice in a hybrid cryptosystem, Bob does the following:

1) Obtains Alice's public key.

2) Generates a fresh symmetric key for the data encapsulation scheme.

3) Encrypts the message under the data encapsulation scheme, using the symmetric key just generated.

4) Encrypt the symmetric key under the key encapsulation scheme, using Alice's public key. Send both of these encryptions to Alice.

To decrypt this hybrid ciphertext, Alice does the following:

1) Uses her private key to decrypt the symmetric key contained in the key encapsulation segment.

2) Uses this symmetric key to decrypt the message contained in the data encapsulation segment.

Perhaps as an improvement, your company would consider using AES instead of 3DES?

Answer 2

Terry Chia gave a fine answer about how and why you combine RSA with a symmetric key encryption scheme and I agree you should do that.

However, one other thing you said in your question stood out to me so I wanted to address that. You said "It decrypts data packets with a public key that was given(entered) by someone in to the machine(where the decryption happened)." I'm not sure what you meant by that, but I'll answer as if you mean something that changes what the "right" answer is.

One of the toughest problems with all the encryption systems is keeping the keys secure. The best solution, used in high-security applications, is to store the keys in a specialized encryption/decryption unit that has electronic and physical security such that the key cannot be extracted from the device. Either the key is generated within the device from a cryptographically secure random number generator or it is programmed into the device in a secure way. After that, all encryption and decryption that needs to be done with that key is done by the device itself and the security of the key is then managed by physically securing the device.

So if you are using a device like that to manage the RSA public and private keys, then that dictates a lot of how you handle other aspects of the overall encryption system. And if that device can also manage 3DES keys and encryption, then that changes things, too. Plus you may be in a regulatory and/or systems integration environment that dictates the use of 3DES in certain ways. All of these factors need to be taken into account in designing the solution, so please fill us in if any of them apply in such a way as to make Terry's solution somewhat inapplicable to your situation.

In general, though, a hybrid crytposystem using RSA to exchange a 256 bit AES key that is newly generated for each communication by a cryptographically secure random number generator and then encrypting the rest of the communication using AES with that key would be an excellent solution. Please do take care to make sure that your key generation mechanism is cryptographically secure, though. Many, many key generation schemes have been broken, generally due to weaknesses in the random number generators. You need to use one that has been audited and certified for cryptographic use.