Key distribution and key exchange for simple secure FTP implementation

Question

I'm developing a simple secure file transfer protocol for a University project using openssl.

I have a Client (let be C) and a Server (let be S).

C will send to S messages like get, put, cd, … and S will send paths or files depending on which messages it has received.

I created a (public, private) key pair. The client has the public key and the server has the private key.

At the beginning of my software there is an exchange of symmetric keys generated by the client. Since I would like to encrypt both the messages sent by the client and the messages sent by the server, is it better use the same private key or two different keys?

Answer 1

What you are trying to recreate is TLS/SSL, which was first published in 1994 (SSL), and has since undergone several revisions, with the most recent specification dated 2011 (TLS) - 17 years of development!

In TLS there are separate keys for sending and receiving (see this question: Separate Read and Write Keys in TLS Key Material) so it would seem sensible for you to do the same. The parties also generate read and write IV's for use with block ciphers.

TLS goes to great lengths to combat known attacks, such as including send and receive packet counts and sequence numbers as part of the HMAC. Some of the known attacks are discussed in the RFC (5246 TLS 1.2), so you might want to read those before you begin work on your own protocol.

You will also need some way of verifying the servers key (or certificate, as it is known in TLS) and this will require a priori knowledge on the client side, such as it's SHA fingerprint. You will also need to have the server prove that it is indeed the holder of the corresponding private key by sending some form of challenge (in TLS this is described in section F.1.1.2 of the RFC linked above).

Answer 2

You are right to generate symmetric keys at the beginning of the session. More precisely, the normal method is to generate one symmetric key (a session key), with a protocol that ensures that both parties generate the same key, so that this shared key can be used on both sides. A popular protocol for such a key exchange is Diffie-Hellman. (Despite the name, this isn't an exchange of symmetric keys: it's an exchange of DH public keys, which leads the two sides to be able to generate the same secret material.) You don't need either party to have an asymmetric key pair to do this and establish a secure channel.

Where you do need at least one of the sides to have a private key and the other side to have the corresponding public key, is for authentication. With Diffie-Hellman, the two parties can be confident that no third party is going to breach their communication, but they cannot know the identity of the other party, so a man-in-the-middle attack is possible: all the attacker has to do is run the session start protocol with the client and the server independently, and from then on relay messages (or not).

If one of the parties isn't willing to communicate with just anyone, it must authenticate the other parties. There are two major methods to do this: with a shared secret (a password), or with public-key cryptography. Password authentication isn't straightforward because if e.g. the client sends its password to the server, this is dangerous unless the client is sure that it is communicating with the legitimate server. Otherwise, in case of a man-in-the-middle attack, the attacker would obtain the password and could send it onward to the server in order to impersonate the client.

Public key cryptography solves this problem. In order for the client to authenticate the server, the client must know the server's public key. The client can send a message that is encrypted with that public key; only the legitimate server will be able to decrypt the message. A simple protocol is for the client to send its password, plus some key material, in this encrypted message.

It is possible for both sides to use a public/private key pair. In this case, each party will have its own private key. Private keys are never shared between parties, otherwise there would be no point in not using symmetric ciphers.

Suggested exercices:

1. Implement what you think is a secure channel protocol.
2. Once your implementation works, play with it. Describe your protocol in writing or with diagrams. Try to find a situation where your implementation breaks — where it allows a third party to eavesdrop on the communication between the client and the server, or to impersonate one of the parties, or to inject illegitimate commands or responses. Cryptography is hard, so don't be discouraged if you got it wrong — that's almost the point of the exercise.
3. It is said that anyone can build a cryptographic protocol that they cannot break, if they tried hard enough. So if you can't break your own protocol, look up the literature on TLS, which is the industry standard for secure channels. Find a feature in TLS that protects against something that you don't. Figure out how to break your protocol that way.