Recommended temporary key size for communication

Question

What I'm going to do is use AES-256 encrypted network communication to agree upon a pseudo-random key for further communications between two peers, so that every "session" utilizes a separate key, ergo cracking the key once doesn't unlock all sessions between the two peers.

My question: what is the "recommended" or "secure" length that my key should be? I was thinking 512, but I know making it too low makes it insecure, and making it too high makes it bad for performance.

P.S. I want to importance of the data not to matter, since it could be argued that it shouldn't.

I hope this isn't too broad. Thanks in advance.

Answer 1

First things first:

Assuming this is all symmetric encryption (rather than something like ephemeral Diffie-Hellman), you're not really achieving anything. You're just kicking the problem up a level. An attacker can still "unlock all sessions between the two peers" (or at least all future sessions) by cracking the "AES-256 encrypt network communication" key one time. This can even be done long after the fact, so long as they recorded the encrypted communication. Once an attacker gains access to any given key, they can decrypt all traffic - including new session keys - encrypted with that key.

There's also the question of how you're establishing this "AES-256 encrypted network" in the first place. How do you agree on the initial key? Does that key ever change? It's obviously the weak point in the protocol - the master key that, if compromised, lets the attacker break everything else - so you'd better think very carefully about how it's to be generated/derived, exchanged, and used.

You need a key exchange algorithm of some sort in there to establish the initial key (and, if you have that, you should probably use that for establishing the session keys too). Of course, such key exchange algorithms usually require some way to secure them...

---

To actually answer your question:

what is the "recommended" or "secure" length that my key should be

Which key? The one for AES-256? What do you think the "256" part of AES-256 means? AES only allows three key lengths - 128, 192, and 256 - and there's no reason to use any but the longest with modern hardware. If you need to support very weak / outdated hardware without hardware support for AES then there might be an argument for shorter keys, but it'd have to be very weak hardware indeed and also you might want a different cipher altogether - one more amenable to software optimizations - in that case.

If you mean for some other cipher (e.g. an asymmetric cipher like RSA) then what key length you should use will depend on the cipher in question. RSA requires very long keys to be secure. 512 bits is laughably weak in RSA; I recommend at least 3072 bits, considered roughly 128 bits of entropy, but if you want it to be long-term secure don't use RSA at all. On the other hand, X25519/Ed25519 requires much more modest 256-bit keys for the same level of security (and doesn't even support 512 bit keys, although there may exist some other curve that you could use with EdDSA that does?).

Note that while 128 bits of entropy might not seem like much, in practice, it's secure against anything humanity is likely to come up with in the next 30 years or so, hardware-wise, unless quantum computing gets a lot better. With modern hardware, it's de facto unbreakable. On the other hand, 256 bits of entropy would - for AES, at least - be secure for the next few decades (assuming Moore's Law or equivalent stays on course) even in the face of useful-scale quantum computers (or for centuries without them).

---

Obligatory closing note:

You appear to be attempting to roll your own cryptosystem. Don't do that! While using established primitives is much better than trying to create your own, the fact that you had to ask this question makes it clear that you lack the expertise to build a secure cryptosystem even with established primitives. Cryptography is hard! Use an established cryptosystem (or better yet, a complete protocol) that is widely reviewed, and use a widely-reviewed implementation rather, than trying to come up with your own.

If you don't think it's that hard, see if you have answers to the following questions (bearing in mind that the ability to do so is necessary, but not sufficient, to create a secure cryptosystem):

• What pseudo-random function are you using for key generation?
• What mode of operation are you going to use AES in, and why?
• How are you generating nonces/IVs?
• How do you plan to prevent tampering with the ciphertext?
• How do you plan to prevent replay attacks?
• How does your system exchange the initial key (bootstrap) to initiate the encrypted communication?
• How are you authenticating the counterparty to the communication?
• Is there a central authority for key management, and if so, how does it work and how do clients authenticate it and how does it authenticate them?
• Is your protocol forward-secret? If not, why not?
• Do you support session resumption, and if so, how and what information is necessary for it?
• Do you support non-repudiation? If so, how? If not, why not?
• Do you support protocol flexibility? How do you update the network?
• How long (in time or data) do your sessions last before re-keying, and do you re-key using the established session or via the session-establishment channel? Why?
• Do you support communicating with multiple other parties at once (one-to-many messaging)? If so, how is that done?

And those are just the questions for the cryptosystem and transport layer of the protocol; it doesn't even get into authenticating a user or device on the network, handling multiple devices, the storage of keys, the protecting and/or distributing of message history, handling of lost keys, handling authentication errors, etc. Even if you're just doing this as a learning project with no intent to publish, please get somebody who knows applied cryptography to look over the full design, and definitely don't publish anything using this system until the system, and ideally also the implementation, have passed significant public review.

Answer 2

@CBHacking's answer is spot-on, as usual [+1].

Embedded in this question is an interesting 'subquestion' of whether it is possible for Bob and Alice to communicate securely, with forward secrecy (and perhaps even break-in recovery) starting with just a shared secret. One way of achieving this might be as follows:

Bob creates a long-term private/public key pair, then takes an HMAC of his public key using the shared secret as the HMAC key. Then, Bob sends both the public key and the HMAC to Alice. Alice then authenticates that this is indeed Bob's true and correct public key by verifying the HMAC, again using the same shared secret as the HMAC key.

Then, Alice does the same, and sends her long-term public key to Bob, and Bob authenticates that this is indeed Alice's true and correct public key.

At this point, Alice and Bob have each other's true and correct long-term public keys. The shared secret is no longer needed going forward.

If the two are in a synchronous environment: Each time one wants to send a message to the other, the two can setup a mTLS session using the two public keys to commence the session. For each session, Bob must verify that the public key that Alice is using is the same as the one in the step above, and vice-versa Alice must do the same for Bob. Once the public keys have been mutually verified, the message can be sent through the secure TLS channel, then the session is torn-down so that all ephemeral keys used during the session are discarded. This provides secrecy, integrity, authenticity, and protection against replay attacks. In addition, it provides forward secrecy - meaning that even if an attacker records all of the cyphertext going over the wire between Alice and Bob, and later Alice's and/or Bob's long-term private key is leaked, the attacker still cannot use this to go back and decrypt all of the past messages.

If the two are in an asynchronous environment, then the double ratchet protocol can be used - again using the two public keys to commence the session. This also provides all of the above security features.

Related: In a synchronous environment, does Signal offer any security benefits that can’t be achieved with a modern version of TLS?