I have a master server generating large files for other (known) servers to download.

Those files:

  • Are quite big. From 20MB to 5GB.
  • Are written once by the server and then it forgets about them.
  • Nobody should be able to read the content of the file except the server it has been generated for.
  • Will be downloaded over HTTPS using a library such as RequestJS

Based on this knowledge, I planned to generate an RSA key pair for each server, store the public key in the database of the master server along with the servers' specific data (like unique identifier) and only keep the private key on the server it belongs to.
I thought I would just have to encrypt the whole 5GB file using the server's public key and only it would be able to decrypt it.

However, after looking around on the internet, I noticed that it's an extremely bad idea to encrypt big things using RSA. It's not meant for this purpose at all.

I also heard a lot about AES and decided to take a look at the way the TLS works and noticed it uses RSA encryption for the symmetric AES keys.
Now, I'm thinking that I should encrypt my files using AES with a randomly generated key per file but I need to transmit the key to the server.

My idea is the following:

  • Generate a random 256 bits key per generated file.
  • Use this key to encrypt the whole file.
  • Use the public key of the target server to encrypt this key.
  • Prepend the encrypted key to the file so that it's a whole package.
  • Sign the whole thing with the master private key (but I haven't thought about that yet so I'm open to suggestions regarding the algorithm)

And, on the reception side, the server would just read the first 256 bits, decrypt with its private key and use the decrypted payload as a key to decrypt the file itself.

Since I have no deep knowledge of security, I would like to be sure that it a good idea. I see no real downside but I may not know what I'm talking about enough to judge that.

At first I thought HTTPS would be sufficient but I don't want anyone to be able to get the file so I still need more encryption than the connection itself.
Moreover, I wouldn't want a "slave" server to be able to decrypt the content of another. And I should be able to revoke a server by changing its key pair.

  • 1
    Using an asymmetric cipher to encrypt a per-message/ key is pretty standard - this is effectively how TLS works, for one example.
    – Matthew
    Jan 4 '19 at 14:46
  • Does it stay safe even if I pack the encrypted key along with the encrypted message itself?
    – Telokis
    Jan 4 '19 at 14:55
  • Safe against what? If the private key of the target server gets compromised, every message where the message key was encrypted with the corresponding public key is exposed, but that's normal for PKI where the keys aren't dynamically generated. Safe depends on the threat model. You might look at security.stackexchange.com/questions/187694/… which is effectively "HTTPS where only specific target can download", although you'd need to handle working out which data to return to a given client.
    – Matthew
    Jan 4 '19 at 15:13
  • 1
    .... I think everyone hopes so... I guess the only answer to your question is: You've done an excellent job researching how TLS works and duplicated the process effectively in one direction. So, as long as you keep the private keys safe, your process should ensure the expected level of security. ... The only thing I would add to your process would be a sha256sum of the file which can be used by the receiving server to quickly ensure that the transmitted large file+key has been received properly and without modification... a little better, have the master server PGP sign the sha256sum Jan 4 '19 at 15:17
  • WannaCry ransomware is known to encrypt files exactly this way.
    – defalt
    Jan 4 '19 at 17:29

You are reinventing the wheel by trying to solve your problem at a low level. You are trying to combine RSA and AES encryption primitives to create something that will asymmetrically encrypt a file. What you should be doing is looking for a solution that can asymetrically encrypt a file. For example,

  • 1
    I'm not sure I wholly agree with your assessment... The crypto itself is not being re-invented. If I'm not mistaken, GPG encrypts large files symmetrically and encrypts the symmetric key with the recipient public key... the OP has demonstrated both the willingness to discover and aptitude to understand the underlying complexities of what is required to ensure secure transmission and reception of data... Short and incomplete answers are not conducive to a learning environment such as StackExchange was designed to provide..... a good decent gpg --list-packets might be in order here. Jan 5 '19 at 3:52

What you've defined in your process is called Key Encapsulation. It's how TLS functions as well as GPG and many other cryptographic data exchange protocols.

GPG is particularly easy to demonstrate, since there is an option to list detailed information regarding what is contained within the PGP message. This option is --list-packets, and along with a little information from the standard, RFC4880, it's fairly easy to dissect an encrypted PGP message.

Here are the applicable areas of RFC4880:

4.3.  Packet Tags

   The packet tag denotes what type of packet the body holds.  Note that
   old format headers can only have tags less than 16, whereas new
   format headers can have tags as great as 63.  The defined tags (in
   decimal) are as follows:

       0        -- Reserved - a packet tag MUST NOT have this value
       1        -- Public-Key Encrypted Session Key Packet
       2        -- Signature Packet
       3        -- Symmetric-Key Encrypted Session Key Packet
       4        -- One-Pass Signature Packet
       5        -- Secret-Key Packet
       6        -- Public-Key Packet
       7        -- Secret-Subkey Packet
       8        -- Compressed Data Packet
       9        -- Symmetrically Encrypted Data Packet
       10       -- Marker Packet
       11       -- Literal Data Packet
       12       -- Trust Packet
       13       -- User ID Packet
       14       -- Public-Subkey Packet
       17       -- User Attribute Packet
       18       -- Sym. Encrypted and Integrity Protected Data Packet
       19       -- Modification Detection Code Packet
       60 to 63 -- Private or Experimental Values

5.1.  Public-Key Encrypted Session Key Packets (Tag 1)

   A Public-Key Encrypted Session Key packet holds the session key used
   to encrypt a message.  Zero or more Public-Key Encrypted Session Key
   packets and/or Symmetric-Key Encrypted Session Key packets may
   precede a Symmetrically Encrypted Data Packet, which holds an
   encrypted message.  The message is encrypted with the session key,
   and the session key is itself encrypted and stored in the Encrypted
   Session Key packet(s).  The Symmetrically Encrypted Data Packet is
   preceded by one Public-Key Encrypted Session Key packet for each
   OpenPGP key to which the message is encrypted.  The recipient of the
   message finds a session key that is encrypted to their public key,
   decrypts the session key, and then uses the session key to decrypt
   the message.

So, what we're looking for is a packet with Tag 1: Public-Key Encrypted Session Key Packet, which contains the encrypted symmetric key that was used to encrypt message.

Let's generate an Elliptic Curve25519 test key and experiment:

gpg --expert --full-gen-key

pub   ed25519 2019-01-05 [SC]
uid           [ultimate] Demo Key 2 (Delete Me) <demo2@domain.tld>
sub   cv25519 2019-01-05 [E]

Here's a listing of the generated key:

gpg --list-key --keyid-format long demo2@domain.tld

pub   ed25519/8D1D7EACEC02106D 2019-01-05 [SC]
uid                 [ultimate] Demo Key 2 (Delete Me) <demo2@domain.tld>
sub   cv25519/7B2F2DF3D9ABA877 2019-01-05 [E]

Now create an encrypted message:

echo -e "Hello There.\n" | gpg -o message.gpg -er demo2@domain.tld

Let's look at what's in the encrypted message:

gpg --list-packets message.gpg

gpg: encrypted with 256-bit ECDH key, ID 7B2F2DF3D9ABA877, created 2019-01-05
      "Demo Key 2 (Delete Me) <demo2@domain.tld>"
# off=0 ctb=84 tag=1 hlen=2 plen=94
:pubkey enc packet: version 3, algo 18, keyid 7B2F2DF3D9ABA877
    data: [263 bits]
    data: [392 bits]
# off=96 ctb=d2 tag=18 hlen=2 plen=73 new-ctb
:encrypted data packet:
    length: 73
    mdc_method: 2
# off=117 ctb=a3 tag=8 hlen=1 plen=0 indeterminate
:compressed packet: algo=2
# off=119 ctb=cb tag=11 hlen=2 plen=20 new-ctb
:literal data packet:
    mode b (62), created 1546740749, name="",
    raw data: 14 bytes

Yes, indeed there is a packet with Tag 1. This packet contains an encrypted symmetric session key which was used to encrypt the message.

It's useful to note that there is very little effective difference between the process of how GPG encrypts messages and how TLS functions at the cryptographic level. The protocol is a bit different and the trust anchors are different, but the mechanism used to encrypt the data and decrypt it at the far end are nearly the same. In both cases, there is a symmetric session key that is used to encrypt the message. In both cases, the symmetric message key is encrypted with the public key of the receiver and sent along with the encrypted message itself.

It's clear that your described algorithm matches what is routinely done programmatically by many modern cryptographic messaging systems.

So, yes GPG satisfies the requirement on the transmission side. However, how can you be sure that the transmitted message or file has not been replaced?

And this is where the server signatures on the file become important. Many software developers have adopted the method of generating a SHA256SUM hash of the distributed file and then signing the hash sum list with the developer private key. The signed sums are distributed via TLS enabled web pages. This completes the loop. The receiver of the file can now ensure that the received file is the same file distributed by the server by checking the downloaded file's hash against the server displayed and signed hash list.

  • Thank you for the detailed explanation. Are you suggesting that I use GPG instead of my system? I don't think NodeJS has native GPG support. That's the main reason why I tried to figure something out by myself, I'm not very fond of using a third party dependency for something so risky.
    – Telokis
    Jan 6 '19 at 16:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.