5

Is there any way to encrypt a file incrementally? The problem is that I receive a file a chunk at a time via Internet (it's a client-server application and I send byte arrays via a WCF service) and I don't know when the next chunk is going to arrive (if any) and what is it's size. Everything I know is that I can't store it into RAM, I immediately need to cypher it and append it to the previous chunks.

Constraints:

  • I can't impose a chunk size;
  • I would like (if possible) to decrypt the file like if it was encrypted as a single file;
  • I can't store the chunks without cyphering them;
  • I can't store the various chunk sizes nor store the chunks into different files;
  • I'm using an AES algorithm;
  • I can't change my job.

Any ideas or suggestions?

Improve this question
2
  • 4
    Why not use a stream cipher mode of AES, like CFB or OFB?
    – Stephen Touset
    Nov 5, 2012 at 17:39
  • 4
    You should get someone with experience to implement security systems. A lot can go wrong with cryptographic implementations.
    – rook
    Nov 5, 2012 at 19:30

1 Answer 1

Reset to default
9

Yes. Most symmetric-key encryption algorithms support a streaming mode, where you encrypt the message incrementally as it arrives. This includes CTR mode, CBC mode, and most other modes.

Similarly, most message authentication codes (MACs) support a streaming mode of operation when generating the MAC. On the other hand, when verifying a MAC, while you can process data in a streaming fashion, you cannot check whether the MAC is valid until you've processed all of the data, so you cannot act on the contents of any chunk until you've received them all and verified the MAC on the entire file is valid.

In your situation, there's another solution that's probably simpler: encrypt each chunk of data as soon as it arrives, and store all of the encrypted chunks. When you want to decrypt, decrypt each chunk, then concatenate the results. If you follow this approach, make sure to include a sequence number and the filename in each chunk, so that they cannot be re-ordered. Also, make sure to use authenticated encryption (as always). This approach is probably simpler and thus may be a better match for your needs.

Improve this answer
4
  • MACs are actually pretty difficult for streaming. Since you should only use the data after verifying it. Which collides with streaming use if you're using a simple MAC.
    – CodesInChaos
    Nov 5, 2012 at 20:04
  • With the chunk approach, using a key for multiple files might cause issues, since with a naive approach where each chunk only contains a sequence number, the attacker can mix chunks from different files. So you need to include some per file value, such as the per-file IV in each MAC>
    – CodesInChaos
    Nov 5, 2012 at 20:06
  • @CodesInChaos, great points! Thank you! On MACs: It sounds like Fabio needs to encrypt incrementally, but maybe he can decrypt everything as a whole in one go. There's an asymmetry between MAC generation and MAC verification: you can do MAC generation in a streaming fashion, whereas for MAC verification (as you say) you need to process all the data before you can verify the MAC or act on any of it.
    – D.W.
    Nov 6, 2012 at 0:49
  • @D.W. There are also hash tree approaches to allow MAC verification of individual packets to arbitrary granularity, but I'm not sure how practical they actually are, you still to receive most of the tree before you can begin checking and the overhead can be considerable.
    – Thomas
    Nov 6, 2012 at 2:46

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .