46

According to the ownCloud documentation, if you enable encryption, file sizes can be ~35% larger than their unencrypted forms.

From my understanding of encryption, the file sizes should be more-or-less identical (perhaps some padded 0 bits at the end to make it a multiple of the key size).

Is that incorrect? If not, why?

15
  • 2
    depends on the format that is used for encryption. Data expansion varies from 28 bytes to much more if you put more stuff in headers and footers (algorithm identifiers, key derivation parameters, salts, encrypted keys, ...)
    – SEJPM
    Mar 10, 2016 at 18:41
  • 1
    @SEJPM but those are all small-ish (at least compared to, say, a video file), and do not scale with the amount of data being encrypted. I could see that stuff being ~35% of a single photo, but not ~35% of a folder full of photos, or a video file. Mar 10, 2016 at 18:53
  • 4
    Possible duplicate of Why does openssl enc -aes-256-cbc -a -salt increase the file size?
    – dr jimbob
    Mar 10, 2016 at 19:11
  • 3
    I wouldn't want a file to be the same size encrypted. That discloses information. Even if the size is consistently almost the same, it discloses information about the contents.
    – kojiro
    Mar 11, 2016 at 0:58
  • 9
    @kojiro So does a file size that's consistently about 35% bigger.
    – user253751
    Mar 11, 2016 at 7:56

3 Answers 3

82

Most likely, the encrypted file is base64 encoded which would account for 33.3% file increase (you encode three bytes of data in four bytes of base64 data). Inserting a new line every 64 characters to make it easier to read (as is done by ASCII armor in openssl, GPG, PGP) will increase the size by 65/64.

Combining these two effects results in the new file being (4/3)*(65/64) = 135.4% of the size of the original or an increase in file size of 35.4%.

I've gone through the calculation in this answer here.

You are correct though that encryption should not need to significantly change the file size. It possibly adds a couple blocks of data if there is a header, an initialization vector/nonce, some padding to make it a full block and/or MAC to check integrity, though these changes will be insignificant for large files (e.g., adding four blocks to an AES encoded file that is 1 MB would make the file 0.006% larger).

However, to not increase the files size, you need to be fine with storing and passing around the encrypted data as an arbitrary binary. Arbitrary binaries are often blocked over email to prevent spreading computer viruses, and are often difficult to open outside of hexeditors. Base64 encoded files are easier to pass around and is a more portable format than binary files of an unknown file type.

13
  • 13
    Why does owncloud do this?
    – Moby Disk
    Mar 10, 2016 at 21:19
  • 10
    @DeerHunter - I've never used OwnCloud. Briefly looking at their source code it seems they use the poorly documented PHP function openssl_encrypt to do the bulk of their encryption work. The fourth parameter $options is hard-coded to false in owncloud's source code. The parameter $options used to be called $raw_output and when it's set to false it base64 encodes the ciphertext output.
    – dr jimbob
    Mar 11, 2016 at 0:50
  • 9
    Playing around with openssl_encrypt in a PHP fiddle, it seems to base64 encode the data (and not insert linebreaks every 64 characters), but as php.net calls this an undocumented function, it wouldn't surprise me if this changed between PHP versions (if it's not 100% clear, I am not a PHP fan). Without the linebreaks, I'd expect encrypted files to be consistently 33.33% larger up to 1-3 blocks for padding, an IV, and a MAC. So saying 35% may just be rounding up for safety safe (and I shouldn't have assumed line breaks).
    – dr jimbob
    Mar 11, 2016 at 0:56
  • 22
    @drjimbob Being utterly clueless about details like this pretty much ensures I will never trust ownCloud for any kind of security guarantees ever. Mar 11, 2016 at 1:22
  • 2
    @drjimbob No need to speculate, or make assumptions about PHP changing behaviour in incompatible ways; the default for that function has always been to output in base64. The boolean was changed to a flags param to avoid lengthening the signature, in this commit, in a completely backward-compatible way: github.com/php/php-src/commit/… The intention was probably to match hash functions (md5, sha1, etc) which output in printable form by default.
    – IMSoP
    Mar 11, 2016 at 16:54
7

If the files are being compressed then you might see this discrepancy.

Compression algorithms work best on non-random data. Encryption aims to generate randomness from information. Information is generally easy to compress as it has patterns. However, if you encrypt it, you are generally erasing any patterns (and information).

Example: 2.75GB of email archive files can be easily compressed down to <.5GB. If these email archives were encrypted, however, then the compressed version would be much closer to 2.75GB.

8
  • 6
    This is true, but you can get around that by just encrypting the compressed collection of files. The fact that it increase by roughly 35% indicates its base-64 encoding with linebreaks to avoid sending passing around arbitrary binary files of encrypted data. If it had to do with compression, there's no reason to suspect it would always be 35%. (Instead of sometimes being 90% difference as in your example).
    – dr jimbob
    Mar 10, 2016 at 19:20
  • Definitely agree. But ignoring the specific product and answering OP's question, this is one example where encryption might be leading to larger files.
    – d1str0
    Mar 10, 2016 at 19:27
  • 1
    Owncloud is an enterprise file-sharing application (which will store arbitrary types of documents, which may be highly redundant or not). Their documentation says that "Encrypting files increases their size by roughly 35%, so you must take this into account when you are provisioning storage and setting storage quotas. User quotas are based on the unencrypted file size, and not the encrypted file size." So this isn't talking about their specific files, but user files being 35% larger.
    – dr jimbob
    Mar 10, 2016 at 19:43
  • 1
    Well this is sec.se not crypto and I read and answered the question in the title "Why would an encrypted file be ~35% larger than an unencrypted one?". Though I guess you fairly answered the question in the body "From my understanding of encryption, file sizes should be more-or-less identical (perhaps some padded 0 bits at the end to make it a multiple of the key size). Is that incorrect?"
    – dr jimbob
    Mar 10, 2016 at 19:50
  • Lol. I'm an idiot. The app doesn't desperate the two sites very well.
    – d1str0
    Mar 10, 2016 at 19:50
4

Normally, the % mark says that the file might be Base64 encoded after encrypting, and also might get some checksum over each block to prevent corruption. Base64 encodes characters of 8 bits into characters of 6 bits, which means the file in question gets about 30 % larger due to more charachters required to render the whole file. Add a per-block checksum and you are up to 35 %.

Normally, the encryption itself adds some overhead. Normally, the overhead is header+footer, eventual encrypted key, parameters, salts, checksum, and also one block size minus 1, because if the encrypted data is not evenly dividable with the block size, you would have to pad with up to block size - 1.

But all those data in the previous sentence would add a static amount of data to every file, regardless of its size, even if its 1 or 100 GB large.

The data enlargement expressed in % says its a reencoding process like base64 or something similiar.

7
  • 1
    This says so too: github.com/owncloud/core/issues/10831 . But you've mixed something up: Base64 makes 3 byte => 4 chars, it won't encode 1 byte in 6 bit. And it's 25%, not 20%.
    – deviantfan
    Mar 10, 2016 at 19:05
  • "Base64 encodes 8 bits into 6 bits, which means the file in question gets about 20 % larger." This doesn't make sense. If b64 encodes 8bits to 6bits then the file would get smaller.
    – d1str0
    Mar 10, 2016 at 19:07
  • 1
    @d1str0 6 bits per charachter. Which means the file gets larger as more charachters is required to render the file. Mar 10, 2016 at 19:10
  • That doesn't clarify your answer. "Base64 encodes 8 bits into 6 bits" suggests it is reducing the size.
    – d1str0
    Mar 10, 2016 at 19:12
  • 1
    @d1str0 "8 bits encode to 6 bits" doesn't mean 2 bits get discarded. Every leftover 2-bits get collected to create additional 6-bit "characters. So for every 3 8-bit characters, there are 4 6-bit characters generated. The valid 6-bit characters that are allowed to come out of base64 encoding are characters that consist of bit patterns that are also valid 8-bit characters; they're simply the 8-bit characters that have 2 bits that are both zeros. It so happens that each of those characters is one that is "printable" ASCII, and that makes the output file acceptable to SMTP. etc. Mar 13, 2016 at 13:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .