15

Situation:

  • User has several folders worth of GBs which they want to keep in sync on a cloud service (for example Mega or Dropbox) in an encrypted form;

  • User does not want to sync a single encrypted volume, because this would mean, obviously, re-uploading the single volume every time even the smallest file in it changes;

It is clear that this rules out, for different reasons, VeraCrypt or EFS.

What should be the solution?

  • 1
    Much better question, and resolves the X/Y problem. – schroeder Feb 9 '17 at 12:30
  • Are you cool with an initial, one-time upload of an encrypted volume? – schroeder Feb 9 '17 at 12:31
  • OK, I'm happy you're happy :)) Yes I assume the initial upload is necessary. I've been doing this with a rar encrypted file in the past which I would create via a script, but then syncing meant overwriting it each time, which eventually became impossible. – nico Feb 9 '17 at 12:34
  • Would you accept a solution that encrypted your files locally before uploading them, e.g. a Dropbox service which zero-knowledge about your files on the server side? – Pascal Feb 9 '17 at 13:19
  • @Pascal, sorry I missed your comment. I normally use Mega which allegedly encrypts files on their servers... However what I want is having control on the encryption locally, so what you hint at seems ideal. Please elaborate. )) – nico Feb 9 '17 at 13:48
11

this would mean, obviously, re-uploading the single volume every time even the smallest file in it changes;

It is clear that this rules out, for different reasons, VeraCrypt or EFS.

That's not true. VeraCrypt does not re-encrypt the whole encrypted volume on each change of the files inside and Dropbox does not upload the whole volume if only a small part of it changes:

Dropbox tries to achieve the best possible performance by trying to be as smart as possible about what data it transfers. It only uploads changes made to the file instead of the entire file.

  • 1
    Maybe. Normal file change patterns differ from filesystem change patterns. Maybe the Dropbox algorithm won't be able to handle change patterns occuring on a filesystem container efficiently. You'd need to test it in order to see if the sync algorithm Dropbox uses is smart enough. – Pascal Feb 9 '17 at 13:17
  • 4
    @nico Sorry, but if you don't use Dropbox then don't write the title "Keep encrypted files in sync on a dropbox service" and don't ask "(for example Mega or Dropbox)". You asked a question and I answered the question. – techraf Feb 9 '17 at 13:47
  • 1
    @techraf, you are right. I've changed the title as soon as I read your comment. Mistakenly I wanted to avoid mentioning Mega since I didn't want people to tell me that they encrypt their files anway. Nothing wrong with your answer though because I can just as well change my cloud provider if it's worth it. I will test both with an encrypted volume, in fact. – nico Feb 9 '17 at 13:51
  • 2
    @nico: If you're dead set on not using Dropbox, you can roll your own by using the rsync command line tool (available both for Unices and Windows) to periodically rsync your Veracrypt container to the destination. You'll need a virtual cloud server with enough storage, which will cost you about 5-10 $ a month, to rsync your files to. Unfortunately, there won't be a ready-made up to use on your phone or tablet... – Pascal Feb 9 '17 at 14:20
  • 3
    Yes, but Dropbox (and service with delta uploads) will work for you. When you pack the files using RAR, the whole archove will be different in the end. If you modify only one file in the VeraCrypt container, only some blocks will be different and the application will only sync those block which were affected by the change. – JohnEye Feb 9 '17 at 16:38
4

You could use one of the zero-knowledge file sync services that exist online. Examples would be http://www.spideroak.com, https://www.boxcryptor.com/en and so on. Pick any one that earns your trust.

Basically, they offer a Dropbox-like service, but they promise to encrypt all your files locally, on your own machine, using a password that never leaves your computer, before the files are uploaded to their servers.

Some of them also offer web access to your files, but I'd probably steer clear of that, since any solution that works on the web will likely be implemented using javascript, which can't protect your credentials.

Of course you'll never be completely sure that these services don't have a backdoor somewhere in their software, but that's the price of using a finished product, instead of rolling your own.

https://librevault.com/ might be a way to go if you want to inspect source code to make sure there's no backdoors, but it doesn't look like it's ready for prime time yet. Maybe there's other zero-knowledge open-source solutions around. If there were, I'd favour one of these.

  • 1
    Unfortunately I don't trust easily, which puts me a bit in a bind here. Ideally, IMO, you should rely on one entity to encrypt, and a completely different entity to upload. – nico Feb 9 '17 at 14:11
  • Then with your requirements I don't think there's a solution except what @schroeder suggests - e.g. rolling your own encryption. FWIW, I still think using an encrypted Veracrypt volume over Dropbox or any other file sync service that does partial file syncs, like techraf suggests, would be the way to go. – Pascal Feb 9 '17 at 14:15
  • Yes I think I will try that way first, and if it fails I will go Schroeder's way. – nico Feb 9 '17 at 14:20
  • 2
    Actually, Boxcryptor works that way: it doesn't handle the file syncronization, just the encryption. You point it at a folder under your dropbox/OneDrive/whatever root, setup your keys. The client regular software handles the sync and it handles the encryption. That said, unless you're willing (and able) to perform a full code review yourself of all software involved, you're going to have to trust SOMEONE at one point. – Stephane Feb 9 '17 at 15:13
  • @Stephane: I am usually prone to trusting open source projects which are participated by enthusiasts. But Boxcryptor does seem a very good possibility, and I did consider it. But then I discarded because it doesn't support Mega which is where I have my files now. Then I started thinking: why this software should support only certain providers and not others, when all it does is encrypting files in a folder?? This would never happen in an open source project. – nico Feb 9 '17 at 16:03
2

You could use a "shadow copy" design pattern where you copy all new files (and delete files) to a duplicate location, then encrypt the individual files in the shadow copy. The backup/sync solution is aimed at the encrypted shadow copies.

This can be done with scripts, though comes with risks and inefficiencies.

Using this pattern, your local email client is unaffected, and you only backup/sync encrypted changes.

  • ..you need to be a bit more detailed that this. What piece of software would I use to encrypt the files individually in the copied folder? Obviously I need something that can be called via script (batch file) and can recursively encrypt the content of several folders? As a sideline, can I say that it seems to me incredible that there is not a comprehensive software to target this particular need, which after all must be quite common. – nico Feb 9 '17 at 12:41
  • 1
    You could use a wide range of software, including PowerShell. The market is chock full of encrypting programs, many accessible from the commandline. As I say, there are risks and inefficiencies in the approach and most people trust the cloud service's encryption and if they don't, spin up their own file server for the job. – schroeder Feb 9 '17 at 12:44
  • Or they use an online mail service where the users maintain the encryption keys. – schroeder Feb 9 '17 at 12:45
  • The people who have this problem are people dealing with massive amounts of data that makes it infeasible to transfer in bulk. Mostly, those people are large companies. They have tools to do what you want, but it's basically what I laid out. – schroeder Feb 9 '17 at 12:46
  • 1
    @nico a certificate is just a pair of keys, it's quite easy to generate and allow you to send your public key so someone can transmit you an ecrypted file you only can decrypt. One in many example tool is gpg4win.org you can get some information and inspiration from there. – Tensibai Feb 9 '17 at 13:03
2

I asked this question a few years ago on Superuser. The solution I came up with at the time was to encrypt the files using BoxCryptor (which does exactly what you're asking for), then upload the encrypted files onto Dropbox.

Note that this approach leaks some information, such as how many files you have and how large they are. Also, the file-names are only encrypted in the paid-version.


I've since moved my files to SpiderOak, after making a comprehensive spreadsheet of backup services†. It's much more convenient, and the zero-knowledge backup ultimately provides the same (or better) security. I now only use Boxcryptor for things I rarely need to decrypt, like tax documents.

Note: This spreadsheet is several years old, and the information may be outdated, especially the pricing

  • Yes, but if you want to encrypt files on your local drive then maybe SpiderOak isn't that much more convenient. – zylstra Oct 20 '17 at 23:32
1

This is a problematic question, requiring an overly detailed explanation. Assuming you're talking about Encrypted Volumes as 1 file objects.

The reasoning can boil down to one tiny little detail concerning file encryption. Basically, a 'good' encryption methodology will encrypt the file in its completeness, and not in 'chunks'. Therefore, any small changes (one byte) would generate an entirely new file content.

If you're proposing solutions that get around this fact by chunk-encrypting files, then you're actually missing the whole point of long-term encrypted storage.

That's the philosophical answer to the technical problem above. Therefore, what you need to consider is re-organizing your file volumes to minimize size and changes to a subset of encrypted files which would be easier to synchronize and move around.

But, technically speaking, you cannot synchronize part of an encrypted file. It goes against good security practices if you actually do. I would seriously study the underlying cryptography before investing in a solution that allows incremental updating of encrypted files. I have yet to see a "good" solution to this problem that doesn't sacrifice security.

  • How about encrypting each file individually, and uploading them one by one (rather than uploading a single encrypted file container)? I'm not thinking about large files here, but rather a large folder. What would be the best way to achieve this? – nico Feb 10 '17 at 5:23
0

I'm afraid your assumptions are incorrect. VeraCrypt DOES in fact use a format that is delta-change friendly. This means that you can use it perfectly well with DropBox and similar cloud storage. I know this because I do! I have some large volumes that are synchronised to the cloud.

The important thing is to ensure that you are using a cloud storage sync client that uses delta-processing (e.g. it checks for changes at a block level and only synchronises changed blocks) - which I suspect all do now, that wasn't always the case.

0

I use cryptomator for this and have been mostly satisfied. If anything, it's written in Java, so it'll pull in 280mb of dependencies on java runtimes, and the gui won't integrate especially well with your desktop or scale nicely on HDPI.

Other than that it does a good job (note that I haven't audited the encryption code or anything). It's free, opensource and will encrypt file names.

0

If you can "mount" the cloud service as a local path, borgbackup (BSD license) is a pretty good solution. It's a python based tool that makes encrypted backups. The server sees only large-ish blobs of "data", so your file structure or meta data is also not revealed. (I say "mount", because borg requires the server also to run borg, which for simple file hosting on cloud is not possible).

If you cannot do that, you can investigate "duplicity" (http://duplicity.nongnu.org/; GPL license if I recall), which does not require anything specific on the cloud, and supports about 20 different cloud providers so far. I used duplicity for years, and only switched to borg because borg has better support for pruning old history (even automatically, if you wish). Duplicity scores badly on that front.

But for cloud I currently use rclone (MIT license), which supports an even larger number of cloud providers, and can encrypt locally to any of them (if I understand things correctly; I only used google drive so far). Rclone works very much like rsync, and is pretty darn fast too, in my experience. Rclone of course reveals file metadata like file size and directory structure. I do not care about that so it's OK for me; if you do care, then it won't work for you.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.