28

I'm planning on encrypting a large media library of mine. Have looked into different solutions, but thought I should ask the community what your thoughts are.

  • I don't want to encrypt each file individually, I'd like to create a container which I decrypt once when mounting.
  • I have 1.4 TB of data consisting mainly of 2-4 GB files, though some files exceed 4 GB (meaning zipping isn't an option).
  • Ideally I'd like the solution to be cross platform (even if that means command line only).

My current best solution is to use GPG. I would create a tarball of my 1.4 TB directory then encrypt that file with my GPG key. Though I don't have much expertise in compression, I would think this isn't ideal to tar 1.4 TB. Which also means browsing the files will be cumbersome, unless I untar which would double the storage needed along with many other downsides.

Do you have any suggestions for other ways I should go about this? I don't need to solution to be incredibly secure, prioritising on convenience for this use case.

More Info: I currently use OSX's built in encryption method of creating a .dmg and mounting it to access the contents. Hoping for a cross platform version of this.

closed as off-topic by Jedi, Stephane, Xander, Matthew, Steve Jun 30 '17 at 14:36

This question appears to be off-topic. The users who voted to close gave this specific reason:

If this question can be reworded to fit the rules in the help center, please edit the question.

  • 2
    Adversitement for my own open source project: github.com/netheril96/securefs. It may suit your use case. – Siyuan Ren Jun 28 '17 at 7:27
  • 7
    Note: tar has nothing to do with compression... (well, some versions do have options to compress built-in, but they're just equivalent to piping the tar output through gzip or similar :) ) – psmears Jun 28 '17 at 13:39
  • 9
    Should probably be on softwarerecs.stackexchange.com – Chloe Jun 28 '17 at 17:10
  • 3
    Consider using hard drives with integrated full-disk encryption (FDE). The Samsung Evo Pro range of SSDs work great. I guess for multiple TB you want a mechanical hard drive. Not sure on current state of play, but a few years back numerous manufacturers had FDE options that were only a little more expensive. – paj28 Jun 28 '17 at 19:37
  • can you explain why "some files exceed 4 GB (meaning zipping isn't an option)" – Filip Bartuzi Jun 30 '17 at 12:18
59

Why not use a commonly used application to do it? VeraCrypt is a good choice as it replaced the respected TrueCrypt application and allows you to create an encryption container that you mount as a drive.

  • 1
    I'm trying to stay away from methods that are reliant on an application, and do it manually - as I'd feel more in 'control'. That being said I can't really fault the ease of use of VeraCrypt, so I will probably go with a solution like that. Thanks for the suggestion! – Huckleberry Finn Jun 27 '17 at 14:12
  • 1
    Just a suggestion. I went for ease of use and reliable security. :) – ISMSDEV Jun 27 '17 at 14:27
  • 43
    You're always much better off using well-implemented protocols done by professionals. – mikebabcock Jun 27 '17 at 20:28
  • 72
    @HuckleberryFinn Your zip program, gpg program etc. are applications too. – deviantfan Jun 27 '17 at 21:11
  • 5
    I ended up going down the VeraCrypt route, as mounting and unmounting is not very time consuming. As well, I can use VC on all the platforms I currently need. Thanks again for this one. – Huckleberry Finn Jun 28 '17 at 8:19
41

I find some of your comments curious. Particularly,

I'm trying to stay away from methods that are reliant on an application, and do it manually - as I'd feel more in 'control'.

and

I don't need to solution to be incredibly secure, prioritising on convenience for this use case.

do seem somewhat at odds with each other.

First, you really should spend some time to think about your threat model. Are you mainly concerned with theft of the physical device, or are you concerned with attacks through a network while your system is up and running, or are you concerned about attacks by a motivated adversary who is targetting you specifically and have the ability to repeatedly gain physical access to your systems, or what?

For most people, the two biggest threats to be considered tend to be:

  • data loss, due to media errors and similar conditions ("hard disk crashes"), and
  • loss of control of data, e.g. due to theft of a system that has been shut down

The first point is easily handled by making regular backups and verifying that they are restorable. Encryption can add an additional layer of complexity to this, but don't fundamentally change much of anything; for example, with encryption in the picture, you need to consider how you'd handle the case where you lose the decryption key and/or passphrase.

The second point is basically what full disk encryption is designed to solve.

So you should probably look at full disk encryption (FDE) solutions. I will specifically refrain from recommending specific products, but there are choices available for all major operating systems.

That way, you will (though the terminology tends to vary slightly) "open" a "container" (which requires you to provide the corresponding passphrase or similar), use your data however you like, and then "close" the container which renders the data inaccessible without the passphrase associated with the container.

Every time you store something on a computer, even if you delete it afterwards, unless special measures are taken, some or all of the data actually remains. This is data remanence, and it is a big enabler of digital forensics. If all you do is click "delete" in the user interface and maybe empty the trash, then the data will very likely remain trivially recoverable using off-the-shelf tools that anyone can download and use, or buy for a few tens of dollars.

If you use full disk encryption properly (for example, not copying the data to an unencrypted location), then while the data will still be on disk, it will be so in encrypted form. That way, even if someone is able to analyze what's written to the storage media, unless they happen to know the passphrase they won't be able to figure out the meaning of the data.

The upside of it is that while you are using the data, you won't really have to think about the fact that it's encrypted at all; it just is.

You can still do something like your homegrown GnuPG encrypt/decrypt scheme if you want to, and for some files it can even make sense to do so to protect against an online attack. (An "online" attack, in this context, is for example someone who is able to trick you into executing code of their choosing which copies files off of your system while the encrypted container is opened thus enabling plaintext access.)

  • 2
    Thank you very much for this comprehensive answer! To clarify my two points, I liked the simplicity typing gpg -d -o /unlocked /locked. That is easy for me to execute, as well the applications required are minimal and widely supported. Whereas I've not only had to install VeraCrypt on my Mac, but also the OSXFuse package (Not the end of the world). Another thing to note, is this will be on an external drive with other uses, so FDE won't be an option in this scenario. My main concern is theft of the physical media or snooping from other persons with access, will probably go with VC for this. – Huckleberry Finn Jun 27 '17 at 15:17
  • 1
    @HuckleberryFinn VeraCrypt supports files, partitions and whole disks as container. If you choose one of the former, you can store other data on the disk as well. – CodesInChaos Jun 27 '17 at 16:29
  • if you have to decrypt the data to use them, and you need to decrypt more than it fits in your RAM, there will be an unencrypted copy in your disk at a certain moment. Unless you use a safe delete option every time you get rid of that you are in trouble. – pqnet Jun 29 '17 at 15:33
5

I use LUKS for this, which is extremely well integrated into my Linux distribution of choice. Decryption is possible during boot, or using cryptsetup luksOpen [...] if the system is already running. You may add an encryption layer around physical disks (/dev/sda), partitions (/dev/sda1), RAIDs (/dev/md0), LVM-LVs (/dev/mapper/vg-lv).

I have no idea how/if this is supported on non-Linux platforms, though.

  • According to Wikipedia it is compatible with LibreCrypt on Windows. Disclaimer: I have no experience with LibreCrypt personally. – Jon Bentley Jun 30 '17 at 14:44
2

As others have pointed out, you should use a Full Disk Encryption scheme.

GPG encrypt files by creating another file with the (en|de)crypted contents. So, as you pointed out, you need to process the whole 1.4 Tb directory, for encrypting, and process it again for decrypting Even if you were decompressing on-the-fly in order to avoid the space requirements (eg. < store.tar.gpg gpg -d | tar -x specific-file), you would need on average to traverse half the container for each extraction.

On the other hand, solutions for these are based on a given key and then each block is based on that (such as using CTR mode). The disk key is stored in a metadata block, protected with a password, etc. but in overall, there is a disk key in memory which allows random access to the disk. Which solves the issue of browsing the collection, leaving traces of the temporary uncompressed files and having to double the necessary storage space.

0

GPG for large tarball will be extremely slow. Definitely it's not the best way if you have some daily use-cases.

As others said, FDE is a choice. If you need a container-like solution, take a look at Tomb:

Not the answer you're looking for? Browse other questions tagged or ask your own question.