4

What would be a safer bet/standard choice for encrypted archived data?

Presently I use a dmcrypt + LUKS container for my sensitive data. For some stuff (like music mp3s), it would not matter if someone accesses them. But regarding private emails, financial statements and even family pictures I prefer when they remain private.

I worry that, if I store and forget the data, it would not be possible to open the container any more at some point in time (5-10 years). Another issue is whether the whole container could be damaged by a couple of flipped bits here and there.

How do things like LUKS actually last?

2

I can see different questions here:

  1. How to build long term archives (whatever the content)

    This is a common problem and has common solutions. The princips are simple

    • information must be on at least 2 different places (ideally with an important distance between them but it is not always possible), and must be stored off line to be immune to hackery and thunder on the power supply
    • information be be re-written on a timely base (coherent with the information support)
    • information support must be stored in a physically safe way (to be protected from physical theft or destruction)

    You may fully ignore those rules, but my advice is to considere them up to the threats you want to be protected against

    This guarantees that the low level byte sequence will be later available.

  2. How to make sure that the information will be useable.

    In medium range terms (years to decades) it is not a problem. In really long term ranges (several decades) you should control on a timely base that the information can still be accessed and decoded. The more standard the formats, the less problems to be expected. As an example, a PostScript or ASCII text document will be easy to process, but who could still read a Textor document from the 80's...

  3. How to make sure that the crypto will be secure

    You cannot. Even if you use state of the art implementations, you cannot presume of a future flaw discovery, or a new technology that will gain orders of magnitude in processing time. That's the reason why a physically secure storage is important.

TL/DR: if you only want a reasonably secure archiving for short to medium range terms (few decades) you can store the applications that process you data along with the data itself and use any correct quality support. If you really want long term archiving (several decades), you must be prepared to re-write the information many times and eventually convert it to new formats.

2

It is safe to store encrypted data for 5-10 years, but not all drives last that long.

I worry that, if I store and forget the data, it would not be possible to open the container any more at some point in time (5-10 years).

There is no reason why the encryption itself would fail to work after 5 or 10 years. As long as you remember the password and the drive is still working, you will be able to decrypt it. The algorithms that are used have a very long life. Even old algorithms like DES from more than 10 years ago are very easy to use. Simply put, encryption algorithms do not care at all about the current date.

Think about it this way: A 10 year old hard drive with a SAS interface (2004) encrypted with AES (2001) and formatted with the NTFS v3.1 filesystem (2001) would not only be trivial to access on modern hardware, it would not even be out of place to see today! Even a 20 year old hard drive with a PATA interface (1986) encrypted with 3DES (1998) and formatted with the FAT32 filesystem (1996) could be easily accessed today, in 2018. And you're worried about only 5 to 10 years?

However, I need to point out that 10 years is a long time for a hard drive. Even if you are keeping redundant backups (e.g. by using RAID 1), you are risking the hard drive itself dying from old age. Encryption will not prevent the drive it is stored on from dying. If you do need long-term archival, there are plenty of resources online to learn how to properly store data for decades.

Another issue is whether the whole container could be damaged by a couple of flipped bits here and there.

That depends on which bits are flipped. All block ciphers (such as AES) require a mode of operation to work properly. Modern LUKS setups typically use a mode called XTS. Being a narrow block mode, a single bitflip will always corrupt 16 bytes (128 bits) of data. This may be enough to cause corruption in the underlying filesystem, but a corrupt filesystem can almost always be fixed, usually using automated repair tools. It is important to realize, though, that it is more likely for an entire sector (usually 512 or 4096 bytes on modern hard disk drives) to fail. It is fairly uncommon for only a single bit to be flipped, but not uncommon for a sector to "go bad" all at once. The hard drive will make a best effort to recover the sector's contents and copy them to a fresh sector, but it can only do so much. If it is unable to recover the sector, it is marked as damaged and is lost.

LUKS also contains a feature called "anti-forensic key splitting" (see page 10 of this document). This is a cluster of data spanning typically 250 KiB and contained in the 2 MiB header. If a single bit is corrupted in that region, the entire volume can be lost*. Its purpose is to make it easier to erase data quickly in the case that erased data can be only partially recovered, but it also increases the risk of accidental data loss. Unfortunately, this feature cannot be disabled in the current version of LUKS. Unless the header itself is damaged, the rest of the volume should remain intact.

Because this feature cannot be changed when using LUKS, it may be better to use different disk encryption software. VeraCrypt takes the opposite approach of LUKS, not only making its header as compact as possible, but keeping a copy of the header just in case the primary header is damaged. Linux fully supports VeraCrypt when a modern copy of cryptsetup is used. It does not require any extra software and even uses dm-crypt as a backend, just like LUKS.

How do things like LUKS actually last?

I would only start worrying when the data approaches half a century of age. At that point, you run a high risk of not having access to any hardware which is capable of interfacing with your storage device or executing the LUKS software. After that much time, it's possible that even the LUKS specification will be lost, making it necessary to reverse engineer the format. If you are only storing data for 5 to 10 years, you do not have to worry about that kind of thing at all. I have storage drives with data that old and have never had any problem. The only thing you need to worry about is your storage medium failing due to age, which you can deal with by keeping redundant backups.


Overall, there are three things you need to keep in mind when archiving data that long:

  1. Algorithm support. Encryption algorithms that are well known will continue to be supported for a very, very long time. There is no reason to worry about an algorithm like AES becoming ancient history after a mere 5 to 10 years. In fact, it will likely still be the most popular!

  2. Storage medium compatibility. Over time, interfaces for storage devices become obsolete and the hardware required to interface with them becomes harder to find. This starts to become an issue as the age of the medium begins to be measured in decades. Even with this issue, for a popular interface (like SATA), chances are you will find plenty of retrocomputing fans who will know how to access data on even the oldest of devices.

  3. Storage medium health. One thing you cannot prevent is the progression of time slowly damaging hardware. While some types of media last longer than others, everything will break down. I would be very uncomfortable keeping extremely important data on a hard disk drive or a solid state drive, but I would not feel too worried keeping it on an archival DVD. It's important to periodically test your backups to make sure the hardware still works.

To reduce the risk of loss of data loss when archiving, you should follow the 3-2-1 rule. This rule states that you should have at least three separate copies of your data, of which two should be kept on different types of media, and one copy should be kept at a remote location. The specific kind of media you will want to use depends on multiple factors. In general, you should know that:

  • High-capacity tape drives last for a long time, but require expensive hardware.

  • Archival DVDs last a long time, but have a low storage capacity.

  • Archival hard drives don't last as long as some solutions, but have a high capacity.

  • Solid state drives begin to lose data in the span of years to decades.

* While a single corrupt bit in the header can corrupt the entire volume, it is still possible to recover it by using a brute force attack. If more than a single bit is corrupted, the chances of recovery become significantly lower. If an entire 512 byte sector is lost, which is the normal failure mode of modern hard drives, there will be absolutely no way to recover the encrypted data if that sector was part of the LUKS header itself.

There is a lot of misinformation about archival DVDs out there, in part because "archival" is not a formal term and anyone can slap that label on their products. If you buy one, do your research first! Otherwise you may find yourself buying a sub-par product. In particular, be skeptical when "gold discs" are mentioned.

0

Regarding long-term storage (independent of LUKS), there are several points to consider:

Hardware
Storing some semi-cheap consumer hard disk 10+ years somewhere, while "forgetting" it, it not recommendable.

Either switch mediums (tape...) or, at very least, take the disk every few years and rewrite the whole content (refreshing the magnetic stuff, using the motor a bit, etc.).

Backups
Never store some important data only one time, for too many reasons to count them.
Also don't store the second hard drive together with the first.

Bit flips
They can be of the sort that is noticable immediately (not mountable, half of the files missing etc.) - in this case, use the backup.

Or they are more subtle - for this, a file system that has integrity checks might be a good choice (random example Btrfs, but not ext4). (Or, if wanted, a list of file hashes can be maintained. Or ...). Again, take broken files from the backup.

=> In any case, bit errors can be mitigated by having backups.

Necessary software for the file system
While a valid factor to consider, it is most likely the least of the problems.

Some examples: FAT was introduced 1977, ext2 1993 ... and to have a less known one: Minix from 1993. All three will be mounted happily by a 2018 standard Debian install, as well as Ubuntu, Arch, etc.etc.

Point is, even if something falls out of use, it isn't gone. And even if something gets so old and unused that a standard install won't mount it anymore, getting the source from some archive and recompiling it will still be possible for a long while.


And about encrypted partitions specifically:

Bit flips on LUKS
will destroy their whole hard disk sector (eg. 512 byte), because one changed bit doesn't stay just one bit after decrypting. And now it can not only be in file data or filesystem metadata, but also in the LUKS header (which would make the whole content unusable with just 1bit difference).

Nonetheless, the answer stays the same: Backups, solved.
And about the LUKS header, a few flipped bits in there could even be corrected manually if necessary (not very straightforward, but trying what bit was wrong by always trying to decrypt is entirely possible).

LUKS software
This is actually the same as the filesystem software:
It won't go away tomorrow, next year, or even in 10 years.


Tldr: Proper hardware usage/selection, and backups, then you will be fine.

  • 1
    You are incorrect about bitflips destroying a whole sector. LUKS uses XTS which is a tweakable narrow block cipher mode, not a wide block mode. A single bitflip will destroy 16 bytes of data. – forest Jul 24 '18 at 3:29
  • @forest Let me disagree again and point you to specs and implementation. Neither LUKS nor plain dmcrypt must use XTS in any way; rather there is a long list of choices for encryptions, block modes, IV generators, etc. ... it even is extensible with custom things. (And, just to say, I also do not say it is always 512 or something like this. See the "eg.") – deviantfan Jul 24 '18 at 15:27
  • 1
    A sector is always either 512 or 4096 bytes. And there are no wide block cipher modes supported in any distro which uses dm-crypt anyway. Other modes like CBC do not destroy an entire 512 bytes. – forest Jul 25 '18 at 0:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.