Does the destruction of sensitive information limit the choice of hard drives to non-flash based devices?

Question

Working with a non-profit organization,it's common to reuse hard drives that have previously stored highly sensitive information such as medical and financial records. This is primarily driven by cost-saving measures to reduce purchasing new hard drives.

If the destruction of sensitive information is the first requirement, does this limit the choice in selecting the type of storage medium?

For example, do non-flash based devices provide a higher level of assurance in the destruction of data using ATA Secure Erase and a single wipe in comparison to SSDs including self-encrypting drives?

Answer 1

Data destruction is a technique of last resort. If you are planning to use a new storage device, you should use full disk encryption. This allows you to either destroy the encrypted master key or simply forget the password, effectively rendering all data unrecoverable, despite no data actually being wiped. Encryption is a solution for both solid state and standard hard drives. Use a strong algorithm like AES.

If you absolutely need to use a hard drive without full disk encryption, you should get one which supports SED, which is transparent hardware encryption. SED transparently encrypts all data written to the drive, but keeps the encryption key stored in a special area. When you initiate secure erasure, this key is all that is destroyed. This feature is supported on most modern SSDs and HDDs. If you do not know if a drive supports it, you can often conclude that it is supported if the estimated ATA Secure Erase time is showing as only two minutes, regardless of how large the drive itself is.

There is nothing intrinsic to the data storage methods used by solid state media that makes it hard to perform data destruction, but their firmware makes it impossible for the operating system to overwrite specific sectors due to wear leveling, a feature that spreads writes around the drive to decrease the wear and tear on individual flash cells (each of which has a finite lifespan). This does mean that you cannot overwrite data on SSDs reliably. You can still use SED if the drive implements it, and you can use ATA Secure Erase as well, but if you need to manually overwrite a range of sectors, use an HDD.

Note that, if you do use an SSD and are using full disk encryption and you have TRIM enabled, the drive will leak a limited amount of metadata, as explained in this excellent blog post. You can usually disable TRIM at a small performance penalty, but you will avoid metadata leakage. Whether or not the exact metadata leaked is problematic depends on your specific threat model.

Answer 2

Tl;dr: Because you can never trust all storage drives to securely wipe themselves, you must plan as if none of your drives can be securely wiped.

Placing a dependency on the type of media is not the right way to approach the problem, because the technology is always evolving and changing, and you can never be in 100% control of all IT spend. Remember that disks were never designed for secrecy first - they are designed for the opposite: reliable access. (Some disk makers like to maximize profits by selling their products as “security solutions”, but that still doesn’t make them the best choice for the job.)

For example, Shadow IT (aka the boss’s kid) is good at buying consumer equipment like SSDs, and installing it in the department desktops without asking permission. Or a non-profit might have to accept a generous donation of a hundred drives from some corporate sponsor (for political or marketing reasons), but that don’t support Secure Erase. Decent corporate laptops don’t even offer spinny disks as an option anymore, while wear-leveling algorithms ensure that SSDs always risk leaking some data in the slack spaces of the drive.

Instead, look to something that is designed to solve this exact security problem, and is something that you can control enterprise-wide, such as installing encrypted file systems that can be wiped as quickly as deleting the key. For example, in a Windows shop enforcing BitLocker via Group Policy would protect all the drives, not just the special ones you ordered.

Answer 3

Firstly, my understanding is that an SSD that properly implements the secure erase command will erase the unallocated blocks, although it may be unable to erase retired/failed blocks (these are blocks that have worn out and no longer operate correctly) and in theory these could contain recoverable data.

Secondly, HDDs also include reserved space. Most notably, this is used when a sector on the disk fails (a "bad sector") and the data must be relocated elsewhere. The original data is left behind in the bad sector which is no longer in use. Some disks also use additional reserved space as working room to rearrange other sectors so that the data from the bad sector can be physically located in a more optimal place and the disk may not erase the reserved space that was used either. In theory though, an HDD that properly implements the secure erase command will erase both bad sectors (if possible) and reserved space.

However, as other answers have pointed out, this is all relying on the proper implementation of the secure erase command and the ability for failing/failed parts of the media to be erased. The best solution may lie in something that is not dependent on the drive's own firmware supporting a particular operation.

With full disk encryption (or even file-level encryption, although be careful of filenames revealing information) you don't need to erase the actual data, just the encryption keys. In that case, the data is encrypted before it is written to disk and the disk (should) only ever contain the encrypted data. As the encryption keys are required to decrypt the encrypted data, having access to the encrypted data on the disk is useless without the encryption keys. As long as your encryption keys are securely erased or, even better, stored on a separate disk/memory device (e.g. a smart card or hardware key) the data is effectively unreadable.

Note that a lot of HDDs and SSDs are now offering "self-encryption". This is where the disk's own firmware generates an encryption key and stores it in the disk controller's internal memory, and encrypts the data itself before it is written to disk and decrypts it again after reading. The computer sees the disk as an unencrypted disk but the data actually stored on the media is encrypted. Such disks typically implement the secure erase command by deleting the encryption key from the controller's memory rather than overwriting the disk. Personally I avoid self-encrypting disks because they have a bad history of firmware bugs leading to either gaping security vulnerabilities or data loss, but the concept is the same as OS-level full disk encryption.

Answer 4

Solid state disks are definitively to be preferred. Note that they are not without their troubles either, since sometimes implementors just suck (and Windows/Bitlocker sucks, too).

Traditional disk drives have been "encrypting" (or rather mixing) data weakly since pretty much forever to distribute bits better, but this doesn't help much in protecting data. More recently, there exist harddrives which are self-encrypting disks (SED), but as harddisks they are kinda "prestige" products and outrageous in price. I haven't so far owned one.

Solid state disks are practically always SED, but the feature set, and more importantly, the quality of the implementation differs a lot. As you can read in the linked article, for example, earlier models from Crucial used an encryption that was total bollocks. The user's password is compared to a hash by the firmware to "unlock" the drive's encryption key as opposed to e.g. Samsung's drives which use PBKDF2 to derive the key from the password. Which, in terms of actual versus misleading security is worlds in between.

Luckily, in any case, and regardless of bad implementations, the security-while-used is much more affected than the security-after-erased. Well... luckily, I don't know if that's a good wording, actually systems should always be secure. But at least it doesn't suck beyond.

There exist the notion of "master password" in the ATA standard, so any such thing as unlocking verus deriving an encryption key is -- even not considering that someone might find a way to read out the storage -- catastrophic. It basically means nothing is encrypted at all in a meaningful way.

Secure erase on a SED means erasing the disk encryption key, rendering the contents of the complete disk unreadable. So, unless one assumes a maliciously-built drive (which tells you it did a secure erase, but secretly still holds a copy of the key), this is secure even in presence of a broken implementation, and even in presence of someone cracking open the controller chip or such.

Secure erase on a traditional harddisk means the disk will overwrite every sector. I've recently done that with a pre-fail (SMART showing errors) Seagate Barracuda that was to be RMAed.
And guess what, secure erase is all nice and well, but a pre-fail disk will simply refuse to do the job. It'll start, whack around for a few minutes, and terminate with "error blah blah" after erasing approximately 10% of the disk. That wasn't an issue in my case since the data on the disk was from a RAID with software encryption on top, so any contents was basically useless anyway (wiping not really necessary). But, you get the idea. If you didn't use an encrypted filesystem, there's now no way to erase the data!

Generally, wear-levelling (both on traditional disks and SSDs) may make overwriting stuff much less possible than you are maybe inclined to believe.

Also, restoring overwritten data on a magnetic disk is possible. Yes, it is much, much harder than it was 15-20 years ago when data density was much lower (back then, it was pretty much a routine job). But it is still... generally possible.

So, if the data is truly super-sensitive (as in medical records), either one should layer software encryption on top, which eliminates the need to wipe the disk (though it doesn't hurt to do it anyway), or one should not donate the drives but use one of these to be sure.

Answer 5

The way ssd drives handle their secure erase is by encrypting the drive and then securely overwriting the encryption key. Technically the data is still there it's just theoretically not possible to access it. I used a more manual way.

I securely erased a couple thousand SSD drives when I worked for $LARGE_INTERNET_VIDEO_COMPANY.

NOTE: these drives were provisioned with 20% empty space using the ata max_lba command to prevent the pathological case of thrashing the same blocks as you fill up the drive.

I used the linux command-line tool called shred. The default is 3x random re-write (perfectly safe as far as my research showed). I also added another pass with zeros. Something like

shred -z /dev/sd$DRIVE_NUMBER

Making a last pass with zeros is nice because it's dead simple to verify it worked. If you sum any given blocks you should get zero.

You could create a linux live usb boot device. With one command and a few minutes you can securely erase the drive of any machine. (and you can verify it) Alternatively you could do this on any drive you plug in.