23

If the goal is to make data no longer retrievable, how secure is it to format the disk? I assumed formatting the disk overwrites free space (thus making it a safe bet no one's going to be able to retrieve the data) but according to webopedia this is not the case. How much more secure is it to delete files with an erasure utility (using something like Schneier's method)than formatting the drive? How does formatting the drive not erase all data: formatting involves recreating the file system so this seems to imply the data is not retrievable.

UPDATE: I generally do not choose the "quick format" option.

  • 1
    Note that the answer is very different depending on whether it is a conventional hard disk (spinning platters) or an SSD. – sleske Apr 28 '16 at 7:00
  • 1
    Maybe note that the so called "Schneier's method" is in fact based on a misunderstanding. Originally, experiments were conducted to see which of several methods would prove beneficial in wiping data. This included writing all 0's, writing all 1's, as well as writing random data. The conclusion was that writing random data two or more times is absolutely sufficient. The original paper was subsequently often misinterpreted as stating that all examined methods should be carried out in sequence. – JimmyB Apr 28 '16 at 11:46
45

Quick-formatting a hard disk simply erases the filesystem's structures and tables and writes new ones in place, giving the illusion of a brand new disk. Old data is simply overwritten as and when needed, but it still remains on the disk. File carving utilities can go through the disk data and recover fragments of files, then stitch them back together without needing the original filesystem entries. This is commonly offered in commercial "undelete" applications, but more comprehensive methods are available in forensics packages.

Wiping a disk with a single pass of random data (or zeroes, or whatever really) is sufficient to fully remove all traces of the data from the overwritten sectors. Multiple passes are pointless on modern disks, even against the perceived threat of hardware-level recovery attacks. I refer you to this question for details, but the short answer is that old techniques like magnetic force microscopy (MFM) were never really effective at recovering overwritten data in the first place on low-density devices, and newer magnetic disks have such high densities that it's physically impossible. Multi-pass overwrites are there to help people validate their need for über-security, or sell magic disk wiping software, despite it being pointless and detrimental to disk longevity.

The only exception is flash (e.g. USB flash drives and SSD), which have additional wear-leveling sectors to increase the lifespan of the device. The physical sectors are exposed as a logical map to the system, which makes it impossible to directly overwrite all of the data. Even if you overwrite all the logical sectors, old data might remain in the wear-leveling sectors. In order to combat this, some flash device specifications include an encryption requirement to increase the difficulty of recovery (because it is no longer possible to directly read data from the memory chips using hardware probes).

In SSDs, encryption can be used to speed up the ATA Secure Erase feature. All sectors on the disk are encrypted using a key stored in hardware, and this key can be discarded and a new one generated when a Secure Erase command is sent to the device, thus rendering all data on the disk (including slack / wear-leveling areas) unreadable. Secure Erase is also possible without encryption, but then the drive must actually delete all memory cells, which may take a while.

  • 8
    "(...) rendering all data on the disk unreadable" - the old data is still there, but encrypted. If the discarded key can be cracked (yeah, hundreds of years) then the data is available agian. – Mindwin Apr 27 '16 at 16:08
  • 4
    What if the quick-format option is not checked? – Celeritas Apr 27 '16 at 19:28
  • 2
    Note that spinning disks perform sector reallocations, whilch have effects similar to the wear leveling of SSDs. So overwriting all sectors of the disk does not erase all data, because of reallocated sectors (and the HPA). Still, it will typically erase 99% of the data. The safest option is SATA Secure Erase. – sleske Apr 28 '16 at 8:04
  • 2
    @Polynomial (...) as of now? yes. Tomorrow? XXII century? Moore's law? New math? New computers? who knows. And I stated "hundreds of years" in my comment above. Nobody is worrying, but since this is a theoretical discussion, we have to consider all possibilities. – Mindwin Apr 28 '16 at 17:35
  • 2
    @Nayuki Apparently since Vista, full formatting does a complete wipe, see e.g. support.microsoft.com/en-us/kb/941961 – JimmyB Apr 29 '16 at 15:00
4

One part of your question seems not to have a good answer, so I want to add to this:

By formatting a drive, you typically choose to do a "quick format". This does not overwrite the data, but only clears the information where each file is stored. Therefore, it is as easy as starting a program to recover a hard drive that has just been "quick formated".

Maybe this comparison helps you to understand this better: Your hard drive is like a book - and the partition(s) and the file system(s) are the table of contents. A quick format only erases the ToC. This is faster than erasing each single page - but the information in the book stays recoverable.

At least for classical hard drives, overwriting each single sector should provide enough security for most cases.

  • 2
    To add a quick note to the metaphor - a quick format removes the Table of Contents of the book, allowing you to write a new Table of Contents later which overwrites the existing content of the book. But after only a Quick Format, a data recovery program can easily still read the book. Once you overwrite the data, not so much. – Jake Apr 27 '16 at 16:56
  • 1
    @Jake yes exactly! – Lukas Apr 27 '16 at 21:35
1

If "disk" happens to mean "solid state disk", you are somewhat lucky. You can instantly make retrieval of data (almost) impossible by using the manufacturer's secure erase tool. This process is both very fast and secure due to the way these drives work: They encrypt all data that is written to the drive, no exception made. Doing a "security erase" simply tosses the key overboard, and the data, while still present, is unrecoverable without the key.

Otherwise (i.e. "disk" means "harddisk"), the only safe method of making data unrecoverable is by disassembling the device and physically destroying the platters.

The reason for that is two-fold. First, overwritten data can still be read. Data density has admittedly gone up considerably during the last decade, and some people argue that what's overwritten once is basically unrecoverable, but if it is really, really important, I wouldn't bet my right hand onto that. I haven't tried to recover erased data during the last 15 years, but it used to be that recovering 5-6 times overwritten data was kinda tedious, but otherwise absolutely no problem. Maybe that's different now, maybe you only need to overwrite twice, or maybe once. But there's a bigger problem ahead.

The second, and much more important issue is that you don't know if and when you overwrite something. This is a very serious problem.

Modern drives do transparent wear-levelling and sector reallocation, and maybe even caching on MCL (hybrid drives). You have absolutely no knowledge, or control, what gets written when and where, or what gets overwritten.

Thus, even a "secure erase" that overwrites the same file ten times with different patterns might indeed overwrite ten different sectors on the disk, none of them belonging to the original file (and you might possibly still have an old copy of the data stored in a retired, now inaccessible block).

The only way of knowing for sure is disassembling the drive, putting a neodym magnet onto the platters for a few minutes, and giving the board and platters a little treatment with the hammer afterwards.

  • 10
    "It used to be that recovering 5-6 times overwritten data was kinda tedious, but otherwise absolutely no problem." From what I've read, this is more of an urban legend than fact, with no more support than a single researcher's theoretical claims to support it. Do you have any references to support it ever being "tedious but no problem" to recover data from a single-pass randomized overwrite? This is the best resource I have found supporting the opposite, but I'm happy to be proven otherwise: web.archive.org/web/20121110053501/http://grot.com/wordpress/… – loneboat Apr 27 '16 at 18:34
  • 1
    There are several mistakes here: Not all SSD drives use encryption; spinning platter disks can be (quite) safely erased without destruction (ATA Secure Erase); "it used to be that recovering 5-6 times overwritten data was kinda tedious," - I don't think this has ever been done. etc. etc. – sleske Apr 28 '16 at 8:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.