Latest Forum Posts

Latest News Posts
Coming Soon!
Social
Go Back   Techgage.com > Archives > Reviews and Articles

Reviews and Articles Discussion for Techgage content is located here. Only staff can create topics, but everyone is welcome to post.

Reply
 
Thread Tools
Old 03-05-2010, 02:25 AM   #1
Rob Williams
Editor-in-Chief
 
Rob Williams's Avatar
 
Join Date: Jan 2005
Location: Atlantic Canada
Posts: 13,337
Default Too TRIM? When SSD Data Recovery is Impossible

It goes without saying that solid-state drives are well-worth the investment in order to give your PC some responsiveness, but with all the benefits they can offer, there's one lesser-known issue that we'll talk about here. That issue is simple. As soon as you delete a file on a TRIM-enabled SSD, the data is gone, for good.

You can read our full report here and discuss it here.
__________________
Intel Core i7-3960X, GIGABYTE G1.Assassin 2, Kingston 16GB DDR3-2133, NVIDIA GeForce GTX 770 2GB
Kingston HyperX 3K 240GB SSD (OS, Apps), WD VR 1TB (Games), Corsair 1000HX, Corsair H70 Cooler
Corsair 800D, Dell 2408WFP 24", ASUS Xonar Essence STX, Gentoo (KDE 4.11. 3.12 Kernel)

"Take care to get what you like, or you will be forced to like what you get!" - H.P. Baxxter
<Toad772> I don't always drink alcohol, but when I do, I take it too far.


Rob Williams is offline   Reply With Quote
Old 03-05-2010, 02:47 PM   #2
Optix
Basket Chassis
 
Optix's Avatar
 
Join Date: Dec 2009
Location: New Brunswick, Canada
Posts: 1,677
Default



While I don't own a SSD and I can't see this being an issue for me since I always considered what I move from my recycle bin as being lost forever and am careful enough with my work it was really neat to read this. I had no idea that TRIM had a shortcoming. Well done!
Optix is offline   Reply With Quote
Old 03-05-2010, 04:03 PM   #3
Rob Williams
Editor-in-Chief
 
Rob Williams's Avatar
 
Join Date: Jan 2005
Location: Atlantic Canada
Posts: 13,337
Default

Wow.

I saw a new post in this thread, so I clicked on the thread from the main forum index, then minimized the browser to deal with something else while it loaded. I ended up forgetting about it, so the next time I opened up the browser, and saw that dude staring me in the face, I roared laughing. Definitely caught me off guard. Good stuff.

Thanks for the compliments also ;-)
__________________
Intel Core i7-3960X, GIGABYTE G1.Assassin 2, Kingston 16GB DDR3-2133, NVIDIA GeForce GTX 770 2GB
Kingston HyperX 3K 240GB SSD (OS, Apps), WD VR 1TB (Games), Corsair 1000HX, Corsair H70 Cooler
Corsair 800D, Dell 2408WFP 24", ASUS Xonar Essence STX, Gentoo (KDE 4.11. 3.12 Kernel)

"Take care to get what you like, or you will be forced to like what you get!" - H.P. Baxxter
<Toad772> I don't always drink alcohol, but when I do, I take it too far.


Rob Williams is offline   Reply With Quote
Old 03-05-2010, 04:26 PM   #4
Optix
Basket Chassis
 
Optix's Avatar
 
Join Date: Dec 2009
Location: New Brunswick, Canada
Posts: 1,677
Default

Brightening your day one stupid Google searched image at a time.
Optix is offline   Reply With Quote
Old 03-06-2010, 10:33 AM   #5
Unregistered
Guest Poster
 
Posts: n/a
Default sounds perfect

another reason to upgrade to SSD!
  Reply With Quote
Old 03-06-2010, 10:08 PM   #6
Unregistered
Guest Poster
 
Posts: n/a
Default Nilfs

My response to this is two-fold:

1) Back up your data to protect against accidental deletion; and

2) Use Linux with nilfs: http://www.nilfs.org/en/

nilfs, being a log-structured filesystem, is almost like a version control system used in software engineering, except that it's done for the entire filesystem. So if you nuke a file, you can simply revert the change as if it never happened.

Nilfs is also being optimized specifically for SSDs in some of the lower level routines, so you should see good performance on SSDs once it is stable. It's still in the development phases as of now, but with nilfs on a TRIM drive, you can still issue the command: rm -rf / (as root) and recover from it using an out-of-band system (just mount the nilfs drive and revert).

Even on a mechanical drive, using nilfs is better than relying on random chance with deleted files: even a very good undelete program will not be able to recover deleted files if you've done any serious amount of disk writes after you deleted your files. A sufficiently high number of writes have a very high probability to map a block or two into the places that have just been freed up, making your data unrecoverable unless you go to the electromagnetic experts referred to in the article.

Nilfs, on the other hand, will "fill up" your disk with filesystem revisions, and only start deleting the oldest revisions after the disk is totally full. Obviously, the log structure can go deeper depending on what percentage of your disk is full: if you write over a 1GB file with totally new data ten times, that will occupy at least 10GB of space with logs (previous versions). But the more data your disk has to keep track of in the current version, the less space is available for those previous versions; the oldest files will start getting kicked off the disk (permanently deleted) to make room for files in the current version.

Nilfs also makes redundant a lot of popular program features, such as the "backup file" used by many text ediitors, where the editor appends or prepends a ~ character and saves that as the previously saved copy. Heck, it even makes version control redundant to a degree, except that true software engineering version control systems keep an unlimited amount of history (assuming sufficient disk space) and allow for special features like merging, commit messages and branches.
  Reply With Quote
Old 03-08-2010, 10:01 PM   #7
Rob Williams
Editor-in-Chief
 
Rob Williams's Avatar
 
Join Date: Jan 2005
Location: Atlantic Canada
Posts: 13,337
Default

The first of your two recommendations is the easier one ;-)

Most people aren't going to move over to a different OS just because of a feature like this. It's important to note regardless that file recovery under Linux with most file systems is near-impossible to begin with, and if you do get the data back, it's only going to be after a lot of hard work, and even then you'll likely get broken files with random file names. I am not sure if ext4 is that much different from ext3 to make file recovery easier, but I doubt it.

As for NILFS, that's still a file system that most would consider bleeding-edge, or experimental, so I'm not sure even I would be willing to trust it on an OS drive. But from what you described, it sounds excellent, and I'd love to give it a test at some point in the future. I'm really curious about the performance though.
__________________
Intel Core i7-3960X, GIGABYTE G1.Assassin 2, Kingston 16GB DDR3-2133, NVIDIA GeForce GTX 770 2GB
Kingston HyperX 3K 240GB SSD (OS, Apps), WD VR 1TB (Games), Corsair 1000HX, Corsair H70 Cooler
Corsair 800D, Dell 2408WFP 24", ASUS Xonar Essence STX, Gentoo (KDE 4.11. 3.12 Kernel)

"Take care to get what you like, or you will be forced to like what you get!" - H.P. Baxxter
<Toad772> I don't always drink alcohol, but when I do, I take it too far.


Rob Williams is offline   Reply With Quote
Old 03-15-2010, 11:23 PM   #8
Ron
Guest Poster
 
Posts: n/a
Red face Mr. Ron N.

Rob: Thanks for an excellent article backed by your individual research. But I couldn't tell by looking at the screenshots of the recovery programs you used what was actually going on. In other words, I was unable to recognize or interpret the evidence you provided to support your hypothesis. Of course that may be because I have never used a data recovery program but, even so, the screens were not very self explanatory. However, I'll take your word for your conclusions.

Ron N.
  Reply With Quote
Old 03-17-2010, 12:55 PM   #9
Rob Williams
Editor-in-Chief
 
Rob Williams's Avatar
 
Join Date: Jan 2005
Location: Atlantic Canada
Posts: 13,337
Default

Hi Ron:

The screenshots weren't helping me out too much is half of the problem. As it appears in those, even on the TRIM drive, the SSD is loaded with data, but really, it isn't. Back in December, when I began doing all this testing, the TRIM-affected drive basically showed as 100% empty (minus the required file system space), but when doing it again for this article, the drive showed up all pink. I'm not quite sure why, but overall, the TRIM drive simply had no data, despite listing a couple of folders and files.

The thing I pointed out in the article to look at in both screenshots was the "Specific File Documents". You can see that on the non-TRIM drive, there were 288,000 files or traces of files found, while on the TRIM drive, there was just 488. And again, despite that, nothing is recoverable.

Hope this helps.
__________________
Intel Core i7-3960X, GIGABYTE G1.Assassin 2, Kingston 16GB DDR3-2133, NVIDIA GeForce GTX 770 2GB
Kingston HyperX 3K 240GB SSD (OS, Apps), WD VR 1TB (Games), Corsair 1000HX, Corsair H70 Cooler
Corsair 800D, Dell 2408WFP 24", ASUS Xonar Essence STX, Gentoo (KDE 4.11. 3.12 Kernel)

"Take care to get what you like, or you will be forced to like what you get!" - H.P. Baxxter
<Toad772> I don't always drink alcohol, but when I do, I take it too far.


Rob Williams is offline   Reply With Quote
Old 05-05-2010, 09:44 AM   #10
TheFocusElf
Obliviot
 
Join Date: Mar 2005
Posts: 49
Default

I have seen similar forays into the flaws of Trim coupled with SSDs, not just in data loss but in deterioration of speed and access as well. Scary. I did enjoy the Asheron's Call 1 world builder though! =D
TheFocusElf is offline   Reply With Quote
Old 05-05-2010, 06:00 PM   #11
Kougar
Techgage Staff
 
Kougar's Avatar
 
Join Date: Mar 2008
Location: Texas
Posts: 2,652
Default

There aren't any "flaws" in TRIM that cause deterioration of speed or increased latency... whenever that happens it's the flaws in the underlying controller itself being exposed.
__________________
Core i7 4770k 4.2Ghz
Gigabyte Z87X-UD5H
Crucial Ballistix Sport LP 1600MHz 32GB
EVGA GTX 480 HydroCopper FTW
ASUS Xonar DX
Corsair Neutron GTX 240GB | Windows 7 64-bit
Apogee XT + MCP655 & Thermochill Triple 140mm Radiator
Corsair AX1200 PSU | Cooler Master HAF-X

Kougar is offline   Reply With Quote
Old 05-06-2010, 02:03 AM   #12
thehailo
Obliviot
 
thehailo's Avatar
 
Join Date: Apr 2010
Location: US
Posts: 14
Default

Well, having a background in IT security let me fill in some of the gaps. This is kind of from a different angle than the original article though; more about making sure data is actually deleted rather than trying to make sure it’s recoverable.

The fundamental principal that makes recovery of data from magnetic media so possible is that it’s imperfect. The fact is modern hard drives are so dense manufactures are literally dealing with quantum randomness. This means just because a head writes over a track doesn’t mean it writes over every single molecule in that track.

Now, to be practical, writing over data once is enough to foil almost anyone. Past that you need time, skill, money, and an electron microscope. Even then you’re digging only for fragments of data at a very slow pace (100s of MB/h). Unless you piss NSA off you’re good with a single-pass wipe. The best practice today for government/military use is an eight-pass wipe, which pretty much beats even the electron microscope. Some people advocate as much as 35-pass wipes; but this is just overkill given modern recovery technology.

As far as SSDs, we won’t really know for a few years until the forensics field matures more. Until you get someone like FBI/NSA interested, who can take equipment like an electronic microscope and huge budgets and analyze them, we won’t know to what extent, if any, data may be recovered after being overwritten. In the past few years we’ve had minor breaks in data recovery from traditional RAM, but this is hit and miss even under the best circumstances such as having physical access immediately after shutting down and having something on hand to immediately cool the RAM down. Of course, NAND is by design more permanent, and MLC and SLC have fundamental differences, so once again, we just don’t know yet.

Either way, for forensics purposes recovering overwritten data doesn’t come into play much. Most people leave traces everywhere on their system. “Deleted” files, page and hibernation files, web browser history and cache, Windows thumbnails, System Restore, and Shadow Files are just a few of the places ripe for finding information users thought was long gone. I prefer EnCase for these purposes, as it’s extraordinary thorough in finding traces of long lost data.

One more note here deals with “secure wipe” tools. The fact is they don’t work, at least not on modern equipment. Journaling file systems don’t necessarily map directly to the hard drive. That is, because the file system points to sector 1 doesn’t mean that’s where the data is. The file system is duplicating bits for redundancy, moving things around for performance, and lots of other small tasks to keep things running. On top of this, modern hard drives also obfuscate the physical layer to some extent, quietly moving data around to bypass bad sectors for example. This all means that no software tool can know exactly where a file is on the physical drive, or where previous traces of it are. The only sure way to securely wipe data is either encrypt your entire drive or fill the entire drive with data, thus wiping all deleted files out. The Windows cipher command works well for this.
thehailo is offline   Reply With Quote
Old 05-09-2010, 01:52 PM   #13
Rob Williams
Editor-in-Chief
 
Rob Williams's Avatar
 
Join Date: Jan 2005
Location: Atlantic Canada
Posts: 13,337
Default

Quote:
Originally Posted by thehailo
Either way, for forensics purposes recovering overwritten data doesn’t come into play much. Most people leave traces everywhere on their system. “Deleted” files, page and hibernation files, web browser history and cache, Windows thumbnails, System Restore, and Shadow Files are just a few of the places ripe for finding information users thought was long gone. I prefer EnCase for these purposes, as it’s extraordinary thorough in finding traces of long lost data.
The lesson here is that it's just easier to get rid of the actual storage device, rather than run these tools... if you are THAT worried. Thermite, anyone?

Quote:
Originally Posted by thehailo
One more note here deals with “secure wipe” tools. The fact is they don’t work, at least not on modern equipment. Journaling file systems don’t necessarily map directly to the hard drive.
I think the best method from both a time/effectiveness perspective is just to do a secure erase, like with the HDD Erase tool. It goes beyond simple writes and rewrites and runs vendor-specific algorithms across the drive in order to truly purge the data. After running HDD Erase on a drive, the absolutely only thing you should see from a recovery drive map is: nothing.

That's not to say that data still couldn't be recovered by those who really take it seriously, but if I were to choose one method for the best chance of purging data, it would be with a secure erase.
__________________
Intel Core i7-3960X, GIGABYTE G1.Assassin 2, Kingston 16GB DDR3-2133, NVIDIA GeForce GTX 770 2GB
Kingston HyperX 3K 240GB SSD (OS, Apps), WD VR 1TB (Games), Corsair 1000HX, Corsair H70 Cooler
Corsair 800D, Dell 2408WFP 24", ASUS Xonar Essence STX, Gentoo (KDE 4.11. 3.12 Kernel)

"Take care to get what you like, or you will be forced to like what you get!" - H.P. Baxxter
<Toad772> I don't always drink alcohol, but when I do, I take it too far.


Rob Williams is offline   Reply With Quote
Old 05-10-2010, 01:17 AM   #14
thehailo
Obliviot
 
thehailo's Avatar
 
Join Date: Apr 2010
Location: US
Posts: 14
Default

Quote:
Originally Posted by Rob Williams View Post
I think the best method from both a time/effectiveness perspective is just to do a secure erase, like with the HDD Erase tool. It goes beyond simple writes and rewrites and runs vendor-specific algorithms across the drive in order to truly purge the data. After running HDD Erase on a drive, the absolutely only thing you should see from a recovery drive map is: nothing.

That's not to say that data still couldn't be recovered by those who really take it seriously, but if I were to choose one method for the best chance of purging data, it would be with a secure erase.
That's how I use cipher actually. The tool has a few functions, but cipher /W wipes the free space of a drive with a three-pass wipe; once with 0x00, once with 0xff, and once with random numbers, respectively.

When I decommission a drive I typically just format it, attach if as a secondary drive, then run cipher on it. It's quick and easy, and since cipher is built into Windows it's always on hand and absolutely free.
thehailo is offline   Reply With Quote
Old 08-26-2010, 11:44 AM   #15
meathead
Obliviot
 
Join Date: Aug 2010
Posts: 1
Default

Not only is this completely untrue, data that has been written to an SSD is still recoverable until more than enough data has been written to the disk to completely fill it.

The TRIM command doesn't erase blocks, it lets the drive know that blocks are no longer in use. The drive is then free to reallocate those blocks as it sees fit. Actually overwriting the physical data is unnecessary, and in the context of an SSD with a limited number of writes, wasteful.

In other words, if a file occupies block 10, a normal file delete wouldn't actually touch that block, just the file system structures that associate it with a file. In a mechanical disk, that's enough, since the file system is responsible for file placement.

An SSD uses virtual blocks, a look up table, and wear leveling to increase drive lifetime. Virtual block 10, which is what the OS sees may be physically located in block 45 on the disk, and any read request for block 10 would get redirected, using the look up table, to physical block 45. Next time the OS writes to virtual block 10, it might be writing to physical block 38.

With TRIM, the drive is notified right away that a block is available for reuse. The SSD can then remove that block from the look up table and return it to the pool of available blocks. It's still physical block 45, but it's no longer associated with virtual block 10. While the data is no longer available to the OS, it still physically exists.

Any read of virtual block 10 will now return empty data, but the data that was written to virtual block 10 still exists on physical block 45.

Wear leveling prefers least recently used blocks, so the original data will stay intact until enough data has been written to the disk to completely fill it up. It's even worse than that. SSDs have additional reserved space. An 80GB drive may actually have 90GB of storage space. The extra space exists both to compensate for blocks that have gone bad and to make it easier for the wear leveling algorithm to reduce fragmentation.

Add to that the fact that wear leveling algorithms are proprietary and not strictly LRU, and the only way to be sure the data is gone is to completely overwrite the disk at least twice.
meathead is offline   Reply With Quote
Reply

Tags
None

Thread Tools

Posting Rules
You may not post new threads
You may post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is On

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Intel Divulges Information on TRIM for Linux Rob Williams Storage Devices and Memory 2 11-20-2009 03:12 PM
Personal SSD Usage: TRIM and Linux Rob Williams Storage Devices and Memory 0 11-16-2009 03:43 PM
Is OnLive's Technology Impossible to Pull Off? Rob Williams Gaming 1 04-03-2009 08:12 AM
Windows XP's SP3 Makes Reverting to IE6 Impossible Rob Williams General Software 8 05-22-2008 12:37 PM


All times are GMT -4. The time now is 07:58 AM.