Backups aint backups, Saul

rat156 · #1 12-12-2013, 09:48 PM

Hi All,

My daughter has learnt three valuable lessons over the past week or two.

Her school laptop had to be returned to the school (Fujitsu, quite bad, don't buy laptops from people who make air conditioners!!), so they had to remove all their personal stuff. She used a portable HDD for the purpose.

Lesson 1: Windows 8 didn't copy her photos. It copied everything else, but missed all her photos. She was quite distraught as these were her only copy of her school trip to France earlier this year. Make sure the files you think are backed up are actually backed up!.

Lesson 2: Just because you have deleted a file doesn't mean it's gone. I retrieved her SD card from her camera, which she had wiped. A little bit of software later (and $89US) and my Mac had most of the photos back. Much relief!

Lesson 3: Her Dad is an IT guru!!

Of course the lesson for all here is to test the integrity of your backup system.

Cheers
Stuart

hobbit · #2 13-12-2013, 10:55 AM

with SD cards being cheap as chips these days I never wipe mine. When one gets full I just get a new one and keep the other as a backup.

ZeroID (Brent) · #3 13-12-2013, 01:42 PM

Quote:

Originally Posted by hobbit

with SD cards being cheap as chips these days I never wipe mine. When one gets full I just get a new one and keep the other as a backup.

Geez !! if I did that with the SONY at 24 megapixels I'd have a bucket load of cards ! I run multiple drives in the big PC and a USB offline HD unit with swappable drives. Fill it up, throw in another.

2stroke (Jay) · #4 13-12-2013, 03:35 PM

Thats great you got her photos back, nothing worse then loosing personal doc's and images. Just for future ref invest in some good backup software or hard copy files onto a personal NAS that has raid 0. In my HTPC now i run raid 0 because of HD fail rates these days. Anyhow good work rat, haha hope she remembers fathers day

Astro_Bot · #5 13-12-2013, 03:58 PM

There's a sure fire way to make sure you miss nothing, no matter where it may be hiding on your hard drive - clone the drive.

Another advantage is that if your internal hard drive fails, you can replace it with the external drive (extracted from its enclosure, and assuming it's the same form factor and physical interface) and boot off it as if nothing had happened.

It's easy: get an external hard drive the same size (or larger). Get any recent Linux distro's LiveCD and burn to CD (it's free). Plug in the external hard drive, boot from said LiveCD, open a terminal and issue the following commands:

Code:

su -
dd if=/dev/sda of=/dev/sdb bs=1M

If it takes a long time (as it might with a USB2 drive) then leave it running overnight - it'll be done by morning.

The only thing you need to check first is the identity of the targets, so I'll explain what they are:
if is the Input File, which, in this command, is assigned to device sda, the first hard disk detected.
of is the Output File, which, in this command, is assigned to device sdb, the second hard disk detected.

Your system will detect the hard disks in the same order each time and 99.9% of the time the internal hard disk will be sda, but it pays to check. There's usually a Disk Utility to tell you the parameters of each disk device or you can simply look at what's on them - as long as you can confirm that sda is your internal drive, you're good to go.

(There's a very small chance that your system could reverse the order of the drive designations, so it's possible, however unlikely, that you could wipe everything instead of backing up. Hey, with great power comes great responsibility. Of course, if you want to wimp out and buy Windows software to do this, then I suppose that's another way to go.)

2stroke (Jay) · #6 13-12-2013, 05:08 PM

Quote:

Originally Posted by Astro_Bot

There's a sure fire way to make sure you miss nothing, no matter where it may be hiding on your hard drive - clone the drive.

Another advantage is that if your internal hard drive fails, you can replace it with the external drive (extracted from its enclosure, and assuming it's the same form factor and physical interface) and boot off it as if nothing had happened.

It's easy: get an external hard drive the same size (or larger). Get any recent Linux distro's LiveCD and burn to CD (it's free). Plug in the external hard drive, boot from said LiveCD, open a terminal and issue the following commands:

Code:

su -
dd if=/dev/sda of=/dev/sdb bs=1M

If it takes a long time (as it might with a USB2 drive) then leave it running overnight - it'll be done by morning.

The only thing you need to check first is the identity of the targets, so I'll explain what they are:
if is the Input File, which, in this command, is assigned to device sda, the first hard disk detected.
of is the Output File, which, in this command, is assigned to device sdb, the second hard disk detected.

Your system will detect the hard disks in the same order each time and 99.9% of the time the internal hard disk will be sda, but it pays to check. There's usually a Disk Utility to tell you the parameters of each disk device or you can simply look at what's on them - as long as you can confirm that sda is your internal drive, you're good to go.

(There's a very small chance that your system could reverse the order of the drive designations, so it's possible, however unlikely, that you could wipe everything instead of backing up. Hey, with great power comes great responsibility. Of course, if you want to wimp out and buy Windows software to do this, then I suppose that's another way to go.)

Cheers for this will be doing my notebook factory drive with it before a custom install. I take this grabs raw data so partions and boot rant effect when you recopy back to original drive?

Astro_Bot · #7 13-12-2013, 05:13 PM

It copies every byte, including boot blocks, MBR, partition tables, etc. regardless of whether there's anything there or not. If there's deleted files hidden, it'll copy those too.

RickS (Rick) · #8 13-12-2013, 05:33 PM

I reckon there is a fair chance of running into problems doing a raw disk copy unless both drives have exactly the same geometry. At the very least you are risking poor performance if the second disk has cylinder boundaries in different places.

Cheers,
Rick.

Astro_Bot · #9 13-12-2013, 05:57 PM

Quote:

Originally Posted by RickS

I reckon there is a fair chance of running into problems doing a raw disk copy unless both drives have exactly the same geometry. At the very least you are risking poor performance if the second disk has cylinder boundaries in different places.

That's not the case anymore. Cylinder boundaries are an artificial construct in modern hard drives. Modern access modes in disk firmware (and as supported by your OS) obfuscate physical geometry.

I've cloned dozens (hundreds?) of times and not yet had a problem, including booting off the cloned drive many times.

RB (Andrew) · #10 13-12-2013, 06:03 PM

Quote:

Originally Posted by Astro_Bot

That's not the case anymore. Cylinder boundaries are an artificial construct in modern hard drives. Modern access modes in disk firmware (and as supported by your OS) obfuscate physical geometry.

I've cloned dozens (hundreds?) of times and not yet had a problem, including booting off the cloned drive many times.

Would this work on old IDE and SATA drives running XP?
Thx

RickS (Rick) · #11 13-12-2013, 06:31 PM

Quote:

Originally Posted by Astro_Bot

That's not the case anymore. Cylinder boundaries are an artificial construct in modern hard drives. Modern access modes in disk firmware (and as supported by your OS) obfuscate physical geometry.

I've cloned dozens (hundreds?) of times and not yet had a problem, including booting off the cloned drive many times.

Another case of modern computing losing the plot, then

Despite all the buffering in the world, the physical geometry of a moving head HDD does have an impact on performance.

kelaga (Gary) · #12 14-12-2013, 12:12 AM

Quote:

Originally Posted by 2stroke

Thats great you got her photos back, nothing worse then loosing personal doc's and images. Just for future ref invest in some good backup software or hard copy files onto a personal NAS that has raid 0. In my HTPC now i run raid 0 because of HD fail rates these days. Anyhow good work rat, haha hope she remembers fathers day

Hmmm.... I hate to be the devil's advocate. RAID 0 does not have any redundancy. It only does stripping across two drives.

Peter.M · #13 15-12-2013, 11:33 AM

Quote:

Originally Posted by kelaga

Hmmm.... I hate to be the devil's advocate. RAID 0 does not have any redundancy. It only does stripping across two drives.

He is referring to raid 1.

noeyedeer (Matt) · #14 15-12-2013, 09:54 PM

what about making an image .. iso of your hard drive?

h0ughy (David) · #15 15-12-2013, 10:33 PM

Quote:

Originally Posted by RB

Would this work on old IDE and SATA drives running XP?
Thx

not wanting to fix Lazarus are you

RB (Andrew) · #16 16-12-2013, 01:34 AM

Quote:

Originally Posted by h0ughy

not wanting to fix Lazarus are you

Yep, can't let go......

multiweb (Marc) · #17 16-12-2013, 08:28 AM

Quote:

Originally Posted by Peter.M

He is referring to raid 1.

I try to stay away from RAID 1 or 5. For small volumes it's more time and trouble to resync than just using stand alone drives with no dependencies. RAIDs work well in huge distributed systems or NAS but for an individual I reckon keep it simple. Just ghost individual drives. No guessing involved.

Astro_Bot · #18 16-12-2013, 03:52 PM

Quote:

Originally Posted by RB

Would this work on old IDE and SATA drives running XP?

The OS on the disk shouldn't matter - it'll be faithfully copied - it's a byte-by-byte copy and it doesn't matter what's in the byte.

As to booting off a new SATA drive after cloning an old IDE drive .... well, I've never tested it.

The changes from IDE to SATA were designed to be fairly transparent to the user and I can't presently think of a reason why it wouldn't work.

If you're thinking of transplanting to a new machine (new motherboard) then you will definitely have problems, and I doubt the machine would get far in the boot process due to wrong drivers and wrong absolute addresses.

For a cloned HDD put back in the same machine, it's worth a try.

Keep in mind that cloning does not increase the size of partitions/volumes. For example, if you clone a 100GB drive to a 1000GB drive, you will still only use 100GB! You will have to add either a new volume to use the remaining 900GB (from within your OS) or expand the exisiting partitions/volumes to use more space - that can be risky and a fair percentage of partition/volume resizes have been known to fail.

RB (Andrew) · #19 20-12-2013, 09:13 PM

Quote:

Originally Posted by Astro_Bot

The OS on the disk shouldn't matter - it'll be faithfully copied - it's a byte-by-byte copy and it doesn't matter what's in the byte.
....

Thank you for the info Astro_Bot.

Astro_Bot · #20 01-01-2014, 03:34 PM

Quote:

Originally Posted by Astro_Bot

Modern access modes in disk firmware (and as supported by your OS) obfuscate physical geometry.

I thought I'd post a little more on this, just in case anyone was still labouring under the misapprehension that modern hard disks were simple things:

Hard disk hacking

It's a very interesting article, especially when you realise how much "hidden" software there is between you (your application) and the hard drive platter.