Sugioarto

2015-03-03

How to make an efficiently compressed drive image

Many people use dd to make an image of their harddrives. They tend to be huge consuming lots of precious space, even when compressed. There is a small trick how to make such an image as small as possible, but still have a fully working result.

The trick

Before creating an image with dd on the raw drive make sure you write zeroes to all unallocated space on the drive. For this on some filesystems, like ZFS which uses snapshots, you should also destroy all snapshots that you don't need.

Then execute a command similar like this:

dd if=/dev/zero of=/zerofile bs=65536

The command above will make the rootfs full with zeroes. This can take a while (it depends on how large your rootfs is). The erase it again:

rm /zerofile

Do this on all filesystems that belong to the harddrive which you want to do a dd of. You can find out which filesystems you need to fill with such a izerofile using the command mount.

In case you are on Microsoft Windows, you can use this Java program to write a zerofile to your drive:

import java.io.*; public class NullWriter { public static void main(String[] arg) { if (arg.length == 1) write(arg[0]); else { System.out.println("java NullWriter FILENAME"); } System.exit(0); } public static void write(String filename) { byte[] buf=new byte[1024*1024]; for (int i = 0; i < buf.length; i++) buf[i]=0; FileOutputStream fos = null; try { fos= new FileOutputStream(filename); while (true) { fos.write(buf); } } catch (IOException ex) { try { if (fos != null) fos.close(); } catch (IOException exx) {} } } }

Compile it with:

javac NullWriter.java

Run it like this:

java NullWriter c:\zerofile

If you have more than only drive C: on your harddrive, write on the other drives, too. Here you should also erase the file again after the drive is full.

Now proceed with making an image

You can proceed to make an image the good old way. You should use a Linux live filesystem for this step.

For example on a second mounted drive (assumed it is mounted on /mnt/second-hdd and the drive you want to make an image of is /dev/sda; you need to be root):

dd if=/dev/sda bs=65536 | gzip -c > /mnt/second-hdd/sda-image.img

Or pipe it over SSH to some other remote host:

dd if=/dev/sda bs=65536 | gzip -c | ssh myremoteuser@remotehost 'cat - > sda-image.img'

It does not matter what system is installed on the drive. It will be a full image and because you wrote a lot of zeroes into the free space, gzip will compress it very efficiently. It can take a lot of time to make an image of a huge drive.

Please make sure that you actually understand what the commands above do and more importantly, make sure that you've made an image of the drive which you actually planned to be backuped. Mistakes happen, but it will make you very sad, if you lose data.

Testing integrity

It is a good idea to test if the image can be uncompressed at least:

gunzip -c sda-image.img > /dev/null

If you see an error, something is wrong. The command also takes a lot of time.

Image recovery

First, you can only recover an image on exactly the same drive. It won't work with a different type, even it is from same vendor and even if it has the same size (in many cases!). Be careful!

Attention! Make sure the drive you write an image to does not contain any data that you need. Double-check it! Also make sure that you know exactly which device is which. Please be aware that sometimes device names can change after a reboot, even from the same live filesystem CD you used before!

I use the same name as above (/dev/sda) which was the source drive, but I just warned you that you could destroy data in the following step, right?

Write an image back to the drive from a second mounted harddrive in directory /mnt/second-hdd:

gunzip -c /mnt/second-hdd/sda-image.img | dd of=/dev/sda bs=4096

Or back from a remote host using SSH:

ssh myremoteuser@remotehost 'cat sda-image.img' | gunzip -c | dd of=/dev/sda bs=4096

That's it!

With the trick above your image should be maximally as large as the allocated space in the filesystem. The free area on the filesystem won't be backuped and is not going to make the resulting image too huge.

It is a good idea to make an image using this method for your Microsoft Windows installation, after directly you bought a new laptop and don't want to lose the system with all recovery partitions and so on, when you sell it later to someone else.