Cloning disks or partitions using dd

<yambe:breadcrumb self="Cloning disks or partitions using dd">Dd|Dd</yambe:breadcrumb>

Cloning disks or partitions using dd

We can backup entire hard-disk or partition or clone hard-disks using dd and nc command combinations. To clone hard-disk from one machine to another we can use:

On destination machine:

1. Boot using Linux live CD
2. If Live CD is cached in RAM then use 'eject /dev/scd0' to eject cd
3. Use 'dhclient eth0' to get IP from DHCP server. Note that IP address obtained.
4. Start data receiver using:

   nc -l -p 9000 | gunzip -4 | dd of=/dev/sda bs=40000000

On source machine:

1. Boot using Linux live CD
2. If Live CD is cached in RAM then use 'eject /dev/scd0' to eject cd
3. Use 'dhclient eth0' to get IP from DHCP server. Note that IP address obtained.
4. Start data sender using:

   dd if=/dev/sda bs=40000000 | gzip -4 | nc <IP_of_destination> 9000

Note:

• The default compression level used by gzip is 6. Any compression level higher than that will cause too much CPU usage and very less network usage. Hence use compression less than or equal to 6. Compression of 4 used above is good enough for 100mbps networks, especially if same source is serving three destinations and source has dual core CPU with good RAM. On gigabit networks one may want to avoid compression or reduce it to -1 or -2.
• The above process can also be used for copying VM images from one machine to another machine. Copying sparse VM images would cause ballooning on destination. Then we can use 'cp --sparse=always' to reduce the ballooned image to normal size if required. It should be noted that this finishes faster than rsync with '-Sz' combinations

Seeing progress

To see progress of how much data is copied we can use:

ps -C dd
while true; do kill -USR1 <process_id>; sleep 1; done

on any of source and destination. In case source is serving multiple destinations than preferably do it on destination.

Performance

It is better to use udpcast in case you want to copy same source to more than one destinations

When using same source to copy data to three destinations parallely, try to start source to send data to all three soon one after other. If there is large gap while trying to send same data to three destinations the data read from hard-disk into RAM cache may get replaced and hence require re-reading of hard-disk. This may lead to hard-disk becoming bottle-neck and not network. It was observed that copying to more than three destinations at same time does not works well and causes machines to hang for some unknown reason.