CentOS 8.x Copying or backup of documents using rsync

Home > CentOS > CentOS 8.x > CentOS 8.x Backup tools > CentOS 8.x rsync > CentOS 8.x Copying or backup of documents using rsync

rsync is very powerful tools for copying files within the same system or across systems (most often over ssh). For this rsync should be installed on both source and destination machines. Further, it makes sense to have similar (not needed to be exact same) versions of rsync on both machines so that various options work properly.

While using rsync it is important to keep track of ending forward slash (/) after directory path. If a directory name is not ending with forward slash (/) then it means that directory should be copied to destination. If directory name or path ends with forward slash (/) then the contents should be copied. I find using terminating both source and destination paths with forward slash(/) easiest and less confusing.

For example:

• rsync /opt/ root@10.1.1.1:/opt/ - This will copy contents of /opt/ on source to /opt/ on destination. That is /opt/a.txt on source will be copied as /opt/a.txt on destination
• rsync /opt/ root@10.1.1.1:/opt/abcd-opt-backup/ - This will copy conents of /opt/ on source to /opt/abcd-opt-backup/ on destination. That is /opt/a.txt will be copied as /opt/abcd-opt-backup/a.txt on destination machine.

For remote rsync any one (either source or destination) argument can be remote.

Various useful options for rsync are:

-v

Verbose. This will print path of each and every file/folder being copied. This allows seeing which file is being copied right now.

-t

Preserve timestamp. This ensures that timestamps are preserved across machines

-r

Copy recursively. Useful while copying folders with sub-folders and files.

-p

Copy permissions. Useful along with (-t) to ensure both permissions and timestamps are preserved.

-a

Archive mode. Equals -rlptgoD. Useful while copying from one system to another along with ownership information.

--progress

Show progress. This is useful while copying large files and we want to see the progress

--inplace

Copy inplace. Useful if you want to copy directly to destination file. Normally rsync creates a temporary file and renames after successful copy. This is an issue if rsync stops in between. The temporary file is deleted and copying starts again from scratch. With --inplace the copying happens inplace and hence even if copying stops in between. Later it can resume at same location. Note that rsync uses diff algorithm to copy large files. So it can transfer checksums of large blocks of data (eg every 1MB) from source to destination to quickly validate whether source and destination are same. If they are different then only relevant block is copied. This is very useful if you want to rsync directly a device to another device as that must be done inlpace only.. Ideally we should use dd for such copying.

--sparse

Treat sparse files properly. If there is consecutive blocks with just zeros, it is converted to sparse block properly.

-s

Preserve spaces in path. Since rsync happens between machines it is tricky to do rsync with path having spaces. Refer https://unix.stackexchange.com/questions/104618/how-to-rsync-over-ssh-when-directory-names-have-spaces

-z

Compress files during transmission. Useful if network is slower 100mbps while there is enough CPU usage free. Also if files are text/html etc. that can be compressed.

-H

Preserve hardlinks. That is files a.txt and b.txt on source side are hard-links then on destination also they should be hard-links. Useful while copying backups created via CentOS 7.x rsnapshot.

--delete

Delete extra files. Normally rsync only copies files from source to destination but does not deletes any extra files on destination. This ensures that extra files are deleted so that source and desination become identical.

-n

Dry run. This simulates what changes will happen if rsync is run without -n. This helps in seeing what files will get copied / deleted etc. before making any change.

Common invocations

Overall I most often use these two options:

rsync -vtrp

While copying to current user where ownership of files should change to current user. Typically for copying files.

rsync -vaHz --delete

While copying between machines specially while migrating data between machines where ownership must be preserved.

Older article on this is at Backing up documents using rsync

rsnapshot

You can consider using Rsnapshot for taking backups. It internally uses rsync anyway.

Home > CentOS > CentOS 8.x > CentOS 8.x Backup tools > CentOS 8.x rsync > CentOS 8.x Copying or backup of documents using rsync