Various methods to improve rsync speeds

rsync is a command line tool to transfer data between a source and a destination. A destination or source can be remote.

Avoid the following: 

  • -z - try to avoid using compressing network traffic.
  • --append-verify - resume an interrupted transfer. This sounds like a good idea, but it has the dangerous failure case: any destination file the same size (or greater) than the source will be IGNORED. Also, it checksums the whole file at the end, meaning no significant speed up over --no-whole-file while adding a dangerous failure case.

Use the following in the command: 

  • -a - Archive mode - recursive, preserves owner, preserves permissions, preserves modification times, preserves group, copies symlinks as symlinks, preserves device files.
  • -v - Increase verbosity
  • -A - Preserves ACLs
  • -H - Preserves hard-links
  • -X - Preserves extended attributes
  • -W - Copy files whole - always use this if you don't want it to compare differences;
    rsync is designed to compare differences and only update the changes. Remove this if you require the compare.
  • -S - Handles sparse files (files that are for instance holding a file system and that just take the space needed by their actual size)
  • --no-compress - This may have a big impact on speed gain.
  • --exclude-from - Exclude files you might not need.
$ rsync -avAXEWSlHh /source /destination --no-compress --info=progress2 --dry-run

Test using --dry-run, then remove this to execute the sync.