Rsync for Backups and File Synchronization: How I Actually Use It

By /

rsync is one of those tools I reach for constantly. Backups, deployments, syncing configs across servers, copying large directory trees — it handles all of it well. The basic invocation is simple, but there are a few flags and patterns that make a huge difference once you know them.

The Basic Idea

rsync transfers files efficiently by comparing source and destination and only sending what’s different. The “delta algorithm” means if you’ve already synced 100GB and change 10 files, only those 10 files transfer next time. This makes it ideal for recurring backups — fast after the first run.

Basic Syntax

rsync [options] source destination

Local copy:

rsync -av /home/user/documents/ /backup/documents/

The trailing slash on the source matters. /source/ means “copy the contents of source”, while /source (no slash) means “copy the source directory itself”. I forget this constantly and get an extra level of nesting.

Remote sync over SSH:

rsync -avz user@remote:/var/www/html/ /local/backup/html/

Flags I Actually Use

-a (archive) — this is the combination flag I always include. It enables recursion, preserves symlinks, permissions, timestamps, owner, and group. Equivalent to -rlptgoD.

-v — verbose. Shows which files are being transferred.

-z — compress data during transfer. Useful over slow or metered connections. On fast local networks it can actually be slower due to compression overhead.

-P — combines --progress (shows transfer progress) and --partial (keeps partial transfers so they can be resumed). I use this for large transfers.

--delete — deletes files in the destination that no longer exist in the source. Essential for real backups — without it, deleted files accumulate in your backup directory forever.

-n (dry run) — shows what would happen without actually doing it. I always run this first when trying a new rsync command.

--exclude — skip files matching a pattern. More on this below.

-e — specify the remote shell. Usually ssh, but I sometimes use this to pass SSH options.

Backup Script Template

Here’s the basic backup script structure I use:

#!/bin/bash
set -euo pipefail

SOURCE="/var/www/html/"
DEST="/backup/www-html/"
LOG="/var/log/rsync-backup.log"

echo "$(date): Starting backup" >> "$LOG"

rsync -av \
  --delete \
  --exclude="*.log" \
  --exclude="*.tmp" \
  --exclude=".git/" \
  --exclude="node_modules/" \
  "$SOURCE" "$DEST" >> "$LOG" 2>&1

echo "$(date): Backup complete" >> "$LOG"

Throw this in a cron job:

0 2 * * * /usr/local/bin/backup-www.sh

Incremental Backups with –link-dest

Plain rsync gives you the latest snapshot. But what if you want to keep multiple snapshots without using tons of disk space? Hard links solve this.

The --link-dest flag creates hard links to unchanged files from a previous backup, so each snapshot looks like a full copy but only stores new/changed files:

#!/bin/bash
set -euo pipefail

SOURCE="/var/www/html/"
BACKUP_ROOT="/backup/www"
DATE=$(date +%Y-%m-%d_%H-%M-%S)
DEST="$BACKUP_ROOT/$DATE"
LATEST="$BACKUP_ROOT/latest"

rsync -av \
  --delete \
  --link-dest="$LATEST" \
  "$SOURCE" "$DEST"

# Update the 'latest' symlink
rm -f "$LATEST"
ln -s "$DEST" "$LATEST"

This creates dated backup directories. Each one contains all files, but files that haven’t changed are just hard links to the previous snapshot — they take up no extra space. You can browse any date directory and see the full state of the source at that time.

To clean up old backups, just delete old directories:

find /backup/www -maxdepth 1 -type d -mtime +30 -exec rm -rf {} \;

Syncing to a Remote Server

Pushing a deployment to a remote server:

rsync -av --delete \
  -e "ssh -i ~/.ssh/deploy_key -p 2222" \
  ./dist/ \
  deploy@production.server.com:/var/www/html/

-e "ssh -i ~/.ssh/deploy_key" lets you specify a non-default key. Useful when you have different keys for different purposes.

Pulling from remote:

rsync -av user@remote:/etc/nginx/ /backup/nginx-configs/

Exclude Patterns

Excludes are powerful but the syntax trips people up. A few examples:

# Exclude a specific directory
--exclude=".git/"

# Exclude all .log files
--exclude="*.log"

# Exclude multiple things
--exclude={"*.log","*.tmp","cache/"}

# Exclude from a file
--exclude-from="/etc/rsync-excludes.txt"

The exclude file format (one pattern per line):

*.log
*.tmp
.git/
node_modules/
__pycache__/
.DS_Store

Order matters when using both --include and --exclude. rsync processes them in order, first match wins. This is the source of much confusion. If you need fine-grained include/exclude logic, test with -n first.

Bandwidth Limiting

On production servers, I don’t want a backup job saturating the network:

rsync -av --bwlimit=10000 source/ dest/

--bwlimit is in KB/s. 10000 = ~10 MB/s. Useful for transfers over shared links or when you want backups to run quietly in the background without impacting traffic.

Syncing Entire Server Configs

I occasionally rsync configs from production to staging for testing:

rsync -av \
  --exclude="/proc/" \
  --exclude="/sys/" \
  --exclude="/dev/" \
  --exclude="/tmp/" \
  --exclude="/run/" \
  --exclude="/mnt/" \
  --exclude="/media/" \
  root@production:/etc/ \
  /staging/etc-backup/

Not a full system clone, but useful for keeping config files in sync.

rsync vs. cp vs. scp

  • cp — fast local copy, no delta sync, no remote
  • scp — encrypted remote copy, transfers everything every time
  • rsync — delta sync, efficient, works locally and remotely

For anything that runs more than once, rsync beats scp significantly. For one-time transfers of small files, it doesn’t matter much.

Common Issues

“Permission denied” on destination — check that the user running rsync has write access to the destination directory, or run with sudo.

Files not deleting with –delete — verify you have the trailing slash on the source. Without it, rsync syncs the directory itself into the destination, which changes the comparison.

SSH key issues — if rsync to remote fails with authentication errors, test the SSH connection directly first: ssh user@remote. Fix SSH issues before debugging rsync.

Symlinks not followed — -a preserves symlinks (doesn’t follow them). To follow symlinks and copy the actual files: add -L.

Large file transfers failing partway — use -P (partial transfers) so the transfer can resume where it left off.

Monitoring Backup Completeness

I check backup logs regularly. My script appends to a log file, and I scan it with a weekly cron:

grep -i "error\|failed\|rsync error" /var/log/rsync-backup.log | tail -20

Also worth using a monitoring tool or setting up alerts so you know immediately if backups start failing rather than discovering weeks later when you need them. Certificate expiry alerting and backup monitoring belong in the same category of things you don’t want to notice by accident — the SSL certificate monitoring guide has a similar philosophy around setting up proactive alerts.

Related Posts

Scroll to Top