rsync Footguns¶
Mistakes that corrupt syncs, delete production data, or waste hours debugging. Every one of these has happened in production.
1. Trailing Slash Confusion (src/ vs src)¶
You run rsync -av /opt/myapp /backups/ expecting to back up the app directory contents. Instead, rsync creates /backups/myapp/ with everything nested one level deeper than expected. Your restore script points at /backups/ and finds nothing where it expects files. Or worse, you run rsync -av --delete /opt/myapp /var/www/ and it deletes everything in /var/www/ that is not the myapp directory.
The rule: a trailing slash on the source means "the contents of this directory." No trailing slash means "this directory itself."
# These produce DIFFERENT results:
rsync -av /opt/myapp/ /backups/ # /backups/file1, /backups/file2
rsync -av /opt/myapp /backups/ # /backups/myapp/file1, /backups/myapp/file2
Fix: Always be explicit about trailing slashes. When in doubt, add --dry-run and check the file list. Adopt a team convention (e.g., "always use trailing slashes on both source and dest") and enforce it in scripts.
2. --delete Without --dry-run First¶
You add --delete to a sync command to mirror source to destination. But your source path is wrong -- maybe it points to an empty directory, or a parent directory you did not intend. rsync obediently deletes everything on the destination that is not in the (empty or wrong) source. Terabytes of data gone.
# Disaster: SOURCE is empty or wrong, wipes destination
rsync -av --delete /mnt/empty_by_mistake/ /production-data/
Fix: Always run --delete with --dry-run first. Make it a muscle-memory sequence:
# Step 1: ALWAYS
rsync -avn --delete source/ dest/
# Step 2: Read the output. Look for "deleting" lines.
# Step 3: Only if the dry run looks correct:
rsync -av --delete source/ dest/
For critical paths, add --max-delete=100 as a safety net. rsync will abort if it would delete more than 100 files.
3. Forgetting -a (No Permissions/Ownership Preserved)¶
You sync an application directory with rsync -r source/ dest/ instead of rsync -a source/ dest/. The files arrive but with wrong ownership (everything owned by root or the rsync user), wrong permissions (maybe world-readable secrets), and no symlinks preserved. The application fails to start because it cannot read its own config files.
# Bad: recursive only, no metadata
rsync -rv source/ dest/
# Permissions: -rw-r--r-- root root (wrong)
# Expected: -rw-r----- appuser appgroup
# Good: archive preserves everything
rsync -av source/ dest/
Fix: Use -a (archive mode) as your default. It includes -rlptgoD. Add -H if you need hard links, -A for ACLs, -X for extended attributes. Never use bare -r unless you specifically want to strip metadata.
4. --exclude Order Matters (First Match Wins)¶
You want to sync only Python files but exclude __pycache__. You write:
# BROKEN: excludes everything, including .py files
rsync -av --exclude='*' --include='*.py' source/ dest/
rsync evaluates rules in command-line order. --exclude='*' matches first, so everything is excluded. The include rule never fires.
# CORRECT: include rules must come BEFORE the catch-all exclude
rsync -av \
--include='*/' \
--include='*.py' \
--exclude='*' \
source/ dest/
Note the --include='*/' -- without it, rsync will not descend into subdirectories (they are excluded by *) and will miss nested Python files.
Fix: Always put include rules before exclude rules. Test complex filter sets with --dry-run. For complicated patterns, use --filter with a file:
5. No --partial Over Slow Links (Restart from Scratch on Failure)¶
You are transferring a 50 GB database dump over a VPN link. At 47 GB, the connection drops. Without --partial, rsync deletes the 47 GB partial file and starts over from zero. This cycle repeats until you rage-quit.
# Bad: network drop at 90% means starting over
rsync -avz bigfile.tar.gz remote:/backups/
# Good: resume from where it left off
rsync -avzP bigfile.tar.gz remote:/backups/
# -P = --partial --progress
Fix: Always use -P (or --partial --progress) for large file transfers over unreliable links. For cleaner destinations, use --partial-dir=.rsync-tmp to keep partial files in a hidden directory.
6. -z on Already-Compressed Data (Wasted CPU)¶
You add -z to every rsync command by habit. When syncing directories full of .jpg, .mp4, .gz, or .zip files, rsync burns CPU compressing data that will not compress. On fast networks, this actually slows down the transfer.
# Wasteful: compressing JPEGs achieves nothing
rsync -avz /photos/ remote:/photos/
# Better: skip compression for already-compressed formats
rsync -av --compress --skip-compress=gz/jpg/mp4/zip/bz2/xz/png/rar/7z \
/mixed-content/ remote:/mixed-content/
Fix: Use -z only when the data is compressible (text, logs, code, uncompressed databases) and the link is slow enough that CPU time is cheaper than bandwidth. On a 10 Gbps LAN, -z is almost always counterproductive. On modern rsync (3.2+), --compress-choice=zstd is faster than the default zlib.
7. Permission Denied Mid-Sync Leaving Partial State¶
rsync encounters a file it cannot read (wrong permissions, mandatory access control, encrypted filesystem). It logs an error and continues. The sync "completes" with exit code 23 (partial transfer). Your monitoring shows green because you only check for exit code 0 vs non-zero. The destination is missing files, and nobody notices until a restore fails.
# Sync completes with partial transfer -- some files skipped
rsync -av /data/ /backup/
# rsync: send_files failed to open "/data/secrets.db": Permission denied (13)
# rsync error: some files/attrs were not transferred (code 23)
Fix: Always check rsync's exit code and handle code 23 (partial transfer) and code 24 (vanished files) explicitly:
rsync -av /data/ /backup/
RC=$?
case $RC in
0) echo "OK" ;;
23) echo "WARNING: partial transfer -- some files not synced" >&2 ;;
24) echo "WARNING: some source files vanished during sync" >&2 ;;
*) echo "ERROR: rsync failed with code $RC" >&2; exit 1 ;;
esac
Before syncing, verify you can read all source files: find /data/ ! -readable -ls
8. Symlink Handling Surprises (--copy-links vs -l)¶
With -a (which includes -l), rsync copies symlinks as symlinks. If the symlink target is outside the transfer tree, the destination gets a dangling symlink. Your application follows the symlink and gets "No such file or directory."
# Source has: /app/config -> /etc/myapp/config.yml
# rsync -av /app/ dest/
# Destination gets: dest/config -> /etc/myapp/config.yml (dangling if /etc/myapp/ doesn't exist)
With --copy-links (-L), rsync follows symlinks and copies the actual file content. But now if you have recursive symlinks, rsync loops. And if you have symlinks to large files outside the tree, you get unexpected data.
# Dereference symlinks -- copies file content instead of the link
rsync -avL /app/ dest/
# Destination gets: dest/config (actual file, not a symlink)
# But watch out for symlink loops or symlinks to huge files outside the tree
Fix: Know your symlink landscape before syncing. Use find /source -type l -ls to audit symlinks. Choose deliberately:
-l(default in-a): preserve symlinks. Correct when source and dest have the same filesystem structure.-L/--copy-links: dereference. Correct when you need self-contained copies.--safe-links: ignore symlinks that point outside the tree. Good defensive default.--copy-unsafe-links: dereference only symlinks pointing outside the tree.
9. --delete Removing Files That Are in --exclude List¶
You exclude logs/ from the sync and use --delete to mirror. You expect logs on the destination to be left alone. But --delete removes files from the destination that are not in the transfer set -- and excluded files are not in the transfer set. So --delete wipes the destination's logs/ directory.
# Surprise: --delete removes dest/logs/ because it's excluded from the transfer
rsync -av --delete --exclude='logs/' source/ dest/
# The logs/ directory on dest is deleted because rsync sees it as
# "extraneous" (not in the source file list, which excludes logs/)
Wait -- this is actually a common misconception. rsync's --delete does NOT delete excluded files by default. Excluded files are protected from deletion. The actual footgun is the reverse: using --delete-excluded when you do not mean to.
# SAFE: --delete preserves excluded files on destination
rsync -av --delete --exclude='logs/' source/ dest/
# dest/logs/ is preserved
# DANGEROUS: --delete-excluded actively removes excluded files on dest
rsync -av --delete --delete-excluded --exclude='logs/' source/ dest/
# dest/logs/ is DELETED
Fix: Understand the distinction. --delete removes files that are on the destination but not on the source -- but excluded files are invisible to both sides and thus protected. --delete-excluded explicitly opts into removing excluded files on the destination. Only use --delete-excluded when you truly want to clean up files matching exclude patterns on the destination (e.g., removing build artifacts everywhere).
10. Not Using --checksum When Timestamps Are Unreliable¶
You sync files between NFS mounts, or between Linux and a FAT-formatted USB drive. Timestamps have different granularity or get mangled. rsync compares mtime+size, sees "no change," and skips files that have actually changed. Your backup is silently incomplete.
FAT timestamps have 2-second resolution. NFS can have clock skew. git checkout sets mtimes to "now," not the original commit time. Build tools may regenerate identical files with new timestamps (opposite problem: unnecessary transfers).
# Unreliable: timestamps on FAT/NFS may not reflect actual changes
rsync -av /nfs-mount/data/ /local-backup/
# Reliable: compare by checksum regardless of timestamps
rsync -avc /nfs-mount/data/ /local-backup/
Fix: Use --checksum (-c) when dealing with FAT/exFAT filesystems, NFS mounts, or any situation where timestamps are unreliable. Accept the performance cost (rsync reads every file to compute checksums). For large datasets where --checksum is too slow, consider --size-only as a compromise -- it skips timestamp comparison and transfers only if size differs.
11. Running as the Wrong User (UID Mapping Issues)¶
You run rsync as root to sync files to a remote server. The files on the source are owned by appuser (UID 1000). On the destination, UID 1000 maps to a different user (or no user). rsync faithfully preserves UID 1000, but now the files are owned by the wrong person. Or the application runs as a different UID on the destination and cannot read its own files.
# On source: appuser is UID 1000
# On dest: UID 1000 is "testuser" or doesn't exist
rsync -av /app/ remote:/app/
# Result: files owned by UID 1000 on dest, which is the wrong user
Fix: Options depending on the situation:
# Option 1: Use --chown to remap ownership on the destination
rsync -av --chown=destuser:destgroup /app/ remote:/app/
# Option 2: Use --numeric-ids to preserve UID/GID numbers
# (correct when UIDs match across systems)
rsync -av --numeric-ids /app/ remote:/app/
# Option 3: Use --no-owner --no-group to skip ownership preservation
# (files will be owned by the rsync user on dest)
rsync -av --no-o --no-g /app/ remote:/app/
Verify UID mapping before syncing: ssh remote id appuser
12. Cron rsync Without File Locking (Overlapping Syncs)¶
Your cron job runs rsync every 15 minutes. One sync takes 20 minutes because of a large batch of new files. Now two rsync processes are writing to the same destination. At best, you get wasted bandwidth. At worst, one rsync deletes files that the other is actively writing, and you end up with a corrupted partial sync.
# BAD: no locking, overlapping runs will fight
*/15 * * * * rsync -av --delete /data/ /backups/data/
# GOOD: flock prevents overlapping runs
*/15 * * * * flock -n /var/run/backup-sync.lock \
rsync -av --delete /data/ /backups/data/
Fix: Always wrap cron rsync in flock -n (non-blocking). The -n flag means "if the lock is held, exit immediately" rather than waiting. Combine with timeout protection:
*/15 * * * * flock -n /var/run/backup.lock \
timeout 1800 rsync -av --delete /data/ /backups/data/ \
>> /var/log/backup-sync.log 2>&1
13. Using --delete with the Wrong Source Path Variable¶
Your backup script uses a variable for the source path. The variable is unset or empty due to a bug. rsync resolves this to the current working directory or the root directory. Combined with --delete, it either mirrors junk to the destination or (if it resolves to empty) deletes everything on the destination.
#!/bin/bash
# BUG: if SOURCE is unset, this becomes: rsync -av --delete / /backups/
rsync -av --delete "${SOURCE}/" /backups/data/
Fix: Use set -euo pipefail and explicitly validate variables before rsync:
#!/bin/bash
set -euo pipefail
SOURCE="${SOURCE:?ERROR: SOURCE variable is not set}"
if [ ! -d "$SOURCE" ]; then
echo "ERROR: SOURCE directory does not exist: ${SOURCE}" >&2
exit 1
fi
# Extra safety: refuse to sync if source has fewer files than expected
FILE_COUNT=$(find "$SOURCE" -type f | wc -l)
if [ "$FILE_COUNT" -lt 10 ]; then
echo "ERROR: SOURCE has only ${FILE_COUNT} files, expected more. Aborting." >&2
exit 1
fi
rsync -av --delete "${SOURCE}/" /backups/data/
14. Ignoring Exit Code 23 in Monitoring¶
rsync exits with code 23 when some files could not be transferred (permission denied, vanished files, I/O errors). Many monitoring setups treat only exit code 0 as success and everything else as failure. But some treat "non-zero" as a single failure category without distinguishing between "a few temp files vanished" (code 24, usually harmless) and "disk is full" (code 11, critical).
Fix: Handle rsync exit codes with granularity in your monitoring:
rsync -av /data/ /backup/
RC=$?
case $RC in
0) STATE="OK" ;;
24) STATE="WARNING" ;; # vanished source files (usually fine)
23) STATE="WARNING" ;; # partial transfer (investigate)
*) STATE="CRITICAL" ;; # actual failure
esac
echo "rsync_backup_status ${STATE} exit_code=${RC}"
Do not silently swallow code 23. It means some files were not backed up. Investigate which files and why.