Using rsync Correctly: Deployments, Mirroring, Excludes and Dry Runs

1. Understanding rsync: Delta Transfer and Checksums

rsync does not simply transfer files, it computes deltas. The rsync protocol splits files into blocks, computes checksums for each block, and transfers only the blocks that have changed. For large files that have only been partially modified, that is the decisive performance advantage over scp or cp. Anyone who wants to use rsync correctly first needs to understand when rsync applies this delta algorithm and when it falls back to the faster metadata comparison.

By default, rsync compares files by size and modification time. If both match, the file is considered identical, without comparing the actual content. That is sufficient for most deployments and noticeably faster than a full checksum comparison. For critical scenarios, such as backup verification or migration after a file server crash, plain metadata comparison is not reliable enough. The --checksum flag then forces a full content comparison. Anyone who wants to use rsync correctly chooses the appropriate comparison strategy deliberately for each use case.

The protocol also has an important quirk with path arguments: a trailing slash on the source path changes the semantics significantly. rsync src/ dst/ copies the contents of src into dst, without creating an additional src directory. rsync src dst/, on the other hand, creates dst/src/. This difference is one of the most common mistakes when trying to use rsync correctly for deployments: an accidentally missing slash changes the entire directory structure on the target.

2. The Most Important rsync Flags for Deployments

The combination rsync -avz --progress is the most commonly used rsync invocation and is sufficient for many cases. The -a flag (archive mode) enables recursive copying and preserves symlinks, permissions, timestamps, owners and groups, it is shorthand for -rlptgoD. The -v flag prints every transferred file. The -z flag compresses the transfer, useful over slow networks, but often counterproductive over local networks or fast links, since the CPU load can exceed the network speed.

For deployments to web servers, --no-perms --no-owner --no-group is often important: the web server on the target runs under a different user than the deploy process, and rsync should not transfer permissions that the target server user then cannot read. The -u flag (update mode) skips files on the target that are newer than the source, useful in situations where the target server has modified files locally (for example uploaded user content). When you use rsync correctly in a deployment context, --delete and an --exclude-from file are almost always added as well.

#!/usr/bin/env bash
# deploy-rsync.sh - Production deployment via rsync with full error handling
set -euo pipefail
IFS=$'\n\t'

readonly REMOTE_HOST="${REMOTE_HOST:?Set REMOTE_HOST}"
readonly REMOTE_USER="${REMOTE_USER:-deploy}"
readonly REMOTE_PATH="${REMOTE_PATH:?Set REMOTE_PATH (e.g. /var/www/myapp)}"
readonly LOCAL_SOURCE="${LOCAL_SOURCE:-./dist/}"
readonly EXCLUDE_FILE="${EXCLUDE_FILE:-./rsync-excludes.txt}"
readonly SSH_KEY="${SSH_KEY:-$HOME/.ssh/id_ed25519}"

# Validate source directory exists and has trailing slash
[[ -d "$LOCAL_SOURCE" ]] || { echo "[ERROR] Source directory not found: $LOCAL_SOURCE" >&2; exit 1; }

rsync \
  --archive \
  --verbose \
  --compress \
  --delete \
  --exclude-from="$EXCLUDE_FILE" \
  --checksum \
  --human-readable \
  --progress \
  --stats \
  --rsh="ssh -i $SSH_KEY -o StrictHostKeyChecking=yes" \
  "$LOCAL_SOURCE" \
  "${REMOTE_USER}@${REMOTE_HOST}:${REMOTE_PATH}/"

echo "[OK] Deployment complete: $LOCAL_SOURCE -> ${REMOTE_USER}@${REMOTE_HOST}:${REMOTE_PATH}"

3. --delete: Exact Mirrors Without Orphaned Files

The --delete flag is the essential tool for using rsync correctly as a true mirror. Without --delete, files on the target that no longer exist in the source are never removed. The target grows monotonically: deleted configuration files, old asset versions, and removed PHP files stay behind on the production server. With --delete, the target is an exact copy of the source after the transfer.

The --delete-after variant removes orphaned files only after all new files have been transferred, important for web servers where a brief state of missing files would cause 404 errors. An even safer variant is --delete-delay, which collects deleted files in a list and removes them all at once at the end of the transfer. When you use rsync correctly in high availability setups, the order of transfer and deletion is not a minor detail, it is business critical. The common pattern is --delete-after combined with a server side reload hook that only fires after the transfer has fully completed.

Be careful when combining --delete with --filter and --exclude: excluded files on the target are also deleted by --delete unless they are explicitly protected with --filter "protect ...". In practice this regularly leads to unwanted deletion of upload directories, local configuration files, or cache files that are meant to exist on the target but are not present in the source. This is why a full dry run before the first production run is mandatory whenever you use rsync correctly with --delete.

4. --exclude-from: Clean Exclusions for Deployments

The --exclude-from=file.txt option reads exclusion patterns from a file, one pattern per line. That is the scalable alternative to multiple --exclude flags on the command line, which quickly become unwieldy as projects grow. A central rsync-excludes.txt can be kept under version control, used consistently by the whole team, and included in CI/CD pipelines. Anyone who wants to use rsync correctly in a team setting always centralizes exclusion patterns in a file.

The syntax in the exclude file supports wildcards (* for any characters except /, ** for paths across directory boundaries), anchors (/ at the start for absolute patterns relative to the source, / at the end for directories), comments (#), and negation (! for include exceptions inside an exclude rule). The processing model matters: rsync checks every file against the rules in the order they appear in the file, the first matching rule wins. Exceptions therefore need to come before the broader exclusion rules.

# rsync-excludes.txt - Deployment exclude patterns for rsync --exclude-from
# Syntax: one pattern per line, # for comments, / prefix = anchored to root, ! = include exception

# --- Version control ---
/.git/
/.gitignore
/.gitattributes

# --- Development tools ---
/node_modules/
/.npm/
/vendor/           # Composer vendor - deploy pre-built assets, not sources

# --- Build artifacts to INCLUDE (exception before broader rule) ---
!/dist/
!/build/

# --- Local config files (kept on server, not overwritten) ---
/config/local.php
/config/env.php
/.env
/.env.local

# --- Upload directories - protect from --delete ---
/pub/media/
/var/

# --- Logs, cache, temp ---
/var/cache/
/var/log/
*.log
*.tmp
*.swp
.DS_Store
Thumbs.db

5. --dry-run: Simulating Deployments Safely Beforehand

The --dry-run flag (short form: -n) is the single most important safety net when you use rsync correctly, before ever running a new deployment script in production. A dry run performs every rsync calculation, checksums, exclude matching, delete detection, and prints out what would be transferred and deleted, without changing a single file. The output of a dry run with --verbose shows exactly which files would be newly transferred, which would be overwritten, and which would be deleted on the target.

A professional deployment script implements rsync correctly with a built in dry run mode: DRY_RUN="${DRY_RUN:-1}" as the default, meaning the dry run is active by default, and the actual transfer must be explicitly enabled with DRY_RUN=0. That prevents a new or changed script from accidentally reaching production without a prior simulation. In CI/CD pipelines the dry run runs as a separate pipeline step, its output is stored as an artifact, and it is either approved manually or checked automatically for critical deletion patterns before the actual deploy step runs.

6. --checksum: Integrity Instead of Metadata Comparison

The --checksum flag fundamentally changes the comparison strategy: rsync computes MD4 checksums for every file and only transfers files whose checksum differs, regardless of size and timestamp. That is noticeably slower than the default metadata comparison, but it is the only reliable way to detect silent file corruption. Beyond using rsync correctly for regular deployments, --checksum is the right mode for backup verification, migrations after hardware failures, and the first synchronization after a longer interruption.

In practice, --checksum is rarely used for every deployment, because the increased CPU and I/O overhead on both sides can multiply transfer time for large file sets. The sensible pattern when you use rsync correctly: regular deployments without --checksum, but periodic verification runs (for example weekly) with --checksum --dry-run, which log divergent files without transferring anything. This verification run also works between two local directories and is therefore ideal for RAID consistency checks.

7. --link-dest: Atomic Deployments with Snapshot History

The --link-dest=directory flag is the tool for atomic deployments with version history, a pattern made possible by learning to use rsync correctly, without needing additional tools like Capistrano or Deployer. rsync creates a new deployment directory and creates hard links for files that have not changed compared to the referenced previous release directory. Only new or changed files are transferred. The result: a complete deployment directory that only needs disk space for the files that actually changed.

The atomic switchover then happens with a single symlink swap: ln -sfn /deployments/release-20260509 /var/www/current. This symlink swap is atomic on Unix file systems, the web server never sees an inconsistent state at any point. Rollbacks are trivial: point the symlink back at an earlier release directory. When you use rsync correctly with --link-dest, the cleanup logic for old release directories is part of the deployment script, typically the last 5 to 10 releases are kept and older ones are deleted.

#!/usr/bin/env bash
# atomic-deploy.sh - Atomic deployment with rsync --link-dest and symlink swap
set -euo pipefail
IFS=$'\n\t'

readonly REMOTE_HOST="${REMOTE_HOST:?}"
readonly REMOTE_USER="${REMOTE_USER:-deploy}"
readonly DEPLOY_BASE="/var/www/releases"
readonly CURRENT_LINK="/var/www/current"
readonly KEEP_RELEASES=5

RELEASE_DIR="${DEPLOY_BASE}/$(date +%Y%m%d-%H%M%S)"
PREVIOUS_LINK="${CURRENT_LINK}"

log() { printf "[%s] %s\n" "$(date +%T)" "$*"; }

log "Starting atomic deployment -> $RELEASE_DIR"

# Step 1: Transfer to new release directory, hardlink unchanged files from current
rsync \
  --archive \
  --compress \
  --delete \
  --exclude-from=./rsync-excludes.txt \
  --link-dest="${PREVIOUS_LINK}" \
  --rsh="ssh -o StrictHostKeyChecking=yes" \
  ./dist/ \
  "${REMOTE_USER}@${REMOTE_HOST}:${RELEASE_DIR}/"

log "Transfer complete. Swapping symlink..."

# Step 2: Atomic symlink swap (single syscall, no downtime window)
ssh "${REMOTE_USER}@${REMOTE_HOST}" \
  "ln -sfn '${RELEASE_DIR}' '${CURRENT_LINK}' && echo 'Symlink: ${CURRENT_LINK} -> ${RELEASE_DIR}'"

# Step 3: Prune old releases - keep only KEEP_RELEASES most recent
ssh "${REMOTE_USER}@${REMOTE_HOST}" "
  ls -1dt '${DEPLOY_BASE}'/* | tail -n +$((KEEP_RELEASES + 1)) | xargs -r rm -rf
  echo 'Pruned old releases, kept last $KEEP_RELEASES'
"

log "Deployment complete."

8. --bwlimit: Bandwidth Limiting for Production Servers

The --bwlimit=KBPS flag caps the transfer rate of rsync at the given value in kilobytes per second. When you use rsync correctly on production servers, this flag is indispensable for deployments during business hours: an uncontrolled rsync transfer can consume all available network bandwidth and interfere with concurrent HTTP requests, database operations, or other production traffic. With --bwlimit=10240 (10 MB/s), enough bandwidth remains for ongoing operations.

The bandwidth limit is not an exact ceiling but an average over short time intervals, rsync regulates the transfer in bursts. For nightly backups and synchronizations outside peak hours, --bwlimit can be omitted to make use of maximum transfer speed. The professional pattern when you use rsync correctly in production environments: make the bandwidth limit configurable as an environment variable and set it dynamically in the deployment script depending on the time window (night versus day, weekday versus weekend). That way the same script can be optimally configured for different contexts.

In addition to --bwlimit, the --timeout=60 option is worthwhile, it terminates rsync if no data has been transferred for 60 seconds. Without a timeout, rsync hangs indefinitely on network problems, which leads to zombie processes in cron jobs that block each other. The deployment script should additionally have an external timeout guard using the timeout builtin: timeout 3600 rsync ... caps the entire transfer at one hour, independent of rsync's own timeout logic.

9. rsync Flags Compared

Choosing the right rsync flags depends heavily on the use case. This overview shows the most important options for using rsync correctly across different scenarios.

For a complete production deployment, these flags are combined: --archive --compress --delete-after --exclude-from --link-dest --bwlimit --timeout --stats. The result is a deployment that mirrors the target server exactly, controls bandwidth, enables an atomic switchover, and produces detailed statistics for the deployment log. Learning to use rsync correctly in practice means combining these flags to fit the situation, not using all of them for every use case.

10. Summary

Learning to use rsync correctly means combining the right flags for the given use case. --delete turns rsync into a true mirror that removes orphaned files on the target, always with a dry run and a well thought out --exclude-from file to prevent unwanted deletions. --checksum enforces full integrity checking for critical transfers. --link-dest enables atomic deployments with zero downtime and version history for rollbacks. --bwlimit protects production operations from bandwidth bottlenecks during ongoing deployments.

The single most important practical rule when you use rsync correctly: test every new deployment script with --dry-run against the production environment before going live. The combination of an atomic --link-dest deployment, a maintained rsync-excludes.txt under version control, and a dry run gate in the CI/CD pipeline is the complete approach to professional rsync deployments.

11. FAQ: Using rsync Correctly