Shell Scripts for Deployments: Checklists, Guards and Rollbacks

1. What a deployment script must actually do

A professional deployment script is more than a wrapper around rsync and a web server reload. It has to make sure a deployment only starts once every prerequisite is met. It has to prevent two deployments from writing to the same server at once. It has to leave the target server in a consistent state after the transfer: no partial state, no brief window with an inconsistent file structure. And it has to fall back automatically to a safe state whenever any step fails, without requiring manual intervention.

In many projects these requirements are never spelled out until the first time a deployment goes wrong. That is when it becomes clear whether the deployment script covered these scenarios or not. A deployment that leaves production in a broken state for 30 minutes because the automatic rollback was missing costs far more than the time it takes to build a robust deployment script in the first place. This article walks through the building blocks every production-ready deployment script in Bash should contain.

The architecture of a professional deployment script always follows the same phase structure: pre-deploy (checklists, guards), transfer (atomic file copy), post-deploy (health checks, service reload), verification (HTTP check, log inspection), and rollback if needed. These phases are not optional: every phase you skip is a risk. The deployment script is the gatekeeper between the build artifact and the production server, and that gatekeeper has to verify each of its steps explicitly.

2. Pre-deploy checklists: checking prerequisites systematically

A pre-deploy checklist is a collection of guard conditions checked before the first transfer step. The deployment script aborts if any of these conditions is not met, with a clear error message that immediately tells the operator what is missing. Typical pre-deploy checks: existence and readability of the configuration file, reachability of the target server, available disk space on the target server, validity of the SSH connection, existence of the build artifact, and whether the deploy branch matches the expected branch.

Implementing this as a run_preflight_checks() function is the recommended pattern for deployment scripts: all checks bundled into one dedicated function that runs at the start of the script. Each individual check is a condition with an explicit error message and an exit code that tells monitoring which check failed. The script fails loudly and early, not silently and late, once the transfer is already half done. "Fail fast" is not a stylistic choice for deployment scripts, it is an operational necessity.

#!/usr/bin/env bash
# deploy.sh: Production deployment with preflight checks, guards and rollback
set -euo pipefail
IFS=$'\n\t'

# --- Configuration ---
readonly REMOTE_HOST="${REMOTE_HOST:?Set REMOTE_HOST}"
readonly REMOTE_USER="${REMOTE_USER:-deploy}"
readonly DEPLOY_BASE="${DEPLOY_BASE:-/var/www/releases}"
readonly CURRENT_LINK="${CURRENT_LINK:-/var/www/current}"
readonly BUILD_DIR="${BUILD_DIR:-./dist}"
readonly KEEP_RELEASES="${KEEP_RELEASES:-7}"
readonly HEALTH_URL="${HEALTH_URL:?Set HEALTH_URL for post-deploy check}"
readonly LOCK_FILE="/tmp/deploy-${REMOTE_HOST}.lock"
readonly TS="$(date +%Y%m%d-%H%M%S)"
readonly RELEASE_DIR="${DEPLOY_BASE}/${TS}"

log()  { printf "[%s] [INFO]  %s\n" "$(date +%T)" "$*"; }
warn() { printf "[%s] [WARN]  %s\n" "$(date +%T)" "$*" >&2; }
err()  { printf "[%s] [ERROR] %s\n" "$(date +%T)" "$*" >&2; }

# --- Preflight check suite ---
run_preflight_checks() {
  log "Running preflight checks..."

  # 1. Build artifact exists and is non-empty
  [[ -d "$BUILD_DIR" ]] || { err "Build directory not found: $BUILD_DIR"; exit 10; }
  [[ -n "$(ls -A "$BUILD_DIR")" ]] || { err "Build directory is empty: $BUILD_DIR"; exit 11; }

  # 2. Remote host reachable via SSH
  ssh -o ConnectTimeout=5 -o BatchMode=yes \
    "${REMOTE_USER}@${REMOTE_HOST}" "true" 2>/dev/null \
    || { err "SSH connection failed to ${REMOTE_USER}@${REMOTE_HOST}"; exit 12; }

  # 3. Sufficient disk space on remote (require 500 MB free)
  local free_kb
  free_kb=$(ssh "${REMOTE_USER}@${REMOTE_HOST}" \
    "df --output=avail /var/www | tail -1")
  (( free_kb >= 512000 )) \
    || { err "Insufficient disk space on $REMOTE_HOST: ${free_kb}KB free"; exit 13; }

  # 4. No other deployment in progress (checked by guard, but verify here too)
  ssh "${REMOTE_USER}@${REMOTE_HOST}" \
    "test ! -f /tmp/deploy.lock || exit 14" \
    || { err "Deployment already in progress on $REMOTE_HOST"; exit 14; }

  log "All preflight checks passed."
}

3. Guards: preventing concurrency and double starts

A guard mechanism in the deployment script prevents two deployment processes from writing to the same server at the same time. Without a guard, CI/CD environments with parallel pipelines regularly run into this: one deployment is running, a second push triggers another deployment at the same time, both rsync processes write into the same release directory simultaneously, and the symlink swap runs twice with different target directories. The result is a production server left in an undefined state.

The robust pattern for deployment scripts is a two-tier guard: a local flock-based lock for multiple concurrent runs of the same script on the same machine, and a remote lock on the target server for deployments coming from different sources. The remote lock is a file created at the start of the deployment and removed at the end, inside the EXIT trap. Anyone relying on PID-file patterns here has to implement the cleanup logic very carefully: a crash without cleanup leaves behind an orphaned lock file that blocks every future deployment. flock, on the other hand, hands the lock off to the operating system automatically, the lock file is released automatically when the process ends.

#!/usr/bin/env bash
# guards.sh: Local flock guard + remote lock for deployment serialization
set -euo pipefail

readonly LOCAL_LOCK="/tmp/deploy-local.lock"
readonly REMOTE_LOCK="/tmp/deploy-${REMOTE_HOST}.lock"
readonly REMOTE_USER="${REMOTE_USER:-deploy}"
readonly REMOTE_HOST="${REMOTE_HOST:?}"

REMOTE_LOCK_ACQUIRED=0

cleanup() {
  local code=$?
  # Always release remote lock if we acquired it
  if [[ $REMOTE_LOCK_ACQUIRED -eq 1 ]]; then
    ssh "${REMOTE_USER}@${REMOTE_HOST}" "rm -f '$REMOTE_LOCK'" 2>/dev/null || true
    log "Remote lock released"
  fi
  [[ $code -eq 0 ]] && log "Deployment completed successfully" \
                     || err "Deployment failed with exit code $code"
}
trap cleanup EXIT

# --- Local guard: prevent parallel deployments from same machine ---
exec 9>"$LOCAL_LOCK"
flock -n 9 || { err "Another deployment is already running locally"; exit 13; }
log "Local lock acquired"

# --- Remote guard: prevent deployments from different machines ---
ssh "${REMOTE_USER}@${REMOTE_HOST}" "
  if [ -f '$REMOTE_LOCK' ]; then
    echo 'LOCKED' >&2
    exit 1
  fi
  touch '$REMOTE_LOCK'
" || { err "Deployment already in progress on $REMOTE_HOST (remote lock exists)"; exit 14; }

REMOTE_LOCK_ACQUIRED=1
log "Remote lock acquired on $REMOTE_HOST"

4. Atomic deployment: symlink swaps and version history

An atomic deployment script makes sure the web server never points at an incomplete or inconsistent file state, at any point in time. The pattern: every deployment gets its own timestamped release directory. rsync transfers all files into this new directory. Only after the transfer is fully complete and verified does a symbolic link get repointed at the new directory. This symlink swap is a single syscall (rename(2) under the hood of ln -sfn) and is therefore atomic, the web server never sees an intermediate state.

Version history is a byproduct of this pattern: every earlier release directory stays intact until rotation deletes it. That makes rollbacks trivial: just point the symlink at the desired older release directory. No new deployment, no waiting for a transfer. A rollback takes as long as a single ln -sfn call. That is the defining trait of a professional deployment script: rollbacks are not an emergency procedure, they are a prepared, tested action that runs in seconds.

5. Post-deploy health checks: verifying success

After the symlink swap, the deployment script verifies that the production server actually responds with the new version and returns no error output. The simplest health check: an HTTP request against the defined health endpoint using curl --silent --max-time 10 --fail. The --fail flag makes curl exit with a non-zero code when the HTTP status is 4xx or 5xx. That combines a reachability check and a correctness check for the response format into a single command.

For Magento deployments and more complex applications, a simple HTTP 200 check is not enough. The deployment script should implement several layers of health checks: HTTP status for basic reachability, content inspection for specific markers (for example a version number in the response, or the absence of PHP error pages), and log inspection for errors that surface shortly after deployment. A health check that only runs 60 seconds after the symlink swap gives application caches time to warm up and catches errors that only appear once the first real requests come in.

6. Rollback mechanisms: automatic and manual

The deployment script implements two rollback variants: automatic on a failed health check, and manual via a separate rollback script. The automatic rollback lives in the EXIT trap: if the health check fails and the script exits with an error code, the EXIT trap points the symlink back at the previous release directory. That requires remembering the previous release directory at the start of the deployment: PREVIOUS_RELEASE="$(readlink -f "$CURRENT_LINK")".

The manual rollback is a separate deployment script (rollback.sh) that either takes the desired release directory as a parameter or lets you choose interactively from the available releases. This script performs the same symlink swap and the same health checks as the deployment itself. The single most important property of a rollback script: it is the script you run least often, yet the one with the highest reliability requirements. It has to work reliably in the stressful moment of a production incident. That calls for regular testing, not just during development but also as a planned exercise on the production environment.

#!/usr/bin/env bash
# rollback.sh: Automated and manual rollback with health check verification
set -euo pipefail
IFS=$'\n\t'

readonly REMOTE_HOST="${REMOTE_HOST:?}"
readonly REMOTE_USER="${REMOTE_USER:-deploy}"
readonly DEPLOY_BASE="${DEPLOY_BASE:-/var/www/releases}"
readonly CURRENT_LINK="${CURRENT_LINK:-/var/www/current}"
readonly HEALTH_URL="${HEALTH_URL:?Set HEALTH_URL}"

log() { printf "[%s] [INFO]  %s\n" "$(date +%T)" "$*"; }
err() { printf "[%s] [ERROR] %s\n" "$(date +%T)" "$*" >&2; }

list_releases() {
  ssh "${REMOTE_USER}@${REMOTE_HOST}" \
    "ls -1dt '${DEPLOY_BASE}'/*/ 2>/dev/null | head -10"
}

get_current_release() {
  ssh "${REMOTE_USER}@${REMOTE_HOST}" \
    "readlink -f '$CURRENT_LINK' 2>/dev/null || echo 'none'"
}

perform_rollback() {
  local target_release="$1"

  log "Rolling back to: $target_release"
  log "Current release: $(get_current_release)"

  # Validate target release exists on remote
  ssh "${REMOTE_USER}@${REMOTE_HOST}" \
    "test -d '$target_release'" \
    || { err "Release directory not found: $target_release"; exit 1; }

  # Atomic symlink swap
  ssh "${REMOTE_USER}@${REMOTE_HOST}" \
    "ln -sfn '$target_release' '$CURRENT_LINK' && echo 'Symlink updated'"

  # Post-rollback health check
  log "Verifying rollback via health check..."
  local attempt
  for attempt in 1 2 3; do
    if curl --silent --max-time 10 --fail "$HEALTH_URL" > /dev/null 2>&1; then
      log "Health check passed after rollback (attempt $attempt)"
      return 0
    fi
    warn "Health check attempt $attempt failed, retrying in 5s..."
    sleep 5
  done

  err "Health check failed after rollback, MANUAL INTERVENTION REQUIRED"
  err "Current symlink points to: $(get_current_release)"
  exit 1
}

# --- Main: use provided release or let user choose ---
if [[ $# -eq 1 ]]; then
  TARGET_RELEASE="$1"
else
  log "Available releases:"
  list_releases
  read -rp "Enter release path to roll back to: " TARGET_RELEASE
fi

perform_rollback "$TARGET_RELEASE"
log "Rollback complete. Active release: $(get_current_release)"

7. Notifications and deployment logs

A professional deployment script logs every step and sends notifications on both success and failure. The minimum log content: start and end of the deployment with timestamps, each step with its duration, and every error message with exit code and line number. The log is written to a dated log file and printed to the console at the same time. The pattern exec > >(tee -a "$DEPLOY_LOG") 2>&1 at the top of the script sends every output stream, stdout and stderr alike, to both the terminal and the log file simultaneously.

Notifications via webhook (Slack, Teams, Mattermost) or email keep the team informed without anyone needing to watch logs. The deployment script sends a notification at the end of the EXIT trap containing: deployment status (success, failure, or rollback), release directory, deployment duration, and, on failure, the last error message from the log. A simple webhook notification via curl -X POST with a JSON body takes only a few lines and is considerably more reliable than email. Putting the notification logic in the EXIT trap ensures a message is sent even if the deployment aborts.

8. CI/CD integration: deployment scripts in pipelines

The deployment script inside a CI/CD pipeline is not an autonomous program, it is a step in a larger workflow. It receives its configuration through environment variables (secrets for the SSH key and remote host, the build artifact path, the health URL), runs its sequence, and reports success or failure back to the pipeline via exit codes. The pipeline then decides, based on that exit code, whether subsequent steps run or the whole workflow is marked as failed.

The recommended pipeline structure for a deployment script: build step, test step, a ShellCheck step on the deployment script itself, staging deployment step, staging health check step, a manual approval step (for production deployments), production deployment step, production health check step. This pattern ensures a broken deployment script is caught on staging first, before it ever reaches production. And the manual approval step gives the team control over exactly when critical deployments go live.

9. Deployment strategies compared

Different deployment script strategies come with different trade-offs in downtime, rollback capability and complexity. The right choice depends on your requirements for availability and rollback speed.

For most single-server web deployments, the atomic symlink swap is the optimal strategy: no downtime, rollback in seconds, moderate implementation effort. A deployment script using rsync with --link-dest plus a symlink swap covers this strategy completely. For multi-server setups behind a load balancer, blue-green is the next sensible step up, with the load balancer swap taking over as the atomic element that replaces the symlink swap.

10. Summary

A professional deployment script in Bash consists of five essential building blocks. Pre-deploy checklists verify every prerequisite before a single byte is transferred to the target server. Guard mechanisms prevent parallel deployments through local flock and remote locks. Atomic deployment via rsync with --link-dest and a symlink swap ensures the web server never sees an inconsistent state. Post-deploy health checks verify that the new version actually responds correctly. Automatic rollback in the EXIT trap immediately returns to the last good release if the health check fails.

The most important quality trait of a deployment script: it is tested, not just once manually during development, but regularly and automatically. ShellCheck in the CI pipeline verifies static correctness. BATS tests verify the preflight checks and the rollback logic. And occasional rollback drills on the production environment confirm that the rollback mechanism actually works when it is needed, not for the first time during a real incident.

11. FAQ: Shell Scripts for Deployments