Zero-Downtime and Database Backups: When to Back Up Before a Release

1. Why a Symlink Rollback Alone Is Not Enough

The symlink rollback in Magento deployments is fast and safe at the code level. It switches the current symlink back to an older release directory, instantly restoring the old code. What it cannot do is undo database changes. If setup:upgrade in the new release altered tables, added columns, or written configuration data, those changes still exist in the database, even after a code rollback.

The result of a symlink rollback without a DB rollback after schema-changing migrations: the old code runs against the new database schema. Depending on the type of migration, this can be harmless (additive columns that the old code simply ignores) or critical (renamed columns or changed types that the old code still tries to access). Without a current database backup, there is no way to restore a consistent state made up of old code and an old database.

The backup job in the GitLab pipeline is therefore not an optional nice to have but an explicit safety step for every release that includes database migrations. It must run before the migration step, not after. A backup created after a failed migration preserves the broken state, not the clean starting point.

2. When a Backup Before Release Is Mandatory

A database backup before release is mandatory whenever the release meets one or more of the following conditions: it contains database migrations (Setup/Patch/Schema files in custom modules or Magento core updates). It contains setup:upgrade calls that change the database schema. It contains data transformations that modify or delete existing records. It contains configuration changes that create or modify entries in the core_config_data table.

A backup is optional but recommended for pure code releases with no database interaction. Even here, a backup can make sense if the release is deployed to a system without reliable automated backup infrastructure. The underlying principle holds: no deploy to production without a known, verified backup point in time. The backup does not necessarily have to be created at the moment of deployment; a verifiable backup from the last few hours can be sufficient for pure code releases.

3. Backup Methods: mysqldump vs. Percona XtraBackup

mysqldump is the standard tool for MySQL backups in Magento deployments. It produces a SQL dump file containing all tables, data, and indexes. The downside: for large Magento databases (several gigabytes), mysqldump can take several minutes and generate increased database load during that time. With the --single-transaction flag, the dump runs inside a single transaction, which guarantees consistent data without table locks, provided that all tables use InnoDB.

Percona XtraBackup creates hot backups without interrupting normal database operations and is significantly faster for large production databases. For Magento setups with databases under 5 GB, mysqldump --single-transaction is usually sufficient and easier to configure. For larger databases or high-availability setups, XtraBackup is the more professional choice. The decision depends on database size and the acceptable backup duration within the deployment window.

# Database backup job: runs before any deployment with DB migrations
backup:database:
  stage: deploy
  variables:
    # Backup retention: keep 7 release backups
    BACKUP_DIR: "/var/backups/magento/releases"
    BACKUP_RETENTION: "7"
  script:
    - eval $(ssh-agent -s)
    - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add -
    - echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts
    - BACKUP_TS=$(date +%Y%m%d-%H%M%S)
    - BACKUP_FILE="magento-pre-release-$BACKUP_TS.sql.gz"
    - |
      ssh "$DEPLOY_USER@$DEPLOY_HOST" bash -s << SSH
      set -euo pipefail
      mkdir -p "$BACKUP_DIR"
      cd "$DEPLOY_PATH/current"
      # Read DB credentials from Magento env.php
      DB_HOST=\$(php -r "
        \\\$env = include 'app/etc/env.php';
        echo \\\$env['db']['connection']['default']['host'];
      ")
      DB_NAME=\$(php -r "
        \\\$env = include 'app/etc/env.php';
        echo \\\$env['db']['connection']['default']['dbname'];
      ")
      DB_USER=\$(php -r "
        \\\$env = include 'app/etc/env.php';
        echo \\\$env['db']['connection']['default']['username'];
      ")
      DB_PASS=\$(php -r "
        \\\$env = include 'app/etc/env.php';
        echo \\\$env['db']['connection']['default']['password'];
      ")
      # Create consistent backup using single transaction (InnoDB safe)
      mysqldump \
        --host="\$DB_HOST" \
        --user="\$DB_USER" \
        --password="\$DB_PASS" \
        --single-transaction \
        --quick \
        --routines \
        --triggers \
        --add-drop-table \
        "\$DB_NAME" | gzip -9 > "$BACKUP_DIR/$BACKUP_FILE"
      echo "Backup created: $BACKUP_DIR/$BACKUP_FILE"
      ls -lh "$BACKUP_DIR/$BACKUP_FILE"
      SSH
  needs: []
  when: manual
  only:
    - tags

5. Backup Verification: Actually Checking the Backup

A backup that has not been verified is not a backup, it is a file with unknown contents. Verifying a mysqldump backup happens on two levels. The first level is file integrity: calculate and store a checksum, and sanity check the file size (a compressed backup smaller than 1 MB is suspicious for a Magento production database). The second level is content verification: check the compressed SQL file for compression integrity with gunzip -t, and check the first lines of the dump file for a valid SQL header.

A full restore verification, meaning restoring the backup into a test database and checking it for completeness, is too costly for production release pipelines. It should exist as a separate, regularly scheduled job, not as part of the deployment process. What is sufficient within the deployment process is file integrity, compression integrity, and a plausible file size. These three checks take seconds and protect against the most common backup failures (empty file, aborted dump, compression error).

6. Backup Timing: Immediately Before the Migration

The backup must be created immediately before the migration step. An hour between the backup and the migration is not an acceptable gap on an active production system; in that time, thousands of orders, customer records, and configuration changes could have been written to the database. Rolling back to a backup from an hour earlier means losing that data.

In the GitLab pipeline, this timing is enforced through job ordering and the needs directive: the backup job must be defined as a direct prerequisite of the migration job. No migration job may start before the backup job has completed successfully. This dependency protects against the most common mistake: creating the backup only after a problem has already occurred, once the database is already in a damaged state.

# Deploy pipeline with mandatory backup before DB migration
stages:
  - build
  - test
  - backup    # Backup stage runs before deploy
  - deploy
  - verify
  - rollback

# Backup job: must complete before any deploy with DB migration
backup:pre-release:
  stage: backup
  script:
    - eval $(ssh-agent -s)
    - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add -
    - echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts
    - BACKUP_TS=$(date +%Y%m%d-%H%M%S)
    - |
      ssh "$DEPLOY_USER@$DEPLOY_HOST" bash -s << 'SSH'
      set -euo pipefail
      BACKUP_FILE="/var/backups/magento/releases/pre-release-$BACKUP_TS.sql.gz"
      mkdir -p /var/backups/magento/releases
      # Run backup using credentials from env.php
      cd "$DEPLOY_PATH/current"
      php bin/magento setup:backup --db
      # Also create external backup for faster restore
      mysqldump --defaults-file=/etc/mysql/backup.cnf \
        --single-transaction --quick --routines --triggers \
        magento | gzip -9 > "$BACKUP_FILE"
      # Verify backup integrity
      gunzip -t "$BACKUP_FILE" && echo "Backup integrity OK"
      ls -lh "$BACKUP_FILE"
      # Record backup location for rollback job
      echo "$BACKUP_FILE" > /var/backups/magento/latest-pre-release.txt
      SSH
  when: manual
  only:
    - tags

# Deploy job must explicitly wait for backup
deploy:production:with-migration:
  stage: deploy
  needs:
    - backup:pre-release  # Cannot start without successful backup
  script:
    - echo "Backup verified, starting deployment with DB migration"
    # ... deploy steps with setup:upgrade
  when: manual
  only:
    - tags

7. DB Rollback: How a Database Rollback Works

A database rollback is more costly than a code rollback and should therefore only be performed when a code rollback alone is not enough. The decision must be made quickly: once a deployment has failed and the database is in an inconsistent state, every minute counts. The DB rollback process must be documented, rehearsed, and automated through a script so that it can be executed correctly even under pressure.

The sequence of a DB rollback: enable maintenance mode to prevent further database writes. Perform the code rollback (switch the symlink back to the previous release). Restore the database from the pre-release backup. Verify the backup restore (check the table count and critical configuration values). Disable maintenance mode. For a mysqldump-based backup, the entire process typically takes 5 to 15 minutes, depending on database size.

8. Backup Retention and Rotation

Pre-release backups do not need to be kept indefinitely, but they must remain available for as long as a rollback still makes sense. A retention period of 7 release backups is sufficient for most Magento projects: it covers the last seven deployments, which typically corresponds to several weeks. Older backups are rotated out automatically to save storage space.

Rotation should be implemented as part of the backup job, not as a separate cron job: immediately after creating the new backup, all backups except the last seven are deleted. This keeps storage under control and avoids the situation where the backup storage fills up and new backups can no longer be created. Backup file sizes should also be logged so that unusual size changes can be flagged as an anomaly.

9. With vs. Without a Backup Job: A Direct Comparison

The difference between a deployment process with and without a backup job only becomes visible in an incident. In normal deployments, the backup is never needed. In an incident, such as a failed migration step, a damaged database, or an unexpected code error that corrupts the database, the backup is the only way back to a consistent state.

The cost of the backup job is low: the mysqldump command with --single-transaction takes between 1 and 5 minutes for a typical Magento database (1 to 5 GB). That is a manageable investment in the ability to respond in a controlled way when an incident occurs. A deployment without a backup is a bet that everything will go well. A deployment with a backup is a controlled process.

10. Summary

Database backups before Magento releases are not optional for deployments that include database migrations. They are the only safeguard against data loss when a migration step fails. The backup must be created immediately before the migration, its integrity must be verified, and it must be defined as an explicit prerequisite of the migration job in the GitLab pipeline.

The DB rollback process must be documented, rehearsed, and automated through scripts. During an incident, minutes determine the extent of the damage. A team that runs the DB rollback process for the first time during an actual incident is not prepared. A team that regularly rehearses it on staging and has automated it through the pipeline can act in a controlled way even under pressure. That is the core of zero-downtime design: not the hope that nothing goes wrong, but preparation for the case that something does.

11. FAQ: Database Backups Before Magento Releases