Preparing Release Directories on the Server: shared, var, pub/media, app/etc

1. Why the directory structure decides everything

The most common reason zero downtime deployments for Magento fail in practice is not the CI/CD pipeline, it is the server itself. Teams build the symlink based release structure on a server that was never prepared for this model: pub/media sits directly in the web root, app/etc/env.php is hardcoded, and var/log grows uncontrolled inside every single release directory. Once the symlink points to the new release, either the media files or the configuration is missing.

The solution is a clear separation between files that are allowed to change with every release and files that must persist across all releases. This persistent data lives in the shared directory and is linked into every new release through a symlink. This is not a Magento specific concept, it is the classic Capistrano deployment model, applied to GitLab CI/CD with shell scripts.

What matters most is that this structure is prepared once, completely, before the first pipeline runs. A deployment script that tries to create missing directories on the fly is prone to race conditions and to diverging states between staging and production. The server init script belongs in the repository, is run once, and is not touched again unless a deliberate structural change is required.

2. The complete base structure on the server

The target structure on the production server follows a simple but strict pattern. The application root directory contains three areas: releases/ for all historical and current releases, current as a symlink to the active release, and shared/ for all persistent files and directories. The web server points to current/pub, the publicly accessible part of Magento.

Inside shared/ lives everything that is not part of the build: the Magento configuration app/etc/env.php and app/etc/config.php, the media folder pub/media, the session files in var/session, and the log files in var/log. This structure guarantees that a rollback to an older release has no effect at all on media files or configuration, the symlink simply points to a different release directory, and the shared data stays untouched.

# Expected server directory structure after init
# /var/www/magento/
# ├── current -> /var/www/magento/releases/v1.4.2   (symlink, updated per deploy)
# ├── releases/
# │   ├── v1.4.0/                                    (old release, kept for rollback)
# │   ├── v1.4.1/                                    (old release, kept for rollback)
# │   └── v1.4.2/                                    (current active release)
# └── shared/
#     ├── app/
#     │   └── etc/
#     │       ├── env.php                             (persistent: DB, Redis, secrets)
#     │       └── config.php                          (persistent: module list)
#     ├── pub/
#     │   └── media/                                  (persistent: uploaded images)
#     └── var/
#         ├── log/                                    (persistent: application logs)
#         └── session/                                (persistent: user sessions)

# Nginx document root points to: /var/www/magento/current/pub

3. Shared directories: what stays, what changes

The most important design decision in the server structure is the distinction between release specific and persistent data. Release specific data is the source code, the compiled PHP classes in generated/, the frontend build in pub/static/, and the composer vendor folder. This data arrives fresh on the server with every release and is part of the build artifact.

Persistent data, on the other hand, is everything that must stay consistent across releases: media that users have uploaded, configuration files that contain server specific database connections, session data for logged in users, and log files needed for traceability. This data must never be overwritten or deleted by a release. The symlink mechanism guarantees that: every release directory only contains a symlink to the shared directory, never the data itself.

4. Server init script: run once, then never touch again

The server init script is the only script that is run manually and only once on the target server. It creates all the required directories, sets the correct owner and permissions, and places the initial env.php in the right location. After this first run the complete directory structure is in place, and every following deployment runs fully automated through GitLab CI/CD.

# scripts/server-init.sh - Run ONCE on a new server to prepare the directory structure
#!/usr/bin/env bash
set -euo pipefail

# Configuration, override via environment variables before running
APP_PATH="${APP_PATH:-/var/www/magento}"
APP_USER="${APP_USER:-www-data}"
APP_GROUP="${APP_GROUP:-www-data}"

echo "[INIT] Creating base directory structure at ${APP_PATH}"
mkdir -p "${APP_PATH}/releases"
mkdir -p "${APP_PATH}/shared/app/etc"
mkdir -p "${APP_PATH}/shared/pub/media"
mkdir -p "${APP_PATH}/shared/var/log"
mkdir -p "${APP_PATH}/shared/var/session"

# Set ownership, the web server must be able to write to shared directories
chown -R "${APP_USER}:${APP_GROUP}" "${APP_PATH}"

# Protect env.php with restricted permissions, no world readable secrets
chmod 660 "${APP_PATH}/shared/app/etc" || true

echo "[INIT] Directory structure created successfully"
echo "[INIT] Next step: place env.php at ${APP_PATH}/shared/app/etc/env.php"
echo "[INIT] Then run your first GitLab pipeline with a release tag"

5. Setting symlinks correctly in the deploy step

The deploy step consists of three operations. First, the build artifact is transferred into the new release directory via rsync. Second, all shared directories and files are linked into the new release with symlinks. Third, the current symlink is switched atomically to the new release. The atomicity of the third step is essential: the command ln -sfn creates the new symlink and replaces the old one in a single, indivisible operation, so the web server never sees an inconsistent state.

The order matters: symlinks to shared directories must be in place before the current symlink is switched. Otherwise a request that already uses the new release could hit missing configuration files or a missing media folder. That brief window is enough to produce errors that are hard to reproduce and hard to diagnose.

deploy:production:
  stage: deploy
  environment:
    name: production
    url: https://shop.example.com
  script:
    # Step 1: Transfer build artifact to new release directory
    - |
      ssh "${DEPLOY_USER}@${DEPLOY_HOST}" "mkdir -p '${DEPLOY_PATH}/releases/${CI_COMMIT_TAG}'"
    - rsync -az --delete --exclude='.git' \
        ./ "${DEPLOY_USER}@${DEPLOY_HOST}:${DEPLOY_PATH}/releases/${CI_COMMIT_TAG}/"
    # Step 2: Link shared files before switching current symlink
    - |
      ssh "${DEPLOY_USER}@${DEPLOY_HOST}" bash -s <<'SSH'
      set -euo pipefail
      RELEASE="${DEPLOY_PATH}/releases/${CI_COMMIT_TAG}"
      SHARED="${DEPLOY_PATH}/shared"
      # Remove placeholder directories created by rsync
      rm -rf "${RELEASE}/app/etc" "${RELEASE}/pub/media" "${RELEASE}/var"
      mkdir -p "${RELEASE}/app" "${RELEASE}/pub" "${RELEASE}/var"
      # Create symlinks to shared persistent data
      ln -sfn "${SHARED}/app/etc"    "${RELEASE}/app/etc"
      ln -sfn "${SHARED}/pub/media"  "${RELEASE}/pub/media"
      ln -sfn "${SHARED}/var/log"    "${RELEASE}/var/log"
      ln -sfn "${SHARED}/var/session" "${RELEASE}/var/session"
      # Step 3: Atomic switch of current symlink (zero downtime)
      ln -sfn "${RELEASE}" "${DEPLOY_PATH}/current"
      echo "[OK] Switched to release ${CI_COMMIT_TAG}"
      SSH
  rules:
    - if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/
      when: manual

6. Assigning permissions and ownership correctly

Permission problems are the second most common cause of failed Magento deployments, right after a broken directory structure. The web server process must be able to write to pub/media, create sessions in var/session, and log to var/log. At the same time, configuration files such as app/etc/env.php must not be world readable, because they contain database passwords and API keys.

The recommended setup is: all files in the release directory belong to the deploy user that runs the pipeline. The web server user (for example www-data) is a member of the deploy user's group. Directories such as pub/media get permission 775 so the web server can write to them. The env.php gets 640, so only root and the associated group can read it. This configuration must be built into the init script and the deploy job. None of it should ever be "fixed" manually.

7. Verify job in GitLab: checking the structure automatically

After every deployment a verify job checks whether the server structure is correct. This is not just a confirmation that the deployment worked, it is also an early warning system for structural drift. If a symlink is missing or a directory has the wrong permissions, the verify job fails before the first real user request runs into the problem.

verify:server-structure:
  stage: verify
  script:
    - |
      ssh "${DEPLOY_USER}@${DEPLOY_HOST}" bash -s <<'SSH'
      set -euo pipefail
      CURRENT="${DEPLOY_PATH}/current"

      # Verify that current symlink exists and points to a valid release
      test -L "${CURRENT}" || { echo "[FAIL] current is not a symlink"; exit 1; }
      test -d "${CURRENT}" || { echo "[FAIL] current target does not exist"; exit 1; }

      # Verify that shared symlinks are in place
      test -L "${CURRENT}/app/etc"    || { echo "[FAIL] app/etc symlink missing"; exit 1; }
      test -L "${CURRENT}/pub/media"  || { echo "[FAIL] pub/media symlink missing"; exit 1; }
      test -L "${CURRENT}/var/log"    || { echo "[FAIL] var/log symlink missing"; exit 1; }

      # Verify env.php exists and is readable
      test -f "${DEPLOY_PATH}/shared/app/etc/env.php" \
        || { echo "[FAIL] env.php not found in shared"; exit 1; }

      echo "[OK] Server structure verified for $(readlink ${CURRENT})"
      SSH
  rules:
    - if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/
      when: on_success

8. Comparison: flat structure vs. release model

Many Magento projects start with a flat server structure: the code lives directly in /var/www/magento, and deployments overwrite files directly via rsync or git pull. That works, until the first failure. Then it becomes clear that no rollback is possible because the previous release no longer exists, and no atomic switch is possible because there is no symlink mechanism.

9. Summary

A correctly prepared server structure is the foundation of every zero downtime deployment for Magento. The server init script creates releases/, current and shared/ with all the required subdirectories. The deploy job transfers the build artifact, sets symlinks to all shared directories, and switches the current symlink atomically. The verify job checks after every deployment that the structure is correct.

Anyone who sets this up cleanly once will not need to intervene manually again: every release lands in its own directory, persistent data stays untouched, rollbacks take seconds, and the deployment history can be read directly from the directory names on the server.

10. Common mistakes with the server structure

The classic mistake is forgetting pub/media in the shared directory. It does not show up immediately, only once a user uploads an image, the new release gets deployed, and the image is no longer present in the new release directory. Then it becomes clear that pub/media is not a symlink to shared/pub/media, it is a real directory sitting inside the release folder.

Another common mistake is setting symlinks with an absolute path instead of a relative one, or vice versa without a consistent approach. If the base path of the deployment changes, for example during a server migration, all existing symlinks point to paths that no longer exist. In general we recommend always setting symlinks with absolute paths, but sourcing those paths from environment variables so they can be adjusted centrally during a migration.

11. FAQ: Release Directories and Server Structure for Magento