SSH Bastion, Jump Hosts and Separate Deployment Users in GitLab Pipelines

1. Why a bastion host instead of direct SSH

In many Magento projects the production server is directly reachable over a public IP via SSH. The GitLab runner connects with a stored SSH key straight to port 22 and runs the deploy scripts. That works, right up until a runner is compromised, a key is lost, or a team member leaves the project. Without an intermediate layer there is no way to cut off access selectively without rotating the entire set of SSH keys on the server.

A bastion host is a dedicated entry point into the internal network. The production server is no longer directly reachable from the internet; only the bastion host is exposed, and every path leads through it. GitLab runners connect to the bastion first and then hop, via ProxyJump, to the target server. This reduces the attack surface, enables centralized logging of all SSH connections and makes access paths auditable.

Separate deployment users, meaning dedicated Linux accounts without sudo rights and with tightly scoped authorized_keys restrictions, ensure that a compromised deployment key does not grant root access. Together these three components form a reasonable security architecture for automated Magento deployments through GitLab.

2. Architecture: bastion, jump host and target system

The typical network topology for secured GitLab deployments consists of three layers. The GitLab runner runs either as a shared runner on gitlab.com or as a self-hosted runner on a separate network. It only has access to the bastion host, which sits in a DMZ and only opens port 22 to the outside. The actual target server, the web server holding the Magento deployment path, is only reachable from the internal network and has no direct connection to the internet.

The jump host is identical to the bastion host in many setups. The difference is purely terminological: a bastion is the first exposed point, a jump host is the forwarding instance. In practice a single server takes on both roles. The GitLab runner uses SSH ProxyJump to connect through the bastion to the target server in a single SSH command, without the bastion requiring a separate login. This reduces complexity while keeping security intact.

3. Creating and securing separate deployment users

A deployment user is a dedicated Linux account that exists solely for automated deploy processes. It has no password, no sudo rights and no interactive shell access beyond what is necessary. On the target server it is created with adduser --disabled-password deploy. The ~/.ssh/authorized_keys file for this user contains only GitLab-specific deployment keys, never personal developer keys.

For additional security, every authorized_keys entry can be given a command= restriction: the SSH key is then only allowed to execute a specific script, not arbitrary commands. That can be too restrictive for simple deployments, but it is the right approach for high-security environments. On the bastion host you likewise create a dedicated user that the runner jumps through; the bastion user itself needs no write access on the target server.

stages:
  - build
  - deploy
  - verify

variables:
  GIT_STRATEGY: fetch
  # SSH agent is initialized in before_script
  SSH_OPTS: "-o StrictHostKeyChecking=yes -o BatchMode=yes"

.ssh_setup: &ssh_setup
  before_script:
    # Load private key from GitLab CI variable (file type)
    - eval $(ssh-agent -s)
    - chmod 600 "$SSH_PRIVATE_KEY"
    - ssh-add "$SSH_PRIVATE_KEY"
    # Write known_hosts for bastion and target
    - mkdir -p ~/.ssh && chmod 700 ~/.ssh
    - echo "$SSH_KNOWN_HOSTS_BASTION" >> ~/.ssh/known_hosts
    - echo "$SSH_KNOWN_HOSTS_TARGET" >> ~/.ssh/known_hosts
    - chmod 644 ~/.ssh/known_hosts
    # Configure ProxyJump transparently for all SSH calls
    - |
      cat >> ~/.ssh/config <<EOF
      Host $DEPLOY_HOST
        ProxyJump $BASTION_USER@$BASTION_HOST
        User $DEPLOY_USER
        IdentityFile $SSH_PRIVATE_KEY
        StrictHostKeyChecking yes
      EOF
    - chmod 600 ~/.ssh/config

deploy:production:
  stage: deploy
  <<: *ssh_setup
  script:
    - ssh $SSH_OPTS $DEPLOY_USER@$DEPLOY_HOST "bash -s" < scripts/deploy.sh
  environment:
    name: production
  only:
    - tags

4. Configuring ProxyJump in GitLab CI/CD

The ProxyJump directive in ~/.ssh/config replaces the older ProxyCommand approach with ssh -W. Both solve the same problem; the difference lies in readability and fault tolerance. ProxyJump has been available since OpenSSH 7.3 and is present on all modern Linux systems. The directive tells the SSH client: connect to the bastion host first, and use that connection as a tunnel for the actual target server. From the application layer's point of view it looks like a direct SSH command.

In GitLab pipelines you write ~/.ssh/config dynamically in before_script, because runner environments are reset after every job. The relevant values, bastion hostname, bastion user, deploy hostname and deploy user, come from GitLab CI/CD variables. That way .gitlab-ci.yml contains no hardcoded hostnames and can use the same configuration for staging and production, separated by environment scopes.

5. Managing known hosts for bastion and target server

Setting the StrictHostKeyChecking flag to yes is mandatory in automated deployments. Setting it to no to work around connection issues opens the door to man-in-the-middle attacks. Instead, store the SSH fingerprints of both servers, bastion and target, as GitLab CI/CD variables and write them into the runner's ~/.ssh/known_hosts in before_script.

You obtain the fingerprints on the server with ssh-keyscan -H hostname. The output contains one or more lines in known_hosts format and is stored directly as the variable SSH_KNOWN_HOSTS_BASTION or SSH_KNOWN_HOSTS_TARGET in GitLab. During server migrations or key rotations these variables must be actively updated, otherwise the pipeline fails with a host verification error, which is the desired behavior.

6. GitLab variables and the SSH agent in the runner

For SSH bastion setups the pipeline needs the following variables: SSH_PRIVATE_KEY as a file variable (not a masked string), SSH_KNOWN_HOSTS_BASTION, SSH_KNOWN_HOSTS_TARGET, BASTION_HOST, BASTION_USER, DEPLOY_HOST, DEPLOY_USER and DEPLOY_PATH. The SSH private key must be stored as a file variable because the ssh-add command expects a file path, not a string variable. GitLab writes file variables to a temporary file and passes the path as an environment variable.

The SSH agent in the runner has to be started fresh for every job; it does not survive between jobs. The eval $(ssh-agent -s) call in before_script starts a new agent, and ssh-add loads the key. Importantly, the agent process dies with the job container, so the key does not leak into other jobs or runners. In Docker executors this cleanup happens automatically; for shell executors, trap "ssh-agent -k" EXIT is recommended as a safety net.

# Required CI/CD variables (set in GitLab project settings):
# SSH_PRIVATE_KEY     : Type File, deploy key without passphrase
# SSH_KNOWN_HOSTS_BASTION : bastion host fingerprint (ssh-keyscan output)
# SSH_KNOWN_HOSTS_TARGET  : target server fingerprint
# BASTION_HOST        : hostname or IP of the bastion server
# BASTION_USER        : SSH user on the bastion (e.g. jump)
# DEPLOY_HOST         : internal hostname of the target server
# DEPLOY_USER         : deployment-only user on the target server
# DEPLOY_PATH         : absolute base path, e.g. /var/www/magento

deploy:staging:
  stage: deploy
  before_script:
    - eval $(ssh-agent -s)
    - ssh-add "$SSH_PRIVATE_KEY"
    - mkdir -p ~/.ssh && chmod 700 ~/.ssh
    - printf '%s\n' "$SSH_KNOWN_HOSTS_BASTION" "$SSH_KNOWN_HOSTS_TARGET" \
        >> ~/.ssh/known_hosts
    - |
      cat > ~/.ssh/config <<EOF
      Host $DEPLOY_HOST
        ProxyJump ${BASTION_USER}@${BASTION_HOST}
        User $DEPLOY_USER
        StrictHostKeyChecking yes
        BatchMode yes
      EOF
    - chmod 600 ~/.ssh/config ~/.ssh/known_hosts
  script:
    - ssh "$DEPLOY_USER@$DEPLOY_HOST" "echo 'Connection via bastion: OK'"
    - scp scripts/deploy.sh "$DEPLOY_USER@$DEPLOY_HOST:/tmp/deploy_$CI_JOB_ID.sh"
    - ssh "$DEPLOY_USER@$DEPLOY_HOST" "bash /tmp/deploy_$CI_JOB_ID.sh && rm -f /tmp/deploy_$CI_JOB_ID.sh"
  environment:
    name: staging
    url: https://staging.mironsoft.de
  only:
    - main

7. Direct SSH vs. bastion routing compared

Comparing direct SSH access with bastion routing shows where the key differences in security and maintainability lie. Both approaches enable automated deployments, but with fundamentally different risk profiles.

The initial extra effort for a bastion setup pays off quickly: when a runner key needs to be rotated, a single change on the bastion is enough instead of updating every target server. On growing infrastructures with multiple staging and production servers, that is a considerable operational advantage.

8. Typical error patterns and diagnosis

The most common problem in bastion setups is known_hosts management. If the bastion host or the target server gets a new host key version, after a migration, an OS upgrade or a deliberate key change, StrictHostKeyChecking=yes fails. The error message is unambiguous: Host key verification failed. The fix is updating the GitLab variable SSH_KNOWN_HOSTS_BASTION or SSH_KNOWN_HOSTS_TARGET. If you cannot find the variable, you most likely forgot to create it with the right environment scope.

A second common problem: the SSH agent is not active in the runner because eval $(ssh-agent -s) is missing or the key was never loaded with ssh-add. This shows up as Permission denied (publickey) even though the variable is set correctly. To diagnose it, check the pipeline log to see whether ssh-add -l lists a key before the deploy step. A third failure mode involves ProxyJump configuration mistakes in ~/.ssh/config: typos in the hostname or missing whitespace after directives lead to cryptic connection errors.

9. Rollback via the bastion connection

Rollback jobs in a GitLab pipeline use the same SSH configuration as deploy jobs. The only difference: instead of the deploy script, a rollback script is invoked that points the current symlink to the previous release directory. Since the connection chain runs through the bastion, the rollback job must include the same before_script block with SSH agent and ProxyJump configuration.

A proven pattern is a separate rollback stage with when: manual in GitLab. The job appears in the pipeline view and can be triggered with a single click. The rollback target release is passed either as a hardcoded value or as a variable. Whoever tests rollback regularly in the staging environment knows, when a production incident happens, that the path actually works, and that is the single most important prerequisite for genuine zero-downtime deployment.

rollback:production:
  stage: rollback
  when: manual
  allow_failure: false
  before_script:
    # Same SSH setup as deploy job, bastion jump must work for rollback too
    - eval $(ssh-agent -s)
    - ssh-add "$SSH_PRIVATE_KEY"
    - mkdir -p ~/.ssh && chmod 700 ~/.ssh
    - printf '%s\n' "$SSH_KNOWN_HOSTS_BASTION" "$SSH_KNOWN_HOSTS_TARGET" \
        >> ~/.ssh/known_hosts
    - |
      cat > ~/.ssh/config <<EOF
      Host $DEPLOY_HOST
        ProxyJump ${BASTION_USER}@${BASTION_HOST}
        User $DEPLOY_USER
        StrictHostKeyChecking yes
        BatchMode yes
      EOF
    - chmod 600 ~/.ssh/config ~/.ssh/known_hosts
  script:
    # Determine previous release and switch symlink atomically
    - |
      ssh "$DEPLOY_USER@$DEPLOY_HOST" bash -s <<'REMOTE'
      set -euo pipefail
      cd "$DEPLOY_PATH"
      # List releases sorted newest-first, pick second (previous) entry
      PREV=$(ls -1dt releases/*/ | sed -n '2p' | tr -d '/')
      if [[ -z "$PREV" ]]; then
        echo "[ERROR] No previous release found for rollback" >&2
        exit 1
      fi
      ln -sfn "$DEPLOY_PATH/$PREV" "$DEPLOY_PATH/current"
      cd "$DEPLOY_PATH/current"
      bin/magento cache:flush
      echo "[OK] Rolled back to $PREV"
      REMOTE
  environment:
    name: production
  only:
    - tags

10. Summary

SSH bastion servers and jump hosts are not a luxury reserved for large infrastructures, but a reasonable security measure from the moment automated deployments start touching production systems. The combination of a bastion as the sole exposed entry point, ProxyJump for transparent SSH routing, and dedicated deployment users without sudo rights creates a clear, auditable connection architecture. GitLab CI/CD can be configured for this model with just a few lines in before_script, without complicating the pipeline structure.

The operationally most important step is regularly testing the entire path, not just from staging to staging, but the complete bastion, jump, deploy, verify, rollback cycle on a production-like environment. Whoever does this knows the connection chain and can act confidently in an incident instead of having to improvise under pressure.

11. FAQ: SSH Bastion and Jump Hosts in GitLab