SSH in Automation: known_hosts, Agent, Keys, and Security

1. Why SSH automation requires care

The simplest fix for "SSH interactively asks me to confirm the host" is -o StrictHostKeyChecking=no. This option shows up in countless CI scripts, tutorials, and deployment configurations, and in most contexts it is a security problem. It disables the very mechanism that prevents man in the middle attacks. In a production SSH automation setup, that means anyone who controls or can redirect the DNS name or IP address of the target server can intercept a connection without the script ever noticing. Credentials, deployment artifacts, and sensitive configuration get shipped straight to the attacker.

The actual problem that StrictHostKeyChecking=no is meant to solve is a workflow problem: the known_hosts file of the automation account does not contain the target server's fingerprint. The correct fix is not to disable the check, but to provision the fingerprint ahead of time. In SSH automation, that means calling ssh-keyscan during server provisioning, writing the fingerprint into the automation account's known_hosts or into a project-specific known-hosts file, and storing that file as a secret in the CI configuration. This solves the workflow problem without any security compromise.

Another common mistake in SSH automation: private keys get stored as plaintext files in repositories, container images, or environment variables that end up in logs. The correct approach: keep keys only in protected secret stores (GitHub Secrets, Vault, AWS Secrets Manager), write them to temporary files with restricted permissions (chmod 600) during the build, and delete them securely right after use. These three principles, no StrictHostKeyChecking=no, no keys in repositories, no keys in logs, form the foundation of secure SSH automation.

2. known_hosts: provisioning fingerprints in advance

The known_hosts file stores the public host key of every SSH server the client has connected to. On the first connection, SSH interactively asks whether to accept the fingerprint, a mechanism that simply does not work in SSH automation. The fix is ssh-keyscan: this tool queries a server's public key without establishing an authenticated connection and prints it in known_hosts format. The output can be written straight into the known_hosts file: ssh-keyscan -H hostname >> ~/.ssh/known_hosts. The -H flag hashes the hostname, which prevents the known_hosts file from being used as a network topology leak.

For SSH automation in CI pipelines, a project-specific known-hosts file is preferable to the global ~/.ssh/known_hosts. The file is created during server provisioning, stored as a CI secret, and written temporarily into ~/.ssh/ or a temp path before every SSH connection. In the SSH options it is referenced explicitly with -o UserKnownHostsFile=/path/to/known_hosts. That keeps known-hosts management independent of the CI runner environment, and it can be versioned and audited. Rotating host keys on the server side (after an incident or on a regular schedule) then requires updating the CI secret, which is exactly the deliberate, conscious step you want.

#!/usr/bin/env bash
# ssh-automation-setup.sh - provision known_hosts and agent for automation
set -euo pipefail
IFS=$'\n\t'

KNOWN_HOSTS_FILE="${1:?Usage: $0 <known_hosts_file> <host1> [host2...]}"
shift

# Collect host fingerprints securely (with hashed hostnames)
provision_known_hosts() {
  local hosts=("$@")
  local tmpfile
  tmpfile="$(mktemp)"
  trap 'rm -f -- "$tmpfile"' EXIT

  local host
  for host in "${hosts[@]}"; do
    echo "Scanning: $host" >&2
    # -T timeout, -H hash hostname, query ed25519 and rsa keys
    ssh-keyscan -T 10 -H -t ed25519,rsa "$host" >> "$tmpfile" 2>/dev/null || {
      echo "[WARN] Could not scan $host" >&2
    }
  done

  # Deduplicate and sort for stable output
  sort -u "$tmpfile" > "$KNOWN_HOSTS_FILE"
  chmod 600 "$KNOWN_HOSTS_FILE"
  echo "[OK] Written ${#hosts[@]} host(s) to $KNOWN_HOSTS_FILE" >&2
}

provision_known_hosts "$@"

# Verify: connect with strict checking enabled
verify_connection() {
  local host="$1"
  ssh -o StrictHostKeyChecking=yes \
      -o UserKnownHostsFile="$KNOWN_HOSTS_FILE" \
      -o BatchMode=yes \
      -o ConnectTimeout=10 \
      "$host" 'echo OK' 2>&1 | head -1
}

3. Configuring StrictHostKeyChecking correctly

The SSH option StrictHostKeyChecking has three values that matter for SSH automation: yes (the default in newer OpenSSH versions) refuses the connection if the host is not in known_hosts or the key has changed. no accepts any host and any key without checking anything, never use it in production. accept-new (OpenSSH 7.6+) is the sensible middle ground for scenarios where new hosts appear during regular operation: new hosts are accepted and added to known_hosts, but a changed key from an already known host is still treated as an error. That is the recommended value for dynamic infrastructure, where new server instances need to be connected to automatically.

An often overlooked aspect: when a server changes its host key (after a reinstall, key rotation, or migration), SSH automation fails with a warning. That is intentional, but in automated deployment pipelines without monitoring, the warning shows up in logs nobody reads and the script fails with a cryptic error. The fix: explicitly remove the known old key from known_hosts (ssh-keygen -R hostname) and add the new one via ssh-keyscan, as part of the server reprovisioning process. That makes the process explicit and traceable instead of silently failing.

4. ssh-agent in shell scripts and CI

The ssh-agent holds decrypted private keys in memory and makes them available to the SSH client over a Unix socket. In SSH automation it removes the need to type the key passphrase on every connection. In interactive shells, the agent is often started by the desktop environment or the login session and made accessible via the SSH_AUTH_SOCK environment variable. In non-interactive contexts (cron jobs, CI runners) the agent has to be started explicitly: eval "$(ssh-agent -s)" starts a new agent process and sets the environment variables in the current shell. The key is then loaded with ssh-add /path/to/key.

In CI pipelines for SSH automation there is an important security decision to make: should the private key be stored directly as a CI secret, or should a passphrase-free key be used for CI? For high security: keys with a passphrase, the passphrase stored separately as a CI secret, and the agent with automatic ssh-add. For practicality: dedicated CI keys without a passphrase, with restricted permissions on the target server (a command= restriction in authorized_keys). Either way, the agent process must be stopped at the end of the script (ssh-agent -k), and the key content must never appear in logs. Always use ssh-add - (reading from stdin) instead of echo "$key" | ssh-add, the latter can become visible in process listings.

#!/usr/bin/env bash
# ssh-agent-ci.sh - start ssh-agent, load key, cleanup after use
set -euo pipefail
IFS=$'\n\t'

# Private key from CI secret (never echo to stdout/stderr)
SSH_PRIVATE_KEY="${SSH_PRIVATE_KEY:?SSH_PRIVATE_KEY must be set}"

# Start agent and ensure cleanup
start_agent() {
  # eval sets SSH_AGENT_PID and SSH_AUTH_SOCK in current shell
  eval "$(ssh-agent -s)" > /dev/null
  # Register agent kill in cleanup
  trap 'ssh-agent -k > /dev/null 2>&1 || true' EXIT
}

load_key() {
  local key_content="$1"
  # ssh-add from stdin: key never appears in ps output or shell history
  printf '%s\n' "$key_content" | ssh-add - 2>/dev/null
  echo "[OK] SSH key loaded ($(ssh-add -l | wc -l) key(s) in agent)" >&2
}

start_agent
load_key "$SSH_PRIVATE_KEY"

# Now SSH connections use the agent, no password prompts
ssh -o StrictHostKeyChecking=yes \
    -o BatchMode=yes \
    -o ConnectTimeout=15 \
    deploy@production.example.com \
    'bash -s' < deploy-commands.sh

# Agent is killed automatically via trap EXIT

5. BatchMode: non-interactive SSH connections

The SSH option BatchMode=yes is essential in SSH automation. It disables every interactive prompt: no password entry, no fingerprint confirmation, no passphrases. When SSH fails in BatchMode, it returns a clear non-zero exit code instead of a blocking prompt. Without BatchMode, a non-interactive script can hang on an SSH prompt, potentially forever, until an external timeout kicks in. In CI pipelines that leads to jobs that never end and keep blocking runner slots.

To complement BatchMode, ConnectTimeout is worth setting for SSH automation: an explicit timeout stops an unreachable host from blocking the script for the length of the TCP connection timeout (which on some systems can be several minutes). ServerAliveInterval and ServerAliveCountMax detect a lost connection to a running remote process and enable clean failover instead of endless hanging. The combination BatchMode=yes ConnectTimeout=30 ServerAliveInterval=60 ServerAliveCountMax=3 is a solid default for robust SSH automation in deployment scripts.

6. SSH key management for automation

Key management is the most critical security aspect of SSH automation. The most important ground rule: create dedicated keys for automation that carry no additional access rights. Do not share private keys between people and automation accounts, and do not share a single automation key across different environments (staging and production must have separate keys). The authorized_keys entry on the target server can be restricted with options: command="only-this-command" allows only one specific command to run, no-pty prevents pseudo-terminal allocation, and no-agent-forwarding and no-port-forwarding limit further SSH features.

In modern setups for SSH automation, private keys are not stored statically at all, they are generated dynamically by a secret manager and issued with a time limit. HashiCorp Vault has an SSH secrets engine that issues a short-lived certificate for every connection, expiring after a few minutes or hours. That eliminates the key rotation problem entirely: there are no long-lived private keys left to steal. For environments without Vault, the minimum is: keys generated with ssh-keygen -t ed25519 (a modern, secure algorithm choice), regular rotation (at least yearly), and immediate revocation on any suspicion of compromise.

7. Jump hosts and bastion proxies

In many production environments, target servers are not reachable directly from the internet or the CI runner, only through a bastion host (jump host). The classic, insecure fix: copy the private key onto the bastion host and connect to target servers from there (-A agent forwarding). That is problematic: an attacker who compromises the bastion host can use the forwarded agent socket to connect to any target server as long as the agent session is active. For SSH automation the secure alternative is ProxyJump (or the older ProxyCommand variant): every connection is tunneled through the bastion host, but the agent socket stays on the original client.

The OpenSSH option -J user@bastion, or ProxyJump configured in ~/.ssh/config, creates a direct TCP tunnel through the bastion host. The cryptographic material (private keys) never leaves the CI runner, the bastion host only ever sees the encrypted data stream. For complex topologies, multiple jump hosts can be chained: -J bastion1,bastion2. In ssh_config the jump-host configuration can be set differently per hostname pattern, cleanly separating environments (staging via bastion A, production via bastion B).

#!/usr/bin/env bash
# ssh-jump-deployment.sh - deploy through bastion with ProxyJump
set -euo pipefail
IFS=$'\n\t'

readonly BASTION="${BASTION_HOST:?BASTION_HOST must be set}"
readonly TARGET="${TARGET_HOST:?TARGET_HOST must be set}"
readonly DEPLOY_USER="${DEPLOY_USER:-deploy}"
readonly KNOWN_HOSTS_FILE="${KNOWN_HOSTS_FILE:?KNOWN_HOSTS_FILE must be set}"

# Build common SSH options (no agent forwarding, strict host checking)
SSH_OPTS=(
  -o "StrictHostKeyChecking=yes"
  -o "UserKnownHostsFile=${KNOWN_HOSTS_FILE}"
  -o "BatchMode=yes"
  -o "ConnectTimeout=30"
  -o "ServerAliveInterval=60"
  -o "ServerAliveCountMax=3"
  -o "ForwardAgent=no"       # Never forward agent - security risk
  -o "LogLevel=ERROR"        # Suppress banner noise in CI output
)

# Connect to target through bastion via ProxyJump
# Private key never touches the bastion host
run_remote() {
  local cmd="$1"
  ssh "${SSH_OPTS[@]}" \
      -J "${DEPLOY_USER}@${BASTION}" \
      "${DEPLOY_USER}@${TARGET}" \
      "$cmd"
}

# Copy deployment artifact through bastion
deploy_artifact() {
  local local_file="$1"
  local remote_path="$2"
  scp -o "StrictHostKeyChecking=yes" \
      -o "UserKnownHostsFile=${KNOWN_HOSTS_FILE}" \
      -o "BatchMode=yes" \
      -o "ForwardAgent=no" \
      -J "${DEPLOY_USER}@${BASTION}" \
      "$local_file" \
      "${DEPLOY_USER}@${TARGET}:${remote_path}"
}

echo "Deploying to $TARGET via $BASTION..."
deploy_artifact "app-release.tar.gz" "/tmp/app-release.tar.gz"
run_remote "bash /opt/deploy/install.sh /tmp/app-release.tar.gz"
echo "[OK] Deployment complete"

8. Setting up SSH securely in CI/CD pipelines

SSH automation in CI/CD pipelines has specific security requirements that go beyond a plain shell script. The private key is stored as a CI secret (GitHub Secrets, GitLab CI variables marked Masked and Protected), written from the secret into a temporary file with chmod 600 in a before-step, and deleted right after the deployment step. The known-hosts file is likewise stored as a CI secret. It does not contain any secrets itself, but its integrity is critical for the security of the connection. If the known-hosts file can be modified, an attacker can bypass the host key check entirely.

An important security aspect for SSH automation in multi-stage pipelines: deploy keys should only be accessible to the deploy step, not to every pipeline job. In GitHub Actions that means setting the key as an environment secret scoped to the specific job, not as a repository secret available to every workflow. In GitLab CI: scope environment-specific variables to the production environment so that feature-branch pipelines have no access to production deployment keys. The principle of least privilege, every job gets only the credentials it actually needs, is the single most important defense against credential leakage from a compromised supply-chain component.

9. SSH options compared on security

For every insecure shortcut in SSH automation there is a secure alternative that satisfies the same workflow need without giving up security.

The table shows: every insecure practice in SSH automation has a direct, secure alternative. The most common objection, "but that's more complicated to set up", is true for the initial setup only. After that, running secure SSH automation is no more effort than running an insecure one, and the risk profile is fundamentally better. command= restrictions in authorized_keys in particular punch above their weight: even if a deployment key is compromised, the attacker can only run the one configured command, not get free shell access.

10. Summary

Secure SSH automation in shell scripts and CI/CD pipelines rests on a few principles applied consistently: never StrictHostKeyChecking=no, instead populate known-hosts files during server provisioning and manage them as a CI secret. Always BatchMode=yes and ConnectTimeout for non-interactive connections. Replace agent forwarding with ProxyJump, which never lets cryptographic material touch the bastion host. Private keys only ever as CI secrets, never in repositories or container images. command= restrictions in authorized_keys limit the damage if a key is compromised.

These measures take a bit more effort to set up the first time than the simple, insecure alternatives. But running them afterward is no more complicated, and the risk profile is fundamentally better. Especially in deployment pipelines that operate with root-equivalent permissions on production servers, SSH automation security is not an optional feature, it is a baseline requirement for responsible operations practice.

11. FAQ: SSH in Automation, known_hosts, Agent, Keys, and Security