Securing SSH Connections: Keys, Config, Hardening

1. Why SSH is the most important attack surface

On practically every production Linux server, SSH is the only open administrative access point facing the internet, and that is exactly why it is the preferred target of automated attacks. Anyone watching the logs of a freshly provisioned server with port 22 open sees the first brute force attempts from botnets within minutes, systematically trying default usernames like root, admin, or ubuntu against common passwords. These scans are not an edge case, they are permanent background noise on the internet.

The good news: the most effective countermeasures are not exotic extra tools, but a handful of well documented configuration changes to sshd_config and a consistent switch from password to key based authentication. A server that only allows public key auth, blocks root login over SSH, and only offers modern encryption schemes reduces the realistic attack surface to almost zero, even if port 22 stays open and publicly reachable. The following sections walk through the concrete implementation, from the keypair to a production ready sshd_config.

2. Key-based authentication instead of passwords

An SSH keypair consists of a private key that never leaves your own machine, and a public key that gets placed on the target server in ~/.ssh/authorized_keys. For new keys, the Ed25519 algorithm is the right choice: shorter keys than RSA, faster signature computation, and no known practical weaknesses. RSA with at least 4096 bits remains a sensible fallback when target systems are old enough that they do not support Ed25519.

The key should always be protected with a passphrase, so that a stolen private key alone is not enough. ssh-copy-id transfers the public key cleanly without disturbing existing permissions, and is preferable to manually appending it with cat because it avoids duplicates. Correct file permissions matter: the ~/.ssh directory needs 700, the authorized_keys file needs 600. If these permissions are off, sshd silently refuses key authentication by default, without any indication on the client side.

#!/usr/bin/env bash
# Generate a modern Ed25519 keypair (recommended over RSA for new keys)
ssh-keygen -t ed25519 -a 100 -C "deploy@mironsoft.de" -f ~/.ssh/id_ed25519_mironsoft

# Copy the public key to the target server (appends to authorized_keys)
ssh-copy-id -i ~/.ssh/id_ed25519_mironsoft.pub deploy@server.mironsoft.de

# Enforce strict permissions manually if the key was copied by other means
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chmod 600 ~/.ssh/id_ed25519_mironsoft
chmod 644 ~/.ssh/id_ed25519_mironsoft.pub

# Verify which key the client actually offers during the handshake
ssh -vv deploy@server.mironsoft.de 2>&1 | grep "Offering public key"

3. Hardening sshd_config: PermitRootLogin and PasswordAuthentication

The central configuration file /etc/ssh/sshd_config controls the behavior of the SSH server, and two directives have the biggest security impact. PermitRootLogin no blocks any direct root login over SSH, so attackers gain no access even with a correct root password; administrative tasks instead go through a regular user with sudo. PasswordAuthentication no disables password logins entirely and allows only public key authentication, which means classic brute force attacks against SSH have nothing left to guess, because no password exists anymore.

Before enabling these settings, make sure at least one working key exists for a sudo capable user, because a mistake here locks you out of the only access point to the server. Best practice: test changes in a second SSH session while the first session stays open, and only close the old session after a successful login with the new key. sshd -t checks the syntax of the configuration file before the service reloads, preventing a typo from taking down the SSH service entirely.

# /etc/ssh/sshd_config - core hardening directives
# Disable direct root login entirely, use sudo instead
PermitRootLogin no

# Disable password authentication, allow public key auth only
PasswordAuthentication no
PubkeyAuthentication yes
KbdInteractiveAuthentication no
ChallengeResponseAuthentication no

# Reject empty passwords outright (defense in depth)
PermitEmptyPasswords no

# Limit authentication attempts and time per connection
MaxAuthTries 3
LoginGraceTime 20

# Restrict SSH access to specific users or groups
AllowUsers deploy admin
# AllowGroups ssh-users

# Validate syntax before reloading the service:
#   sshd -t && systemctl reload sshd

4. Protocol and cipher restrictions

Beyond the authentication method, the choice of allowed encryption, key exchange, and MAC algorithms determines how resilient an SSH connection is against cryptographic attacks. Older OpenSSH versions enable weaker algorithms like diffie-hellman-group1-sha1 or arcfour by default for compatibility reasons, even though they have long been considered insecure. The Ciphers, KexAlgorithms, and MACs directives let you explicitly restrict the allowed list to modern schemes such as chacha20-poly1305@openssh.com and aes256-gcm@openssh.com.

The commands ssh -Q cipher, ssh -Q kex, and ssh -Q mac list every algorithm supported on the local system, and are the fastest way to build a whitelist without consulting outdated documentation. One often overlooked point: moving off port 22 to a different port significantly reduces the volume of automated scan noise in the logs, but it is not a real security measure in the cryptographic sense, only security through obscurity, and should never be treated as a substitute for actual hardening.

# /etc/ssh/sshd_config - restrict to modern, audited algorithms
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com

KexAlgorithms curve25519-sha256,curve25519-sha256@libssh.org,diffie-hellman-group16-sha512

MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com

# Only protocol 2 has been supported since OpenSSH 7.6, listed here for clarity
Protocol 2

# Optional: move off the default port to cut down automated scan noise
# Port 2222

# Disable features rarely needed on a hardened server
X11Forwarding no
AllowTcpForwarding no
AllowAgentForwarding no

5. Client configuration with ~/.ssh/config

On the client side, a well maintained ~/.ssh/config saves typing and also reduces sources of error: instead of manually specifying username, port, and key file for every connection, you define a per-host alias with all the required parameters. IdentityFile determines which private key is used for a host, and IdentitiesOnly yes stops the SSH client from also offering every key loaded in the agent, which can otherwise quickly lock you out on servers with a low MaxAuthTries.

For servers that are only reachable through a bastion or jump host, ProxyJump handles the forwarding automatically and replaces the older, far more error prone ProxyCommand syntax with a manual nc call. HashKnownHosts yes stores host entries in ~/.ssh/known_hosts hashed instead of in plain text, so a stolen known-hosts file does not reveal a list of reachable servers. Include also lets you split larger configurations into team and project specific files that can be version controlled.

# ~/.ssh/config - per-host client configuration
Host prod-shop
    HostName server.mironsoft.de
    User deploy
    Port 2222
    IdentityFile ~/.ssh/id_ed25519_mironsoft
    IdentitiesOnly yes
    AddKeysToAgent yes

Host db-internal
    HostName 10.0.4.12
    User dbadmin
    ProxyJump prod-shop
    IdentityFile ~/.ssh/id_ed25519_internal
    IdentitiesOnly yes

# Global defaults applied to hosts without an explicit block
Host *
    HashKnownHosts yes
    ServerAliveInterval 30
    ServerAliveCountMax 3
    IdentitiesOnly yes

6. SSH agent and key management

ssh-agent keeps decrypted private keys in memory so the passphrase does not need to be re-entered for every connection. That is convenient, but it has an important downside: AddKeysToAgent yes in the client config combined with ssh-add -t 3600 and a time limit ensures keys stay in memory only for a bounded session rather than indefinitely. On shared systems or jump hosts, agent forwarding (ForwardAgent yes) should only be enabled selectively for individual, trusted hosts, never globally, since a compromised intermediate server can otherwise misuse every forwarded key.

The same caution applies to deployment pipelines: deploy keys should be created individually per repository or target system, instead of storing a personal key in CI/CD environments. Rotating keys regularly and consistently removing old entries from authorized_keys keeps the attack surface small even as team membership changes. The command ssh-keygen -y -f key checks whether a private key actually matches the stored public key before an old entry is removed.

# .github/workflows/deploy.yml - load a dedicated deploy-only key via ssh-agent
name: Deploy
on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Start ssh-agent and load the deploy key
        run: |
          eval "$(ssh-agent -s)"
          ssh-add - <<< "${{ secrets.DEPLOY_KEY_ED25519 }}"
          echo "SSH_AUTH_SOCK=$SSH_AUTH_SOCK" >> "$GITHUB_ENV"

      - name: Deploy over SSH
        run: ssh -o StrictHostKeyChecking=yes deploy@server.mironsoft.de "bin/deploy.sh"

7. Complementary protection: fail2ban against brute force

Even with password authentication disabled, an extra layer of protection against the sheer volume of automated connection attempts is worthwhile: fail2ban parses /var/log/auth.log or journalctl -u ssh, detects repeated failed login attempts from the same IP address, and blocks it for a configurable time via iptables or nftables. This substantially reduces log noise and takes load off the SSH daemon itself, because connection attempts from banned IP addresses never even reach the TCP handshake.

For SSH, the bundled jail template is usually enough, enabled via /etc/fail2ban/jail.local with an adjusted maxretry and bantime. Important: fail2ban does not replace any of the previous measures, it complements them, because without working key authentication, a single patient attacker with a low attempt frequency would still be a risk. Fully configuring fail2ban with custom filters and multiple jails is a topic of its own and goes beyond the scope of this article.

8. Monitoring and logging SSH access

A hardened configuration alone is not enough if nobody notices when the access pattern changes. journalctl -u ssh --since "1 hour ago" shows recent login attempts directly from the systemd journal, while grep "Failed password" /var/log/auth.log | wc -l gives a quick overview of the number of failed attempts. Successful root logins, if they are even still possible, should trigger an immediate alert in any monitoring system, since they almost always mean either a misconfiguration or a successful attack.

For centralized logging across multiple servers, it is worth forwarding the auth logs to a central syslog server or into a log aggregator such as Loki or the ELK stack family, so patterns become visible across the entire infrastructure instead of disappearing into isolated individual logs. last and lastlog also give a quick overview of recent successful logins and help spot unexpected access times or unknown source IPs early, before they turn into a larger incident.

9. SSH hardening in direct comparison

The following overview summarizes the most important settings where the difference between the default configuration and a hardened setup is particularly large.

Anyone who consistently implements all five rows has already reduced the realistic attack vectors against SSH to a minimum. The remaining risk, such as zero-day vulnerabilities in OpenSSH itself, can only be covered by timely patching and an up to date package state, not by configuration alone.

10. Summary

The most important measures for securing SSH connections work together: key based authentication with Ed25519 keys replaces passwords entirely and renders classic brute force attacks useless. PermitRootLogin no and PasswordAuthentication no in sshd_config close the two biggest entry points right at the source. Restricted ciphers, key exchange algorithms, and MACs prevent outdated, insecure encryption schemes from even being an option. A well maintained ~/.ssh/config makes daily work both safer and more convenient at the same time.

fail2ban complements these measures as an additional layer against the sheer mass of automated scans, but it does not replace any of the previous steps. Continuous monitoring through journalctl, centralized logs, and alerts on successful root logins ensures that deviations from the normal access pattern do not go unnoticed for weeks. Anyone who consistently implements all of these building blocks reduces the realistic attack surface of a Linux server over SSH to a minimum, regardless of whether the default port stays open or not.

11. FAQ: Securing SSH connections