when the server room itself is affected
An offsite backup is the copy that survives a fire, theft, or ransomware infection at the local site. The 3-2-1 rule gives a simple but effective formula for this: three copies of the data, on two different media types, at least one physically separated from the main site. This article shows how to implement that rule on Linux servers in an automated, encrypted, and monitored way.
Table of Contents
- 1. The 3-2-1 rule explained: three copies, two media, one offsite target
- 2. Why a second local backup alone is not enough
- 3. Implementing media diversity in practice
- 4. Automating offsite transport with rclone
- 5. Calculating bandwidth and cost
- 6. Immutable copies against ransomware
- 7. Encryption and access control for offsite targets
- 8. Monitoring and alerting for missed offsite syncs
- 9. Offsite options compared
- 10. Summary
- 11. FAQ
1. The 3-2-1 rule explained: three copies, two media, one offsite target
The 3-2-1 rule is the most widely used formula for resilient backup strategies and the starting point for any offsite backup concept. It requires three copies of the data: the original in production and two additional backups. Those copies should live on at least two different media types, for instance local disk and object storage, so a failure in one technology does not automatically affect every copy. And at least one copy must be offsite, physically separated from the main site.
The offsite backup is therefore not an optional extra, but the core of the rule, without which the other two copies become worthless during a local event. A server room that burns down takes out both the production system and a locally sitting backup NAS at the same time. An offsite backup, whether at another branch, a second data center, or cloud object storage, survives that scenario because it is not physically located in the same building.
2. Why a second local backup alone is not enough
Many teams confuse a second local backup with a complete backup strategy. A NAS in the same server room as the production server reliably protects against a single disk failure, but not against the events that most commonly cause complete data loss in practice: ransomware spreading across the network to every reachable share, a power outage with surge damage, theft, or water damage in the server room. An offsite backup is indispensable for exactly these cases.
Ransomware is the most relevant threat scenario of recent years. Modern encryption trojans actively search for reachable network drives and backup shares and deliberately encrypt them first, before attacking the actual production data, specifically to prevent the recovery an offsite backup would otherwise enable. An offsite backup that is not permanently mounted as a writable network drive, but only briefly authenticated for the transfer, escapes exactly that attack pattern.
3. Implementing media diversity in practice
The second pillar of the 3-2-1 rule, two different media types, maps well onto a typical Linux server setup. The first copy usually sits locally on a RAID array or a second disk in the same server for fast restores of minor issues. The second copy should live on a structurally different technology: a NAS on the local network with a different filesystem, or directly on cloud object storage such as S3, Backblaze B2, or Wasabi. In practice, that exact cloud target usually doubles as the offsite backup as well.
It matters that media diversity exists not only physically but also in the access path. Two backup targets that are both reachable with the same credentials and the same network path share a common risk: compromised credentials hit both at once. A robust offsite backup therefore uses its own, as restricted as possible, credentials that only have write permission for new objects, but no delete permission for objects that already exist.
4. Automating offsite transport with rclone
rclone is the standard tool for syncing an offsite backup from a Linux server to cloud object storage. It supports practically every relevant S3-compatible provider through a unified command line interface and offers, with rclone sync, an incremental sync that only transfers changed or new files. For a reliable offsite backup, rclone belongs in a systemd timer rather than a plain cron entry, because systemd offers better log integration and error handling.
A common mistake in automation: rclone sync mirrors the target exactly onto the source state, meaning it also deletes files in the offsite backup that were deleted locally. That is dangerous in case of accidental local data loss, because the mistake otherwise propagates straight into the offsite backup. The fix is rclone sync with a backup directory option that moves deleted or overwritten files into a separate folder instead of removing them for good, combined with versioning on the object storage side.
#!/usr/bin/env bash
# offsite-sync.sh — 3-2-1 offsite backup with rclone
set -euo pipefail
readonly LOCAL_SOURCE="/var/backup/nightly"
readonly REMOTE="offsite-s3:mironsoft-backups/$(hostname)"
readonly DELETED_ARCHIVE="offsite-s3:mironsoft-backups-deleted/$(hostname)/$(date +%Y%m%d)"
readonly LOG_FILE="/var/log/offsite-sync.log"
# --backup-dir keeps deleted/overwritten files instead of removing them outright
rclone sync "$LOCAL_SOURCE" "$REMOTE" \
--backup-dir "$DELETED_ARCHIVE" \
--transfers 8 \
--checkers 16 \
--fast-list \
--log-file "$LOG_FILE" \
--log-level INFO
exit_code=$?
if [[ $exit_code -ne 0 ]]; then
echo "[ERROR] Offsite sync failed with code $exit_code" >&2
exit "$exit_code"
fi
echo "[OK] Offsite backup synced to $REMOTE"
5. Calculating bandwidth and cost
A realistic offsite backup budget has to consider bandwidth, storage cost, and egress fees together. The first full transfer of an offsite backup, the seed backup, is the most bandwidth intensive step and can take days on multi terabyte data sets depending on the uplink. After that, the daily volume usually shrinks to a fraction through incremental syncs, as long as payload data does not change completely.
When choosing a provider for the offsite backup, egress cost, the fee for downloading data during an actual restore, deserves close attention. Some providers price storage cheaply but charge steep fees for retrieval during a restore, which makes the offsite backup expensive at exactly the moment it matters most. Providers with egress free models are often the more economical choice for disaster recovery scenarios, even if the pure storage fee is slightly higher.
6. Immutable copies against ransomware
The most effective protection for an offsite backup against ransomware is immutability, implemented technically as Object Lock or WORM (Write Once Read Many). An uploaded object is locked against overwriting and deletion for a defined period, even with the account's admin credentials. An attacker who gains access to the backup credentials could theoretically try to overwrite the existing offsite backup with new, encrypted data, but the locked older versions remain untouched.
Object Lock is available at most S3-compatible providers as a compliance or governance mode and has to be enabled when the bucket is created, retroactive activation is usually not possible. For an offsite backup with ransomware protection, the retention period should be chosen at least as long as the time that realistically passes between a compromise and its discovery, in practice often 30 to 90 days.
#!/usr/bin/env bash
# verify-object-lock.sh — confirm offsite bucket enforces immutability
set -euo pipefail
readonly BUCKET="mironsoft-backups"
readonly TEST_KEY="object-lock-test-$(date +%s).txt"
echo "test payload" > /tmp/lock-test.txt
# Upload with a 30-day compliance retention lock
aws s3api put-object \
--bucket "$BUCKET" \
--key "$TEST_KEY" \
--body /tmp/lock-test.txt \
--object-lock-mode COMPLIANCE \
--object-lock-retain-until-date "$(date -u -d '+30 days' +%Y-%m-%dT%H:%M:%SZ)"
# Deletion must fail while the retention period is active — that failure is the proof
if aws s3api delete-object --bucket "$BUCKET" --key "$TEST_KEY" 2>/dev/null; then
echo "[FAIL] Object was deletable — object lock is NOT enforced" >&2
exit 1
else
echo "[OK] Deletion correctly blocked — object lock is enforced"
fi
rm -f /tmp/lock-test.txt
7. Encryption and access control for offsite targets
An offsite backup leaves the physical control of your own data center, which is why encryption here is not an optional extra but a basic requirement. Transport should happen exclusively over TLS, which rclone ensures by default against S3-compatible endpoints. For the data itself, client side encryption before uploading is recommended, for instance with rclone crypt as an encrypting remote layer, so the cloud provider itself never sees unencrypted data.
Access control for the offsite target should follow the principle of least privilege: dedicated IAM credentials per server, restricted to exactly one bucket prefix, with write permission for new objects but no delete permission, combined with the Object Lock described in section six. Credentials for the offsite backup should never sit in plain text in scripts, but be loaded through environment variables from a secret store or at least from a file with restrictive permissions.
; rclone.conf — encrypted offsite remote with restricted credentials
; File permissions must be 600, owned by the backup service user only
[offsite-s3]
type = s3
provider = Other
access_key_id = AKIA_OFFSITE_RESTRICTED_KEY
secret_access_key = ${OFFSITE_SECRET_FROM_ENV}
endpoint = s3.eu-central-1.example-cloud.com
region = eu-central-1
acl = private
[offsite-crypt]
type = crypt
remote = offsite-s3:mironsoft-backups-encrypted
filename_encryption = standard
directory_name_encryption = true
password = ${RCLONE_CRYPT_PASSWORD_OBSCURED}
8. Monitoring and alerting for missed offsite syncs
An offsite backup that has not run successfully for weeks without anyone noticing is a silent failure of the entire backup strategy. Monitoring for the offsite backup has to check at least three things: did the last sync complete successfully, does the last successful sync date fall within the expected frequency, and does the transferred data volume roughly match the source. A sync that runs without error but accidentally syncs an empty directory otherwise goes unnoticed for months.
In practice a dead man's switch pattern works well: after every successful offsite backup, the script sends a signal to an external monitoring service such as Healthchecks.io or a self hosted equivalent. If that signal fails to arrive within a defined window, the service automatically triggers an alert, regardless of whether the backup script itself is still running or the entire server has failed.
#!/usr/bin/env bash
# offsite-sync-monitored.sh — dead man's switch for offsite backup health
set -euo pipefail
readonly HEALTHCHECK_URL="https://hc-ping.com/YOUR-UNIQUE-CHECK-ID"
readonly LOCAL_SOURCE="/var/backup/nightly"
readonly REMOTE="offsite-s3:mironsoft-backups/$(hostname)"
# Signal "start" so a hung sync is also detected, not only a missing one
curl -fsS -m 10 --retry 3 "${HEALTHCHECK_URL}/start" || true
if rclone sync "$LOCAL_SOURCE" "$REMOTE" --transfers 8 --checkers 16; then
# Compare local and remote size as a sanity check against silent empty syncs
local_size=$(du -sb "$LOCAL_SOURCE" | cut -f1)
remote_size=$(rclone size "$REMOTE" --json | jq -r '.bytes')
if (( remote_size < local_size / 2 )); then
echo "[WARN] Remote size ($remote_size) far below local ($local_size)" >&2
curl -fsS -m 10 "${HEALTHCHECK_URL}/fail" || true
exit 1
fi
curl -fsS -m 10 --retry 3 "$HEALTHCHECK_URL" || true
echo "[OK] Offsite backup verified and monitored"
else
curl -fsS -m 10 "${HEALTHCHECK_URL}/fail" || true
exit 1
fi
9. Offsite options compared
Several offsite backup options are available for implementing the 3-2-1 rule in practice, and they differ substantially in cost, automatability, and restore speed.
| Offsite option | Automation | Ransomware protection | Typical cost |
|---|---|---|---|
| rsync over SSH to a second site | Very good, standard tooling | Medium, without Object Lock | Only hosting the second site |
| rclone to S3-compatible object storage | Very good, systemd timer | High, with Object Lock | Storage plus possible egress |
| Physical tape drive (LTO) | Low, manual handling | Very high, air-gapped | High upfront investment |
| Replicated second site (own hardware) | Good, with configuration effort | Medium, depends on segmentation | High, own infrastructure |
For most mid-sized setups, rclone to S3-compatible object storage with Object Lock enabled is the best trade-off between automatability, ransomware protection, and cost. A tape drive remains relevant for particularly high security requirements, because a truly physically separated, not permanently networked copy is not an attack target on the network.
Mironsoft
Linux server operations, disaster recovery, and backup strategy
No reliable offsite backup following the 3-2-1 rule?
We set up an automated, encrypted, and monitored offsite sync, with Object Lock against ransomware and dead man's switch monitoring, so a local event never becomes a total loss.
3-2-1 audit
Reviewing your existing backup landscape against the 3-2-1 rule
rclone & Object Lock
Setting up an automated, immutable offsite sync
Monitoring & alerting
Implementing a dead man's switch for missed syncs
10. Summary
The 3-2-1 rule remains the most reliable base formula for backup strategies because it addresses three independent failure sources at once: hardware failure through multiple copies, technology failure through media diversity, and local disasters through an offsite backup. A second local backup alone does not fully protect against any of these scenarios, especially not against ransomware that deliberately encrypts reachable network drives along with everything else.
A modern offsite backup combines automated transport with rclone, Object Lock for immutability, client side encryption, and dead man's switch monitoring that immediately reports a missed sync. Anyone who consistently implements these four building blocks has an offsite backup that not only exists, but is actually usable in an emergency.
Offsite Backup Following the 3-2-1 Rule — Key Takeaways
Three copies, two media, one offsite
The 3-2-1 rule covers hardware, technology, and site failure at the same time.
Automated transport
rclone with a backup directory and a systemd timer instead of manual copying.
Object Lock against ransomware
An immutable retention period of 30 to 90 days protects even compromised credentials.
Do not skip monitoring
A dead man's switch reports a missed sync before months pass unnoticed.