Snapshot Lifecycle Management: Setting Up Automated Backups

1. Why manual snapshots do not hold up as a backup strategy

A manually triggered snapshot is essentially a one-off act of good will, not a reliable process. Once responsibility for regular backups gets delegated to a single person or an unmonitored cron script, typical gaps appear: a forgotten snapshot right before a major deployment, a failed snapshot whose error nobody saw, or a repository that goes months without cleanup and eventually blows through available storage.

Snapshot Lifecycle Management shifts that responsibility away from an individual human and into a declarative policy the cluster itself executes. That turns an error-prone manual routine into a reproducible, monitorable process that runs on a fixed schedule regardless of whether anyone happens to be thinking about it, and that additionally removes old snapshots automatically according to clearly defined rules.

2. Setting up a snapshot repository: object storage versus a shared filesystem

Before an SLM policy can take effect at all, the cluster needs a registered repository as the target for its snapshots. Two variants dominate in practice: an S3-compatible object storage repository for cloud environments, and a shared filesystem repository, mounted under the exact same path on every cluster node, for on-premises setups. Both require the matching repository plugin and correctly set access permissions, without which every snapshot attempt fails with a permission error.

An S3 repository brings additional benefits around scaling and durability, since object storage exists independently of the cluster infrastructure and usually already carries its own redundancy. A filesystem repository is simpler to set up, but demands its own careful protection against data loss, for example through RAID or an additional external mirror, since a failure of the underlying storage can otherwise take down both the cluster and its backups at the same time.

PUT _snapshot/s3-backup-repo
{
  "type": "s3",
  "settings": {
    "bucket": "myshop-es-snapshots",
    "region": "eu-central-1",
    "base_path": "production/backups",
    "server_side_encryption": true
  }
}

3. SLM policy structure: schedule, naming pattern and configuration

An SLM policy defines four central building blocks: a cron-based schedule, a naming pattern for the resulting snapshots, the target repository and, optionally, a configuration for which indices should be included. The naming pattern can include date variables, so every snapshot automatically gets a unique, chronologically sortable name such as daily-snap-2026.08.08, without needing any extra logic outside the policy itself.

Index selection can be controlled through patterns or explicit lists, and in practice it usually pays off to deliberately include system-critical indices such as the Kibana configuration index, since a restore scenario without them would bring back the data but not the associated visualizations and dashboards. A deliberate, documented selection matters more here than a blanket rule applied to every index.

PUT _slm/policy/daily-snapshots
{
  "schedule": "0 30 1 * * ?",
  "name": "<daily-snap-{now/d}>",
  "repository": "s3-backup-repo",
  "config": {
    "indices": ["*", "-.monitoring*"],
    "include_global_state": true
  },
  "retention": {
    "expire_after": "30d",
    "min_count": 5,
    "max_count": 50
  }
}

4. Retention rules in detail: expire_after, min_count and max_count

The retention configuration of an SLM policy controls how many old snapshots are kept before being deleted automatically. The expire_after field sets a plain time boundary, after which a snapshot is generally marked for deletion. The min_count and max_count fields step in as complements: min_count prevents too few snapshots from remaining even when the time boundary has long passed, while max_count sets a hard upper limit regardless of age.

Together these three rules prevent two opposite problems: an uncontrolled growth of the repository from snapshots that never get deleted, and an accidental loss of all backups if snapshot creation fails for an extended period and no new snapshot arrives to replace the expired ones. A typical pattern for production clusters keeps snapshots for thirty days, but guarantees at least five even if they are older than thirty days.

5. Monitoring snapshot execution with the SLM stats API

Since SLM policies run in the background, regularly checking execution status is essential. The SLM stats API and the get policy API both report the last successful run, the last failed run including its error message, and cumulative counters for successful and failed executions since the cluster last restarted. These values should be a standard part of a monitoring dashboard, not something only looked up manually when something already feels wrong.

A particularly tricky failure mode is a snapshot that starts but gets skipped because a competing snapshot is already running on the same repository. Since SLM by default allows only one snapshot at a time per repository, an unusually long-running snapshot can cause several scheduled runs in a row to fail silently, unnoticed without active monitoring, until eventually no current snapshot exists at all.

6. The restore test as a mandatory exercise, not a backup illusion

A successfully created snapshot only proves that the write operation worked, not that a restore will actually succeed. Corrupted repository metadata, missing read permissions or an incompatible target cluster often go unnoticed at creation time, surfacing only in an emergency, once it is already too late. A regular, documented restore test into a separate test environment is therefore not an optional bonus, it is the only proof that a backup strategy actually works.

In practice a monthly restore test works well, where a randomly chosen, recent snapshot is loaded into an isolated test cluster and spot-checked for data completeness and consistency. Anyone who skips this test is relying on a pure assumption that backups would work in an emergency, and that exact assumption regularly turns out, in practice, to be the most expensive mistake in the entire backup concept.

# Restore a snapshot into a separate test cluster
curl -X POST "test-cluster:9200/_snapshot/s3-backup-repo/daily-snap-2026.08.08/_restore" \
  -H 'Content-Type: application/json' -d '{
    "indices": "product-catalog-*",
    "rename_pattern": "(.+)",
    "rename_replacement": "restored_$1",
    "include_global_state": false
  }'

7. Cross-cluster replication as a complement, not a replacement for snapshots

Some teams mistake cross-cluster replication for a backup solution, since both mechanisms keep data redundantly available. The critical difference lies in how errors propagate: an accidental deletion or a corrupted indexing operation gets replicated to the target cluster almost in real time, so both clusters end up showing the exact same broken state. A snapshot, by contrast, is a point-in-time capture that stays untouched by that exact problem, as long as the error happened after the snapshot was taken.

For a complete backup concept, both mechanisms complement each other: cross-cluster replication reduces recovery time in a full data center outage, while SLM-driven snapshots protect against logical errors such as accidental deletions, faulty bulk updates or ransomware scenarios. Anyone relying on replication alone is, knowingly or not, giving up exactly that second layer of protection.

8. Handling failed snapshots

A failed snapshot is usually a symptom of a deeper problem: a full repository, a broken network connection to object storage, expired access keys, or shards currently relocating and therefore unable to be captured consistently. The snapshot status API returns a detailed error message per affected shard for every failed attempt, which speeds up root cause analysis considerably, provided those messages actually get reviewed.

A proven pattern is alerting that escalates automatically after two consecutive failed SLM runs, rather than only reacting once snapshots have been missing entirely for several weeks. Since SLM by default keeps running and simply retries the next run at its scheduled time, a repeatedly failing snapshot can easily go unnoticed for a long stretch without active alerting.

9. Verifying repository integrity and cleaning up orphaned data

A snapshot repository should be checked regularly for structural consistency instead of blindly trusting successful snapshot notifications. The verify repository API lets every cluster node confirm it can actually read from and write to the repository, surfacing configuration mistakes early, for example a node with missing access permissions that would otherwise only show up during an actual snapshot or restore attempt.

On top of that, aborted snapshot deletions or interrupted uploads can leave orphaned data fragments behind in the repository, referenced by neither a current nor a historical snapshot, yet still occupying storage and therefore driving up cost. The cleanup API removes exactly those orphaned fragments without endangering any referenced, valid snapshot, and should therefore be scheduled at regular intervals as its own maintenance step, independent of the actual SLM policy.

curl -X POST "localhost:9200/_snapshot/s3-backup-repo/_verify"
curl -X POST "localhost:9200/_snapshot/s3-backup-repo/_cleanup"

10. Summary

11. FAQ: SLM in Elasticsearch: the essentials at a glance