The Rollover API: Practical Strategies for Growing Indices

1. Why a single, endlessly growing index is problematic

An index that grows without interruption for years accumulates several structural problems at once. Shard count gets fixed when an index is created and cannot easily be changed afterward, so an undersized index eventually runs into a hard per-shard ceiling under sustained growth. On top of that, maintenance tasks such as snapshots, reindexing or mapping changes become increasingly expensive and risky as index size grows, since a mistake there potentially affects the entire historical dataset.

Rollover solves this by splitting the data volume across several manageable physical indices, each bounded by time or size. Every one of those indices stays small enough to run maintenance tasks quickly and with low risk, while the overall dataset remains queryable as a single logically coherent body of data through an alias structure.

2. Rollover basic principle: a write alias plus condition checking

The rollover mechanism builds on a writable alias that initially points to exactly one physical index, marked as the write index. A rollover call checks the configured conditions against the currently active index and, if at least one condition is met, creates a new physical index with an incremented name. Elasticsearch then automatically switches the write alias over to the new index, while the old index stays readable through the same alias, just no longer as a target for new writes.

It matters that rollover runs as an atomic operation: either the new index gets created successfully and the alias switches over completely, or the whole operation fails and the original state stays unchanged. That atomicity prevents an inconsistent intermediate state where, say, two indices would end up marked as the write target at the same time.

3. Conditions in detail: max_size, max_docs, max_age and max_primary_shard_size

The rollover API supports several freely combinable conditions. max_size caps overall index size across every primary shard, while max_primary_shard_size specifically caps the size of the single largest primary shard, which is more precise under uneven shard distribution. max_docs caps document count regardless of actual size, which is sometimes more meaningful than a pure size condition for documents whose size varies widely. max_age, finally, caps the time since the index was created, regardless of size or document count.

In practice most policies combine at least two of these conditions, usually a size condition together with an age condition, to force a sensible rollover point under both unusually heavy and unusually light data volume. Without such a combination, an index would either stay tiny for months under low data volume or grow noticeably larger than intended for efficient handling during a sudden spike in load.

POST reviews-write/_rollover
{
  "conditions": {
    "max_primary_shard_size": "25gb",
    "max_docs": 50000000,
    "max_age": "30d"
  }
}

4. Alias design for seamless application integration

For clean integration into application code, a two-part alias model is recommended: a write alias that always points only to the currently active, writable index, and a separate read alias covering every index in the full rollover sequence, used exclusively for read queries. That separation guarantees writes never accidentally run against multiple indices at once, while read queries can transparently search the entire historical dataset without any change to application code.

For the application code itself, nothing changes with a rollover: it keeps writing against the same write alias name and keeps reading against the same read alias name, regardless of how many physical indices actually exist behind the scenes. That decoupling is the real practical value of the rollover API, since it moves the entire complexity of index management fully out of the application layer and into Elasticsearch's configuration.

5. Manual rollover via API versus automated rollover via ILM

A rollover can be triggered manually through the REST API at any time, which suits one-off, planned events, such as a deliberately triggered rollover right before a large marketing event with an expected traffic spike. For ongoing regular operation, though, that manual control is not very practical, since nobody wants to continuously watch current condition values and estimate the right moment for the next call.

In practice, ILM almost always takes over that job instead: rollover conditions get attached as an action of the hot phase within an ILM policy, and Elasticsearch checks those conditions automatically at regular intervals in the background. A manual rollover call still stays available as an emergency or special-case tool, and works fine running alongside automated ILM control at any time.

6. Practical example: a growing product review index with rollover

A Magento store with a strongly growing product catalog and active customer engagement continuously accumulates new product reviews, whose total volume grows without bound over the years. Without rollover, a single review index would eventually span several hundred million documents, making both reindexing on mapping changes and regular snapshots increasingly impractical. With a rollover strategy triggered after fifty million documents or thirty days, every single physical index instead stays permanently manageable.

The store's application code keeps writing new reviews exclusively against the alias reviews-write, never needing to know which physical index is currently active. The product page keeps querying reviews for a given product against the alias reviews-read, so search works transparently across the entire history of review indices, even when a long-lived product's review history is spread across several physical indices.

PUT _index_template/reviews-template
{
  "index_patterns": ["reviews-*"],
  "template": {
    "settings": {
      "index.lifecycle.name": "reviews-ilm-policy",
      "index.lifecycle.rollover_alias": "reviews-write"
    },
    "aliases": {
      "reviews-read": {}
    }
  }
}

7. Naming conventions for rollover-capable indices

Elasticsearch requires rollover-capable indices to follow a naming pattern ending in a sequential, six-digit number, such as reviews-000001, so every subsequent rollover can automatically increment that number by one. If a date-based naming scheme is preferred instead, for example for daily log indices, the rollover API also supports date math expressions in the index name, so a rollover automatically produces a name with the current date, without needing any external logic for it.

A consistent naming convention across every rollover-capable index family in a cluster not only makes monitoring considerably easier but also prevents mix-ups during manual maintenance tasks, for example when an administrator accidentally edits an archived index instead of the currently active one. Especially with many rollover sequences running in parallel in a larger cluster, clear, uniform naming pays off substantially over time.

8. Rollover and mapping changes: the ideal moment for adjustments

An often overlooked benefit of the rollover strategy is the ability to introduce mapping changes with low risk. Since every rollover creates a completely new, empty physical index, the underlying index template can be adjusted before the next rollover, so the change only takes effect in the next physical index, without requiring an expensive reindex of existing data. Existing indices stay unchanged with their original mapping.

This approach works great for additive changes such as new fields, but hits limits once existing fields would need an incompatible change, for example switching a field type from text to keyword. Such cases still require a classic reindex of the already existing, affected indices, though rollover at least avoids that task entirely for every index created from that point forward.

9. Monitoring and troubleshooting rollover problems

A rollover can fail if the target index name already exists, for instance after a manual test run that accidentally produced the same name, or if the configured alias fails to uniquely reference exactly one write index. The rollover API returns a clear error message in such cases, but that only helps if the call is actually being monitored, which is why an automated, ILM-driven rollover should always be paired with a look at the ILM explain API.

Another practically relevant problem arises when rollover conditions were chosen too generously and therefore rarely or never trigger, letting a single index keep growing unnoticed despite having a rollover configuration in place. A regular check of the actual size and age of the currently active index against the configured thresholds reliably surfaces such misconfigurations before they turn into a real performance problem.

10. Summary

11. FAQ: Rollover API in Elasticsearch: the essentials at a glance