Zero-Downtime Reindexing with Alias Swapping

1. Why reindexing in Elasticsearch is needed at all

An Elasticsearch mapping is effectively frozen for most changes once the index exists: switching an existing field from text to keyword, changing an analyzer configuration or adjusting the shard count is impossible without reindexing. The only way to make a fundamentally changed mapping effective is to create a new index with the desired mapping and then copy every document from the old index into the new one.

Without a structured process for this reindexing, every mapping change means downtime: the application has to be paused, data is migrated, and only afterward is traffic released again. For systems with continuous write traffic, such as an e-commerce product search or a log aggregator, that is unacceptable. A maintenance window of several hours for a mapping change is rarely tolerable in production environments.

The solution for zero-downtime reindexing rests on a simple idea: clients never address a physical index directly, they always address an alias. The alias initially points at the old index while a new index with the correct mapping is populated in the background. Only once the migration is fully validated is the alias switched to the new index in a single atomic operation, without any application ever changing its configuration.

2. The _reindex API in detail

The _reindex API copies documents from a source index into a destination index, entirely inside the cluster, without data ever leaving the server. The endpoint accepts a source block with an optional filter query, a dest block with the target index, and optionally a script to transform every document during the copy. For a pure migration reindex without transformation, the minimal form with just source.index and dest.index is enough.

The parameter wait_for_completion matters for production use. By default the _reindex request blocks until every document is copied, which causes timeouts on large indices. With wait_for_completion=false the API returns a task ID immediately, whose progress can be polled through the task API. This asynchronous pattern is the right choice for any reindexing job beyond a few thousand documents.

POST _reindex?wait_for_completion=false
{
  "source": {
    "index": "products_v1",
    "size": 1000
  },
  "dest": {
    "index": "products_v2",
    "op_type": "create"
  },
  "conflicts": "proceed"
}

The parameter op_type: create prevents documents that already exist in the target index from being overwritten, which matters when the same reindexing job is rerun after an interruption. conflicts: proceed ensures that version conflicts on individual documents do not abort the entire job, they are merely counted and reported in the result.

3. Write alias and read alias as the base pattern

The central architectural pattern for zero-downtime reindexing is that applications never write or read against a physical index name such as products_v1 directly, they always work against a stable alias such as products. When the first index is created, the alias is set up immediately, so a layer of indirection exists from the very start. This indirection is the prerequisite that makes a later reindexing possible at all without any client change.

In many setups a single alias for reading and writing is enough, in systems with heavy write load teams often separate a dedicated write alias from one or more read aliases, to steer read traffic deliberately toward the old or the new index during the migration phase. The example below shows the base alias setup right at index creation time.

PUT products_v1
{
  "aliases": {
    "products": { },
    "products_write": { "is_write_index": true }
  },
  "mappings": {
    "properties": {
      "sku": { "type": "keyword" },
      "name": { "type": "text" }
    }
  }
}

Every write operation from the application targets products_write, every read operation targets products. This separation later allows the read alias to be switched gradually, while the write alias only switches after full validation, which significantly reduces the risk of a faulty reindexing run.

4. The atomic alias swap

The decisive step of any zero-downtime reindexing operation is the alias swap through the _aliases endpoint using the remove and add actions inside a single request. Elasticsearch executes both actions atomically, at no point does the alias point at nothing or at both indices simultaneously. That distinguishes the alias swap fundamentally from two separate requests, between which requests could theoretically fall into a gap.

POST _aliases
{
  "actions": [
    { "remove": { "index": "products_v1", "alias": "products" } },
    { "add":    { "index": "products_v2", "alias": "products" } },
    { "remove": { "index": "products_v1", "alias": "products_write" } },
    { "add":    { "index": "products_v2", "alias": "products_write", "is_write_index": true } }
  ]
}

After this swap, every client working against the alias products sees the new index immediately, without any configuration file being changed or any deployment being triggered. The old index products_v1 stays in place for now and can serve as a fallback if unexpected problems with the new mapping surface after the swap. Only after an observation period, typically several days, is the old index deleted for good.

5. Reindexing with transformation and ingest pipelines

Plain copying is rarely enough, the data structure often has to be transformed during reindexing as well, for example to rename a field, normalize a value or merge several fields. The _reindex API supports a script object with Painless code for that, executed for every document before it is written to the target index. Alternatively an ingest pipeline can be referenced, which is especially useful when the same transformation logic should also apply to newly arriving documents outside the reindexing run.

POST _reindex
{
  "source": { "index": "products_v1" },
  "dest": { "index": "products_v2" },
  "script": {
    "lang": "painless",
    "source": "ctx._source.sku = ctx._source.remove('legacy_sku'); ctx._source.price_cents = (int)(ctx._source.price * 100)"
  }
}

For more complex transformations it is advisable to test the ingest pipeline separately before wiring it into a production reindexing run. An error in the Painless script causes individual documents to fail, and depending on the conflicts setting the behavior either stops the whole job or skips the failing documents and lists them as failures in the result report.

6. Throttling, batch size and slicing

An uncontrolled reindexing run at full speed can noticeably strain a production cluster, because it competes with regular search queries for CPU, I/O and heap. The parameter requests_per_second limits the rate at which batches are processed and allows the reindex to be throttled deliberately, so production traffic keeps priority. The parameter size inside the source block controls the batch size per scroll request.

For large indices with many shards, slices speeds up reindexing considerably by splitting the job into several parallel sub-tasks, each handling a portion of the source shards. With "slices": "auto" Elasticsearch determines the number automatically based on the shard count of the source index. This parallelization noticeably reduces total duration for large data volumes, but also increases system load, which is why slicing and throttling are usually configured together.

# Throttled, parallelized reindex with progress polling
curl -s -X POST "https://es.mironsoft.de:9200/_reindex?wait_for_completion=false" \
  -H "Content-Type: application/json" -d '{
    "source": { "index": "products_v1", "size": 500 },
    "dest": { "index": "products_v2" },
    "slices": "auto"
  }'
# Response contains a task id, e.g. "task": "node1:123456"

curl -s "https://es.mironsoft.de:9200/_tasks/node1:123456" | jq '.task.status'

7. Validation before the swap

Before executing the alias swap, the new index must be carefully validated, because after the swap every client sees the new data immediately. The simplest check is a document count comparison between source and target via GET products_v1/_count and GET products_v2/_count. If the numbers differ without expected conflicts, that points to an incomplete reindexing run that must be investigated before the swap.

Beyond sample comparisons, it pays off to run targeted test queries against the new index that simulate typical application requests, such as aggregations over newly mapped fields or full text searches using the changed analyzer. Comparing the result structure between old and new index reveals mapping errors that a plain count comparison cannot catch, for example when a field keeps the same name but received a different data type and therefore groups aggregations differently.

8. Rollback strategy for a failed reindexing run

Because the alias swap is atomic and takes effect instantly, an equally fast rollback option must exist in case the new mapping turns out to be faulty. The big advantage of the alias pattern is that a rollback is simply a second _aliases request in the opposite direction: the alias moves from the new index back to the old one, provided the old index was not deleted in the meantime. As long as products_v1 still exists, a rollback is possible within milliseconds.

It gets trickier once new write operations against the new index have already happened between the swap and the rollback. Those documents would additionally have to be backported into the old index after a rollback, which complicates the rollback considerably. That is why a short observation period with reduced write traffic right after the swap is recommended, along with monitoring that immediately flags unusual error rates, so a reindexing rollback can be triggered as early as possible.

9. Reindexing under active write traffic

The biggest challenge with zero-downtime reindexing is an index that keeps receiving writes during the migration. The _reindex API copies a snapshot as of the start time, documents written to the source index afterward do not automatically land in the target index. The usual solution is a two-phase approach: first an initial _reindex over the entire dataset, then one or more delta reindex runs with a query that captures only documents since the last run, for example filtered by an updated_at timestamp.

For very write-heavy systems, dual-write is a more robust alternative: the application writes into both indices in parallel during the migration phase, so both stay in sync at all times before the read alias is switched. Dual-write requires changes in application code, but it eliminates the risk of missed delta documents entirely and is the preferred strategy for systems with strict consistency requirements.

10. Summary

Zero-downtime reindexing is not a special feature, it is a necessary consequence of the fact that Elasticsearch mappings are largely immutable once an index exists. The _reindex API copies documents efficiently into a newly mapped index, while the write alias / read alias pattern ensures that clients never need to know a physical index name. The atomic alias swap through _aliases with remove and add in a single request makes the switch to the new mapping invisible to users.

For production systems with continuous write traffic, an additional strategy for delta data is needed, whether through timestamp based catch-up runs or dual-write during the migration phase. Validation before the swap and the ability to roll back instantly by resetting the alias are the two safety nets that turn a reindexing project from a risky ad hoc intervention into a predictable, repeatable process.

11. FAQ: Zero-Downtime Reindexing