Reindex From Remote: Migrating Data Between Clusters

1. The classic path: snapshot, restore, and its limits

The traditional way to move data between two Elasticsearch clusters goes through a snapshot repository: the source cluster backs up its indices into a repository reachable by both sides, such as a shared filesystem or a cloud storage bucket, and the target cluster restores the indices from it. That works reliably, but it requires both clusters to see the same repository and the snapshot formats to be compatible across the versions involved.

Especially with a jump across several major versions, or a change of hosting provider, a jointly reachable repository is not always practical, for instance when the source and target clusters live in separate networks or with different cloud providers. For cases like that, the Reindex API's remote option offers an alternative that does not need a shared storage target.

2. The Reindex API in its local baseline form

In the simplest case, POST _reindex copies documents from one index into another index of the same cluster, driven by a JSON object with source and dest fields. Optionally, source.query selects a subset of documents, script applies a transformation during the copy, and dest.op_type controls whether existing documents in the target get overwritten or skipped.

This local baseline case is the foundation for the remote case: instead of pointing at a local source index, source.remote points at an external cluster whose REST endpoint is reachable. All other options, such as query filtering, script transformation during the copy, or slicing for parallelization, work identically regardless of whether the source is local or remote.

POST _reindex
{
  "source": { "index": "products_v1" },
  "dest": { "index": "products_v2" }
}

3. How reindex from remote actually works

During a remote reindex, the target cluster queries the source cluster through its normal search API, paginating internally through a scroll mechanism, and writes every received document straight into its own target index via the bulk API. No intermediate file or snapshot is ever created, data flows directly cluster to cluster over HTTP or HTTPS.

This direct coupling allows migrating across different major versions too, as long as the source cluster's version falls within the range the target's documentation marks as supported. Since access runs through the regular REST API, the same permission checks apply as for any other read access to the source cluster.

POST _reindex
{
  "source": {
    "remote": {
      "host": "https://old-cluster.example.com:9200",
      "username": "migration_reader",
      "password": "changeme"
    },
    "index": "products_v1"
  },
  "dest": { "index": "products_v2" }
}

4. Whitelisting: the prerequisite on the target cluster

For security reasons, Elasticsearch does not allow a remote reindex against arbitrary, unknown hosts by default. The target cluster must explicitly allow the source through the static node setting reindex.remote.whitelist in elasticsearch.yml, as a list of allowed host and port combinations, for example [\"old-cluster.example.com:9200\"]. This setting is static, changing it requires restarting the affected nodes.

If a remote reindex is attempted against a host that has not been allowlisted, the request fails with a clear error message before any connection is even attempted. In production environments, it is a good idea to extend the whitelist only for the duration of the migration and remove the entry again afterward, instead of leaving a permanent open connection path to a foreign cluster.

5. Partial migration through query filtering and parallelization via slicing

For very large indices, it rarely pays off to transfer everything in a single, sequential reindex run. source.query splits the migration into manageable subsets, for example by time range or category, which also allows step-by-step validation of the transferred data before the full dataset gets migrated.

For parallelizing within a subset, the Reindex API supports slicing through the slices parameter, either with a fixed number or the value auto, which automatically adapts the slice count to the source index's shard count. Multiple slices run in parallel and each read only part of the documents, which noticeably raises overall throughput given sufficiently capable hardware.

POST _reindex?slices=auto
{
  "source": {
    "remote": { "host": "https://old-cluster.example.com:9200" },
    "index": "products_v1",
    "query": { "range": { "updated_at": { "gte": "2026-01-01" } } }
  },
  "dest": { "index": "products_v2" }
}

6. A practical workflow for a version upgrade

For an upgrade spanning several major versions, where no direct rolling upgrade path exists, remote reindex is often the most pragmatic route: a new cluster on the target version gets built alongside the existing system, the target indices get created upfront with the correct mapping and desired settings, and then the new cluster pulls the data from the old system via remote reindex.

It matters to define the target index's mapping explicitly before the reindex run instead of relying on dynamic mapping, since field types can differ between versions and an incorrectly inferred type is hard to fix afterward. It also helps to set number_of_replicas to zero during the migration and only raise it again afterward, so write throughput is not needlessly slowed down.

7. A practical workflow for a cloud or provider switch

Moving from a self-hosted environment into a managed cloud service, or between two cloud providers, follows the same core principle, plus network and security considerations: the target cluster needs to be able to reach the source cluster over the network, which, depending on the infrastructure, requires a VPN, an explicit firewall rule, or a publicly reachable endpoint secured with TLS and authentication.

A move away from a different search technology such as Solr does not go through the Reindex API, since it only works between Elasticsearch-compatible clusters. In such a case, a dedicated ingestion process first handles the initial import into a first Elasticsearch cluster, and reindex from remote only becomes relevant later, for consolidations or moves between multiple Elasticsearch clusters.

8. Security: authentication and transport encryption

If the source cluster runs with security enabled, credentials need to be passed in the source.remote block as a username and password, either directly or through an API key. These credentials get sent along with every internal request to the source cluster and should therefore be scoped to minimal necessary read permissions for the duration of the migration, rather than reusing existing administrative access.

If the source runs over HTTPS, the target cluster checks the certificate by default, which can cause connection errors with self-signed certificates in test environments if the certificate chain is not present in the target cluster's truststore. For production migrations, certificate verification should stay enabled and not be disabled across the board, even if that looks simpler in the short term.

9. Monitoring and troubleshooting during the migration

For larger data volumes, wait_for_completion=false is worth using, since it makes the Reindex API return a task ID immediately instead of keeping the HTTP connection open for the entire run. Progress can then be queried through GET _tasks/{task_id}, including counts of documents already processed and remaining, which allows a realistic estimate of remaining time.

If a migration stalls or runs unexpectedly long, the associated task can be canceled specifically through POST _tasks/{task_id}/_cancel without restarting the entire cluster. Common failure causes are expired connection timeouts on slow network links, adjustable through socket_timeout and connect_timeout in the source.remote block, as well as mapping conflicts when the target schema was not correctly defined beforehand.

# Check progress of a running remote migration
curl -s "https://target-cluster:9200/_tasks/oTUltX4IQMOUUVeiohTt8A:12345?pretty"

# Cancel the migration if needed
curl -X POST "https://target-cluster:9200/_tasks/oTUltX4IQMOUUVeiohTt8A:12345/_cancel"

10. Summary

11. FAQ: Reindex From Remote: The Essentials at a Glance