Using Force Merge and Segment Optimization Correctly

1. How Lucene segments form through ongoing indexing

Every Elasticsearch index internally consists of one or more shards, and every shard in turn consists of a collection of immutable Lucene segments. On every refresh cycle, Lucene writes the documents indexed in the meantime as a new, standalone segment to disk instead of modifying existing segments. This immutability is a deliberate design choice, since it enables concurrent reads without expensive locking mechanisms and forms the basis for Lucene's high indexing throughput.

Under continuous indexing this produces a steady stream of new, often very small segments. A background process, the merge policy, automatically combines smaller segments into larger ones once certain thresholds are reached, keeping segment count in a healthy balance. Under very heavy write load, or with poorly configured merge thresholds, this automatic compaction cannot always keep up with the creation of new segments, and noticeably more segments accumulate over time than would actually be necessary.

2. Why fragmented segments cost query performance

A query against a Lucene shard fundamentally has to search every single segment and then merge the partial results together. Every additional segment adds overhead: more file descriptors, more search calls per query, and more work merging partial results into a final, sorted result. On a shard with a few large segments this overhead is small, on a shard with hundreds of tiny segments it can become noticeable.

The effect shows up especially clearly with aggregations and sorting across large document counts, since every segment has to be walked individually before the global results can be merged. An index with heavily fragmented segments can respond considerably slower than a compacted index with the exact same document count, which in practice often gets misdiagnosed as a general performance problem instead of a specific segment problem.

3. What force merge does technically: the max_num_segments target

Force merge manually triggers exactly the compaction process that normally runs automatically in the background, but forces a concrete target result through the max_num_segments parameter. Setting that value to one, for example, makes Elasticsearch fuse every segment of a shard into a single, physically contiguous segment. That reduces management overhead for future queries to a minimum, since only a single segment needs to be searched.

Technically, this process means Lucene rereads all relevant segments in full, permanently removes deleted documents in the process, and writes the result as a new segment to disk. That explains both the benefit, a cleanly compacted index needs less storage and delivers faster queries, and the cost of the operation, since during execution both the old and the new segment data have to briefly coexist on disk at the same time.

POST product-catalog-2026.07/_forcemerge?max_num_segments=1

4. When a manual force merge makes sense: finished indices only

Force merge makes sense almost exclusively for indices that are permanently no longer written to, typically after a rollover, once a time-based index has ended its active life stage. Classic examples are a finished monthly index for log data, an archived order index after fiscal year end, or a snapshot preparation step, where a compacted index produces noticeably more compact and faster-to-create snapshots.

The key reason for that restriction is that a force merge on an index still being written to immediately creates new, small segments again as soon as the next documents arrive. The compaction effort would evaporate again within a short time, while the cost of the operation, especially the I/O load, would already have been paid in full. This is exactly why the combination of rollover followed by force merge in the warm phase of an ILM policy is so widespread.

5. Risks from misuse: I/O spikes on active indices

If force merge accidentally runs against an actively written, production index, a massive I/O spike typically follows: fully rewriting large amounts of segment data competes directly with ongoing write operations for disk bandwidth, which can noticeably lower the indexing rate while simultaneously raising the latency of concurrent read queries. On heavily loaded production clusters, a thoughtlessly triggered force merge can, in the worst case, cause noticeable timeouts for user requests.

Another, often underestimated risk is the temporary storage requirement: during the merge, both the source segments and the newly forming target segment need room on disk at the same time, requiring considerably more free storage than the index occupies at rest. A cluster with tight disk space can be pushed into a critical storage crunch by an ill-considered force merge, potentially blocking further write operations across the entire cluster in the worst case.

6. Force merge in the ILM context as an automated warm phase action

In a well-configured ILM policy, force merge almost never shows up as a manually run command, but as a declarative action in the warm phase, which only kicks in after the guaranteed minimum age since rollover. Since ILM at that point already ensures the index is no longer the current write target, the risk of accidentally applying it to a still active index disappears entirely, provided the policy is correctly linked to the rollover alias.

Even so, deliberate prioritization within cluster resources is worthwhile: a force merge running at the same time as many other ILM transitions across multiple indices can, despite correctly targeting finished indices, heavily strain the cluster's available I/O bandwidth for a short period. This is why staggering execution across the day is often more sensible in many setups than concentrating it into a narrow time window.

7. Force merge versus expunge_deletes: the important difference

Alongside a full force merge with a fixed max_num_segments target, there is the considerably gentler only_expunge_deletes option, which cleans up only segments with a high share of already deleted but physically still present documents, without reducing the total segment count to a fixed target. This matters especially for indices with frequent updates, where Elasticsearch internally marks old document versions as deleted but only actually removes them physically at the next merge.

This targeted cleanup is considerably lighter on resources than a full force merge, since only segments with a high deletion ratio get touched at all instead of rewriting every segment regardless of its state. For indices that still get updated occasionally but are no longer written to at high frequency, only_expunge_deletes is therefore often the better choice over a full force merge down to a fixed segment count.

curl -X POST "localhost:9200/reviews-archive/_forcemerge?only_expunge_deletes=true"

8. Monitoring during force merge: the task API and cluster health

A force merge on large indices can take anywhere from a few minutes to several hours, which is why progress should be actively tracked through the task API rather than treating the operation as fire and forget. The task API shows both the currently running action and, if desired, a blocking call that waits until the operation completes, which is useful for automated scripts that should only continue once the process has fully finished.

It is also worth watching cluster health, and in particular disk usage and I/O load figures, during execution to catch early whether the operation unexpectedly interferes with resources needed by other concurrently running operations. For critical production environments it is additionally advisable to deliberately schedule force merge runs into lower-traffic windows, such as overnight, rather than triggering them during peak business hours.

9. How force merge interacts with cluster disk watermarks

The temporarily increased storage requirement during a force merge can turn into its own problem when combined with cluster-wide disk watermark thresholds. If a node crosses the configured high watermark threshold during the merge, Elasticsearch starts relocating shards away from that node, generating additional network and I/O load exactly while the node is already under heavy strain. If the flood stage threshold gets reached as well, Elasticsearch automatically sets affected indices to read only, which can have serious consequences in a production setup.

Before a planned force merge, it is therefore worth briefly checking the actually available free disk space per node against the size of the segments about to be merged. Under tight capacity, a moderate target such as max_num_segments=5 instead of 1 is a sensible compromise, since it noticeably reduces the temporary storage requirement while still delivering most of the performance gain, without pushing the node close to the critical watermark thresholds.

10. Summary

11. FAQ: Force merge in Elasticsearch: the essentials at a glance