Circuit Breakers in Elasticsearch: Preventing Out-of-Memory Errors Proactively

1. Why Elasticsearch needs circuit breakers at all

The JVM that Elasticsearch runs on operates with a fixed, configured heap size, set when the node process starts. If actual memory demand exceeds that configured heap, the JVM can crash with an out-of-memory error, which for the affected node means an uncontrolled restart including shard recovery, instead of simply rejecting a single, overreaching request.

Without a protective layer in front, a single, poorly designed aggregation or sort on an unsuitable field could already consume enough memory to crash the entire node, potentially affecting other, completely unrelated requests and shards on the same node as well. Circuit breakers prevent exactly this scenario by checking estimated memory demand before actual execution.

2. The parent circuit breaker as an overarching guard rail

The parent circuit breaker sets the top-level limit for the combined, estimated memory consumption of all individual breakers together, and defaults to about 95 percent of the configured JVM heap. It trips as soon as the sum of field data, request, in-flight requests, and other breakers exceeds that overall limit, regardless of whether any single breaker on its own still sits within its own limit.

This overarching limit catches, in particular, cases where several operations, each unremarkable on its own, run at the same time and together would still overload the available heap. A single, isolated breaker value alone therefore often isn't enough for a complete memory diagnosis, the parent breaker status always belongs in the picture too.

3. Field data breaker: risk from sorting and aggregating on text fields

The field data breaker guards against excessive memory consumption from field data, the in-memory data structure needed for sorting, aggregating, and scripted access on analyzed text fields, whenever no suitable keyword field or doc values are used instead. Field data has to be loaded entirely into the heap for every affected field, and it grows proportionally to the number of unique terms in that field.

In Magento contexts, this case typically shows up when an accidental sort or aggregation runs on an analyzed text field like a product description, instead of on the dedicated, unanalyzed keyword subfield. Because field data is disabled by default for text fields, such an attempt usually produces an explicit error right away, instead of silently loading the heap, which makes the problem visible early in practice.

4. Request breaker: protection against a single expensive request

The request breaker estimates the memory demand of a single request while it's still executing, for instance for building large aggregation buckets or extensive sort structures, and aborts the request as soon as the estimated memory load exceeds the configured limit. Unlike the field data breaker, it doesn't only cover text field access, but fundamentally any operation that needs significant, temporary memory within a single request during execution.

Typical triggers are aggregations with very high cardinality, for instance a terms aggregation over a field with hundreds of thousands of unique values without a sensible bucket count limit, or deeply nested aggregations with many levels, whose memory demand multiplies with every additional level. A tripped request breaker is therefore often a direct signal of a poorly scoped, insufficiently bounded aggregation.

5. In-flight requests breaker: protection against network buffer overload

The in-flight requests breaker tracks the memory demand of all transport and HTTP requests currently received but not yet fully processed. It protects against a scenario where a very large number of simultaneously arriving requests with large payloads, for instance bulk indexing requests with large batch sizes, together consume so much buffer memory that the node stumbles before actual processing has even begun.

This breaker matters especially under heavy indexing load, for instance during a full Magento catalog reindex with many parallel bulk requests. If this breaker trips regularly, reducing the bulk batch size or the number of concurrent indexer threads is usually the sensible countermeasure, rather than hastily raising the breaker limit itself.

6. Typical triggers in Magento contexts

Besides sorting on analyzed text fields and high-cardinality aggregations, Magento operations commonly show two more patterns: unbounded facet aggregations in layered navigation on categories with a very large number of distinct attribute values, and very wide scripted metric aggregations for individual price calculations that build a separate script context in memory for every single document.

A sudden spike in simultaneously running, complex facet requests during a traffic peak, for instance on a sales day, can also trip the parent circuit breaker in aggregate, even if every single request looked unremarkable on its own. Such load spikes are a good occasion to review the aggregation structure of layered navigation for sensible bucket limits in general.

7. Diagnosing a tripped circuit breaker

A tripped circuit breaker produces a clearly identifiable CircuitBreakingException in the logs as well as in the HTTP response to the client, including the tripped breaker type, the estimated memory demand, and the configured limit. This information is the first and most important starting point for narrowing down the exact cause, instead of assuming a general memory problem across the board.

The nodes stats API additionally returns detailed counters per node and per breaker type, among them the number of breaker events tripped so far and the currently estimated memory consumption per breaker. A continuous rise in these counters over several days points to a structural problem, while a single, isolated spike points more to one concrete, poorly formulated individual request.

# Query breaker statistics across all nodes
curl "localhost:9200/_nodes/stats/breaker?pretty" \
  | jq '.nodes[].breakers | to_entries[] | {name: .key, tripped: .value.tripped}'

8. Tuning breaker limits: when and how carefully to adjust

The default values of the individual circuit breakers are chosen deliberately conservative and shouldn't be raised across the board as a first reaction to a tripped breaker, since raising a limit merely shrinks the safety margin to an actual out-of-memory error without fixing the underlying, memory-heavy request pattern.

A moderate, well-reasoned adjustment makes sense once diagnosis and monitoring clearly show that the node hardware actually has more heap available than the current limits use, and the triggering requests have already been reviewed and classified as genuinely necessary. In that case, limits get adjusted via the cluster settings API step by step, under close observation, never in one big jump.

PUT _cluster/settings
{
  "persistent": {
    "indices.breaker.request.limit": "50%",
    "indices.breaker.fielddata.limit": "30%"
  }
}

9. Preventive measures instead of pure limit tuning

More sustainable than raising breaker limits is consistently avoiding memory-heavy request patterns in the first place: sorting and aggregating exclusively on keyword fields or dedicated doc values fields instead of analyzed text, plus sensible, explicit size limits on every terms or composite aggregation.

A regular look at the slow query logs also helps spot request patterns early that could potentially lead to a tripped breaker, before they actually escalate under real load. This preventive stance reduces the need to touch breaker limits at all, keeping the built-in safety margin to the JVM memory ceiling fully intact.

10. Summary

11. FAQ: Circuit Breakers in Elasticsearch: The Essentials at a Glance