Dynamic Mapping: Spotting the Risks and Controlling Them Deliberately

1. What dynamic mapping is and how it works

Dynamic mapping is the mechanism by which Elasticsearch automatically creates a mapping for previously unknown fields while indexing a document. When a field arrives without an existing mapping definition, Elasticsearch analyzes the JSON value, infers a suitable data type, and extends the index mapping with exactly that field. This mechanism is active by default and makes Elasticsearch pleasantly easy to use at the start of a project, because no mapping has to be prepared before the first documents can be indexed.

That very convenience, however, is also the root of the problem this article covers. Dynamic mapping makes its decisions per document and without knowledge of the overall data structure. When input data comes from external sources, user input, or evolving APIs, dynamic mapping can silently generate dozens or hundreds of new fields per day. The following sections show how automatic detection works in detail, where the actual danger lies, and which tools rein it in deliberately.

2. Automatic type detection in detail

The type detection of dynamic mapping follows fixed rules. A JSON string is by default indexed both as a text field with an analyzer and as a keyword sub field, so both full text search and exact filtering are possible. A JSON number without a decimal point becomes long, a number with a decimal point becomes float. A boolean value becomes boolean, a nested object becomes object. Strings matching a recognizable date pattern are interpreted as date, provided date_detection is enabled, which it is by default.

These heuristics work well for many cases, but regularly fail on edge cases. A numeric product ID with leading zeros gets detected as a number and loses those zeros. A string like "12345" in a field that was actually always meant to be numeric can be typed inconsistently depending on the first observed value. And a string that happens to look like a date, for example a version number formatted "2024.01", can be mistakenly interpreted as a date field by dynamic mapping, which then causes indexing errors for every following document with a different format.

// First document determines the auto-detected mapping
POST /events/_doc
{
  "user_id": "42",
  "event": "login",
  "score": 3.5,
  "occurred_at": "2026-07-24T10:15:00Z"
}

// Elasticsearch infers roughly this mapping:
// user_id     -> text + keyword sub-field
// event       -> text + keyword sub-field
// score       -> float
// occurred_at -> date

3. The risk of field explosion from unpredictable data

The practically biggest risk of dynamic mapping is what is called field explosion or mapping explosion: when incoming documents have a variable structure that is not known in advance, for example because a JSON object uses a user generated ID as a key, dynamic mapping creates a separate field for every single key. An event tracking system storing metadata as {"attr_" + user_id: value} can produce hundreds of new fields per day this way, without anyone actively intervening.

Every field in the mapping creates additional overhead in cluster state management, which is kept in memory on every node. An index with tens of thousands of fields measurably slows down cluster state updates and can, in extreme cases, destabilize the entire cluster, because every mapping change has to be replicated to all nodes. Elasticsearch therefore caps the field count per index at 1000 by default, but in uncontrolled scenarios with dynamic mapping this limit is often simply raised instead of fixing the underlying problem.

A second, more subtle risk: every automatically generated field remains in the mapping permanently, even if it only occurred in a single document. Because field types are immutable, a single incorrectly typed document that created a wrong field via dynamic mapping can later cause correct documents with the same field name but a different intended type to fail.

4. dynamic: false, ignoring unknown fields

The simplest control measure is the setting "dynamic": false at index or object level. With this setting, fields not explicitly defined in the mapping are silently ignored during indexing: they are stored in the _source document and thus remain retrievable, but they are not indexed and are unavailable for searches, filters or aggregations. This is the right choice when part of the document structure needs to stay variable and does not need to be searchable, for example a generic "extra_data" object for debug information.

"dynamic": false can be set granularly per object field, so that most of the document still benefits from dynamic mapping while a single problematic area is sealed off. This targeted application is often the better choice in practice compared to a blanket deactivation at index level, because it curbs the risk of field explosion without sacrificing the convenience of dynamic mapping for the rest of the well structured data.

PUT /events
{
  "mappings": {
    "properties": {
      "user_id": { "type": "keyword" },
      "event":   { "type": "keyword" },
      "score":   { "type": "float" },
      "metadata": {
        "type": "object",
        "dynamic": false
      }
    }
  }
}

// Any key inside "metadata" is stored but never indexed or searchable
POST /events/_doc
{
  "user_id": "42",
  "event": "login",
  "score": 3.5,
  "metadata": { "session_hash_abc123": "value", "custom_flag_xyz": true }
}

5. dynamic: strict, rejecting unknown fields

Where "dynamic": false silently ignores unknown fields, "dynamic": "strict" goes a step further: a document with a field not defined in the mapping is rejected entirely, and the index request fails with a clear error. This is the right setting for indices where an unexpected field indicates a bug in the writing application and should surface immediately, instead of quietly accumulating new fields in the background.

In practice, strict is particularly suited to systems with a clearly defined, contractually fixed data schema, for example log formats produced by a fixed set of internal services. The downside: every legitimate extension of the data model requires an explicit mapping change beforehand, before new data can be written. This raises the coordination overhead between application development and search infrastructure, but reliably prevents dynamic mapping from growing out of control.

6. Dynamic templates: steering automatic detection deliberately

Dynamic templates are the most nuanced control mechanism: instead of disabling dynamic mapping entirely, they define rules that assign newly detected fields to a specific type. A rule could, for example, specify that any field whose name ends in _id is automatically indexed as keyword instead of text, regardless of the detected JSON type. This combines the flexibility of automatic field detection with the control of explicit type choice.

Dynamic templates are evaluated by criteria such as match_mapping_type (the JSON type detected by Elasticsearch), match (a pattern for the field name) or path_match (a pattern for the full field path in nested objects). Multiple templates are checked in order, and the first match wins. This technique is especially valuable for semi structured data where the rough structure is known but individual field names vary, for example custom attributes in a product catalog.

PUT /catalog
{
  "mappings": {
    "dynamic_templates": [
      {
        "ids_as_keyword": {
          "match": "*_id",
          "mapping": { "type": "keyword" }
        }
      },
      {
        "strings_no_analysis": {
          "match_mapping_type": "string",
          "match": "*_code",
          "mapping": { "type": "keyword" }
        }
      },
      {
        "unmapped_json_as_flattened": {
          "path_match": "custom_attributes.*",
          "mapping": { "type": "flattened" }
        }
      }
    ]
  }
}

7. Field limits as an additional safety net

Beyond content based control through dynamic settings and dynamic templates, Elasticsearch offers hard limits as a last safety net. The index setting index.mapping.total_fields.limit caps the total number of fields per index and defaults to 1000. Once this limit is reached, further indexing attempts with new fields fail, which serves as a clear warning sign that dynamic mapping is generating fields in an uncontrolled way.

Additionally, index.mapping.depth.limit caps the maximum nesting depth of objects, and index.mapping.nested_fields.limit caps the number of nested fields per index. These limits should not be misunderstood as a solution for a poorly controlled data structure, but rather as a last line of defense that makes a mistake visible before it destabilizes the whole cluster. Silently raising a limit without fixing the cause of the field explosion only pushes the problem into the future and makes it bigger there.

8. Spotting field explosion before it becomes a problem

A growing field count often goes unnoticed for a long time, because it does not show up as an obvious error message but gradually degrades cluster performance. The mapping API returns the current field count directly: GET /index_name/_mapping can be parsed by a script and the number of field keys counted. Regular monitoring of this metric, for example daily via a cron job, makes a sudden increase visible before the field limit is hit.

An additional warning sign is the size of the cluster state response via GET /_cluster/state, which noticeably grows and takes longer to transfer with very many fields. In production systems it is worth setting up an alert that triggers automatically when a threshold is exceeded, for example 500 out of 1000 allowed fields. This leaves time to investigate the cause and retrofit a dynamic template or a dynamic: false rule before dynamic mapping actually endangers the cluster.

# Count fields currently defined in an index mapping
curl -s "https://localhost:9200/events/_mapping" -u elastic:changeme \
  | jq '[.. | objects | select(has("type"))] | length'

# Check the configured field limit for an index
curl -s "https://localhost:9200/events/_settings/index.mapping.total_fields.limit?pretty" \
  -u elastic:changeme

9. A pragmatic dynamic mapping strategy

In practice, a combination of several protection layers works better than a single measure. For the core of a document whose structure is known and stable, explicit mapping without dynamic mapping is recommended. For clearly bounded but content wise variable areas like custom attributes, dynamic templates with clear naming patterns are suitable. For truly unstructured supplementary data that never needs to be searched, dynamic: false at object level is the right choice, combined with total_fields.limit as a last safety net.

dynamic: strict pays off mainly in systems with high data quality requirements, where an unexpected field genuinely means a bug in the writing application. The type flattened, available since Elasticsearch 7.3, is an often overlooked alternative for variable key value structures: it indexes an entire object as a single field entry in the mapping regardless of how many keys it contains, structurally preventing field explosion, though with more limited query capabilities compared to full dynamic mapping.

For Magento operators using the Elasticsearch catalog index, it is relevant that custom attributes with dynamic values, for example configurable technical specifications, carry the same field explosion risk as in a custom built index. Anyone regularly creating new product attributes should periodically check how many fields the generated catalog index actually contains, instead of blindly trusting the default Magento configuration.

This comparison shows that none of the settings is universally correct. The choice depends on how predictable the data structure is and how critical searchability is for the specific data area. A deliberate, documented decision per index is always better than unreflectively keeping the default behavior of dynamic mapping.

10. Summary

Dynamic mapping is a useful tool for getting started quickly, but it carries a real risk with variable or external data sources: uncontrolled growth of the field count all the way to field explosion, which endangers cluster performance and stability. Automatic type detection follows fixed but not always fitting heuristics, and because field types are immutable, an incorrectly created field stays in the mapping permanently.

The control mechanisms dynamic: false, dynamic: strict and dynamic templates allow steering behavior deliberately per area of a document, instead of switching dynamic mapping on or off wholesale. Field limits and regular monitoring of the field count form the last safety net. Anyone combining these tools keeps the convenience of automatic field detection where it makes sense and prevents the typical field explosion where input data is unpredictable.

11. FAQ: Dynamic Mapping in Elasticsearch