Nested vs. Object Data Types: The Difference That Matters

1. Why arrays of objects are special in Elasticsearch

Elasticsearch is internally built on Lucene, and Lucene's basic structure has no concept of true nested documents, only flat value lists per field. A single object in a document maps cleanly, but an array of objects, for example several variants of a product each with its own color and size, confronts Elasticsearch with a fundamental design decision: how should the relationship between fields within each individual array element be stored?

This is exactly where the two field types object and nested differ fundamentally, even though both accept the same JSON structure at first glance. Anyone unaware of this difference often builds a mapping that works fine with the first test document and only produces unexpected search hits once several array entries with different values are involved. This article explains what happens internally, when the difference actually becomes relevant, and how to make the right choice.

2. How the object type flattens arrays internally

The default type object stores an array of objects by creating a separate, flat value list for each field inside the objects. A document with an array of two variants, one with color "red" and size "S", one with color "blue" and size "L", is stored internally roughly like this: the field color contains the values ["red", "blue"] and the field size contains ["S", "L"], but with no information whatsoever about which color value belongs to which size value.

This flattening is not an arbitrary limitation, it is a direct consequence of how Lucene indexes documents: a Lucene document consists of named fields, each with a list of values, there is no native support for nested structures with a preserved association. Elasticsearch simulates nested JSON structures for the type object by simply merging the field paths, without preserving the original grouping. For many use cases this is unproblematic, but as soon as a query needs to constrain multiple fields within the same array element at once, the lost association becomes a problem.

PUT /products
{
  "mappings": {
    "properties": {
      "name":     { "type": "text" },
      "variants": { "type": "object" }
    }
  }
}

POST /products/_doc
{
  "name": "T-Shirt",
  "variants": [
    { "color": "red",  "size": "S" },
    { "color": "blue", "size": "L" }
  ]
}

// Internally flattened, roughly equivalent to:
// variants.color -> ["red", "blue"]
// variants.size  -> ["S", "L"]
// The link between red+S and blue+L is lost

3. Cross-object matching: the classic bug in practice

The direct consequence of object's flattening is a phenomenon known in the Elasticsearch community as cross-object matching. A query filtering for "color red AND size L" should, in the example from the previous section, return no result, because neither variant has that combination. With the type object, however, that very query does return a hit, because "red" and "L" both exist somewhere in their respective flat value lists, even though they belong to different array elements.

This bug is particularly insidious because it only shows up with realistic amounts of data involving several array entries per document. A test document with only one variant always works correctly, because there is no second combination for values to blend into. Only in production, once products with three, five or ten variants get indexed, do search hits suddenly appear that make no sense content wise, leaving a team facing a hard to trace bug that is rooted deep in the field type, not in query logic.

// This query with type "object" WRONGLY matches the T-Shirt document
// because "red" and "L" both exist somewhere in the flattened arrays,
// even though they belong to different variants
GET /products/_search
{
  "query": {
    "bool": {
      "must": [
        { "match": { "variants.color": "red" } },
        { "match": { "variants.size": "L" } }
      ]
    }
  }
}

4. How the nested type preserves relationships

The field type nested solves the problem by storing every element of an array as its own, hidden Lucene document within the same segment, instead of merging the fields flat. Internally this creates a main document for the product and a separate hidden document per variant, each with the association between color and size fully preserved within that one element. These hidden documents are linked to the main document via an internal block join mechanism and are not directly visible in normal queries.

This architecture is why nested reliably prevents cross-object matching: a query against a nested field is always evaluated against exactly one of these hidden documents, never against a flat, merged value list. The price for this is structural: every additional array element creates an additional Lucene document, which directly affects storage usage and indexing effort, especially for arrays with many entries.

PUT /products
{
  "mappings": {
    "properties": {
      "name":     { "type": "text" },
      "variants": {
        "type": "nested",
        "properties": {
          "color": { "type": "keyword" },
          "size":  { "type": "keyword" }
        }
      }
    }
  }
}

// Same source document, now correctly kept as separate hidden documents
POST /products/_doc
{
  "name": "T-Shirt",
  "variants": [
    { "color": "red",  "size": "S" },
    { "color": "blue", "size": "L" }
  ]
}

5. The nested query: querying objects individually

A nested field cannot be queried directly with the usual query clauses like match or term, because those are designed for flat fields. Instead, the special nested query is required, which expects a path parameter naming the nested field and an inner query that is evaluated exclusively within a single hidden document. This encapsulation is exactly the mechanism that reliably prevents cross-object matching.

Multiple conditions within the same nested query, for example via a bool clause with several must conditions, must all be satisfied within the same array element to produce a hit. If instead two separate nested queries with the same path are combined within a parent bool, they can again match against different array elements, because each nested query is evaluated on its own. This detail is a common source of mistakes, even when the field type was correctly set to nested.

// Correct: both conditions must match within the SAME variant
GET /products/_search
{
  "query": {
    "nested": {
      "path": "variants",
      "query": {
        "bool": {
          "must": [
            { "term": { "variants.color": "red" } },
            { "term": { "variants.size": "L" } }
          ]
        }
      }
    }
  }
}
// Correctly returns zero hits for the T-Shirt document

6. Aggregating over nested fields

Aggregations over nested fields also need their own syntax, because the hidden documents sit outside the normal aggregation context. The nested aggregation is the entry point that switches the context to the specified path, within which normal sub aggregations like terms or avg can then operate on the fields of the hidden documents. Without this switch, the fields inside a nested object would not be reachable for aggregations at all.

A common use case is counting how often each color occurs across all variants of all products, while a filter condition on the document level remains active, for example only for products in a certain category. The reverse_nested aggregation additionally allows switching back from the nested context to the main document, which becomes important for multi level evaluations, for example when a product has both variants and categories as separate nested fields.

7. Performance and storage cost of nested

Nested solves the cross-object matching problem structurally correctly, but it does not come for free. Every array element creates an additional hidden Lucene document, which for arrays with hundreds of entries per main document can noticeably increase storage usage and indexing time. Elasticsearch therefore caps the maximum number of nested objects per document by default via index.mapping.nested_objects.limit, to catch extreme cases.

Queries against nested fields are also more computationally expensive than against flat fields, because the block join mechanism has to navigate between the main document and hidden documents at runtime. For arrays with few entries, say up to a low double digit number, this overhead is usually negligible in practice. For very large arrays, on the other hand, it is worth carefully checking whether the correctness provided by nested is actually needed, or whether an alternative modeling approach, such as a separately linked index with a join field type or denormalization, is the better choice.

8. Decision criteria: when object, when nested

The choice between object and nested primarily depends on whether queries will ever need to constrain multiple fields within the same array element at once. If an array only contains a single field, for example a simple list of tags, the cross-object matching problem never arises in the first place, because there is no second dimension for values to blend into. In that case object or even a simple keyword array is sufficient and considerably cheaper.

As soon as an array consists of objects with two or more fields and a realistic query could check both fields against each other at once, for example color and size of a variant or price and validity date of a discount, nested is almost always the right choice once more than one array element per document is possible. The only exception is cases where the data volume per array is so large that the performance cost of nested exceeds the correctness benefit, which requires a deliberate trade off decision, not an automatic one.

For Magento operators using the Elasticsearch catalog index, it is relevant that configurable products with multiple options can internally use similar structures. Anyone building custom extensions that need to filter attribute combinations across multiple dimensions, for example color and size together, should check whether the underlying structure is actually modeled as nested, before mistaking unexpected cross-object hits for a bug in their own search logic.

9. Practical example: modeling product variants correctly

A complete example shows the practical implementation. A product catalog with variants, each having a color, size and a variant specific price, needs to correctly answer queries like "available in red and size M under 30 euros". With object, such a query would regularly return incorrect hits as soon as a product has multiple variants with different color, size and price combinations. With nested and the matching nested query, the association stays correct regardless of how many variants a product has.

Fully converting from object to nested, like any field type change, requires a reindex, because field types in the mapping are immutable. Anyone who already knows an array contains several fields that belong together, and that more than one element per document is realistic, should therefore choose nested from the start, instead of having to perform a costly reindex under production pressure once faulty search hits are first noticed.

This comparison makes the core decision tangible: as soon as fields within an array element need to be queried together, nested is the only structurally correct choice, while object stays more resource friendly for simpler cases.

10. Summary

The difference between object and nested is not a stylistic detail, it determines whether queries over arrays of objects return correct results. The type object flattens arrays internally and thereby loses the association between fields of different array elements, leading to cross-object matching and thus content wise incorrect search hits as soon as a document contains more than one array element.

Nested stores every array element as its own, hidden Lucene document and thereby preserves the relationship between fields, but requires the special nested query syntax and incurs more storage and indexing overhead. The right choice depends on whether queries will ever need to check multiple fields within the same element at once, a decision best made before the first production import, because a later switch requires a full reindex.

11. FAQ: Nested vs. Object in Elasticsearch