Configuring Field-Level and Document-Level Security in Elasticsearch

1. The baseline problem: one index, many visibility levels

A central product index shared by purchasing, sales, and customer service saves maintenance effort and guarantees consistent data across every department. That very setup becomes a problem the moment individual fields or entire documents must remain hidden from certain roles, for instance purchase prices, internal supplier terms, or orders belonging to a specific sales region. A second, fully separate index just for sensitive fields creates duplicate upkeep, synchronization effort, and extra joins on the application side, which tends to produce inconsistencies with every schema change.

Elasticsearch solves this directly inside the security API at the role level, without the application itself having to strip fields or clean up result lists afterward. Two mechanisms are available: field-level security operates on the level of individual field names within a document, document-level security operates on the level of entire documents through an automatically attached query filter. Both are maintained centrally in the role definition and apply automatically to every search request, regardless of which client or application issues it.

2. Field-level security: hiding fields per role deliberately

Field-level security is configured through the field_security object in a role definition and consists of two lists: grant defines which fields are visible for this role at all, usually as a wildcard expression like * for every field, while except deliberately removes individual fields from that set again. This combination allows maintaining a blocklist instead of an exhaustive allowlist with just a few lines, which is far less maintenance-heavy on indices with many fields than enumerating every permitted field individually.

Hidden fields appear neither in _source nor can they be searched, sorted, or used in aggregations for that role. They also never show up in highlighting, even if the search term happened to occur in the hidden field. Metadata fields such as _id, _index, or _score remain unaffected, since field-level security only operates on user-defined mapping fields.

PUT /_security/role/sales_role
{
  "indices": [
    {
      "names": ["products"],
      "privileges": ["read"],
      "field_security": {
        "grant": ["*"],
        "except": ["purchase_price", "supplier.*", "margin_calculation"]
      }
    }
  ]
}

3. Document-level security: an access filter through a query

Document-level security works fundamentally differently: instead of hiding individual fields, the role stores a query object that Elasticsearch automatically attaches as an additional filter to every search request made under that role. Technically this behaves like an implicit bool query with a filter clause evaluated ahead of every other query component. Documents that fail to match this query never appear for the affected role in any search, neither in hits, nor in aggregations, nor in the hits.total count.

Document-level security becomes particularly practical through template expressions that can reference the logged-in user's metadata, for example {{_user.metadata.region}}. This way a single role definition can serve many users, while each user, through their individual metadata, still only sees the documents relevant to them, without needing a separate role for every region or department.

PUT /_security/role/customer_service_region
{
  "indices": [
    {
      "names": ["orders"],
      "privileges": ["read"],
      "query": {
        "template": {
          "source": {
            "term": { "region": "{{_user.metadata.region}}" }
          }
        }
      }
    }
  ]
}

4. Practical example: purchase prices visible only to purchasing

In the practical scenario, two roles exist for the same product index. The role purchasing_role receives full field access without restriction, while sales_role hides the purchase price and supplier fields through field_security. Both roles access exactly the same physical documents, there is no second index and no copy of the data, so price changes are immediately consistent for both roles, whether visible or hidden.

If a user is assigned both purchasing and sales, Elasticsearch combines the field-level security grants additively as a union set, so the user ultimately sees every field that at least one of their roles permits. This additive logic matters for role design: an overly broad additional role can inadvertently expose fields that were meant to stay protected by another, more restrictive role.

PUT /_security/role/purchasing_role
{
  "indices": [
    { "names": ["products"], "privileges": ["read"] }
  ]
}

// A search request under sales_role never returns "purchase_price":
GET /products/_search
{ "query": { "match": { "name": "toolbox" } } }
// response contains only the public fields inside _source

5. Combining FLS and DLS: one role, two mechanisms

Field-level security and document-level security are not mutually exclusive, quite the opposite: in practice they are frequently used together within the same role definition. Elasticsearch first evaluates the document-level security query and reduces the result set to the documents visible for that role, then applies field-level security to the remaining hits and hides the configured fields. This ordering ensures that both mechanisms can be maintained independently without one affecting the behavior of the other.

A customer service agent could, through document-level security, see only orders from their own region while, through field-level security, having no access to stored payment data at all. Both rules live in the same role but are maintained independently, so a change to the region filter has no effect on field visibility and vice versa.

6. Effects on aggregations, scoring, and highlighting

Because document-level security applies as an implicit filter ahead of every request, aggregations automatically only consider the documents visible to the respective role. A sum aggregation over order values for the customer service role therefore automatically returns only the sum for that agent's own region, without the application needing to add extra filters into the aggregation request itself. This prevents a common data leak where individual documents are correctly filtered but aggregated metrics are accidentally computed over the entire data set.

For field-level security the rule is: an aggregation or sort on a hidden field fails for the affected role, since the field simply does not exist for it. Highlighting likewise only works on visible fields, a match inside a hidden field is neither highlighted nor factored into the score, so hidden fields cannot influence relevance ranking either.

7. Performance: query rewriting costs and caching of role queries

Document-level security rewrites the stored query into the actually executed query on every search request, which Elasticsearch speeds up through a dedicated bitset cache for role queries. Recurring, identically formulated role queries get cached as a bitset and do not need to be re-evaluated on every request. With a very large number of individually distinct roles, for instance a separate query per single user instead of per department, this cache grows quickly and its hit rate drops, generating additional CPU load.

Field-level security, by contrast, causes practically no measurable runtime overhead, since filtering only happens when assembling the response and does not touch the actual search algorithm at all. It is therefore advisable to model roles as coarsely as possible around functional groups rather than per individual user, both for maintainability and to keep the role query cache effective.

8. Debugging pitfalls: nested fields, runtime fields, stacked roles

Document-level security queries against nested fields must be formulated as a proper nested query, otherwise the filter does not behave as expected and returns either too many or no hits at all, since a plain term query against a nested field structurally fails to match. Runtime fields occasionally collide with field-level security too: a script that internally references a field actually locked for that role can produce inconsistent results, since access protection is not always consistently propagated into every runtime script depending on the Elasticsearch version.

When a user is assigned multiple roles, Elasticsearch combines document-level security queries with a logical or, which can lead to unexpectedly broad visibility if one of the roles is configured more generously than intended. To test a role configuration deliberately, the _security/user/_has_privileges endpoint and running test requests with a dedicated test user are the right tools before a role goes into production use.

9. Rollout strategy and auditing through the security API

A staged rollout is advisable: first define a small number of coarse roles along the main departments, verify them with test users and realistic search requests, and only then refine further by individual field or region. Every new role should be tested against concrete sample documents before going live, to make sure neither too much nor too little is visible.

For ongoing oversight, Elasticsearch offers audit logging, which records which role issued which request and which access decision applied. A regular review of role assignments, especially after organizational changes such as department transfers, closes a common security gap where previously correct roles remain too broadly scoped over time.

10. Summary

11. FAQ: Field-Level and Document-Level Security: The Essentials at a Glance