Security: Configuring Authentication and Roles Correctly

1. Why security is not optional from day one

An Elasticsearch or OpenSearch cluster without active security features is, by default, fully open through the REST API: anyone who reaches the network address can read data, write data, and delete entire indices without authenticating. In the past, exactly these kinds of unsecured clusters, accidentally reachable from the public internet, repeatedly led to major data leaks, because Elasticsearch listens on all network interfaces by default and offers no access protection whatsoever without additional configuration.

Since version 8, Elasticsearch enables security features such as TLS and password authentication by default at initial installation, which has significantly reduced the risk of unintentionally open clusters. At OpenSearch, the OpenSearch Security plugin is likewise preinstalled from the start, but depending on the distribution it must be explicitly enabled and configured. In both cases the rule holds: the default configuration is a starting point, not a finished solution. Roles, users, and network boundaries must be adapted to the specific use case.

This article covers the full chain from authentication methods through role-based access control, API keys, and TLS between nodes, all the way to audit logging and the most common misconfigurations that lead to security problems in practice.

2. Authentication methods overview

Elasticsearch supports several authentication realms in parallel: the native realm with internally stored users and passwords, LDAP and Active Directory realms for connecting to existing corporate directories, SAML and OpenID Connect for single sign-on scenarios, and Kerberos for Windows environments. For smaller deployments, the native realm is often sufficient, while larger organizations usually use LDAP or SAML to keep user management centralized in the existing identity provider instead of maintaining user accounts twice.

Alongside password-based user authentication, there is service-oriented authentication via API keys and service account tokens, designed specifically for machine access from applications, so an application server does not have to store a personal user password in plain text. This separation between human users with a full login and service access with restricted, revocable credentials is a central principle for a clean security architecture.

// Create a native user with the built-in Elasticsearch realm
POST /_security/user/catalog_service
{
  "password": "a-strong-generated-password",
  "roles": ["catalog_writer"],
  "full_name": "Catalog Import Service",
  "email": "ops@mironsoft.de"
}

// Check current realm configuration
GET /_security/_authenticate

3. Role-based access control: roles and privileges

Role-based access control, or RBAC, is the central permissions model in Elasticsearch and OpenSearch. A role defines cluster privileges such as creating indices or managing snapshots, as well as index privileges such as read, write, or delete for a certain set of indices, defined via name patterns. Users or service accounts are then assigned one or more roles, which allows granular permission combinations to be modeled without having to create a separate user type for every combination.

A common mistake is using a single, broad administrator role for all applications and services because it simplifies the initial setup. This practice contradicts the principle of least privilege and creates a significant risk: a compromised application server with administrator rights can manipulate the entire cluster instead of only affecting the indices that this one application actually needs. The correct approach is to define a dedicated, narrowly scoped role for every use case that only contains the privileges actually required.

// Narrow role: read and write only for product-related indices
POST /_security/role/catalog_writer
{
  "cluster": [],
  "indices": [
    {
      "names": ["products-*"],
      "privileges": ["read", "write", "create_index"]
    }
  ]
}

// Read-only role for a reporting dashboard
POST /_security/role/reporting_reader
{
  "cluster": ["monitor"],
  "indices": [
    { "names": ["products-*", "orders-*"], "privileges": ["read"] }
  ]
}

4. Field-level and document-level restrictions

Beyond plain index-level access, Elasticsearch allows fine-grained restrictions within an index: field-level security completely hides certain fields of a document from a role, for example personal data like an email address for a role that only needs aggregated evaluations. Document-level security goes even further and filters which individual documents are visible at all, based on a query stored as part of the role definition, for example only orders from a specific branch for a branch-scoped role.

Both mechanisms are especially relevant in multi-tenant scenarios, where several customers or departments share the same physical index but need strictly isolated views of the data. Instead of creating a dedicated index for every tenant, which would lead to impractical shard overhead with very many small tenants, document separation can be elegantly modeled through document-level security within a shared index, without any tenant ever being able to see another tenant's data.

5. API keys for applications and services

API keys are the recommended authentication method for applications, microservices, and automated scripts, because unlike a shared user password, they can be created individually, assigned their own role, time-limited, and revoked individually at any time without affecting other access, in contrast to a shared user password. Every API key optionally receives its own, even more restricted privileges at creation time, which can never exceed the rights of the creating user, preventing an API key from accidentally gaining more rights than intended.

A proven pattern is to create a dedicated API key with its own minimal role for every application and every environment, meaning staging and production separately. If a key is compromised, it can be revoked in isolation without affecting other applications or requiring a central password change that impacts multiple systems at once. API keys with an expiration date additionally enforce regular rotation, which reduces the risk of long-lived credentials forgotten in configuration files.

// Create a scoped, time-limited API key for a specific application
POST /_security/api_key
{
  "name": "catalog-import-prod",
  "expiration": "90d",
  "role_descriptors": {
    "catalog_writer_scoped": {
      "cluster": [],
      "indices": [
        { "names": ["products-2026-*"], "privileges": ["write", "create_index"] }
      ]
    }
  }
}

// Revoke a compromised or unused API key immediately
DELETE /_security/api_key
{
  "ids": ["VuaCfGcBCdbkQm-e5aOx"]
}

6. TLS between nodes: securing the transport and HTTP layers

Alongside authentication, encryption is the second load-bearing pillar of any security configuration. Elasticsearch distinguishes two separate communication layers, each requiring its own TLS certificates: the HTTP layer, over which external clients and applications communicate, and the transport layer, over which the nodes of a cluster communicate internally with each other. Without TLS on the transport layer, cluster-internal data, including sensitive document content during shard replication, can be read unencrypted on the network, even if the HTTP layer is secured externally.

For the transport layer, a self-signed certificate from a private internal certificate authority is usually sufficient, since only the cluster nodes themselves need to verify these certificates and no external clients are involved. For the HTTP layer, on the other hand, a certificate accepted by external clients without manual exception rules is recommended, for example from a publicly trusted CA or at least an internal CA distributed company-wide. Elasticsearch provides a tool called elasticsearch-certutil that generates both a private CA and node certificates with just a few commands.

# Generate a certificate authority for internal transport encryption
bin/elasticsearch-certutil ca --out /etc/elasticsearch/certs/ca.p12

# Generate node certificates signed by that CA
bin/elasticsearch-certutil cert \
  --ca /etc/elasticsearch/certs/ca.p12 \
  --out /etc/elasticsearch/certs/node.p12

# elasticsearch.yml: enable TLS on both transport and HTTP layers
# xpack.security.transport.ssl.enabled: true
# xpack.security.transport.ssl.verification_mode: certificate
# xpack.security.http.ssl.enabled: true

7. OpenSearch Security compared to X-Pack

OpenSearch uses the OpenSearch Security plugin, which originated from Search Guard, originally developed by Floragunn, and conceptually shares many similarities with Elastic X-Pack Security: role-based access control, TLS configuration, and support for LDAP, SAML, and OpenID Connect are present in similar form. The key difference is that these features are fully free under Apache 2.0 at OpenSearch, while at Elastic certain advanced features like SAML integration or field-level security remain partly reserved for paid license tiers.

The two systems differ in configuration details as well: OpenSearch Security defines roles and role mappings through YAML configuration files, loaded into the cluster with a securityadmin.sh script, while Elasticsearch manages roles primarily through the REST API or the Kibana interface. Anyone switching from Elasticsearch to OpenSearch therefore not only has to remap roles content-wise but also get used to a different management workflow.

# OpenSearch Security roles.yml, loaded via securityadmin.sh
catalog_writer:
  cluster_permissions:
    - "cluster_composite_ops"
  index_permissions:
    - index_patterns:
        - "products-*"
      allowed_actions:
        - "write"
        - "create_index"

8. Audit logging and traceability

Audit logging records security-relevant events such as failed authentication attempts, role changes, and denied access requests in a separate log file, independent of the normal application log. Without audit logging, an intrusion attempt or a misused privilege escalation often goes unnoticed, because neither cluster health nor the normal access logs specifically highlight such events. In regulated industries, audit logging is frequently also a compliance requirement, for example as evidence for an ISO 27001 or GDPR audit.

Enabled audit logging generates a noticeable log volume under high access frequency, which is why in practice usually only selected event categories are logged, for example authentication failures and authorization denials, while successful, routine read access is excluded to avoid flooding the log with unimportant entries. These logs should be exported into a central SIEM system, or at least into a separate, particularly protected Elasticsearch index, so they are available unaltered as evidence in the event of an incident.

9. Cluster hardening: avoiding common misconfigurations

The most common misconfiguration is running the cluster without security for a test and accidentally carrying that state over into production, because the initial setup works faster without authentication. A second classic mistake is permanently using the predefined elastic superuser for application access, instead of using it exclusively for initial administration and creating narrowly scoped, dedicated roles for everyday operations afterward.

A third common mistake concerns network exposure: a cluster listening on all interfaces via network.host: 0.0.0.0, even though only internal applications should have access, unnecessarily increases the attack surface. Combined with a firewall rule that only allows known application server IPs, and a strict RBAC configuration, a layered security model emerges in which the failure of a single protective layer does not immediately lead to full data access.

10. Summary

A resilient security configuration for Elasticsearch and OpenSearch consists of several interlocking layers: appropriate authentication realms for human users, narrowly scoped roles following RBAC principles instead of broad administrator access, API keys with expiration dates for machine access, TLS on both the transport and HTTP layers, and audit logging for traceability in a real incident.

The most important principle remains least privilege: every role, every API key, and every service account should receive exactly the privileges needed for the specific task, no more. Combined with network segmentation and regular review of existing roles, a security model emerges that keeps the damage contained even if individual components are compromised.

11. FAQ: Security, Authentication and Roles