Implementing Autocomplete and Suggest Features in Elasticsearch

1. Why autocomplete is not a simple prefix match

At first glance, autocomplete looks like a trivial feature: the user types "hik" and the application should suggest "hiking boots". In practice, however, this hides a demanding requirement around latency, relevance ranking and error tolerance. A naive wildcard query with a trailing asterisk works for small data volumes, but scales catastrophically, because wildcard queries on large indices practically have to scan every term instead of navigating efficiently through a prepared data structure.

Elasticsearch offers three fundamentally different approaches for autocomplete, each with its own tradeoffs between latency, storage cost, flexibility and implementation effort: the Completion Suggester with a dedicated in-memory data structure, the Edge N-Gram approach with pre-generated prefix tokens in the inverted index, and the newer search_as_you_type field type, which packages edge n-gram logic into a simpler mapping construct. These three approaches for suggest features differ fundamentally in how they work internally, even though they produce similar results on the surface.

Choosing the right approach depends heavily on the use case: does the autocomplete feature need fuzzy matching for typos, does it need to be filterable by context, for instance by category or language, and how high is the expected search volume? The following sections provide a solid decision basis for autocomplete and suggest in Elasticsearch.

2. The Completion Suggester: structure and mapping

The Completion Suggester is the most specialized solution for autocomplete in Elasticsearch. It is based on a special data structure, a finite state transducer (FST), which is kept entirely in memory and therefore delivers extremely low latency for prefix queries, typically in the low single-digit millisecond range. The field type in the mapping is called completion and requires its own dedicated structure with an input array and an optional weight field for ranking.

The Completion Suggester is not queried through the normal query structure, but through the separate suggest endpoint, which uses its own syntax with prefix and field. This separate API concept is an important difference from the other two approaches, which work as normal query clauses.

PUT /products
{
  "mappings": {
    "properties": {
      "title_suggest": {
        "type": "completion",
        "analyzer": "simple",
        "preserve_separators": true,
        "preserve_position_increments": true,
        "max_input_length": 50
      }
    }
  }
}

PUT /products/_doc/1
{
  "title_suggest": {
    "input": ["Hiking Boots", "Trekking Boots", "Outdoor Shoes"],
    "weight": 34
  }
}

POST /products/_search
{
  "suggest": {
    "title-suggest": {
      "prefix": "hik",
      "completion": {
        "field": "title_suggest",
        "size": 5
      }
    }
  }
}

An important drawback of the Completion Suggester: the underlying FST structure exclusively supports prefix matching at the beginning of the respective input entry, no matching in the middle of a word or in arbitrary order. Anyone who types "boots" and wants to find "Hiking Boots" must store "Boots" as a separate entry in the input array, because the suffix alone does not automatically match.

3. The Edge N-Gram approach as an alternative

The Edge N-Gram approach follows a fundamentally different strategy: instead of a dedicated in-memory structure, all prefixes of a word are stored as separate tokens in the regular inverted index at indexing time. From "hiking" this produces the tokens "h", "hi", "hik", "hiki" and so on up to the configured maximum length. A search query for "hik" then directly finds the pre-generated token "hik" in the index, enabling a normal, very fast term search.

The decisive advantage of the Edge N-Gram approach: it works with the completely normal _search query syntax and can be combined freely with other filters, aggregations and bool clauses, which is not possible with the Completion Suggester's separate API. The downside is higher storage cost in the index, because several tokens are stored for every word instead of a single one, plus a tendency toward slightly higher latency compared to the specialized FST structure of the Completion Suggester.

PUT /products_edgengram
{
  "settings": {
    "analysis": {
      "filter": {
        "edge_ngram_filter": {
          "type": "edge_ngram",
          "min_gram": 2,
          "max_gram": 15
        }
      },
      "analyzer": {
        "autocomplete_index": {
          "type": "custom",
          "tokenizer": "standard",
          "filter": ["lowercase", "edge_ngram_filter"]
        },
        "autocomplete_search": {
          "type": "custom",
          "tokenizer": "standard",
          "filter": ["lowercase"]
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "title": {
        "type": "text",
        "analyzer": "autocomplete_index",
        "search_analyzer": "autocomplete_search"
      }
    }
  }
}

GET /products_edgengram/_search
{
  "query": { "match": { "title": "hik" } }
}

The separate configuration of index-time and search-time analyzer is decisive here: the index analyzer generates the edge n-gram tokens, while the search-time analyzer leaves the search term unchanged. If the same analyzer were used for both cases, n-grams would be generated from the search term itself, which would lead to incorrect, far too broad matches.

4. Completion Suggester vs. Edge N-Gram head to head

Both approaches solve the same basic task with different tradeoffs. The Completion Suggester scores on pure speed and lower storage cost for the plain autocomplete function, but loses flexibility because it cannot be combined with regular filters and only supports matching at the beginning of a word. The Edge N-Gram approach is more flexible and integrated into the normal search infrastructure, but costs more index storage and can easily produce misleading partial matches if misconfigured.

A practical decision criterion: if autocomplete needs to be filtered by context, for instance suggesting only products from a certain category or a certain warehouse, the Edge N-Gram approach or search_as_you_type is usually the more practical choice, because normal bool filters can be built directly into the same query. The Completion Suggester does support its own filtering logic via contexts, but this is less flexible than regular filter queries and has to be set up as a separate context field already at indexing time.

5. Fuzzy suggestions and context filtering

Typos are unavoidable in autocomplete, and both main approaches offer different solutions for this. The Completion Suggester supports a built-in fuzzy option directly in the suggest query, allowing a configurable edit distance. The Edge N-Gram approach can add fuzzy matching through a separate fuzzy or match query with fuzziness: AUTO, which however requires an additional request or a combined bool query with should clauses.

Context filtering with the Completion Suggester happens via contexts, a declared field in the mapping of type category or geo, filled with the matching context value at indexing time, for instance the product category. In the search query, this context is passed inside the completion block, so that only suggestions with a matching context are returned, without needing a separate filter query.

PUT /products
{
  "mappings": {
    "properties": {
      "title_suggest": {
        "type": "completion",
        "contexts": [
          { "name": "category", "type": "category" }
        ]
      }
    }
  }
}

PUT /products/_doc/2
{
  "title_suggest": {
    "input": ["Hiking Boots Men"],
    "contexts": { "category": ["shoes", "outdoor"] }
  }
}

POST /products/_search
{
  "suggest": {
    "title-suggest": {
      "prefix": "hik",
      "completion": {
        "field": "title_suggest",
        "fuzzy": { "fuzziness": 1 },
        "contexts": { "category": ["outdoor"] }
      }
    }
  }
}

6. search_as_you_type as a third option

Since Elasticsearch 7.2, the field type search_as_you_type offers a third, simplified option for autocomplete. Instead of manually configuring an edge n-gram analyzer, this field type automatically generates several internal sub-fields with different n-gram resolutions at indexing time, for instance field._2gram and field._3gram, taking over most of the manual configuration work required for the classic Edge N-Gram approach.

Querying happens through the special query type multi_match with type: bool_prefix, which automatically selects the matching sub-field for the current input length. This approach combines the integration into the normal query syntax of the Edge N-Gram approach with significantly reduced configuration effort, making it the pragmatic default choice for many new projects. A deeper comparison between search_as_you_type and the classic Completion Suggester regarding performance details follows in a dedicated, specialized article on this field type.

PUT /products_sayt
{
  "mappings": {
    "properties": {
      "title": {
        "type": "search_as_you_type"
      }
    }
  }
}

GET /products_sayt/_search
{
  "query": {
    "multi_match": {
      "query": "hik bo",
      "type": "bool_prefix",
      "fields": ["title", "title._2gram", "title._3gram"]
    }
  }
}

7. Weighting and ranking suggestions

Autocomplete suggestions without meaningful ranking are not very helpful when multiple entries match the same input. The Completion Suggester offers the built-in weight field for this, an integer value that directly determines the sort order of returned suggestions. In practice, this value is often derived from business metrics such as sales figures, popularity or click-through rate, and updated regularly, so that popular products appear higher in the autocomplete list.

For the Edge N-Gram approach and for search_as_you_type, ranking happens via the regular BM25 score of the underlying query, complemented with optional function score boosts for popularity or recency, exactly like a normal search. This offers more flexibility than the rigid weight field of the Completion Suggester, but also requires more configuration effort to achieve a comparably good ranking result.

8. Multi-language autocomplete strategies

Multi-language autocomplete poses a particular challenge, because language-specific characteristics such as accented characters and different word segmentation need to be taken into account. A proven approach is to set up separate fields per language for both the Completion Suggester and the Edge N-Gram approach, similar to multi-language stemming, and to direct the search query to the matching field depending on the user's language.

An additional aspect for German autocomplete implementations is the normalization of umlauts: an input "ae" should ideally also find matches with "ä" and vice versa. The asciifolding token filter or the more specialized german_normalization filter solve this problem by reducing both spellings to a common normal form at indexing time, regardless of which of the three autocomplete approaches is used.

9. Maintaining the autocomplete index: updates without downtime

Autocomplete data changes frequently, for instance when new products are added or sales figures change and the ranking weight needs to be recalculated. For smaller change volumes, a simple partial update of individual documents via the regular update API is enough. For more extensive restructuring, for instance a change to the analyzer configuration for the Edge N-Gram approach, a mapping update on an existing index is usually not possible, because analyzer settings are immutable after the initial creation of the index.

The robust solution for this is the established alias pattern: a new index with the changed configuration is built in parallel, the data is fully transferred via the Reindex API, and only after successful validation is the search alias atomically switched to the new index. This procedure guarantees that autocomplete requests keep working without interruption throughout the entire migration, because the old index stays active until the very last moment.

10. Summary

Elasticsearch offers three different ways to implement autocomplete and suggest features. The Completion Suggester scores with minimal latency through a dedicated in-memory structure, but is less flexible for filtering and combining with other queries. The Edge N-Gram approach integrates fully into the normal query syntax and allows arbitrary filter combinations, at the cost of more index storage. search_as_you_type combines the advantages of the Edge N-Gram approach with significantly reduced configuration effort and is the pragmatic default choice for many new projects.

Fuzzy matching, context filtering and multi-language support are possible with all three approaches, but differ in implementation effort and flexibility. For index updates to the autocomplete configuration, the alias pattern with the Reindex API has established itself as a robust path for migrations without downtime. The choice of the right approach should always be based on the concrete use case, not on a blanket preference.

11. FAQ: Implementing Autocomplete and Suggest