Optimizing Dense Vector Mapping and Indexing: Dimension, Quantization, Memory

1. What a single vector actually costs in memory

A standard embedding model with 768 dimensions typically stores each value as a 32 bit floating point number, which amounts to roughly three kilobytes of raw data per vector, without counting the additional HNSW graph. For a catalog of five million products with just one embedding field per product, that already adds up to over fourteen gigabytes for the raw vectors alone, and multilingual catalogs with a separate embedding per language multiply that need accordingly.

This memory footprint affects not just disk, but above all working memory, since performant kNN search needs both the vectors and the HNSW graph to be held as fully as possible in the data nodes' memory. Anyone who does not deliberately choose the dimension or the storage strategy risks not just unnecessary cost, but noticeable performance degradation once available memory gets tight and parts of the data have to spill onto slower storage.

2. Dimension: not every model with more dimensions is better

A higher vector dimension generally captures more nuances of the original meaning, but demands proportionally more memory and more compute time for every distance calculation during search. Past a certain point, further increasing the dimension only yields marginal quality gains while memory and compute cost keep growing linearly, a classic diminishing returns phenomenon.

Many modern embedding models therefore explicitly offer reduced dimension variants, such as 384 instead of 768 or 1024 dimensions, produced through a training technique called Matryoshka Representation Learning, which deliberately concentrates the most important information in the first dimensions. For many product search use cases, such a reduced variant delivers nearly the same search quality at significantly lower memory cost, which absolutely should be tested before settling on a final model choice.

3. Quantization: from 32 bit floats to more compact formats

Quantization reduces the precision with which each individual vector value is stored, without changing the number of dimensions itself. Instead of a 32 bit floating point number per value, int8 quantization stores each value as an 8 bit integer, cutting memory cost per vector to a quarter, while int4 quantization at 4 bits per value gets down to an eighth of the original memory cost.

Elasticsearch performs this quantization automatically during indexing once the corresponding index_options type is set, and internally also keeps a more compact representation of the original values available for optional reranking, so that most of the search accuracy is preserved despite the reduced precision.

PUT products
{
  "mappings": {
    "properties": {
      "description_embedding": {
        "type": "dense_vector",
        "dims": 768,
        "index": true,
        "similarity": "cosine",
        "index_options": {
          "type": "int8_hnsw",
          "m": 16,
          "ef_construction": 100
        }
      }
    }
  }
}

4. int8 versus int4: memory savings against accuracy loss

int8 quantization is the solid default path in practice, since the accuracy loss against unquantized vectors is barely measurable for most product search use cases, while memory cost already drops to a quarter. int4 quantization goes a step further and halves memory cost again compared to int8, but demands more careful evaluation, since the additional precision loss can be more noticeable depending on the embedding model and dataset.

A practical approach is to start with int8 as the default and consider int4 specifically for very large catalogs or particularly memory constrained clusters, and before going to production a concrete recall comparison between unquantized, int8, and int4 variants should always be run on a representative sample of your own product data, rather than relying on general vendor figures.

5. Impact on indexing speed for large catalogs

Building the HNSW graph is the most compute intensive part of indexing, since for every new vector the nearest neighbors in the already existing graph have to be found and connections created according to the ef_construction parameter. During an initial full indexing run across several million products, that shows up clearly in the overall runtime, while quantization partially softens this effect because less data per vector has to be moved and compared.

For incremental updates of individual products, as they occur during ongoing Magento operation with every price change or description edit, the impact on a single document is small, but it accumulates with very frequent bulk updates across large parts of the catalog. A proven practice is to apply embedding updates not on every small change, but batched with a sensible batch size through the bulk API, rather than updating individual documents one at a time.

6. Oversampling and reranking to compensate for quantization loss

To offset the accuracy loss caused by quantization, Elasticsearch supports a two stage approach: the initial kNN search runs on the quantized vectors and returns a larger candidate set than actually needed, so called oversampling, and a subsequent reranking step re-evaluates these candidates using the more precise, unquantized values before the final top selection is returned.

This approach combines the speed and memory advantages of quantization for the broad first pass with the accuracy of unquantized values for the final ordering of the few actually returned results, and is often the best compromise between resource efficiency and search quality in practice, especially with int4 quantization, where the pure accuracy loss without reranking is more noticeable.

7. Multiple vector fields per document: planning the cost realistically

In a multilingual Magento shop, it seems natural to create a separate embedding field for every supported language, such as description_embedding_de and description_embedding_en, which multiplies total memory cost per document according to the number of languages. Before making such a decision, it is worth checking whether a single, cross lingually trained multilingual embedding model could deliver the same practical benefit with just one vector field.

If separate language fields are actually needed, for example because language specific models demonstrably deliver better results, cluster capacity planning should account for this multiplied memory need from the start, rather than discovering it only once a production memory bottleneck occurs.

8. Measurement: tracing memory and latency differences concretely

Before deciding on a particular dimension or quantization level, a simple but informative comparison test on a representative slice of your own product catalog pays off, indexing the same dataset once unquantized, once with int8, and optionally once with int4, each time measuring on disk index size, data node memory usage, and actual search latency under realistic query load.

These concrete numbers from your own environment are far more reliable than generic benchmark figures from Elastic's documentation, since index size, vector dimension, catalog structure, and hardware setup vary considerably from shop to shop, and the actual effect of an optimization only shows reliably on your own dataset.

9. Practical recommendation: a pragmatic starting point

For most Magento shops with a medium to large product catalog, a pragmatic starting point is a reduced embedding dimension between 384 and 768, combined with int8 quantization and default HNSW parameters, which already realizes most of the possible memory savings without noticeably degrading search quality.

Further optimizations such as int4 quantization or more aggressive dimension reduction only pay off once concrete cluster capacity limits are actually reached, or once the infrastructure cost calculation explicitly demands further reduction, and should then always be decided based on your own measured data, not on general recommendations alone.

10. Summary

11. FAQ: Optimizing Dense Vector Mapping: The Essentials at a Glance