HNSW and FLAT index for embedding vectors
Semantic product recommendations based on similarity rather than exact attribute filters need a way to efficiently search embedding vectors by closeness. Redis Vector Search, built on top of RediSearch, brings exactly this capability through HNSW and FLAT indexes directly into the Redis infrastructure that is likely already in place.
Table of Contents
- 1. Why Classic Filters Are Not Enough for Semantic Similarity
- 2. Embeddings as the Foundation of Similarity Search
- 3. HNSW Index: Approximate Search for Large Data Volumes
- 4. FLAT Index: Exact Search for Smaller Data Volumes
- 5. Distance Metrics: Cosine, Euclidean, and Inner Product
- 6. Practical Example: Semantic Product Recommendations
- 7. Hybrid Search: Combining Vector Similarity with Classic Filters
- 8. Distinction from Dedicated Vector Databases
- 9. Operating and Scaling Vector Indexes
- 10. Summary
- 11. FAQ
1. Why Classic Filters Are Not Enough for Semantic Similarity
Classic Redis queries, even with RediSearch, filter on exact or numeric criteria: a category, a price range, a tag. For the question of which products are semantically similar to a given product, based on description text, an image, or user behavior, such filters are not enough, because semantic similarity cannot be expressed through exact attribute matching.
Embedding models solve this problem by converting text, images, or other content into high-dimensional vectors, where semantically similar items lie close together in the vector space. Redis Vector Search makes it possible to compute exactly this closeness efficiently, without having to move the vectors into a separate system.
2. Embeddings as the Foundation of Similarity Search
An embedding is a numeric vector of fixed length, typically several hundred to over a thousand dimensions, generated from a piece of content by a machine learning model. Two products with a similar embedding vector are considered semantically similar, measured for example through the cosine distance or the Euclidean distance between the two vectors.
Redis itself does not generate embeddings, it only stores and searches them. The actual embedding generation happens outside, for example through a separate machine learning model that converts product descriptions or images into vectors, which are then stored as an additional field of a hash or JSON document in Redis.
redis-cli HSET product:4711 name "Basic Shirt Blue" embedding "\x3f\x8c...binary..."
3. HNSW Index: Approximate Search for Large Data Volumes
HNSW, short for Hierarchical Navigable Small World, is a graph-based index that organizes vectors across several layers with different connection densities. A search query navigates from a coarse upper layer step by step down to increasingly finer lower layers, quickly finding a good approximation of the actually nearest vectors without having to compare against every vector in the index.
The advantage lies in scalability: HNSW queries stay in the low millisecond range even with several million vectors, because search complexity grows logarithmically rather than linearly with data volume. The price is approximation, HNSW does not always guarantee the mathematically exact nearest neighbors, but in practice it delivers a very good approximation at configurable accuracy.
redis-cli FT.CREATE idx_products_vector ON HASH PREFIX 1 product: \
SCHEMA embedding VECTOR HNSW 6 TYPE FLOAT32 DIM 768 DISTANCE_METRIC COSINE
4. FLAT Index: Exact Search for Smaller Data Volumes
As an alternative to HNSW, Redis Vector Search also offers a FLAT index that performs an exhaustive, exact search over all stored vectors. Every query actually compares against each individual vector in the index, which guarantees the mathematically exact nearest neighbors but grows linearly with the number of vectors.
FLAT is therefore mainly suited for smaller data volumes in the range of a few thousand to the low tens of thousands of vectors, or for use cases where absolute accuracy matters more than maximum speed, such as a carefully curated product selection. For larger catalogs with hundreds of thousands of items, HNSW is usually the more practical choice due to its better scaling.
5. Distance Metrics: Cosine, Euclidean, and Inner Product
Redis Vector Search supports several distance metrics, which fit differently well depending on the embedding model. Cosine distance measures the angle between two vectors and ignores their absolute length, which makes it the most common choice for text embeddings, where the direction of the vector carries the semantic meaning.
Euclidean distance, in contrast, measures the actual geometric distance in the vector space and suits embedding models where the absolute vector length also carries meaning. Inner product distance, finally, is often used in recommendation systems where both the direction and the strength of a match are relevant. Which metric fits depends on the specific embedding model and should follow its documentation rather than a blanket default choice.
6. Practical Example: Semantic Product Recommendations
An obvious use case is a recommendation feature on a product detail page: instead of building recommendations solely from the same category or frequently co-purchased items, vector search can compute content similarity based on product description, image features, or a combination of both. A user viewing a blue summer jacket then also receives stylistically similar jackets in other colors as a recommendation, even if they are filed under different categories.
Technically, this runs through a KNN query, short for K-Nearest-Neighbors, which returns the K most similar vectors in the index for a given vector. This query can additionally be combined with classic RediSearch filters, for example to only allow products within a certain price range or with sufficient stock as a recommendation, instead of showing pure similarity regardless of availability.
redis-cli FT.SEARCH idx_products_vector \
"(@price:[10 100])=>[KNN 5 @embedding $vec AS score]" \
PARAMS 2 vec "\x3f\x8c...binary..." \
SORTBY score ASC DIALECT 2
7. Hybrid Search: Combining Vector Similarity with Classic Filters
The real practical value of Redis Vector Search often only emerges in combination with the classic RediSearch capabilities of text filters, numeric ranges, and tags. A pure vector search without filters returns similar products, but possibly also sold-out or price-wise unsuitable items, while a hybrid query merges semantic similarity and hard business rules into a single step.
This combination reduces the need to run vector search and classic filter logic in separate systems and merge the results in the application afterward. For many mid-sized product catalogs, this is a noticeable architectural advantage over a separate vector database next to the existing search system.
8. Distinction from Dedicated Vector Databases
Dedicated vector databases such as Milvus, Pinecone, or Weaviate are built exclusively for vector search and therefore offer partly more sophisticated index variants, better horizontal scaling across very large vector volumes, and specialized features such as quantization for further memory reduction. For applications with many tens of millions or more vectors and a dedicated machine learning focus, these systems are often the technically more mature choice.
Redis Vector Search, in turn, shines where vector search is only one of several requirements and Redis infrastructure for cache, session, or classic search is already in place anyway. Instead of running an additional system for one part of the application, vector search can be integrated directly alongside the rest of the Redis data types, with the same operational experience and the same operational processes.
9. Operating and Scaling Vector Indexes
Vector indexes are memory intensive: a single embedding vector with 768 dimensions at float32 precision already takes up more than three kilobytes of raw vector data, on top of the HNSW graph overhead for the connection structure between nodes. For a catalog with a million products, this quickly adds up to several gigabytes that must be fully held in the memory of the Redis instance.
For capacity planning, a realistic upfront estimate based on vector dimension, number of products, and the HNSW parameters for connection count and search depth is therefore worthwhile, since these directly affect memory usage and search accuracy. As with the other Redis Stack modules, a dedicated instance for vector-heavy workloads, separate from the classic cache or session backend, is also recommended to avoid mutual interference under memory pressure.
| Criterion | HNSW Index | FLAT Index |
|---|---|---|
| Accuracy | Approximate, configurable via search parameters | Exact, guarantees the nearest neighbors |
| Speed at large volumes | Grows logarithmically, very fast | Grows linearly, slower with many vectors |
| Recommended data volume | From several tens of thousands of vectors upward | Up to the low tens of thousands of vectors |
| Memory overhead | Additional graph overhead per node | Only the raw vector data |
| Updatability | Ongoing inserts and deletes possible | Ongoing inserts and deletes possible |
Mironsoft
Cache layer setup and Magento Redis integration
Magento cache that isn't quite working or is misconfigured?
We set up Redis as a cache and session backend for Magento cleanly, tune memory usage and eviction strategies, and make sure full page cache and session storage work together reliably.
Redis Setup
Configure the cache, session, and FPC backend production-ready for Magento.
Memory Tuning
Match memory usage and eviction policies to the shop's actual load.
High Availability Setup
Set up Redis Sentinel or Cluster for resilient Magento environments.
10. Summary
Redis Vector Search: The Essentials at a Glance
Core Idea
Similarity search on embedding vectors directly through RediSearch
Core Commands
FT.CREATE with a VECTOR field, FT.SEARCH with a KNN query
Typical Use
Semantic product recommendations combined with classic filters
Limits
High memory requirements, dedicated vector databases superior at very large volumes