Fine-Tuning Varnish, Redis and OpenSearch Sidecars for Magento

1. Why sidecar tuning goes beyond default Compose

A typical Magento Compose setup starts Varnish, Redis and OpenSearch as sidecar containers with the default settings from the official image. That is entirely sufficient for local development, but under production load with thousands of concurrent sessions and a large product catalog, the limits of the defaults show up quickly: Varnish caches too aggressively or too cautiously, Redis discards sessions once memory is full, and OpenSearch runs with a heap that is neither too small nor sensibly sized.

Fine-tuning sidecars means configuring each of these three services according to its actual role in the Magento stack, instead of relying on the generic factory defaults meant for arbitrary use cases. A Redis container that serves both sessions and cache at once needs different settings than a Redis dedicated solely to cache. A Varnish in front of a shop with high product variance needs different TTL values than one in front of a shop with few, rarely changing products.

This article walks through the three most important sidecars in the Magento container stack and shows concrete configuration values for Varnish, Redis and OpenSearch that go beyond the default settings and are oriented to the real load of a Magento shop.

2. Fine-tuning Varnish VCL: TTL, grace and hit rate

The default VCL that many Magento Docker images ship with sets a blanket TTL for all cached pages. In practice, category pages with frequently changing stock data should get a shorter TTL than static CMS pages. Varnish sidecar tuning starts by differentiating TTL per content type via the X-Magento-Tags headers that Magento already ships for cache invalidation.

The grace period is an underrated parameter in Varnish tuning: it allows serving an expired but still cached response while a fresh version is fetched in the background. Without a grace period, every request waits for the full backend round trip after TTL expiry, which causes a so called thundering herd problem during traffic spikes when many concurrent requests hit the same expired cache entry.

# default.vcl — differentiated TTL and grace period per content type
sub vcl_backend_response {
    # Category and product pages: shorter TTL, longer grace for stale-while-revalidate
    if (bereq.url ~ "^/catalog/category/view" || bereq.url ~ "^/catalog/product/view") {
        set beresp.ttl = 10m;
        set beresp.grace = 6h;
    }
    # Static CMS pages change rarely: longer TTL is safe
    elsif (bereq.url ~ "^/cms/") {
        set beresp.ttl = 24h;
        set beresp.grace = 24h;
    }
    # Default fallback for everything else
    else {
        set beresp.ttl = 1h;
        set beresp.grace = 1h;
    }

    # Keep serving stale content while Magento generates a fresh copy
    set beresp.keep = 24h;
}

The hit rate of a Varnish sidecar can be observed via varnishstat, in particular the cache_hit metric in relation to cache_miss. A hit rate below 85 percent for a catalog heavy shop almost always indicates TTLs that are too short or too many cache invalidations from product changes, not a fundamental problem with Varnish itself.

3. Varnish storage backend: malloc vs. file

Varnish supports two storage backends for the cache: malloc, which keeps the cache entirely in memory, and file, which uses a file on disk as backing store, managed by the operating system through the page cache. For a Varnish sidecar in a container, malloc is almost always the right choice, because container volumes for file mode bring additional I/O overhead and persistence issues on container restart.

The size of the malloc storage must match the actual cache working set of the Magento shop. An undersized storage causes premature eviction of already cached pages before their TTL has expired, unnecessarily lowering the hit rate. As a rule of thumb: storage size at least twice the sum of all unique page variants requested during a typical day, multiplied by the average response size.

4. Fine-tuning Redis: maxmemory policy and eviction

The single most important parameter in Redis sidecar tuning for Magento is maxmemory-policy. The Redis default value noeviction lets write commands fail with an error once the configured memory is full, which for a session Redis means customers suddenly cannot create a new cart. For cache Redis, allkeys-lru is the right policy, because it automatically evicts the least recently used entries as soon as new memory is needed.

For session Redis, on the other hand, volatile-lru combined with a set TTL on every session key is the safer choice, because it only evicts keys that have a TTL and never deletes keys without an expiry, which could lead to data loss with incorrectly configured sessions lacking a TTL. Anyone using Redis as a sidecar for both purposes at once should run cache and session in separate logical databases with a matching policy each, instead of choosing a single policy for everything.

# redis.conf — separate tuning for cache vs. session workloads
maxmemory 2gb
maxmemory-policy allkeys-lru

# Persistence tuned for a cache-heavy Magento workload
save ""
appendonly no

# Connection and timeout tuning for many short-lived PHP-FPM connections
timeout 300
tcp-keepalive 60
maxclients 10000

5. Redis persistence: AOF vs. RDB in containers

A pure cache Redis for Magento in most cases needs no persistence, because lost cache entries are simply recomputed from MySQL. For this use case, both RDB snapshotting via save and appendonly should be disabled, to avoid CPU and I/O load for unneeded persistence. A session Redis, on the other hand, needs at least moderate RDB snapshotting, so a container restart does not simultaneously delete all active carts and login sessions.

AOF (Append Only File) offers higher data safety than RDB because every write command is logged, but costs measurably more I/O per operation. For a Redis sidecar that holds only sessions, appendfsync everysec is a good compromise between safety and performance, while appendfsync always is only justified for payment relevant data, which in Magento lives in MySQL rather than Redis anyway.

6. Fine-tuning OpenSearch: JVM heap in the container

The most common misconfiguration for an OpenSearch sidecar in containers is a JVM heap that either claims the entire available container memory or is left at a generic default. The recommended rule of thumb is to set the heap to a maximum of 50 percent of the memory allocated to the container, so the remaining memory stays available for the Lucene filesystem cache, which matters for search performance at least as much as the heap itself.

Additionally the heap should never exceed 32 gigabytes, because above this threshold the JVM switches from compressed object pointers to uncompressed ones, which increases the effective memory consumption per object and can worsen performance despite more allocated memory. For most Magento shops a sensible heap for OpenSearch lies between 2 and 8 gigabytes, depending on catalog size and the number of concurrent search queries.

# docker-compose.yml — OpenSearch sidecar with tuned JVM heap
services:
  opensearch:
    image: opensearchproject/opensearch:2
    environment:
      - "OPENSEARCH_JAVA_OPTS=-Xms4g -Xmx4g"
      - "DISABLE_SECURITY_PLUGIN=true"
      - "discovery.type=single-node"
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    deploy:
      resources:
        limits:
          memory: 8g

7. Sizing OpenSearch shards and replicas for the Magento catalog

Magento creates a catalog index per store view with a fixed number of shards by default. Too many shards for a small catalog waste overhead, because every shard brings its own Lucene segments, its own file handles, and its own memory overhead. For most Magento shops with fewer than one million products per store view, a single primary shard per index is entirely sufficient, while larger catalogs can benefit from two to three shards.

The number of replicas affects both fault tolerance and read speed, because search queries can be distributed across all replica shards. For an OpenSearch sidecar running in single node mode, as is common in many Magento Docker setups, a replica count greater than zero makes no sense though, because replicas on the same node as the primary shard offer no failure protection. Only in a multi node OpenSearch cluster does replica configuration become relevant for load distribution and redundancy.

8. Connection pooling and timeouts between Magento and sidecars

Every PHP-FPM worker in Magento opens its own connections to Redis and OpenSearch as needed, which under high concurrency can quickly result in hundreds of simultaneous connections to the respective sidecar. Without a configured connection timeout and without a maxclients limit on the Redis side, hanging PHP-FPM workers can keep connections open until the sidecar itself hits its connection limit and starts rejecting new requests.

In Magento's env.php, the timeout values for Redis cache and Redis session should be set explicitly, instead of trusting the library defaults. A read_timeout of 2 to 5 seconds prevents a single overloaded Redis sidecar from blocking the entire PHP-FPM pool through hanging connections. The same principle applies to OpenSearch: the PHP client should get an explicit connect and read timeout configured, so a single slow search index does not block the entire category page.

9. Tuning parameters and monitoring metrics compared

Each of the three sidecars has its own critical metrics that should be watched in monitoring, to detect in time when the configuration no longer matches the actual load.

These four metrics should be a fixed part of every Magento monitoring setup, because they point to misconfiguration much earlier than symptoms like slow page load times or aborted checkouts, which only appear once a sidecar is already working at its limit.

10. Summary

Fine-tuning Varnish, Redis and OpenSearch as sidecars means above all critically questioning the default settings of the official images and replacing them with values that match the actual role in the Magento stack. Varnish needs differentiated TTLs per content type and a sufficiently sized grace period against thundering herd problems. Redis needs separate eviction policies for cache and session, because both workloads have fundamentally different requirements for data safety.

OpenSearch needs a deliberately sized JVM heap that respects the 32 gigabyte threshold and leaves enough memory for the Lucene filesystem cache. Connection pooling and explicit timeouts between Magento and all three sidecars prevent an overloaded service from dragging down the entire PHP-FPM pool through hanging connections. With these adjustments, the sidecars deliver the performance a Magento shop actually needs under real load.

11. FAQ: Fine-Tuning Varnish, Redis and OpenSearch Sidecars for Magento