Redis Cluster Setup for Scaling Magento Stores

1. When a single Redis instance is no longer enough

Most Magento stores run for years without issues on a single Redis instance for cache and session. That changes once certain thresholds are reached: memory usage approaches the configured maxmemory limit, latency of individual commands rises noticeably, or CPU usage of the single Redis process stays permanently above 70 percent. Because Redis is internally single threaded, one process can only use a single CPU core for actual command processing, regardless of how many cores the server has in total. Beyond a certain traffic volume, that one core becomes the bottleneck, no matter how much RAM is still free.

A second warning sign is mixing different data types in the same instance. When session data, object cache and full page cache all live in the same Redis instance, they compete for the same memory and the same processing thread. A cache flush or a large KEYS operation on the object cache can then briefly block session processing too, because Redis executes commands sequentially. In practice this shows up as sporadic login problems or aborted checkouts exactly when a cache clear is running in the background.

Before considering a real Redis Cluster, it is worth checking INFO memory and INFO commandstats on the existing instance. If used_memory keeps climbing toward maxmemory even though TTLs are set correctly, that points to genuine data growth that needs more RAM or sharding. If instantaneous_ops_per_sec spikes instead, CPU processing is more likely the bottleneck, which can be solved by splitting across multiple processes and not necessarily by adding memory.

2. The first step: role separation instead of a cluster

Before a real Redis Cluster with sharding comes into play, almost every Magento setup should first move to role separation on dedicated instances. Magento uses Redis for three different purposes: the object cache (cache), the full page cache (page_cache), and optionally session storage. These three roles have completely different access patterns: sessions are small, frequently written objects with a short lifetime, the object cache consists of many small to medium entries with a medium TTL, and the full page cache holds few but large HTML blocks.

The simplest scaling step is therefore to run three separate Redis processes on the same host or on different hosts, each with its own maxmemory limit and its own maxmemory-policy. Since every Redis process uses its own thread for command processing, CPU load automatically spreads across multiple cores once the instances run on different ports or hosts. A cache flush on the object cache can then no longer block the session instance, because they are completely separate processes.

In practice this role separation is enough for a large portion of Magento installations to absorb the next one to two years of growth without taking on the complexity of a real cluster. Only once a single role, usually the object cache on very large catalogs, hits the limits of a single instance again does a Redis Cluster with sharding inside that one role become relevant.

; /etc/redis/redis-cache.conf
port 6379
maxmemory 4gb
maxmemory-policy allkeys-lru
save ""
appendonly no
databases 1

; /etc/redis/redis-session.conf
port 6380
maxmemory 2gb
maxmemory-policy noeviction
save ""
appendonly yes
appendfsync everysec
databases 1

; /etc/redis/redis-fpc.conf
port 6381
maxmemory 6gb
maxmemory-policy allkeys-lru
save ""
appendonly no
databases 1

3. Redis Cluster: understanding sharding and hash slots

A real Redis Cluster splits the keyspace into 16384 hash slots distributed across multiple master nodes. Every key is mapped to a slot via a CRC16 hash, and every node in the cluster is responsible for a subset of these slots. That means write and read access automatically spreads across multiple processes and therefore across multiple CPU cores, and ideally across multiple physical servers. Unlike plain role separation, a single logical dataset here, for example the entire object cache, is distributed horizontally across multiple nodes.

The decisive difference from a simple multi instance setup is that a Redis Cluster restricts multi key operations. Commands that touch multiple keys at once, such as MGET or transactions involving several keys, only work if all involved keys live in the same hash slot. Magento internally relies heavily on tag based invalidation using cache tags that reference multiple keys via sets. If these tags and the associated cache entries end up in different slots, certain invalidation operations fail or need to be rewritten to use hash tags.

To keep related keys in the same slot, Redis supports hash tags in curly braces: a key like cache:{tag123}:entity_42 is hashed only on the part inside the curly braces. This allows precise control over which keys end up together on one node. The Cinnamon and Colinmollenhour cache backends that Magento uses for Redis do not support this hash tagging automatically, which is why a Redis Cluster for the Magento object cache in practice requires additional configuration or a cluster aware cache backend.

4. Building a Redis Cluster for Magento

For a minimally production ready Redis Cluster, at least three master nodes and three replica nodes are needed so that automatic failover remains possible if a master fails without losing slots. Every node needs a second port in addition to the normal client port for cluster bus communication, by default the client port plus 10000. These ports must be open between all cluster nodes, but not reachable from outside.

# Six Redis nodes: 3 masters, 3 replicas, minimal production cluster
for port in 7000 7001 7002 7003 7004 7005; do
  mkdir -p /etc/redis/cluster/${port}
  cat > /etc/redis/cluster/${port}/redis.conf <<EOF
port ${port}
cluster-enabled yes
cluster-config-file nodes-${port}.conf
cluster-node-timeout 5000
appendonly yes
dir /etc/redis/cluster/${port}
maxmemory 2gb
maxmemory-policy allkeys-lru
EOF
  redis-server /etc/redis/cluster/${port}/redis.conf --daemonize yes
done

# Create the cluster: first three nodes as masters, last three as replicas
redis-cli --cluster create \
  127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 \
  127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 \
  --cluster-replicas 1

# Verify slot distribution across masters
redis-cli -p 7000 cluster slots
redis-cli -p 7000 cluster nodes

After creation, redis-cli --cluster create automatically distributes the 16384 slots evenly across the three master nodes. Every replica is assigned to a master and automatically takes over the master role on failure, as long as a majority of the remaining master nodes agree to the failover. This consensus mechanism prevents split brain situations, but it requires an odd total number of master nodes at all times so majority decisions stay unambiguous.

5. Configuring Magento for cluster operation

Magento supports Redis Cluster operation through the Colinmollenhour cache backend, activated in env.php via the cluster parameter. Instead of a single server, a list of all cluster seed nodes is provided, through which Magento automatically determines the full cluster topology. It is important that the full page cache and the object cache each use their own cluster, or at least their own slot ranges, to keep a clear role separation.

// app/etc/env.php: Redis Cluster configuration for Magento object cache
'cache' => [
    'frontend' => [
        'default' => [
            'backend' => 'Cm_Cache_Backend_Redis',
            'backend_options' => [
                'seed' => '10.0.1.10:7000,10.0.1.11:7001,10.0.1.12:7002',
                'load_from_slave' => '10.0.1.10:7000',
                'read_timeout' => 10,
                'automatic_cluster' => true,
                'persistent' => 1,
                'compress_data' => 1,
            ],
        ],
        'page_cache' => [
            'backend' => 'Cm_Cache_Backend_Redis',
            'backend_options' => [
                'seed' => '10.0.1.20:7100,10.0.1.21:7101,10.0.1.22:7102',
                'automatic_cluster' => true,
                'compress_data' => 1,
                'compression_lib' => 'l4z',
            ],
        ],
    ],
],

After every change to env.php, cache and configuration must be regenerated so Magento picks up the new cluster topology. A test run with bin/magento cache:status shows whether Magento reaches the configured Redis backends correctly. It also helps to run bin/magento cache:flush deliberately after the switch and then check slot distribution with redis-cli --cluster check to make sure no slots were left unassigned.

6. Redis Sentinel as an alternative for high availability

Not every scaling problem requires sharding. When the data volume per role comfortably fits into a single Redis instance but high availability is required, Redis Sentinel is often the simpler solution compared to a full cluster. Sentinel monitors a master and several replicas, automatically detects a master failure and promotes a replica to the new master, without the data ever needing to be split across multiple slots.

The key advantage of Sentinel over a real cluster is compatibility: all multi key operations continue to work without restriction because the entire dataset lives on a single master. The downside is that Sentinel offers no horizontal scaling of write load, only failover safety. For Magento stores that primarily value availability and whose data volume will continue to fit into one instance, Sentinel is often the more pragmatic choice compared to a cluster with its additional restrictions around cache tags.

; /etc/redis/sentinel.conf: three Sentinel processes watch one master
port 26379
sentinel monitor mymaster 10.0.1.30 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 10000

# env.php pointing Magento at the Sentinel-managed master
# 'backend_options' => [
#     'server' => 'tcp://10.0.1.30:6379',
#     'persistent' => 1,
# ],

7. Sessions in a cluster: avoiding pitfalls

Session data should in practice never be part of a sharded Redis Cluster, but always live in its own dedicated instance or a Sentinel setup. The reason: PHP sessions in Magento are secured through the Cm_Cache_Backend_Redis session handler using locking mechanisms to correctly serialize parallel requests from the same user. This locking relies on individual keys whose consistency is harder to guarantee in a cluster with failover windows than in a single instance with synchronous replication.

Another risk is slot migration during an ongoing cluster rebalance. If a slot holding an active session gets moved to another node in the middle of a checkout process, brief read failures can occur that Magento interprets as session loss. With sessions in a dedicated, non clustered instance using Sentinel failover, this problem does not occur, because no rebalancing happens, only a clearly defined failover switch in an error case.

8. Monitoring and capacity planning

A Redis Cluster or a multi instance landscape without continuous monitoring is flying blind. The most important metrics per node are used_memory relative to maxmemory, the evicted_keys rate as an indicator of tightly sized memory, and connected_clients relative to the configured maxclients limit. In a cluster, monitoring slot distribution is also needed to detect whether individual nodes are more heavily loaded than others due to uneven hash tag usage.

For capacity planning, a simple approach works well: measure average memory usage per active user or per category and product page, and extrapolate the expected total load during traffic spikes such as Black Friday. A Redis Cluster should always be sized with enough headroom so a single node failure does not immediately cause memory pressure on the remaining nodes while failover is in progress.

# Key metrics per node, run against each Redis instance in the cluster
redis-cli -p 7000 info memory | grep -E "used_memory:|maxmemory:"
redis-cli -p 7000 info stats  | grep -E "evicted_keys:|instantaneous_ops_per_sec:"
redis-cli -p 7000 info clients | grep "connected_clients:"

# Cluster-wide slot and node health check
redis-cli --cluster check 127.0.0.1:7000

# Simple capacity forecast: memory per active session times expected peak users
redis-cli -p 6380 info keyspace
# db0:keys=48213,expires=48213,avg_ttl=1780000

9. Cluster vs. dedicated instances compared

The decision between a real Redis Cluster and simple role separation on dedicated instances depends on the concrete bottleneck. The following table summarizes the key differences for the Magento context.

In practice, most Magento stores reach a very solid scaling level with clean role separation on dedicated instances alone. A real Redis Cluster only pays off once a single role, usually the object cache with several million catalog entries, hits its capacity limit again and the added complexity of slot management is justified.

10. Summary

The scaling path for Redis in Magento follows nearly the same order in practice every time: first role separation of cache, session and full page cache onto dedicated instances, then Redis Sentinel for high availability if needed, and only as a last step a real Redis Cluster with sharding for individual roles that outgrow a single instance on their own. Sessions should never be part of a sharded cluster, because locking behavior and slot migrations can collide with session handling.

Following this path avoids premature complexity and keeps control over cache invalidation and multi key operations that Magento relies on internally. Monitoring memory usage, eviction rate and slot distribution is mandatory at every stage so a bottleneck is caught before it causes checkout failures.

11. FAQ: Redis Cluster Setup for Magento