Using Redis for Indexer Lock Coordination in Magento

1. How parallel indexer processes can block each other

Magento's indexers typically work with a version table and a changelog table per indexer, from which a reindex run determines which records to process. If two processes start the same indexer nearly simultaneously, for instance because a manually triggered bin/magento indexer:reindex collides with a regular cron run, both processes can end up processing the same dataset in parallel. In the mild case that just means duplicate work and wasted CPU time; in the worse case both processes can block each other on write locks against the same rows and trigger database deadlocks.

It gets particularly critical with full reindex runs that create temporary tables and, at the end, replace the production index table via an atomic table swap. If two such processes run in parallel, the second, later table swap can overwrite the first one's result or collide with reads still in flight, which in the worst case produces inconsistent or incomplete index data that only gets corrected by the next successful reindex.

2. Magento's existing locking mechanism and its limits

Since Magento 2.3.4, the reindex process protects itself against parallel execution by acquiring a lock named something like indexer_reindex_<indexerId> via LockManagerInterface before the actual run, releasing it again only once the run completes. The default implementation of this interface uses MySQL's GET_LOCK() function, which holds a named lock for the duration of the same database connection and releases it automatically once that connection ends.

This mechanism works reliably as long as all involved processes talk to the same MySQL instance, which is practically always the case in Magento setups. The actual limit isn't in the database itself but in the process coordination around it: for containerized or horizontally scaled deployments with several independently running cron workers, it can make sense to run the lock logic independently of a persistently held MySQL session, for instance to sidestep connection timeouts or connection-pooling quirks.

3. Redis as a distributed lock provider: the SET NX PX pattern

A distributed lock in Redis can be implemented with the single atomic command SET key value NX PX ttl: NX ensures the key is only set if it doesn't already exist, and PX assigns an expiry time in milliseconds so a crashed process can never block the lock forever. The stored value should contain a unique process or request ID, so that releasing the lock can unambiguously verify it is actually deleting its own lock and not accidentally the lock of a different, meanwhile restarted process.

Releasing the lock therefore has to atomically check the value and delete it, which is most reliably implemented via a small Lua script through EVAL, since a separate GET-then-DEL sequence in PHP would leave a race condition open between the check and the deletion.

<?php
declare(strict_types=1);

namespace Mironsoft\IndexerLock\Model;

use Redis;

/**
 * Acquires and releases distributed indexer locks in Redis using
 * the SET NX PX pattern with atomic, value-checked release.
 */
class RedisIndexerLock
{
    private const LOCK_TTL_MS = 300000;

    /**
     * @param Redis $redis Connected phpredis client.
     */
    public function __construct(private readonly Redis $redis)
    {
    }

    /**
     * Attempts to acquire a lock for the given indexer.
     *
     * @param string $indexerId ID of the indexer to lock.
     * @param string $ownerToken Unique identifier of the locking process.
     * @return bool True if the lock was successfully acquired.
     */
    public function acquire(string $indexerId, string $ownerToken): bool
    {
        $key = sprintf('indexer_lock:%s', $indexerId);

        return (bool) $this->redis->set($key, $ownerToken, ['NX', 'PX' => self::LOCK_TTL_MS]);
    }

    /**
     * Releases a lock only if the caller is still its owner.
     *
     * @param string $indexerId ID of the indexer to unlock.
     * @param string $ownerToken Unique identifier of the locking process.
     * @return bool True if the lock was actually removed.
     */
    public function release(string $indexerId, string $ownerToken): bool
    {
        $key = sprintf('indexer_lock:%s', $indexerId);
        $script = <<<'LUA'
if redis.call('GET', KEYS[1]) == ARGV[1] then
    return redis.call('DEL', KEYS[1])
end
return 0
LUA;

        return (bool) $this->redis->eval($script, [$key, $ownerToken], 1);
    }
}

4. Integrating as a LockManagerInterface preference instead of a wrapper script

Instead of only checking the Redis lock inside an external bash wrapper before the actual reindex call, the custom implementation can also be registered directly as a replacement for Magento's LockManagerInterface. To do that, the default preference in di.xml is redirected to the custom Redis class, so not only the manually invoked reindex but every internal call that already relies on the interface gets coordinated through Redis instead of MySQL's GET_LOCK() automatically, without changing anything anywhere else in the code.

This approach is cleaner than a separate wrapper script because it stays within the existing Magento architecture and automatically applies to every caller that programs against the interface, not only manually invoked CLI commands. What matters is that the custom implementation serves exactly the interface's lock(), unlock(), and isLocked() methods, so existing Magento core code keeps working without modification.

<!-- app/code/Mironsoft/IndexerLock/etc/di.xml -->
<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="urn:magento:framework:ObjectManager/etc/config.xsd">
    <preference for="Magento\Framework\Lock\LockManagerInterface"
                type="Mironsoft\IndexerLock\Model\RedisLockManager" />
</config>

5. The Redlock principle and when it actually becomes relevant

The Redlock algorithm, proposed by Redis creator Salvatore Sanfilippo, addresses a problem that does not occur with a single Redis node but becomes relevant once several independent Redis instances are in use for high availability: a lock only counts as successfully acquired once it could be set on a majority of instances, say three out of five, within a tight time window. That protects against a single, isolated, or failing-over node incorrectly reporting a lock as valid while another process already holds it on the remaining nodes.

For the vast majority of Magento installations running a single Redis instance or a simple primary-replica setup without several independently writable nodes, the full Redlock algorithm is overkill. It becomes relevant mainly with Redis Cluster or Sentinel topologies involving several potentially simultaneously writable nodes, where a failover in the middle of a lock operation could otherwise produce contradictory lock states.

6. Practical example: coordinating multiple cron workers across servers

In a typical scaled Magento deployment, several application servers or containers run behind a load balancer, and for historical or availability reasons each node runs its own cron process instead of running cron centrally on a single dedicated node. Without additional coordination, the same indexer reindex job would be triggered on every server in parallel as soon as the cron time is reached.

A Redis-based lock, checked before the actual reindex call inside a wrapper script or custom command, ensures that only the process that acquires the lock first actually runs the indexer, while every other cron instance skips the run for that cycle. That not only reduces collision risk but also saves unnecessary, redundant compute time on the remaining servers.

#!/bin/bash
# Wrapper script: only run the reindex if this server wins the lock
LOCK_KEY="indexer_lock:catalog_product_price"
OWNER="$(hostname)-$$"

ACQUIRED=$(redis-cli SET "$LOCK_KEY" "$OWNER" NX PX 300000)

if [ "$ACQUIRED" = "OK" ]; then
    bin/magento indexer:reindex catalog_product_price
    redis-cli EVAL "if redis.call('GET', KEYS[1]) == ARGV[1] then return redis.call('DEL', KEYS[1]) end return 0" 1 "$LOCK_KEY" "$OWNER"
else
    echo "Reindex already running on another node, skipping."
fi

7. Sizing lock timeout correctly and avoiding deadlocks

Choosing the lock TTL is a balancing act: too short, and a still-running, legitimate reindex process can lose its own lock before it finishes, letting a second process start in parallel while the first is still active. Too long, and a crashed or stuck process blocks the indexer for an unnecessarily long time, even though it no longer exists at all.

A robust solution sets the initial TTL generously enough for the longest realistic runtime of the given indexer, and additionally extends the lock periodically during execution via a separate heartbeat mechanism, as long as the process is provably still active. That way a legitimate, longer-running reindex never runs into premature lock release, while a genuinely crashed process is still released automatically within a clearly defined upper time bound.

8. Monitoring lock contention in production

To see how often different cron workers actually compete for the same lock, simple logging on every failed lock attempt is worth having, including timestamp, hostname, and the affected indexer. If contention piles up noticeably for a given indexer, that's a signal either that the involved servers' cron schedules are timed too tightly, or that the reindex run itself takes too long and already overlaps with the next scheduled execution.

In addition, redis-cli TTL indexer_lock:<name> can be used at any time to check whether a lock is currently held and how much longer it remains valid, giving quick clarity on the actual state during live troubleshooting without having to dig through log files.

9. Limits: Redis as a single point of failure and when Redlock is overkill

If lock coordination runs entirely through Redis, the Redis instance itself becomes a critical component: if it goes down, no new locks can be acquired, and depending on the implementation, this can in the worst case block even regular reindex runs that, as a safety measure, should not start at all without working lock coordination. A sensible fallback is to fall back to Magento's existing, database-based locking when Redis is unreachable, instead of making the entire indexer operation dependent on the availability of an additional component.

For the vast majority of Magento installations running a single production Redis instance, the simple SET-NX-PX pattern is entirely sufficient, and the full Redlock algorithm with several independent instances is unnecessary overhead. It pays off only when the Redis infrastructure is already run as a cluster with several potentially simultaneously writable nodes and that complexity doesn't need to be introduced additionally just for locking.

10. Summary

11. FAQ: Redis Indexer Locks in Magento: The Essentials at a Glance