Avoiding Blocking Redis Commands in Magento Web Processes

1. The synchronous PHP-FPM worker model as a starting point

PHP-FPM maintains a pool of worker processes whose count is capped via pm.max_children. Every incoming HTTP request gets assigned to exactly one free worker, which handles the request from start to finish alone and only becomes available for a new request afterward. There is no cooperative concurrency within a single worker in this model, unlike what Node.js offers with its event loop: as long as a worker is busy with a task, it cannot accept another request during that time.

This architecture suits classic, short-lived request-response cycles well, but assumes that every individual operation within a request completes in a predictable, short amount of time. Any operation that takes longer than expected, or in the worst case never returns at all, blocks not only the current request but effectively reduces the total capacity of the worker pool for every other concurrent visitor.

2. What BLPOP, BRPOP, and friends actually do

BLPOP and BRPOP are the blocking variants of the list commands LPOP and RPOP: instead of immediately returning an empty result when the given list is empty, Redis keeps the connection open and only returns a result once another client inserts a new element via LPUSH or RPUSH, or once an optional timeout is reached. Without an explicit timeout, meaning a value of 0, the call theoretically waits indefinitely.

XREAD BLOCK works similarly for Redis Streams, as does WAIT when waiting for replication acknowledgments: all of these commands are deliberately designed so the calling client waits until a specific event occurs. That waiting is sensible and intentional inside a message queue consumer running as its own long-lived process, but fundamentally out of place inside a PHP-FPM web process.

3. The concrete risk: worker exhaustion and timeout cascades

If a blocking call accidentally ends up in a Magento controller or observer class, for instance because a developer wanted to build a simple custom queue directly on Redis list commands, that single request doesn't just block itself but reduces the available worker capacity for every other visitor. With a tightly sized pm.max_children, even a handful of concurrent, blocked requests can exhaust the entire worker pool, causing new requests to get rejected with HTTP 502 or 504 by the front-facing web server.

What makes this particularly tricky is that the behavior can stay unnoticeable under normal load, as long as the affected queue gets filled regularly, and only under unusual circumstances, such as when the producing process briefly goes down, does it cascade into stuck workers followed by timeouts, which is hard to reconstruct from logs after the fact.

4. Where blocking commands typically end up in Magento-adjacent code

In production Magento core code, blocking Redis commands practically never occur; the risk almost exclusively comes from custom-written code. Typical culprits are hand-built job queues that use Redis lists with BLPOP directly instead of relying on Magento's established message queue framework built on RabbitMQ, or integration modules that want to synchronously wait for an external event, such as a confirmation from a third-party system signaled via a Redis pub/sub channel or a list.

Even seemingly harmless debugging or maintenance scripts that accidentally run inside a regular controller instead of as a separate CLI command can be affected, especially when a developer originally wrote them for the command line, where blocking waits are unproblematic, and they later got carried over unchanged into a web context.

5. Alternative 1: short non-blocking polling instead of waiting

Instead of waiting for an event with BLPOP, a web process can instead repeatedly call the non-blocking command LPOP with a very short, explicit time budget, inserting a brief, controlled sleep call with a defined upper bound between attempts. That keeps a request's maximum blocking time strictly bounded and predictable, even when no result is available, unlike a potentially unbounded BLPOP.

This solution suits cases where a result is genuinely expected within a few hundred milliseconds, for instance while waiting on a very fast, asynchronously running background computation. For longer-running operations, polling inside a web request is still the wrong choice, because it still ties up worker capacity unnecessarily, just with a clear upper bound instead of potentially unbounded.

<?php
declare(strict_types=1);

/**
 * Non-blocking polling with a strictly bounded total wait time,
 * as a safe alternative to BLPOP inside a web process.
 *
 * @param Redis $redis Connected phpredis client.
 * @param string $key List an element is expected from.
 * @param int $maxWaitMs Maximum total wait time in milliseconds.
 * @return string|null The element, or null once the wait time expires.
 */
function pollForResult(Redis $redis, string $key, int $maxWaitMs = 500): ?string
{
    $deadline = microtime(true) + ($maxWaitMs / 1000);

    while (microtime(true) < $deadline) {
        $value = $redis->lPop($key);
        if ($value !== false) {
            return $value;
        }
        usleep(20000); // 20ms between attempts
    }

    return null;
}

6. Alternative 2: using Magento's own message queue framework

For genuinely asynchronous processing, Magento already ships a complete message queue framework built on RabbitMQ by default, configured declaratively via queue.xml, communication.xml, and dedicated consumer classes. A web request simply publishes a message via MessageQueue\PublisherInterface and returns immediately without waiting for the actual processing, while a separate, long-lived consumer process handles the message independently of the web request.

This pattern solves the underlying problem cleanly: the web process stays short and non-blocking, while the actual, potentially long-running work happens in a dedicated, permanently running process that is deliberately allowed to block while waiting for new messages, because it isn't a PHP-FPM worker and blocks no other requests.

7. Drawing the line to consumer processes outside the web process

Consumer processes started via bin/magento queue:consumers:start deliberately run as standalone, long-lived processes outside the PHP-FPM pool, often kept alive permanently through a process supervisor like supervisord. In this context, blocking waits are not just unproblematic but actually the most efficient solution: a consumer waiting for new messages via BLPOP or RabbitMQ's equivalent consumes practically no CPU time during idle periods while still reacting instantly once a new message arrives.

The decisive difference, then, isn't the command itself but the process context it runs in: blocking waits are correct and efficient inside a dedicated, long-lived worker process, but inside a request-bound PHP-FPM worker they're a direct path to worker exhaustion and timeout problems.

# Start a consumer process correctly, outside the web process
bin/magento queue:consumers:start custom.consumer.name --max-messages=1000

# Keep it alive permanently via supervisord (excerpt supervisord.conf)
# [program:magento-consumer]
# command=bin/magento queue:consumers:start custom.consumer.name
# autorestart=true
# numprocs=2

8. Reliably catching blocking calls in code review

Because blocking commands often stay unnoticeable under normal load, a targeted look during code review is more reliable than relying on reactive monitoring in production. A simple but effective measure is a static search for the command names BLPOP, BRPOP, BLMOVE, and XREAD BLOCK across the entire codebase, combined with a manual check of whether the given call sits inside a web controller, plugin, or observer, or inside a standalone CLI command or consumer.

A clear team convention that permits blocking Redis calls only in classes explicitly marked as long-running processes, for instance in a dedicated namespace for consumer and worker code, also helps: an accidental use in regular controller code then stands out during code review, instead of only surfacing as an incident in production.

9. Timeouts as a last safety net, not a primary solution

Even when a blocking call inside a web process seems unavoidable for a good reason, a timeout of 0 should never be used. An explicit, short timeout at least caps the maximum damage of a single blocked request, but it doesn't replace the fundamental architectural decision to implement genuine asynchronous processing through the message queue framework instead of direct blocking Redis calls in the web process.

In addition, PHP-FPM's own request_terminate_timeout should be configured as a global safety net, forcibly terminating a stuck worker after a defined upper bound, even if a single blocking call does end up in production through an oversight.

10. Summary

11. FAQ: Blocking Redis Commands in Magento: The Essentials at a Glance