Symfony Messenger Retry Strategies and the Failed Transport in Detail
AI generated
SF
{ }
Symfony · Messenger · Queues
Symfony Messenger Retry Strategies and the Failed Transport in Detail
How Symfony decides whether a message gets retried or ultimately fails

Once an application processes messages asynchronously through Symfony Messenger, a failing message is not the exception, it is the normal case: an external service becomes briefly unavailable, a database connection drops, or a worker gets restarted mid-processing. Anyone who blindly trusts the default settings either loses messages because they are given up on too early, or overloads an already struggling system with retries that fire too often. This article explains exactly how Messenger's retry strategy works, how the failed transport acts as a second safety net, and how to configure both mechanisms deliberately for a real use case.

14 min read Messenger · Retry Strategies Failed Transport · Commands

1. Why asynchronous messages fail regularly in practice

Symfony Messenger typically processes messages under an at-least-once delivery guarantee: a message stays considered unprocessed until the handler completes successfully and the worker explicitly acknowledges it with the transport. Between dispatch and successful processing lies a window in which any number of things can go wrong: an external payment provider times out, a database connection drops because of a brief network hiccup, or the worker process gets restarted by a deployment while it is still handling a message.

Without a built in retry mechanism, every one of these transient failures would either lose the message entirely or crash the whole worker process, requiring a manual restart. Symfony Messenger solves this with a two stage safety net: a configurable retry strategy decides after every failure whether and when another attempt happens, and a separate failed transport receives messages that could not be processed even after every attempt, so they remain available for later manual or automated handling.

2. RetryStrategyInterface as the central abstraction

At the core of every retry decision sits the interface Symfony\Component\Messenger\Retry\RetryStrategyInterface with two methods: isRetryable(Envelope $envelope, ?\Throwable $throwable = null): bool decides whether another attempt should happen at all, and getWaitingTime(Envelope $envelope, ?\Throwable $throwable = null): int returns the wait time in milliseconds before the next attempt. Each configured transport gets its own instance of this strategy, so different queues can follow entirely different retry rules.

After every failed processing attempt, the worker automatically calls both of these methods before deciding whether to re-enqueue the message with a DelayStamp or hand it straight to the failure transport. This check happens entirely inside the Messenger core, so handler code does not need to know anything about the retry logic unless it deliberately wants to influence it, for example through a dedicated exception type.

3. Configuring the default MultiplierRetryStrategy in detail

Symfony ships with MultiplierRetryStrategy as its default implementation, fully controllable through four parameters in the transport configuration: max_retries sets how many times a message gets retried at most, delay defines the base wait time in milliseconds before the first retry, multiplier determines the factor by which the wait time grows with every further attempt, and max_delay caps that wait time so it does not grow without bound.

In practice, setting these four values directly in framework.yaml per transport is almost always enough, no custom class required. The example below shows a typical configuration for a production async transport together with its associated failure transport, defined separately as its own Doctrine-backed transport.


framework:
    messenger:
        failure_transport: failed

        transports:
            async:
                dsn: '%env(MESSENGER_TRANSPORT_DSN)%'
                retry_strategy:
                    max_retries: 4
                    delay: 1000
                    multiplier: 3
                    max_delay: 30000

            failed:
                dsn: 'doctrine://default?queue_name=failed'

        routing:
            App\Message\SendOrderConfirmationMessage: async

4. How exponential backoff protects the system from overload

With the values from the example above, the wait time before each attempt follows the formula delay * multiplier^attemptNumber, capped by max_delay: the first retry waits one second, the second waits three seconds, the third waits nine seconds, and the fourth would mathematically wait 27 seconds, still below the 30 second cap. Only after that, following four failed attempts in total, does the message move to the failed transport.

This exponential growth is not an academic detail, it actively guards against a thundering herd effect: if an external service becomes briefly overloaded and thousands of messages fail at once, a fixed short wait time would send nearly all of them back at the already struggling service almost simultaneously, making the situation worse. A growing gap between attempts gives the affected system room to recover before the next wave of requests arrives.

5. How a message actually ends up in the failed transport

Once isRetryable() returns false after the last allowed attempt, or once max_retries is reached, the worker automatically moves the message to the transport configured under failure_transport. In doing so, Messenger enriches the envelope with additional stamps, including an ErrorDetailsStamp carrying the exception class, the error message, and the timestamp of the last failure, plus a RedeliveryStamp documenting how many attempts have already happened.

It is important that the failed transport is not consumed automatically by the regular worker, it has to be consumed as its own standalone transport, typically through bin/console messenger:consume failed. That keeps the main worker's processing speed independent from however many messages have permanently failed, and lets a team decide deliberately when and how those messages get revisited.

6. messenger:failed:show, :retry, and :remove in daily operations

Symfony ships three console commands for working with the failed transport. messenger:failed:show lists every failed message with its ID, class, and timestamp, and with a concrete ID as argument (messenger:failed:show 42) the command additionally displays the full exception stack trace and the number of prior attempts, which is usually the first step when debugging a failure.

messenger:failed:retry takes one or more IDs, removes the message from the failed transport, and makes it available again for processing by the original handler, so a transient failure can be retried after a manual root cause check without any code change. messenger:failed:remove, on the other hand, deletes a message permanently, which makes sense once it becomes clear that a message can never succeed because of fundamentally invalid data.

7. UnrecoverableExceptionInterface: when a failure is never worth retrying

Not every failure benefits from a retry. A message carrying an invalid order number or violating a business rule will fail again on the next attempt no matter how long the system waits, and every extra retry cycle just wastes time and resources. For exactly this case, Symfony provides the interface Symfony\Component\Messenger\Exception\UnrecoverableExceptionInterface, which custom exceptions can implement, plus the ready made class UnrecoverableMessageHandlingException as a convenient default.

When a handler throws an exception implementing this interface, Messenger ignores the configured retry strategy entirely and moves the message straight to the failed transport without another attempt. This matters especially for validation errors and permanently invalid payloads, since it drastically shortens the time until the failure becomes visible and avoids wasting load on retries that were never going to succeed.

8. Writing a custom retry strategy for special requirements

When the configurable MultiplierRetryStrategy is not enough, say because different exception types need different handling or the wait time should depend on a rate limit header returned by an external service, you can write a class that implements RetryStrategyInterface directly. Inside isRetryable() and getWaitingTime() the full envelope including every stamp is available, so you could, for instance, read a RedeliveryStamp to factor the number of prior attempts into your own logic.

You register a custom strategy by declaring it as a regular service and, instead of the parameter based configuration, setting its service name under retry_strategy on the relevant transport, for example retry_strategy: App\Messenger\RateLimitAwareRetryStrategy. Symfony then calls it exactly the same way it calls the default implementation, so the rest of the worker flow does not change.

9. Monitoring the failed transport meaningfully in production

A failed transport nobody watches is functionally not much better than a transport where messages silently vanish: it does not get emptied, but without active monitoring a growing pile of failed messages often only becomes visible once customers start complaining. In practice it pays off to poll the size of the failed queue regularly, through a cron job or a monitoring system, either by querying the underlying Doctrine table or through a custom command that fires an alert once a threshold is exceeded.

It also helps to establish a fixed triage routine for accumulated failed messages, for example a daily review where transient failures get retried via messenger:failed:retry and permanently broken messages get removed via messenger:failed:remove once the root cause is documented. That keeps the failed transport an actively maintained safety net instead of a silent graveyard of lost messages.

Parameter Meaning Example value Effect
max_retries Number of retry attempts before the failed transport 4 The 5th failed attempt goes straight to the failed transport
delay Base wait time in milliseconds before the first retry 1000 First retry happens after 1 second
multiplier Factor for exponential growth of the wait time 3 Wait time triples with each attempt
max_delay Upper bound of the wait time in milliseconds 30000 Wait time is capped at 30 seconds
failure_transport Target transport for permanently failed messages failed Message lands in a separate queue

Mironsoft

Symfony architecture, clean domain logic, and legacy modernization

Symfony applications that stay maintainable two years down the line?

We review existing Symfony projects for bloated controllers, missing service abstractions, and untested core logic, then build an architecture that absorbs new features without getting more fragile with every release.

Architecture Review

Checking bundle structure, dependency injection, and service abstractions for maintainability.

Legacy Modernization

Incrementally migrating outdated Symfony versions without a full rewrite.

Testing and Quality Assurance

Setting up PHPUnit, PHPStan, and CI pipelines for lasting code quality.

10. Summary

Messenger Retries and Failed Transport: The Key Points at a Glance

Retry strategy

RetryStrategyInterface decides after every failure whether and when another attempt happens.

Backoff

MultiplierRetryStrategy grows the wait time exponentially, protecting overloaded systems.

Failed transport

Receives permanently failed messages and is managed separately through console commands.

Unrecoverable

UnrecoverableExceptionInterface stops retries immediately for permanently invalid messages.

11. FAQ: Messenger Retries and Failed Transport: The Key Points at a Glance

1What is the difference between RetryStrategyInterface and MultiplierRetryStrategy?
RetryStrategyInterface is the abstraction with the isRetryable and getWaitingTime methods, MultiplierRetryStrategy is the default implementation Symfony ships, configurable through four parameters.
2What happens if max_retries is set to 0?
Then no further attempt happens after the first failure, and the message moves straight to the configured failure transport, provided one is set up.
3Does every transport need its own failure transport?
No, usually a single failure transport is configured globally under failure_transport and shared by all transports, though it can be overridden per transport if needed.
4How does UnrecoverableMessageHandlingException differ from a regular exception?
A regular exception goes through the configured retry strategy normally, while UnrecoverableMessageHandlingException implements UnrecoverableExceptionInterface and therefore skips retries entirely, sending the message straight to the failed transport.
5Can the retry strategy be configured per message type instead of per transport?
Not directly, since the strategy is tied to the transport. In practice this is solved by routing message types with different requirements to different transports, each with its own retry strategy.
6Are messages in the failed transport deleted automatically?
No, they stay there permanently until explicitly retried via messenger:failed:retry or deleted via messenger:failed:remove, which requires deliberate maintenance of the queue.
7Does the regular worker also consume the failed transport?
Only if explicitly started for it, normally messenger:consume runs without the failed transport's name and it gets consumed separately via messenger:consume failed.
8How do you find out why a message ended up in the failed transport?
Through messenger:failed:show with the concrete ID, which displays the full exception stack trace, the error message, and the number of prior processing attempts.
9Is a custom retry strategy worth it for small projects?
Usually not, the configurable MultiplierRetryStrategy covers the vast majority of cases. A custom implementation only pays off for edge cases like exception-specific behavior or external rate limit constraints.
10What is the advantage of exponential backoff over a fixed wait time?
With a fixed short wait time, a larger outage causes nearly all failed messages to hit the affected system again almost simultaneously. Exponential backoff spreads retries over a longer period and gives the system time to recover.