Retry Strategies and Backoff Algorithms in PHP

1. Why naive, immediate retries often make things worse

The most intuitive response to a failed network call is to just try it again right away. For a single client and a rare, random error, that works well enough. But once many clients hit the same overloaded or freshly restarting service at the same time, immediate, uncoordinated retrying produces an effect known as a retry storm: every client fails at the same moment, every client retries at the same moment, and the resulting load spike hits the service exactly when it was supposed to be recovering.

This pattern is particularly nasty because it reinforces itself: the more clients retry at once, the longer the service needs to stabilize, which produces even more timeouts and therefore even more simultaneous retries. A well thought out retry strategy has to do two things at once: grow the wait time between attempts with every failure, and deliberately spread out the retry timing of different clients instead of letting it synchronize.

2. Exponential backoff: the base formula

Exponential backoff solves the first part of the problem by growing the wait time between attempts exponentially with every further failure instead of keeping it constant. The base formula is delay = baseDelay * 2^attempt, where attempt starts at zero and increases by one with every failed try. With a base delay of 100 milliseconds, that produces wait times of roughly 100, 200, 400, 800, and 1600 milliseconds for the first five attempts.

Without an upper bound, this formula would quickly grow into impractically long waits after enough failures, which is why a maxDelay cap always shows up in practice: delay = min(baseDelay * 2^attempt, maxDelay). A maximum number of attempts additionally limits how often a call retries at all, since at some point a call has to be considered definitively failed and reported back to the calling code.

3. Jitter: why plain exponential backoff falls short

Plain exponential backoff solves the problem of growing wait times, but not the problem of synchronized retry waves: if a thousand clients receive the same error at the exact same millisecond, they all compute the exact same delay and hit the service again at the exact same moment, just a bit later. This is exactly where jitter comes in, by adding a random component to the computed delay and spreading out the retry timing of individual clients.

The most common variant is full jitter, where the actual wait time is chosen uniformly at random between zero and the computed exponential backoff value: delay = random(0, min(maxDelay, baseDelay * 2^attempt)). An alternative is equal jitter, where half of the computed delay stays fixed and guaranteed while only the other half varies randomly, giving somewhat more predictable, but still spread out, wait times.

4. A standalone RetryPolicy class in PHP

A reusable RetryPolicy class encapsulates the entire backoff logic behind a single execute() method that accepts an arbitrary callback and automatically retries it with growing, jittered delays on a retryable failure.

For testing purposes it matters that the actual sleeping function is swappable, so a unit test does not literally sleep for several seconds but instead lets the injected replacement return immediately, while still verifying how often it was called and with which values.

<?php

declare(strict_types=1);

namespace App\Resilience;

final class RetryPolicy
{
    /**
     * @param callable(\Throwable):bool|null $isRetryable Decides whether a caught
     *        exception represents a transient failure worth retrying.
     * @param \Closure|null $sleep Injectable sleep function, replaced with a
     *        no-op in tests to avoid real delays.
     */
    public function __construct(
        private readonly int $maxAttempts = 5,
        private readonly int $baseDelayMs = 100,
        private readonly int $maxDelayMs = 10_000,
        private readonly mixed $isRetryable = null,
        private readonly ?\Closure $sleep = null,
    ) {
    }

    /**
     * @template T
     * @param callable():T $operation
     * @return T
     */
    public function execute(callable $operation)
    {
        $attempt = 0;

        while (true) {
            try {
                return $operation();
            } catch (\Throwable $e) {
                $attempt++;
                $retryable = $this->isRetryable !== null ? ($this->isRetryable)($e) : true;

                if (!$retryable || $attempt >= $this->maxAttempts) {
                    throw $e;
                }

                $this->wait($attempt);
            }
        }
    }

    private function wait(int $attempt): void
    {
        $exponential = $this->baseDelayMs * (2 ** $attempt);
        $capped = min($exponential, $this->maxDelayMs);
        $withJitter = random_int(0, (int) $capped);

        $sleeper = $this->sleep ?? fn (int $ms) => usleep($ms * 1000);
        $sleeper($withJitter);
    }
}

5. Distinguishing retryable from non-retryable errors

Not every failure deserves a retry. Transient errors such as timeouts, dropped connections, or server errors with status codes 502, 503, and 504 point to a temporary hiccup and are good retry candidates. Permanent errors such as an invalid input with status code 400, a missing permission with status code 401 or 403, or a violated business rule do not change on a second attempt, a retry there just wastes time and delays an error report that should have happened immediately.

Even more important is the question of idempotency: only idempotent operations such as a GET, a fully replacing PUT, or a DELETE can be safely retried blindly, because repeating them produces the same end state. A non-idempotent call such as a POST that creates a new order must not be retried blindly, because if the original response was merely lost while the operation still succeeded on the server, a blind retry creates a duplicate order. The only safe fix is a client-generated idempotency key that the server recognizes on a repeated call and deduplicates against.

6. Practical example: an HTTP client wrapper with retry decision logic

In practice, a thin HTTP client wrapper that combines RetryPolicy with a clear decision function pays off: server errors and timeouts count as retryable, while status codes 400 and above are surfaced immediately as a permanent failure with no further attempt.

For write calls, an idempotency key is attached, which the server stores as a unique identifier of the original operation and, on a repeated call carrying the same key, simply returns the first, already processed response instead of executing the operation a second time.

<?php

declare(strict_types=1);

namespace App\Http;

use App\Resilience\RetryPolicy;

final class PermanentHttpException extends \RuntimeException
{
    public function __construct(public readonly int $statusCode, string $body)
    {
        parent::__construct("Request failed permanently with status {$statusCode}: {$body}");
    }
}

final class ResilientHttpClient
{
    public function __construct(private readonly RetryPolicy $retryPolicy)
    {
    }

    public function post(string $url, array $body, ?string $idempotencyKey = null): array
    {
        return $this->retryPolicy->execute(function () use ($url, $body, $idempotencyKey) {
            $headers = ['Content-Type: application/json'];
            if ($idempotencyKey !== null) {
                $headers[] = "Idempotency-Key: {$idempotencyKey}";
            }

            $ch = curl_init($url);
            curl_setopt_array($ch, [
                CURLOPT_POST => true,
                CURLOPT_HTTPHEADER => $headers,
                CURLOPT_POSTFIELDS => json_encode($body),
                CURLOPT_TIMEOUT => 5,
                CURLOPT_RETURNTRANSFER => true,
            ]);
            $response = curl_exec($ch);
            $status = curl_getinfo($ch, CURLINFO_HTTP_CODE);
            curl_close($ch);

            if ($response === false || in_array($status, [502, 503, 504], true)) {
                throw new \RuntimeException("Request to {$url} failed with status {$status}");
            }
            if ($status >= 400) {
                // A 4xx client error will not succeed on retry, so it is
                // reported as a permanent failure instead of a transient one.
                throw new PermanentHttpException($status, (string) $response);
            }

            return json_decode($response, true);
        });
    }
}

// Wiring: only exceptions that are NOT a PermanentHttpException are retried.
$retryPolicy = new RetryPolicy(
    maxAttempts: 4,
    isRetryable: static fn (\Throwable $e) => !($e instanceof PermanentHttpException),
);

7. Combining retries with a circuit breaker

Retry logic and the circuit breaker from the previous article solve different parts of the same underlying situation and therefore complement each other well: the retry mechanism absorbs individual, transient failures within a single request, while the circuit breaker recognizes across many requests that a service has failed persistently and stops further calls entirely, instead of letting each one fail repeatedly first.

In terms of ordering, the circuit breaker's state is checked first, and only while it is closed or probing in half open does the retry logic within that single call kick in at all. That way the breaker prevents an already recognized outage from being hit even harder by many parallel retry attempts, while retries continue to absorb ordinary, short blips without tripping the whole breaker.

8. Time budgets and upper bounds: retries must not blow up response time

Beyond a maximum number of attempts, a global time budget for the entire operation pays off too, for example five seconds total, regardless of how many individual attempts happen within it. Without such a budget, a chain of several individually plausible wait times can still add up to a response time that is unacceptable from a user's perspective.

In distributed systems with several hops, it is additionally worth passing the remaining time budget downstream as a deadline, so a deeply nested retry does not consume a budget that was already nearly exhausted higher up the call chain. Without that propagation, overall response time can still grow out of control even though every individual component was configured sensibly on its own.

9. Common mistakes in retry implementations

The most severe mistake is blindly retrying non-idempotent write operations without an idempotency key, which can lead to duplicated orders, duplicated payments, or other inconsistent state. Almost as common is missing jitter entirely, which leaves plain exponential backoff producing synchronized load spikes among many simultaneously failing clients, just delayed a bit instead of happening immediately.

A third common mistake is a missing or far too high cap on the number of attempts, which in the worst case leaves a call running for minutes with ever growing wait times before it finally gives up. And capacity planning for downstream services frequently forgets that a service sized for X requests per second can briefly receive a multiple of that during an outage, once many clients retry the same failing call several times at once.

10. Summary

11. FAQ: Retry and Backoff in PHP: The Essentials