Implementing Idempotency Keys for Payment and Order Endpoints

1. Why network errors are a structural problem for payments

A POST request triggering a payment is, by HTTP definition, not idempotent, meaning a naive repetition of the same request can in principle trigger a second, independent payment, even if the client only repeats because of an unclear network error. At that moment, the client can't reliably know whether the original request actually reached the server and already triggered a payment there before the response was lost, or whether the request itself never arrived.

This uncertainty can't be fully eliminated through better network technology, because lost responses are a fundamental, unavoidable phenomenon in distributed systems. The only robust solution is a mechanism that lets the server unambiguously recognize a repeated request as a duplicate of the original one, regardless of whether the original response reached the client.

2. How idempotency keys actually work

Before the first attempt of a critical request, the client generates a unique idempotency key (typically a UUID) and sends it as a header, such as Idempotency-Key: 7f3e9a2b-..., with every attempt, including all retries of the same logical request. On the first processing of this key, the server stores both the processing status and the complete response, so a later request with the same key isn't processed again but directly receives the cached, original response.

Crucially, this mechanism is entirely client-driven: the server itself can't tell whether two different requests are meant to represent the same logical operation or two deliberately different operations, which is why explicit marking by the client is indispensable, instead of trying to implicitly guess duplicates from payment amount or timestamp.

<?php
declare(strict_types=1);

use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Response;

final class IdempotencyKeyMiddleware
{
    public function __construct(private readonly \Redis $redis) {}

    public function handle(Request $request, callable $next): Response
    {
        $key = $request->headers->get('Idempotency-Key');
        if ($key === null) {
            return $next($request);
        }

        $cacheKey = "idempotency:{$key}";
        $cached = $this->redis->get($cacheKey);
        if ($cached !== false) {
            $data = json_decode($cached, true);
            return new Response($data['body'], $data['status'], $data['headers']);
        }

        $response = $next($request);

        $this->redis->setex($cacheKey, 86400, json_encode([
            'status' => $response->getStatusCode(),
            'body' => $response->getContent(),
            'headers' => $response->headers->all(),
        ]));

        return $response;
    }
}

3. How long cached idempotency responses should be retained

Too short a retention period for the cached response risks a late but legitimate retry (say, after a long mobile connection drop) no longer being recognized as a duplicate and triggering actual double processing. Too long a retention period, on the other hand, needlessly wastes storage for keys that realistically will never be queried again. Stripe, as one of the best-known reference implementations of this pattern, uses a default retention of 24 hours as a proven compromise.

This retention period should be adjusted to the actual retry behavior of your own clients: a mobile app with aggressive offline queueing, which might resend a failed request only hours later, needs a longer retention period than a server-to-server call, where retries typically happen within seconds.

4. Handling different payloads with the same key

An important edge case arises when the same idempotency key is sent again with a substantively different request body, for example because a client mistakenly reuses the same key for two different payments. The server should explicitly treat this case as an error (typically HTTP 422 Unprocessable Entity) instead of silently processing either the original or the new payload, since either behavior would disregard the client's actual intent.

This check requires the server to store not only the response but also a hash of the original request body together with the idempotency key, to detect on a repeated request with the same key whether the body is actually identical, before returning the cached response.

5. Race conditions with concurrent requests using the same key

An aggressively retrying client can theoretically send two requests with the same idempotency key nearly simultaneously, before processing of the first has completed and been stored in the cache, which without extra protection could lead to double processing despite an identical key. An atomic Redis SETNX call (set if not exists) at the start of processing, setting a "processing" marker, prevents this race by letting the second, concurrent request detect that processing is already underway and wait accordingly or respond with a clear conflict status.

This race condition handling is overlooked in many naive idempotency implementations that only check the finished response cache but offer no protection during the ongoing processing itself, undermining the mechanism's actual purpose exactly in the situations where it's needed most.

6. Correctly generating idempotency keys from the client side

The idempotency key must be generated by the client once per logical operation, before the first attempt is sent, and reused identically for all subsequent retries of the same operation, instead of generating a new key on every retry, which would render the entire protection mechanism ineffective. A UUID v4, generated immediately before the first send attempt and stored locally (such as in local application memory or an offline queue) until successful delivery, is the usual approach.

For client-side retry libraries, it's worth anchoring the idempotency key as part of the retry configuration itself, instead of manually resetting it on every API call, to structurally rule out human oversight (accidentally forgetting the key on a manually implemented retry).

7. Using idempotency keys beyond payments

Although idempotency keys are most commonly discussed in the context of payment APIs, the pattern is relevant for any non-idempotent operation with potentially costly duplicate execution: creating orders, sending emails, provisioning external resources. The fundamental question of whether an operation would cause a real problem on accidental double execution decides whether idempotency key protection is justified.

For purely read operations or operations that are already naturally idempotent (a PUT to a known resource ID), no additional idempotency key mechanism is needed, since HTTP semantics already guarantee idempotency here, provided the implementation actually honors that guarantee correctly.

8. Monitoring idempotency usage to detect real network problems

A high rate of actual cache hits (requests recognized as duplicates and answered from the cache) is a valuable signal for the frequency of real network problems or client-side retry events, which would remain invisible without a dedicated metric. An unexpectedly high cache hit rate for a particular client or endpoint can point to underlying infrastructure problems that should be investigated independently of the idempotency mechanism itself.

Equally important is a metric for detected payload conflicts (the same key with a different body), since an elevated rate of these cases points to a bug in the client implementation that would otherwise go undetected and potentially lead to confusing 422 errors for end users, without the actual cause showing up in the client code.

9. Idempotency keys at a glance

The table below summarizes the key design decisions.

10. Summary

11. FAQ: Idempotency Keys: The Essentials at a Glance