Cache Invalidation Strategies in Distributed Systems, Approached Practically

1. The classic problem: only two hard things in computer science

The frequently quoted line that there are only two hard things in computer science, cache invalidation, naming things, and off-by-one errors, is more than an inside joke among developers. It describes a real, structural problem: a cache exists so expensive operations don't need to be repeated, but by definition it is a copy of data that is considered canonical somewhere else. The moment the original data changes, that copy must either be updated or marked invalid, and coordinating that turns out to be remarkably hard to get right in practice.

The difficulty is not in the idea itself but in the edge cases: how does the cache layer even know data has changed, if the change did not go through the exact same code path that populates the cache? What happens when multiple users change the same data at the same time? And how do you handle the window during which stale data gets served before invalidation actually takes effect? These questions have no universal answer; they require a deliberate choice of strategy that fits the consistency needs of the specific application.

Two fundamental strategies that combine well in practice:

1. Time-based invalidation (TTL)
   Cache entry expires after a fixed window, regardless of whether
   the underlying data actually changed.
   Simple to implement, but either too short (more load on origin)
   or too long (stale data visible for longer periods).

2. Event-based invalidation
   Cache entry gets explicitly deleted/updated the moment the
   underlying data changes (e.g. via a message queue event).
   More precise, but requires reliable event delivery across every
   system boundary -- this is exactly where the real complexity
   lives in distributed systems.

2. Time-based invalidation: TTL as the simplest approach

Time-based invalidation via a time-to-live (TTL) is the simplest and most widespread approach: every cache entry gets a fixed validity window, after which it automatically counts as invalid and gets reloaded from the original source on the next request. This approach requires no communication whatsoever between the data source and the cache layer, which makes it robust against network failures and simple to implement, even in heavily distributed systems with many independent services.

The downside lies in its fundamental imprecision: a TTL that is too short increases load on the origin system because data gets reloaded more often than actually necessary, while a TTL that is too long means users see stale information for extended periods, even though the underlying data changed long ago. In practice, TTL works especially well for data with predictable change frequency, such as product categories that change once a day, while it is often unsuitable for highly dynamic data like stock levels or prices.

3. Event-based invalidation: reacting to actual data changes

Event-based invalidation solves the precision problem of TTL by invalidating the cache not after a fixed time window, but exactly when the underlying data actually changes. As soon as a record gets updated in the database, the system publishes an event through a message queue like Kafka, RabbitMQ, or a simpler pub/sub system, which notifies every affected cache layer and either deletes the corresponding entry or overwrites it directly with the new data.

The decisive advantage is near-perfect theoretical consistency between data source and cache, because stale data is only visible during the short window between the data change and event processing. The price is increased system complexity: every code path that changes data has to reliably trigger an invalidation event, which in grown systems with many independent write paths, such as direct database migration scripts or batch jobs, is easily forgotten and leads to hard-to-find inconsistencies.

Typical event-based invalidation flow:

1. Client updates a product's price via the API
2. Application writes the new price to the database
3. Application publishes event: "product.updated" {id: 4711}
4. A cache invalidation service consumes the event
5. Service deletes/updates cache entries tagged
   "product:4711" in Redis, CDN, and application cache at once

Critical point: step 3 (publishing the event) and step 2
(the DB write) must be atomic, or at least reliably
recoverable (outbox pattern), otherwise cache entries end up
that never get invalidated.

4. Cache tagging for targeted partial invalidation

Without a structured link between data and cache entries, teams are left with two bad options: either flush the entire cache on every change, which momentarily creates massive load on the origin system, or manually and error-pronely track individual cache keys. Cache tagging resolves this dilemma by attaching one or more semantic tags to every cache entry, for example product:4711, category:electronics, or user:882, describing what content the cached entry actually depends on.

When a product changes, the system only needs to invalidate every cache entry tagged product:4711, regardless of how many different pages, API responses, or fragments reference that product in some form, say a product detail page, a category overview, and a search results page simultaneously. This targeted, partial invalidation drastically reduces the number of unnecessarily evicted cache entries compared to a full cache flush and keeps the cache hit ratio high even under frequent data changes.

5. Consistency problems across multiple cache layers

In real distributed systems, a single cache layer is rare. Typically a CDN sits between user and application, caching entire HTML pages or API responses at the network edge, with an application cache like Redis in front of or behind it for frequently read database values, and sometimes a query cache inside the database itself. Each of these layers has its own invalidation logic, its own TTL, and, crucially, its own latency when processing invalidation events.

This leads to a subtle but common consistency problem: the application cache gets invalidated correctly and instantly, while the CDN, which operates geographically distributed across multiple edge locations, only propagates the invalidation with a delay, since every individual edge node has to be notified separately. Users at different locations then see different versions of the same page for anywhere from seconds to minutes, which can become especially problematic for price-sensitive or legally relevant content such as stock availability.

6. CDN purge APIs and their limits in practice

CDN providers like Cloudflare, Fastly, or Akamai offer purge APIs that let applications invalidate individual URLs or, with more advanced providers, entire tag groups, similar in principle to application-level cache tagging. These APIs are usually fast, often propagating globally within seconds, but they come with technical and contractual limits: rate limits on the number of purge requests per time window, a maximum number of tags per request, and sometimes extra costs per purge operation at high frequency.

A common mistake in practice is triggering CDN purges synchronously on every single data change, which quickly hits the provider's rate limits on systems with high write frequency and, in the worst case, silently drops purge requests without the application ever knowing. A more robust pattern is batching invalidation events over a short window, say every five seconds, combined with deduplicating multiple changes to the same tags before actually sending the purge to the CDN.

7. Stale-while-revalidate as a pragmatic compromise

The HTTP Cache-Control directive stale-while-revalidate offers a pragmatic middle ground between strict consistency and maximum performance: when a cache entry has expired, the cache still serves the stale version to the user immediately, but triggers a background refresh from the origin system in parallel. The current user briefly still sees the old version but experiences no wait, while subsequent users already get the updated version from cache.

This pattern works especially well for content where brief inconsistency is acceptable, such as product descriptions, blog posts, or category overviews, but less well for content where freshness is business-critical, such as current prices in the checkout flow or availability indicators right before purchase. Combining stale-while-revalidate for non-critical content with event-based invalidation for critical data is, in practice, one of the most effective strategies.

8. Additional challenges in distributed systems

In distributed systems with multiple database replicas, another layer of complexity appears: eventual consistency between the primary database node and its read replicas means an invalidation event can fire before the data change has actually reached every replica. If the cache gets repopulated at that exact moment, it may still read the old version from a not-yet-updated replica, meaning the cache paradoxically ends up stale again despite correct invalidation logic.

A robust fix for this problem is to decouple the cache refresh from the invalidation event with a short, controlled delay matching typical replication lag, or, where available, to deliberately read from the primary database node instead of a replica when the request happens immediately after a known change. Read-your-write consistency patterns, which guarantee a user sees their own latest data right after their own change, solve this problem specifically for the most common and most important use case.

9. Conclusion: a combined strategy instead of a single silver bullet

Cache invalidation in distributed systems cannot be solved with a single universal strategy, because different data types have different consistency requirements. TTL-based invalidation suits rarely changing, non-critical data, event-based invalidation suits data where freshness matters, and cache tagging is what makes both approaches practical in the first place, since it enables targeted, partial invalidation instead of flushing the entire cache on every change.

The most important takeaway for teams working in distributed systems is to treat cache invalidation not as a one-time technical problem but as an ongoing trade-off between consistency, performance, and system complexity. Teams that deliberately choose the right invalidation strategy for each data category, while accounting for the added latency of multiple cache layers and eventual-consistency effects in distributed databases, get considerably closer to a workable solution to the old quote about the two hard things in computer science.

10. Summary

11. FAQ: Cache invalidation in distributed systems at a glance