Data Loss Scenarios in Redis and How to Guard Against Them

1. Why Redis can lose data under certain conditions

Data loss in Redis is rarely the result of a single catastrophic bug, but usually the consequence of a combination of asynchronous mechanisms that work unnoticed in normal operation but reach their limits under stress conditions like a crash, a network partition or a failover. Redis is deliberately optimized for speed, and practically every performance optimization, from asynchronous fsync to asynchronous replication, buys that speed with a clearly defined, but often underestimated, data loss window.

This article brings together the most important data loss scenarios from persistence, eviction and replication and places them into a shared risk model. Instead of looking at each topic in isolation, the goal here is to understand how different failure sources can reinforce each other and which concrete safeguards effectively reduce each specific risk. The article ends with a practical checklist that can be applied directly to production Redis deployments.

2. Scenario 1: crash before the next fsync

The most classic data loss scenario concerns AOF persistence: with appendfsync everysec, the default, fsync runs at most once per second. If the Redis process or the underlying hardware crashes between two fsync calls, every write command that was sitting in the operating system buffer but not yet physically on disk during that last second is lost. At high write throughput, that can easily be thousands of individual operations.

This scenario becomes even more critical with appendfsync no, where the timing of fsync is left entirely to the operating system kernel. Depending on the kernel's dirty page settings, this window can stretch to several minutes. Anyone using this setting without a deliberate risk assessment frequently underestimates the actual data loss potential, especially on instances serving as a primary store for data with no other backup.

3. Scenario 2: replica promotion and unacknowledged writes

Redis replication is asynchronous by default: the master acknowledges a write command to the client as soon as it has been processed locally, regardless of whether a replica has already received the change. If the master fails immediately after this acknowledgment, before the change reaches the replica, and the replica is subsequently promoted to the new master, that already-acknowledged write is irrevocably lost. This scenario is particularly insidious because the client already received a successful acknowledgment and has no reason to suspect a problem.

The WAIT numreplicas timeout command reduces this data loss risk by explicitly making the client wait until a defined number of replicas have confirmed the change before the application proceeds. This turns part of the asynchronous replication into a semi-synchronous acknowledgment, but costs additional latency per write. For data where any loss is unacceptable, WAIT combined with a sufficient number of replicas is one of the most effective safeguards against data loss from replica promotion.

# WAIT command to reduce risk for critical writes
redis-cli SET account:balance:8842 "1500.00"
redis-cli WAIT 1 1000
# Waits up to 1000ms for acknowledgment from at least 1 replica
# Return value: number of replicas that actually acknowledged

# Check how far a replica lags behind the master
redis-cli --no-raw INFO replication | grep -E "slave0|master_repl_offset"
# slave0:ip=10.0.1.12,port=6379,state=online,offset=48213021,lag=0

4. Scenario 3: split-brain and duplicate writes after failover

Split-brain occurs when a network partition causes the old master to keep accepting write commands while, at the same time, Sentinel or a cluster majority has already promoted a replica to be the new master. Both instances now accept writes in parallel without knowing about each other. When the network partition later heals, the old master must yield to the new master's configuration and becomes its replica. Every write the old master accepted during the partition is lost in the process, because it replaces its entire dataset with that of the new master.

This data loss from split-brain cannot be eliminated entirely, but it can be significantly reduced: min-replicas-to-write and min-replicas-max-lag configure the master to refuse writes once fewer than the defined number of replicas are connected with acceptable lag. An isolated old master that has lost connection to its former replicas will, under this configuration, stop accepting new writes on its own, which drastically shortens the window for conflicting writes.

In Redis Cluster, the cluster's own gossip mechanism additionally provides part of this safeguard, because a node that can no longer reach a majority of the other master nodes automatically marks the cluster state as fail and refuses writes. This built-in majority logic is a structural advantage over pure Sentinel monitoring, but does not replace the explicit configuration of min-replicas-to-write at the node level.

# redis.conf: limit the split-brain window through write refusal
min-replicas-to-write 1
min-replicas-max-lag 10

# Check the current configuration and connected replicas
redis-cli CONFIG GET min-replicas-to-write
redis-cli INFO replication | grep connected_slaves

5. Scenario 4: diskless sync interruption and incomplete replicas

Another often overlooked data loss risk arises during the initial full sync of a new replica, especially with diskless replication enabled and repl-diskless-load swapdb. If the network connection drops during the transfer after the master has already started the atomic swap of the replica's dataset, the replica can be left temporarily without a fully functional dataset. If this incomplete replica is mistakenly promoted to master at that moment, for example through a misconfigured Sentinel, significant data loss results, well beyond the usual replication window.

The safeguard against this scenario lies mainly in monitoring the sync state: a replica should never be considered a failover candidate while master_sync_in_progress is active, or while master_link_status has not been up for a sufficient amount of time. Sentinel accounts for this state in principle, but only with correctly configured timeout values that match the actual sync duration of your own environment.

6. Scenario 5: eviction of primary store data through the wrong policy

A structural, but frequently overlooked, path to data loss is the wrong maxmemory-policy on an instance holding business-critical data with no backup in an origin database. If allkeys-lru or allkeys-lfu is accidentally configured on an instance that is actually meant to serve as a primary store, Redis actively evicts data under memory pressure that it should have kept, without any error or warning. This form of data loss differs fundamentally from the other scenarios, because it does not come from a failure, but from regular, correctly functioning behavior operating under the wrong configuration.

The safeguard is clear: primary store instances should always use noeviction, combined with sufficient memory headroom and active alerting on used_memory utilization, instead of relying on any eviction policy. Regular audits of the active maxmemory-policy per instance, especially after configuration changes or migrations, prevent a cache configuration from accidentally ending up on a primary store instance.

# Audit script: check maxmemory-policy per instance against expected role
for instance in primary-store-1 primary-store-2 cache-1; do
  policy=$(redis-cli -h "$instance" CONFIG GET maxmemory-policy | tail -1)
  echo "$instance: $policy"
done
# Expectation: primary-store-* reports "noeviction", cache-* reports allkeys-lru/lfu

7. Sentinel and Cluster: how automatic failover changes the risk

Automatic failover via Sentinel or Redis Cluster significantly reduces downtime, but at the same time changes the data loss risk profile. Without automatic failover, a failed master stays offline until a human intervenes manually, which hurts availability but rules out every one of the race scenarios described above between old and new master. With automatic failover, downtime drops to seconds, but that very speed increases the risk that a promotion happens before the last write has been safely replicated.

Sentinel offers several levers to mitigate this race, such as down-after-milliseconds, failover-timeout and the quorum configuration, but none of them eliminate the risk entirely as long as the underlying replication stays asynchronous. Anyone who needs maximum consistency has to use WAIT and min-replicas-to-write in addition to Sentinel, because Sentinel itself only controls the detection and execution of failover, not the consistency guarantee of individual writes.

An often underestimated aspect is the choice of the quorum value itself: too low a quorum increases the risk of a premature, false failover decision during a temporary network glitch, too high a quorum unnecessarily delays legitimate failover decisions. The right balance depends on the number of Sentinel instances and their distribution across independent network segments or availability zones.

# sentinel.conf: align timeouts and quorum with each other
sentinel monitor mymaster 10.0.1.10 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 60000
sentinel parallel-syncs mymaster 1

8. A practical data loss risk checklist

For a structured assessment of the data loss risk of an existing Redis instance, a systematic review along the following points is recommended: first, which appendfsync strategy is active, and does it match the criticality of the data? Second, does at least one replica exist, and is WAIT used for critical writes? Third, is min-replicas-to-write configured to limit split-brain writes? Fourth, is the maxmemory-policy correctly matched to the instance's role, cache or primary store?

Fifth, is the sync state of replicas continuously monitored, and is it ensured that incompletely synchronized replicas are never considered failover candidates? Sixth, do regular, tested backups exist in addition to replication and persistence, since replication propagates human error, such as an accidental FLUSHALL, just as reliably as legitimate writes? These six points cover the most common practical data loss causes and should be part of every Redis production review.

# Automated risk assessment script (excerpt)
echo "=== Persistence ==="
redis-cli CONFIG GET appendfsync
redis-cli CONFIG GET maxmemory-policy

echo "=== Replication ==="
redis-cli CONFIG GET min-replicas-to-write
redis-cli CONFIG GET min-replicas-max-lag
redis-cli INFO replication | grep -E "role|connected_slaves|slave[0-9]"

echo "=== Backup ==="
redis-cli CONFIG GET save
find /backup/redis -name "*.rdb" -mtime -1 | wc -l
# Expectation: at least 1 recent backup from the last 24 hours

echo "=== Eviction policy per instance role ==="
redis-cli CONFIG GET maxmemory-policy

9. Combining safeguards

No single measure eliminates the data loss risk in Redis entirely, because every safeguard carries its own residual risk: appendfsync always protects against crashes but not against split-brain. WAIT protects against replica promotion loss but costs latency and does not protect against an accidental FLUSHALL. min-replicas-to-write limits the split-brain window but does not prevent data loss from the wrong eviction policy. Only the combination of several complementary measures, aligned with the scenarios described in this article, reduces the overall risk to an economically acceptable level.

The single most important structural safeguard nonetheless remains classic: regular, tested backups outside the Redis replication chain. Replication reliably propagates every error, technical or human, to every connected instance. An isolated, time-delayed backup is the only safeguard that also protects against scenarios none of the previously mentioned data loss measures address, such as an accidentally executed destructive command on the master.

10. Summary

Data loss in Redis typically does not come from a single mistake, but from the interplay of asynchronous fsync timing, asynchronous replication and automatic failover under time pressure. The five central scenarios, crash before fsync, replica promotion, split-brain, diskless sync interruption and the wrong eviction policy, can be significantly contained with targeted, combined measures: appendfsync matched to criticality, WAIT for critical writes, min-replicas-to-write against split-brain, the correct maxmemory-policy per data role, and continuous monitoring of the replication state.

The practical checklist from section 8 offers a structured starting point for a data loss risk audit of any production Redis instance. In the end, one thing stands out: every individual measure reduces one specific risk, but only independent backups outside the replication chain reliably protect against the full range of possible data loss scenarios, including human error.

11. FAQ: Data Loss Scenarios in Redis