Placing Dragonfly as a Drop-in Redis Alternative

1. Baseline: why new Redis alternatives keep emerging at all

Redis' architecture dates back to a time when servers typically had far fewer CPU cores than is common today. The single-threaded model was a deliberate, sensible simplification back then, but it hits a structural limit on modern servers with 32 or more cores, since a single Redis process can never use more than one core for command processing, regardless of how many other cores sit idle.

Dragonfly was designed by former Google and AWS engineers as a complete new implementation, built from the ground up for modern multi-core hardware and cloud environments, rather than being retrofitted with multi-threading on top of the existing Redis codebase the way KeyDB was as a fork. This fundamentally different starting point enables architectural decisions that would hardly have been possible for a fork.

2. Dragonfly's architecture: shared-nothing multi-threading per core

Instead of shared data structures that would need to be coordinated between threads via locks, Dragonfly assigns each available CPU core its own shard of the dataset, operating completely independently of the other shards. Every key is deterministically mapped to exactly one shard, and a core only ever processes commands affecting its own shard, without needing classic mutex locks between threads.

This shared-nothing principle originates conceptually from distributed systems and is applied here within a single machine: instead of many threads competing for the same data structures, several largely independent processing units work in parallel, coordinating only where a command genuinely touches multiple shards at once, for example with certain multi-key operations.

3. A different memory structure: the Dashtable instead of a classic hash table

For actual key-value storage, Dragonfly uses a custom-built data structure called Dashtable, based on academic research into scalable hash tables and specifically optimized for good cache locality and low memory overhead per entry. Unlike Redis' classic hash table implementation with chaining on collisions, the Dashtable uses an open-addressing scheme with several segments that can grow independently of each other.

One practical effect of this structure is noticeably lower memory overhead per stored key compared to Redis, which shows up in a smaller overall memory footprint especially for very many small values, as commonly found in Magento sessions or fragment caches, while offering comparable or better access speed.

4. Snapshot performance: point-in-time backups without a classic fork

Redis classically creates RDB snapshots via a fork() system call that duplicates the entire process memory through copy-on-write, which can briefly cause significantly increased memory usage and noticeable latency spikes on very large datasets when many pages get modified during the snapshot at the same time. Dragonfly avoids this fork entirely and instead implements its own fork-free point-in-time snapshot technique directly at the Dashtable level.

Dragonfly marks entries that have changed since the start of the snapshot directly and writes out consistent versions without having to duplicate the process' entire address space. Published benchmarks show a noticeably reduced memory peak during the snapshot process compared to Redis' fork-based approach, which is particularly relevant for memory-constrained instances.

# Start Dragonfly via Docker with a Redis-compatible port
docker run -d --name dragonfly \
  -p 6379:6379 \
  --ulimit memlock=-1 \
  docker.dragonflydb.io/dragonflydb/dragonfly

# Trigger a snapshot manually, just like with Redis
redis-cli -p 6379 SAVE
redis-cli -p 6379 INFO memory | grep used_memory_peak

5. Compatibility: the Redis and Memcached protocols at the same time

Dragonfly implements Redis' RESP protocol and covers a very large share of the classic command set, from simple strings through hashes and lists to sorted sets, allowing common client libraries such as phpredis to work without code changes. Dragonfly additionally supports the Memcached protocol through the same process, which is interesting for environments that historically ran both systems in parallel.

Not every Redis command is fully covered at this point, particularly some very specific administrative commands, cluster mode details, and a few newer Redis modules that Dragonfly deliberately does not replicate. Before switching, it is therefore worth checking the commands actually used by a given Magento installation against Dragonfly's documented compatibility list.

6. Running it in practice: Docker, Kubernetes, and memory limits

Dragonfly is primarily shipped as a single container image and deliberately replaces the traditional primary-replica-Sentinel topology with a vertically scaling model: instead of spreading many small Redis instances across several cores, a single Dragonfly process runs, using its shared-nothing architecture to exploit the entire machine. For Kubernetes, this means a simpler deployment model with fewer moving parts than a classic Redis cluster setup, though also without the established operator tooling that exists for Redis.

Regarding memory limits, note that Dragonfly manages available memory jointly across all shards and makes its own internal decisions about the split, while classic Redis has a fixed maxmemory limit per instance. For a Magento setup, it is therefore advisable to size the container memory limit deliberately with enough headroom above the expected dataset size and to watch actual utilization through the metrics Dragonfly exposes.

7. Benchmark claims compared to RDB fork-based snapshotting

Benchmarks published by Dragonfly show noticeably higher throughput than comparably sized Redis under parallel workloads with many concurrent connections, particularly on machines with many cores, since the shared-nothing architecture scales nearly linearly with core count as long as multi-key operations spanning several shards remain rare.

It is important to keep in mind that vendor benchmarks are naturally produced under conditions favorable to the vendor's own product. Independent reproductions tend to confirm the general direction of the results, particularly regarding snapshot behavior under memory pressure, but consistently recommend running one's own load tests against the actual access pattern of the given application before treating marketing figures as a decision basis.

8. Maturity assessment: license, community, and production readiness

Dragonfly is released under the Business Source License, a license that makes the source code viewable and permits most forms of use, while restricting offering it as a competing managed service by third parties, and automatically converting to a fully open license after a defined period. For internal operation inside a Magento shop, this license form is not a practical obstacle.

As a comparatively young project, Dragonfly still lacks the decades-long production history of Redis, meaning less operational experience with rare edge cases, long-term stability across very different workloads, and mature failover tooling. The company behind it now offers its own managed service, which additionally signals commercial confidence in its production readiness, but does not replace independent evaluation in a given deployment context.

9. Which Magento shops benefit from a trial, and where caution applies

A trial run of Dragonfly is particularly worthwhile for shops with a very large full page cache or session dataset on machines with many CPU cores, where both the Dashtable's lower memory overhead and the fork-free snapshots could make a measurable difference over Redis, especially if existing Redis snapshots regularly cause noticeable latency spikes.

For business-critical production environments with high availability requirements, it is still advisable to first run a parallel trial in staging, observing cluster mode details, the actual range of commands used, and behavior under realistic load over an extended period before Dragonfly fully replaces the production Redis instance. A gradual switch with a clear rollback path remains the safer approach.

10. Summary

11. FAQ: Dragonfly as a Redis Alternative: The Essentials at a Glance