KeyDB: Placing the Multi-Threaded Alternative to Redis

1. Redis' single-threaded model: reasoning and consequences

Since its inception, Redis has processed all commands in a single main thread that works through incoming requests sequentially via an event loop. This design avoids classic race conditions and removes the need for complex locking mechanisms between parallel threads, since exactly one command is guaranteed to operate on the dataset at any given time, which is the basis for Redis' reputation for predictable, atomic behavior.

The price of this design is that a single Redis process only uses one CPU core for actual command processing, even on servers with sixteen or more cores. Newer Redis versions have introduced I/O threads for the raw reading and writing of network data, but actual command execution remains bound to a single thread, meaning the maximum throughput per instance stays capped by single-core performance.

2. KeyDB's core idea: genuine multi-threading for networking and command processing

KeyDB emerged as a fork of Redis with the explicit goal of breaking the single-core limitation. Instead of a single event loop, KeyDB starts multiple worker threads, each capable of independently accepting client connections, parsing, and executing commands, allowing several CPU cores to work in parallel on command processing rather than only on background tasks such as persistence or network I/O.

To keep this model consistent, KeyDB uses fine-grained locking at the level of individual keys or key ranges instead of a single global lock across the entire dataset. Two threads can therefore work on different keys at the same time, while operations on the same key remain serialized to preserve the atomicity of individual commands that Redis applications expect.

3. Architecture in detail: how locking works across multiple threads

The number of threads used is controlled through the server-threads configuration parameter, and KeyDB recommends matching this value to the number of actually available physical CPU cores rather than over-provisioning it, since additional threads beyond the core count tend to hurt rather than help due to context switching. For transaction blocks via MULTI and EXEC, as well as for Lua scripts via EVAL, KeyDB still guarantees full atomicity by effectively falling back to serialized behavior during their execution.

This trade-off means workloads with many short, independent single commands on different keys benefit the most from parallelization, while workloads with many long transaction blocks or expensive Lua scripts see little speed advantage over classic Redis, since serialization kicks in for those cases regardless.

# Match server-threads in keydb.conf to the number of cores
server-threads 4
server-thread-affinity true

# Check after startup how many threads are actually active
keydb-cli INFO server | grep -i thread

4. Compatibility with the Redis protocol: the drop-in claim in practice

KeyDB explicitly positions itself as a drop-in replacement for Redis, implementing the same RESP protocol and the vast majority of Redis commands with identical signatures, so that common client libraries such as phpredis work against KeyDB without code changes. The RDB format for snapshots also remains compatible, meaning an existing Redis dump file can be loaded directly by KeyDB without conversion.

Differences mainly show up with very recent Redis commands introduced after KeyDB's respective fork point, as well as some internal metrics and configuration parameters specific to KeyDB's multi-threading architecture that have no equivalent in standard Redis. For the bulk of classic data structure operations such as strings, hashes, lists, and sets, compatibility nonetheless remains high.

5. Benchmark results and when using multiple cores actually helps

Benchmarks published by KeyDB itself, as well as independent reproductions, show noticeably higher throughput compared to Redis, particularly for workloads with many parallel connections and short, independent commands on machines with eight or more cores. On machines with few cores, or for workloads already bottlenecked by network latency rather than CPU time, the measured advantage over Redis is markedly smaller.

For a typical Magento shop where Redis primarily serves as a full page cache and session backend, the real-world benefit depends heavily on the number of concurrent PHP-FPM workers accessing the same Redis instance in parallel. For small to mid-sized shops with limited concurrency, the difference is often small, while very high-traffic shops with many simultaneous connections can benefit noticeably from the parallelization.

6. Persistence and memory behavior with several threads active at once

For RDB snapshot creation, KeyDB behaves fundamentally like classic Redis and still relies on a fork()-based copy-on-write mechanism, so the well-known memory spikes on very large datasets can still occur unchanged. Much the same applies to AOF persistence: the write logic itself stays largely independent of command-processing multi-threading, since persistence operations already run in their own background processes or threads.

One difference shows up in per-connection memory overhead: since each worker thread manages its own buffers for incoming and outgoing data, memory demand can end up slightly higher than Redis under very many concurrent connections. For Magento setups with a moderate connection count, for instance through a PHP-FPM connection pool, this effect barely matters in practice, but it should be factored into memory sizing for very large, connection-heavy deployments.

7. Active multi-master replication as an independent additional feature

Besides multi-threading, KeyDB ships a second, independent feature: active multi-master replication, where several KeyDB instances accept writes simultaneously and replicate to each other, instead of Redis' classic primary-replica topology with exactly one writable instance. Conflicts between concurrent writes on different instances are resolved using a last-write-wins strategy.

This feature suits geographically distributed setups where several locations each need to write locally against their own KeyDB instance, but it is rarely relevant for the classic Magento use case with a central database, since Magento itself is not designed for a multi-master cache backend, and last-write-wins semantics for sessions could lead to inconsistent behavior.

8. Risks: project status after the Snap acquisition and maintenance situation

KeyDB was acquired by the backup provider Snap Inc. in 2022, which initially provided the project with additional resources. Public development activity subsequently slowed noticeably, however, and maintenance of the project now rests more with the remaining community than with a dedicated commercial team, reflected in longer response times to reported issues and less frequent major releases.

Anyone running KeyDB in production should therefore regularly monitor the project's status, open security issues, and its currency relative to newer Redis versions themselves, rather than relying on the same level of commercial support that exists for Redis Ltd or established managed services. For mission-critical production environments, this uncertainty is a factor that must be weighed against the pure performance benefit.

9. Practical use cases for Magento: when switching could actually pay off

Switching to KeyDB can pay off when an existing Redis server demonstrably hits the CPU limit of a single core under high concurrency while other cores on the machine remain unused, something that can be verified via redis-cli INFO cpu and per-core system metrics such as top. For very high-traffic Magento installations with many concurrent PHP-FPM workers, this effect can become genuinely noticeable.

For most small and mid-sized Magento shops, however, the bottleneck is rarely Redis' own CPU utilization but rather network latency, memory size, or application logic. In these cases, the additional operational risk of a less established fork outweighs the measurable benefit, which is why a careful load test before any production switch remains essential.

10. Summary

11. FAQ: KeyDB as a Redis Alternative: The Essentials at a Glance