Setting vm.overcommit_memory Correctly: Understanding Memory Overcommit

1. What memory overcommit actually means

When a process requests memory via malloc, the kernel initially reserves only virtual address space, not necessarily physical memory. Only on an actual write access to a memory page does a page fault trigger the assignment of physical memory, a mechanism called lazy allocation, which forms the basis for memory overcommit.

Overcommit means the kernel can promise more virtual memory than is physically available together with configured swap, on the assumption that most processes never actually write their entire requested memory anyway. Many programs, including PHP-FPM and the JVM in some Elasticsearch setups, reserve generous virtual memory regions up front without ever using them in full.

2. The three overcommit modes in detail

The kernel parameter vm.overcommit_memory supports three values. Mode 0, the default, enables a heuristic check: the kernel allows most requests but rejects obviously nonsensical ones, such as a single allocation significantly larger than the available address space. This heuristic is not precisely documented and can shift slightly between kernel versions.

Mode 1 disables any check entirely and allows essentially unlimited overcommit. That is exactly what Redis needs for reliable background snapshots via fork(), since the child process initially references the same virtual address space as the parent when forking, without immediately requiring extra physical memory. Mode 2, on the other hand, strictly caps total commitment at swap plus a configurable share of physical RAM via vm.overcommit_ratio or vm.overcommit_kbytes, and rejects allocations beyond that limit with an error instead of letting them fail later.

# Show the current overcommit mode
cat /proc/sys/vm/overcommit_memory

# Set mode 1 (Redis recommendation) temporarily
sysctl -w vm.overcommit_memory=1

# For mode 2: allowed share of physical RAM in percent
cat /proc/sys/vm/overcommit_ratio

3. The relationship to the OOM killer

In modes 0 and 1, the kernel can promise significantly more memory than is physically available. If many processes simultaneously try to actually write the full amount of memory they were promised, the system reaches a state where neither physical memory nor swap suffices to serve all page faults. That is exactly when the OOM killer steps in and terminates a process to free memory.

Mode 2 shifts this risk: instead of an allocation appearing to succeed and the process being terminated later by the OOM killer, the allocation itself fails immediately with an error as soon as the configured upper limit would be exceeded. Applications that handle allocation failures cleanly benefit from that, though PHP-FPM and most database systems are not prepared for it in every code path and often react to a failed malloc call with a crash as well.

4. Understanding the Redis warning in detail

For persistent background snapshots, both RDB snapshots and AOF rewrites, Redis relies by default on fork() to create a child process that freezes the dataset at the moment of the fork and writes it out at leisure. The parent process keeps running unchanged in the meantime and serves new writes via copy on write, meaning memory pages are only duplicated once actually modified.

In mode 0, the kernel can reject that fork if it heuristically decides there is not enough memory to promise for the child process's full virtual address space, even though copy on write would in practice barely need any additional physical memory. Redis proactively reports this potential risk at startup, which is why the official Redis documentation explicitly recommends mode 1 for production systems.

# Typical Redis warning in the log
# WARNING overcommit_memory is set to 0! Background save may fail under low memory condition.

# Set mode 1 permanently
echo "vm.overcommit_memory = 1" >> /etc/sysctl.d/99-redis-overcommit.conf
sysctl --system

5. Recommendation for database and cache servers

For dedicated Redis hosts, whether used as session storage or cache backend for Magento, mode 1 is the clear, upstream recommended choice, since it reliably enables fork based persistence without kernel interference. The risk that overcommit is actually exploited and the OOM killer has to intervene can be reduced to an acceptable level through generously sized RAM and consistent memory monitoring.

For MySQL or MariaDB servers, which typically operate with a fixed configured innodb_buffer_pool_size and rarely fork, overcommit is less critical, and the heuristic default mode 0 is usually sufficient here. Elasticsearch servers with their JVM heap reservation also gain little from mode 1, since the JVM typically commits its heap fully at startup instead of allocating lazily.

On mixed hosts running both Redis and other memory intensive services, mode 1 should be set system wide once Redis is running, combined with strict memory monitoring via vm.min_free_kbytes and active OOM score tuning for less critical processes, so that the right processes get terminated first in a real emergency.

6. OOM score tuning as a complement to overcommit

Regardless of the chosen overcommit mode, it pays off to deliberately control the OOM priority of critical processes. Via /proc/<pid>/oom_score_adj a value between minus one thousand and one thousand can be set, where lower values protect a process from the OOM killer and higher values make it a preferred target.

For a Redis process serving as a central cache for several Magento instances, a negative oom_score_adj makes sense, so that in an emergency a less important background process gets terminated before the cache itself. This fine tuning does not replace adequately sized RAM, but it does reduce the risk of the OOM killer hitting the wrong process.

# Preferentially protect the Redis process from the OOM killer
REDIS_PID=$(pgrep -x redis-server | head -n1)
echo -500 > /proc/${REDIS_PID}/oom_score_adj

# Check the current effective OOM score
cat /proc/${REDIS_PID}/oom_score

7. Continuously monitoring the overcommit state

The current degree of overcommit can be read from /proc/meminfo: CommitLimit shows the theoretical upper limit for promised memory, Committed_AS the sum of already promised allocations across all processes. If Committed_AS clearly exceeds physically available memory, a closer look at actual memory usage is worthwhile before an OOM event occurs.

For ongoing monitoring, a simple cron based check that logs both values regularly and alerts on a critical ratio works well, combined with checking the system log for OOM killer events, which can be identified via dmesg or the kernel log.

# Show commit limit and currently promised memory
grep -E "CommitLimit|Committed_AS" /proc/meminfo

# Search the kernel log for past OOM killer events
dmesg -T | grep -i "out of memory"

8. Common pitfalls in overcommit tuning

A common mistake is blindly setting mode 1 on every server because a Redis warning suggests it, without considering that other services on the same host also become more heavily overcommitted as a result. On shared hosts running several memory intensive services, the decision should always be made host specifically, not adopted wholesale from a blog recommendation.

Another pitfall is setting the value only temporarily with sysctl -w and forgetting to persist it permanently in /etc/sysctl.d/. After the next reboot the value silently reverts to the default mode 0, and the Redis warning reappears at the next service restart, usually at the least convenient moment.

9. Checklist for production use

Before switching a server, it should first be checked which services actually run on the host and whether any of them actively fork, as Redis does. Only if a fork-heavy service is present does a blanket switch to mode 1 make sense, otherwise the heuristic default mode 0 usually remains the more conservative choice that works just as well.

After every change to vm.overcommit_memory, restarting the affected service belongs into the verification step, to confirm the original warning has actually disappeared. The setting should also be added to server documentation or configuration management, such as Ansible or a simple provisioning script, so it does not get forgotten again on a fresh install.

Finally, a regular look at Committed_AS relative to CommitLimit belongs into production monitoring regardless of the chosen mode, since a ratio persistently close to the limit is an early warning sign of future memory pressure, long before the OOM killer actually has to step in.

# Short check command for server documentation
echo "Overcommit mode: $(cat /proc/sys/vm/overcommit_memory)"
grep -E "CommitLimit|Committed_AS" /proc/meminfo

10. Summary

11. FAQ: vm.overcommit_memory