cgroups v2 Explained: How Docker Implements CPU, Memory, and IO Limits

1. What cgroups are and why Docker needs them

Control groups, cgroups for short, are a Linux kernel feature that bundles process groups and, for those groups, limits, prioritizes, and measures resource usage. Docker does not invent cgroups, it uses them as one of two pillars of container isolation, alongside namespaces. While namespaces make sure a container has its own view of processes, networking, and filesystems, cgroups make sure a container cannot claim unlimited CPU, memory, or I/O bandwidth from the host.

Every started container automatically gets its own cgroup assigned, regardless of whether explicit limits were set. Without limits, that cgroup is effectively unbounded. With a flag such as --memory, the Docker daemon writes concrete numeric values into the corresponding control files of that cgroup. From then on the kernel itself, not Docker, monitors actual resource usage and intervenes once a limit is exceeded, for example by throttling CPU or invoking the OOM killer for memory.

2. The unified hierarchy in cgroups v2

In cgroups v2 there is only a single, unified directory hierarchy under /sys/fs/cgroup, known as the unified hierarchy. Every cgroup is a directory there, containing a set of control files, so-called controller interface files. For a running Docker container this cgroup typically lives at a path such as /sys/fs/cgroup/system.slice/docker-.scope/ when Docker uses the systemd cgroup driver, or /sys/fs/cgroup/docker// with the cgroupfs driver.

Inside that directory sit files such as cpu.max, memory.max, io.max, and many more, each controlling exactly one resource type. Those files can be read with a simple cat command and, with sufficient privileges, written to directly, without going through the Docker CLI at all. That makes the unified hierarchy an excellent debugging tool: whenever a limit does not behave as expected, checking these exact files first is always worthwhile before assuming a more complex cause.

# Determine the cgroup path of a running container
CID=$(docker inspect --format '{{.Id}}' my-container)
docker exec my-container cat /proc/self/cgroup

# On the host: list the container's cgroup directory
ls /sys/fs/cgroup/system.slice/docker-${CID}.scope/
# or, depending on the driver:
ls /sys/fs/cgroup/docker/${CID}/

3. cpu.max: how CPU limits get encoded

The file cpu.max holds exactly two space-separated numbers: quota and period, both in microseconds. The period defaults to 100000 microseconds, i.e. 100 milliseconds. Quota specifies the maximum amount of CPU time processes in that cgroup may consume within one period. A value of 150000 100000 means: within every 100-millisecond window, the processes may consume a combined 150 milliseconds of CPU time, which equals 1.5 CPU cores if they could run in parallel across multiple cores.

When no limit is set, cpu.max simply contains the word max instead of a quota, meaning unlimited CPU usage. This exact mechanism is what sits behind the Docker flag --cpus: docker run --cpus=1.5 translates internally into exactly the values 150000 100000 in the cpu.max file of the container's cgroup. Reading that file directly shows exactly what Docker configured behind the scenes, without going through docker inspect.

# Read a container's CPU limit directly from the cgroup
cat /sys/fs/cgroup/system.slice/docker-${CID}.scope/cpu.max
# example output: 150000 100000  --> 1.5 CPU cores

# For comparison: start Docker with --cpus=1.5
docker run -d --name cpu-test --cpus=1.5 nginx:alpine
docker exec cpu-test cat /sys/fs/cgroup/cpu.max

4. memory.max, memory.high, and memory.swap.max

For memory, cgroups v2 offers several graduated control files instead of just one hard limit. memory.max is the hard limit, matching the Docker flag --memory: if this value is exceeded and no more memory can be reclaimed, the OOM killer fires within that cgroup. Alongside it there is memory.high, a softer limit at which the kernel actively throttles processes and tries to reclaim memory before the hard limit is even reached, a kind of early warning stage.

memory.swap.max separately controls how much swap space may be used in addition to the regular memory limit. That corresponds to the Docker flag --memory-swap, though Docker's arithmetic here takes some getting used to: the value of --memory-swap denotes the combined sum of RAM plus swap, not the swap portion alone. Running docker run --memory=512m --memory-swap=512m effectively disables swap entirely, because no additional swap budget remains beyond the RAM limit.

# Inspect memory limits of a running container's cgroup
cat /sys/fs/cgroup/system.slice/docker-${CID}.scope/memory.max
cat /sys/fs/cgroup/system.slice/docker-${CID}.scope/memory.high
cat /sys/fs/cgroup/system.slice/docker-${CID}.scope/memory.swap.max

# Read current usage and peak usage since container start
cat /sys/fs/cgroup/system.slice/docker-${CID}.scope/memory.current
cat /sys/fs/cgroup/system.slice/docker-${CID}.scope/memory.peak

5. io.max: bandwidth and IOPS limits per device

The file io.max is structurally different from cpu.max and memory.max, because I/O limits are device specific: a host can have several block devices, and a limit meant for one SSD makes no sense for a different disk. That is why every line in io.max starts with the major:minor device number, followed by key-value pairs such as rbps for read bandwidth in bytes per second, wbps for write bandwidth, and riops and wiops for the respective IOPS limits.

Docker sets these values through flags such as --device-read-bps and --device-write-iops, which must explicitly reference a device, because otherwise the kernel would not know which device the value applies to. In practice, I/O limits are used far less often than CPU and memory limits, but they become critical when a single container with heavy write activity could hurt the I/O performance of every other container on the same host, for example with database or logging workloads on shared storage.

# Determine the device number of the root disk
stat -c '%t:%T' -L /var/lib/docker

# Set an IO limit: max 10 MB/s read bandwidth on /dev/sda
docker run -d --device-read-bps /dev/sda:10mb nginx:alpine

# Read the result directly from the cgroup
cat /sys/fs/cgroup/system.slice/docker-${CID}.scope/io.max
# example output: 8:0 rbps=10485760 wbps=max riops=max wiops=max

6. The difference from cgroups v1

In cgroups v1, every controller, meaning CPU, memory, blkio, and others, had its own independent directory hierarchy under its own mount point, such as /sys/fs/cgroup/cpu/ and /sys/fs/cgroup/memory/. A process could theoretically sit in different groups for different controllers, which made administration complex and encouraged inconsistencies. File names differed too: instead of memory.max, v1 used memory.limit_in_bytes, and instead of cpu.max there were separate files cpu.cfs_quota_us and cpu.cfs_period_us.

cgroups v2 replaced that fragmentation with the unified hierarchy: a process belongs to exactly one cgroup, and that one cgroup controls all resource types together through consistently named files. This simplifies not just administration but also closes a class of security issues that arose from being able to manipulate cgroup membership differently per controller. Basic support landed in Docker 20.10, and since Docker 24 it is the default on modern distributions such as Ubuntu 22.04 or newer, provided the kernel and init system offer it.

7. Inspecting a running container live

The concept becomes clearest when starting a container with limits set and watching the cgroup files live while it runs. A simple container with a CPU and memory limit can be started while a watch command reads the current resource usage in parallel. That makes it visible how memory.current rises with the load inside the container and how cpu.stat reports new values for consumed CPU time with every elapsed period.

This live observation is also the most reliable way to verify that a configured limit is actually being enforced, independent of whatever configured values docker inspect reports. It does happen that a limit is set correctly but not actually enforced, due to a misconfigured cgroup driver or an outdated Docker version. Looking directly at the cgroup files on the host is, in such cases, the only way to check the truth independently of the Docker CLI.

# Start a container with limits
docker run -d --name live-test --cpus=0.5 --memory=256m \
  polinux/stress stress --cpu 2 --vm 1 --vm-bytes 200M --timeout 60s

CID=$(docker inspect --format '{{.Id}}' live-test)
CG=/sys/fs/cgroup/system.slice/docker-${CID}.scope

# Watch live, every 2 seconds
watch -n2 "cat $CG/memory.current; echo ---; cat $CG/cpu.stat"

8. What actually changes for users in practice

For everyday Docker work, cgroups v2 changes almost nothing on the surface: the flags --cpus, --memory, and --device-read-bps work syntactically identically to how they did under cgroups v1. The difference only shows up when something does not behave as expected or when deeper debugging is needed, because the paths and in some cases the file names of the underlying control files have changed. Anyone running old debugging scripts or monitoring tools with hardcoded v1 paths like cpu.cfs_quota_us needs to adapt them for v2 hosts.

One practical advantage of v2 is more consistent and often more precise resource reporting, for example through memory.peak, which exposes the peak memory value since the cgroup was created without needing a separate monitoring tool. Tools such as docker stats or cAdvisor also tend to deliver more precise values under cgroups v2, because the underlying kernel interfaces are more consistent. Running docker info quickly shows which cgroup version and driver are active on a host.

# Check the active cgroup version and driver
docker info | grep -i cgroup
# Cgroup Driver: systemd
# Cgroup Version: 2

# Alternative: check directly at the kernel level
mount | grep cgroup2
stat -fc %T /sys/fs/cgroup/

9. cgroups v1 and v2 side by side

The most important structural differences are best summarized in a table, because paths, file names, and the underlying architecture all change at once between the two versions. Anyone still working on an older host running cgroups v1, for example CentOS 7 or older Debian releases, should know these differences in order to translate debugging commands correctly between both worlds.

For new deployments the recommendation is clear: use cgroups v2 wherever the kernel and distribution support it, because resource management is more consistent and debugging through the unified hierarchy is considerably simpler there. The table below lays out the central differences as a quick reference for the next time you switch between a v1 host and a v2 host.

10. Summary

11. FAQ: cgroups v2 Limits: The Essentials at a Glance