Disk Space Analysis with du and df

1. Why df and du belong together

df and du answer two different questions that sound identical at first glance. df (disk free) reads usage statistics directly from the superblock of a mounted filesystem and answers the question: how full is this partition overall? du (disk usage), on the other hand, recursively walks a directory tree and sums up the size of every single file to answer the question: what inside this tree is taking up the space? One command returns a single figure per filesystem, the other a breakdown per directory.

In practice this leads to a fixed workflow: first run df -h to find out which partition is critically full, for example /var or /. Then use du to drill into that specific partition and narrow it down directory by directory until the actual culprit is found. Anyone who only knows one of the two tools loses valuable time during an acute disk space problem, because either the overview or the level of detail is missing.

2. df: measuring disk space at the filesystem level

df -h provides a human-readable overview of all mounted filesystems with total size, used space, free space, and percentage. It is important to specifically look for the line matching the affected mount point, for example / or /var/www, rather than just looking at the first line. On servers with separate partitions for /var/log or /tmp, one of these smaller partitions can be full while the root partition still has plenty of space, the symptom in the logs is then often a "No space left on device", even though df -h / looks harmless.

A second, often overlooked bottleneck is inodes instead of blocks: df -ih shows inode table utilization. If a directory contains millions of small files, for example PHP session files or cache fragments, the inode table can fill up even though there is still plenty of space free on the storage device itself. In this case the system also reports "No space left on device", but df -h incorrectly shows free disk space, only df -ih exposes the actual problem. df -hT additionally shows the filesystem type, which is helpful for mixed setups with ext4, xfs, and tmpfs mounts for overlay containers.

# Human-readable overview of all mounted filesystems
df -h

# Example output
# Filesystem      Size  Used Avail Use% Mounted on
# /dev/sda1        40G   38G  1.2G  97% /
# /dev/sdb1        20G  4.1G   15G  22% /var/log
# tmpfs           2.0G     0  2.0G   0% /dev/shm

# Check inode usage when "No space left" occurs despite free disk space
df -ih

# Example output
# Filesystem      Inodes  IUsed  IFree IUse% Mounted on
# /dev/sda1         2.6M   2.6M    412 100% /

# Also show filesystem type (ext4, xfs, tmpfs, overlay)
df -hT /var/www

# Check a specific partition only, ideal for monitoring scripts
df -h --output=pcent,target /var | tail -n1

3. du: finding the culprit in the directory tree

Once df has identified the affected partition, du takes over the detail work. The classic starting point is du -sh * | sort -rh | head -20 in the suspected root directory, for example /var/www or /var. The -s flag summarizes each entry into a single line instead of listing every subfile individually, -h formats sizes in human-readable form, and sort -rh sorts descending by size while taking units into account. This way the largest directory entry immediately appears at the top, without having to manually scan through the output.

For a step-by-step narrowing, du -h --max-depth=1 /var | sort -rh is ideal, since it only goes one level deep and makes the directories of one level directly comparable. Once the conspicuous subdirectory is found, the same command is repeated one level deeper until the concrete file or file group becomes visible. Important: du by default counts the blocks actually allocated on disk, not the logical file size, which can differ noticeably for heavily fragmented or sparse files. The modern tool ncdu offers the same information interactively with keyboard navigation and is a significant time saver on production servers compared to repeated du invocations.

# Largest directories in the current path, sorted descending
du -sh * | sort -rh | head -20

# Example output
# 4.2G  vendor
# 2.8G  var
# 1.1G  pub
# 340M  app

# Step-by-step narrowing: only one level deep
du -h --max-depth=1 /var | sort -rh

# Example output
# 2.6G  /var/log
# 1.8G  /var/cache
# 210M  /var/lib

# Check the size of individual log files, including the total
du -csh /var/log/*.log

# Interactive alternative for repeated analysis
ncdu /var/www

4. When df shows full but du does not add up

A classic and repeatedly confusing scenario: df -h reports a partition at 98% full, but du -sh /* ultimately sums up to noticeably less than the usage reported by df. The cause is almost always deleted, but still open files. If you delete a file with rm while a process still has it open, the kernel only removes the directory entry, the disk space itself is not released until the last process closes the file descriptor. du no longer sees the file because it has no name left in the filesystem, but df continues to count it because the blocks remain physically allocated.

A typical trigger on PHP servers: a PHP-FPM or Nginx process writes to a log file, an administrator deletes this log file directly with rm instead of rotating it via logrotate, the running process keeps its file descriptor open and happily keeps writing, without the space ever being freed again. With lsof +L1 you can find exactly such open files with a link count of less than one, the output shows the process ID, file size, and the original path with the suffix (deleted). The only clean solution is a restart or a targeted signal to the affected process so it reopens the file, simply deleting it again has no effect anymore.

# Confirm the discrepancy: df reports full, du counts less
df -h /
du -csh /* 2>/dev/null | tail -n1

# Find deleted files that are still held open
lsof +L1

# Example output
# COMMAND   PID    USER   FD   TYPE DEVICE SIZE/OFF NLINK    NODE NAME
# php-fpm  4821 www-data   12w  REG    8,1  3.4G       0  918273 /var/log/php-fpm/error.log (deleted)

# Alternative: filter specifically for "deleted"
lsof / 2>/dev/null | grep deleted

# Clean fix: release the file descriptor by reloading the service
systemctl reload php8.3-fpm

# Emergency only: truncate the file without restarting the process
# (careful, this discards any buffered log lines)
: > /proc/4821/fd/12

5. Log files: the most common space hog

On production PHP servers, log files are by far the most common cause of partitions suddenly filling up. An unusually high error rate in the application, a bug that throws an exception on every request, or a debug mode that accidentally stays active in production, can make error.log or Magento's var/log/exception.log grow to several gigabytes within a few hours. Without logrotate or a comparable rotation mechanism, this file keeps growing unbounded until the partition is full and the entire server, including the database on the same disk, grinds to a halt.

System journals are a second, often forgotten culprit. journalctl persists by default under /var/log/journal and can also take up considerable space without a size limit, especially for services with high logging frequency such as Docker containers. journalctl --disk-usage shows current usage, journalctl --vacuum-size=500M immediately caps it at a defined value. For application logs the same rule applies as for system logs: rotation and compression belong in the deployment from the very beginning, not only once the disk is already full.

# /etc/logrotate.d/magento
# Rotation for Magento and PHP-FPM logs on a production server

/var/www/html/var/log/*.log {
    daily
    missingok
    rotate 14
    compress
    delaycompress
    notifempty
    create 0644 www-data www-data
    sharedscripts
    postrotate
        systemctl reload php8.3-fpm > /dev/null 2>&1 || true
    endscript
}

/var/log/nginx/*.log {
    daily
    missingok
    rotate 30
    compress
    delaycompress
    notifempty
    create 0640 www-data adm
    sharedscripts
    postrotate
        systemctl reload nginx > /dev/null 2>&1 || true
    endscript
}

6. Cleaning up cache and session directories

Cache directories are among the space hogs that grow gradually and therefore rarely stand out immediately. In Magento, tens of thousands of small files accumulate over weeks under var/cache, var/page_cache, and var/view_preprocessed, especially when the full page cache runs on the local filesystem instead of Redis or Varnish. du -sh var/cache var/page_cache var/view_preprocessed quickly gives an overview of how much each of these folders actually occupies, often several gigabytes of pure cache bloat that can be safely deleted at any time, because it is automatically regenerated on the next request.

PHP sessions are the second classic case: the default path /var/lib/php/sessions accumulates expired session files over months if the session garbage collector is missing, often with an inode count in the six-figure range, which stresses the inode table more than the disk space itself. find /var/lib/php/sessions -name "sess_*" -mmin +1440 -delete removes sessions that have not been touched in 24 hours, noticeably more reliable than relying on the built-in PHP garbage collection, which tends to stall at high session counts. To verify the reclaimed space after cleanup, simply compare du -sh var/cache var/page_cache var/view_preprocessed and find /var/lib/php/sessions -name "sess_*" | wc -l before and after the run.

7. Old releases, vendor directories, and deployment artifacts

Anyone working with a symlink-based deployment, where every deployment creates a new directory under releases/ and the active symlink is simply repointed, accumulates a complete new vendor folder with Composer dependencies and compiled frontend code on every deploy. Without a limit on retained releases, this directory grows linearly with the number of deployments, a single Magento release with a complete vendor directory and generated static files can easily amount to several gigabytes. After a hundred deployments this quickly adds up to several hundred gigabytes consisting entirely of old, no longer needed states.

The common solution is a fixed retention limit, usually the last three to five releases, everything older is automatically deleted after every successful deploy. Deployment tools like Deployer or Capistrano already come with this logic built in, for custom scripts a simple find command that sorts releases by modification date and removes everything beyond the desired count is sufficient. The command du -sh releases/* | sort -h shows upfront how big each individual release is, and ls -1dt releases/*/ | tail -n +6 | xargs -r rm -rf removes everything except the five newest. Important: before deleting, check that none of the directories to be cleaned up is currently the target of the active symlink, otherwise you rip away the running production deployment.

8. Automated cleanup routines and monitoring

Manual cleanup actions solve an acute problem, but they do not prevent the same pattern from repeating in four weeks. A lasting solution only comes through automation: a weekly cron job or systemd timer that rotates logs, removes old releases, and deletes expired sessions, combined with a monitoring check that warns early before a partition becomes critical. A simple threshold check via df --output=pcent in a monitoring system like Prometheus with node_exporter, or a simple Nagios check, is already enough to trigger a notification at 85% usage, long before the disk actually chokes at 100%.

For infrastructure managed via Ansible, the same cleanup logic can be defined as a reusable playbook that runs consistently across multiple servers and is executed regularly via a cron trigger or a CI pipeline. The decisive advantage over ad hoc scripts on individual servers: changes to the cleanup logic are centrally versioned and rolled out uniformly across the entire fleet, instead of every server maintaining its own variant of the cleanup script that drifts apart over time.

# ansible/playbooks/disk-cleanup.yml
# Reusable playbook for regular disk space maintenance
---
- name: Disk space cleanup on PHP application servers
  hosts: app_servers
  become: true
  vars:
    releases_to_keep: 5
    session_max_age_minutes: 1440

  tasks:
    - name: Check current disk usage before cleanup
      command: df --output=pcent,target /var
      register: disk_before
      changed_when: false

    - name: Remove old Magento releases beyond retention limit
      shell: |
        cd /var/www/releases
        ls -1dt */ | tail -n +{{ releases_to_keep + 1 }} | xargs -r rm -rf
      args:
        executable: /bin/bash

    - name: Clean up expired PHP sessions
      find:
        paths: /var/lib/php/sessions
        patterns: "sess_*"
        age: "{{ session_max_age_minutes }}m"
        recurse: false
      register: old_sessions

    - name: Delete matched session files
      file:
        path: "{{ item.path }}"
        state: absent
      loop: "{{ old_sessions.files }}"

    - name: Alert if disk usage still above threshold
      debug:
        msg: "WARNING: disk usage still critical after cleanup"
      when: disk_before.stdout | regex_search('([0-9]+)%') | int > 85

9. du and df in direct comparison

Both tools measure disk space, but at fundamentally different levels and with different data sources. Anyone who knows both levels saves several detours in every disk space analysis and finds the actual culprit noticeably faster.

In practice, df and du complement each other in almost every disk space analysis: df delivers the alert, du delivers the diagnosis, and tools like lsof and ncdu close the gaps that neither of the two basic tools covers alone. Anyone who masters this combination needs no additional software for the vast majority of disk space problems on a PHP server.

10. Summary

The disk space analysis with du and df follows a clear pattern: df -h first shows which filesystem is critically full, df -ih additionally uncovers full inode tables that df -h alone misses. du -sh * | sort -rh then narrows things down level by level until the actual culprit becomes visible. When df reports full but du does not add up, there is almost always a deleted but still open file behind it, tracked down with lsof +L1 and resolved by restarting the affected process.

On PHP servers the recurring culprits are almost identical: unrotated logs, growing cache and session directories, and deployment systems that never delete old releases. Anyone who automates these three areas with logrotate, regular cache cleanup, and a fixed release retention, and additionally puts a threshold alert on df, prevents most acute disk space problems before they ever become a middle-of-the-night emergency.

11. FAQ: Disk Space Analysis with du and df