Linux Audit Logging with auditd: Tracking Changes

1. What auditd captures that standard logs do not

syslog and journald only capture what applications choose to log themselves. An SSH login shows up in /var/log/auth.log, but a read access to /etc/shadow by a compromised process leaves no trace there. auditd operates on a different level: it hooks directly into the Linux kernel audit subsystem and logs system calls before the kernel executes them. That means it captures things no application log knows about, for example which process, running under which UID, opened, modified or deleted a file, regardless of whether the application itself writes a log entry for it.

For developers who administer their own servers, this becomes relevant as soon as compliance requirements such as PCI DSS or ISO 27001 come into play, or once a security incident needs to be reconstructed to find out who changed app/etc/env.php or when an attacker abused sudo rights. syslog alone rarely gets you there. auditd provides UID, AUID (the login UID, which persists even after su or sudo), PID, PPID, the full executable path and the exact syscall, timestamped down to the microsecond.

2. Architecture: kernel audit subsystem, auditd and auditctl

The Linux audit subsystem consists of three parts: the kernel module that intercepts syscalls and generates events, the userspace daemon auditd, which reads those events from the kernel ring buffer and writes them to /var/log/audit/audit.log, and auditctl, the command-line tool for loading and managing rules at runtime. Communication between the kernel and auditd runs over a dedicated netlink socket, not through normal syscalls, which makes it harder for an attacker with ordinary user privileges to tamper with the audit pipeline.

On Debian and Ubuntu, apt install auditd audispd-plugins installs the package, on RHEL and Rocky Linux it is usually preinstalled. Once installed, auditd runs as its own systemd service, independent of journald or rsyslog, and starts automatically ahead of most other services so that logging begins as early as possible in the boot process. auditctl -s shows the current status, including the number of loaded rules, the active backlog limit and the enabled flag.

#!/usr/bin/env bash
# Install and enable auditd on Debian/Ubuntu based systems
apt-get update
apt-get install -y auditd audispd-plugins

# Enable and start the daemon, independent from journald/rsyslog
systemctl enable --now auditd
systemctl status auditd --no-pager

# Check current status: rule count, backlog limit, enabled flag
auditctl -s
# AUDIT_STATUS: enabled=1 failure=1 pid=1284 rate_limit=0
# backlog_limit=8192 lost=0 backlog=0

# List all currently loaded rules
auditctl -l

3. Writing audit rules for sensitive files

The basic syntax of a file rule is auditctl -w <path> -p <permissions> -k <key>. The -w flag sets a watch on a path, -p defines the access types being watched: r (read), w (write), x (execute), a (attribute change such as chmod or chown). The -k flag assigns a freely chosen key that later lets you filter events specifically through ausearch, without having to specify the path again. For /etc/passwd, -p wa is usually enough, since read access happens through almost every process and would otherwise inflate the log volume unnecessarily.

For directories with many sensitive files, such as /etc/sudoers.d or the configuration directory of a Magento installation, a directory rule makes more sense than individual file rules, because newly created files inside it are automatically captured too. Important: rules set directly via auditctl are lost on the next reboot, they only serve for quick testing. For production use, rules belong in files under /etc/audit/rules.d/, compiled and loaded via augenrules, as described in section 5.

# /etc/audit/rules.d/10-sensitive-files.rules
# Track changes to core identity and privilege files
-w /etc/passwd -p wa -k identity
-w /etc/shadow -p wa -k identity
-w /etc/group -p wa -k identity
-w /etc/gshadow -p wa -k identity

# Track sudoers changes; the whole directory, not just the main file
-w /etc/sudoers -p wa -k privilege_escalation
-w /etc/sudoers.d/ -p wa -k privilege_escalation

# Application-specific: Magento configuration files
-w /var/www/magento/app/etc/env.php -p wa -k magento_config
-w /var/www/magento/app/etc/config.php -p wa -k magento_config

# Load the rule file immediately for testing
auditctl -R /etc/audit/rules.d/10-sensitive-files.rules

4. Syscall auditing: capturing privilege escalation and process execution

File watches only capture access through the path itself, not system calls such as setuid, execve or ptrace, which are typical steps in a privilege escalation. The syscall rule syntax is significantly more powerful: auditctl -a always,exit -F arch=b64 -S execve -F auid>=1000 -F auid!=unset -k exec_commands logs every program execution by logged-in users, not system services, including the full command line. The filter auid>=1000 separates regular user accounts from system accounts, auid!=unset excludes kernel-owned processes without an assigned login UID.

For privilege escalation, setuid, setgid and the related capability syscalls are of particular interest: a rule on setuid with the key privilege_escalation reliably shows when a process changes its effective UID, a classic pattern in exploited services. It is important to set arch=b32 in addition to arch=b64 on 64-bit systems, because 32-bit compatibility syscalls would otherwise go unobserved, a known bypass route for attackers who deliberately issue syscalls through the 32-bit interface.

# /etc/audit/rules.d/20-syscalls.rules
# Track every command execution by real (non-system) users
-a always,exit -F arch=b64 -S execve -F auid>=1000 -F auid!=unset -k exec_commands
-a always,exit -F arch=b32 -S execve -F auid>=1000 -F auid!=unset -k exec_commands

# Track privilege escalation syscalls, both architectures
-a always,exit -F arch=b64 -S setuid -S setgid -S setresuid -S setresgid -k privilege_escalation
-a always,exit -F arch=b32 -S setuid -S setgid -S setresuid -S setresgid -k privilege_escalation

# Track sudo invocations specifically, by executable path
-a always,exit -F path=/usr/bin/sudo -F perm=x -F auid>=1000 -k sudo_usage

# Track deletion or rename of files inside /etc
-a always,exit -F arch=b64 -S unlink -S unlinkat -S rename -S renameat -F dir=/etc -k etc_delete_rename

5. Making rules persistent with augenrules

Rules set only via auditctl disappear after every reboot, because they only exist in the kernel ring buffer. For persistent rules, .rules files are placed under /etc/audit/rules.d/, usually sorted by topic, for example 10-sensitive-files.rules and 20-syscalls.rules. The command augenrules --load compiles all files in the directory, in the correct order, into /etc/audit/audit.rules and loads them into the kernel immediately, without requiring a restart of the auditd service.

The file /etc/audit/auditd.conf controls the behavior of the daemon itself, independent of the actual rules: the location of the log file, maximum file size, rotation behavior and what happens when the disk fills up. The space_left_action parameter is particularly important, because an audit log filling up without a countermeasure can, in the worst case, impact the entire system through lack of disk space.

# /etc/audit/auditd.conf
log_file = /var/log/audit/audit.log
log_format = ENRICHED
flush = INCREMENTAL_ASYNC
freq = 50

# Rotation: keep 10 files of max 50 MB each
max_log_file = 50
num_logs = 10
max_log_file_action = ROTATE

# Disk space thresholds and what happens when they are hit
space_left = 200
space_left_action = SYSLOG
admin_space_left = 100
admin_space_left_action = SUSPEND
disk_full_action = SUSPEND
disk_error_action = SUSPEND

6. Querying audit logs with ausearch

ausearch is the primary tool for filtering individual events out of an audit.log that can grow to several gigabytes. The key from the rule is the most important entry point: ausearch -k identity -ts today shows all of today's events for the identity rule, meaning changes to /etc/passwd or /etc/shadow. For forensic analysis after a specific incident, you can additionally filter by a specific user ID with -ui or -ue and narrow down to an exact time window with -ts/-te.

The raw output of ausearch consists of several lines per event (SYSCALL, CWD, PATH, PROCTITLE), tied together by the same msg=audit(...) identifier. For scripts and log aggregation, --format json is much more practical in more recent auditd versions, because the result can be fed directly into jq or a log pipeline instead of parsing the multi-line text output manually.

{
  "timestamp": "2026-07-12T09:14:22.412+02:00",
  "serial": 1928374,
  "node": "web01.mironsoft.de",
  "record_type": "SYSCALL",
  "syscall": "openat",
  "success": "yes",
  "exe": "/usr/bin/vim",
  "auid": "1000",
  "auid_name": "deploy",
  "uid": "0",
  "uid_name": "root",
  "key": "identity",
  "path": "/etc/passwd",
  "comm": "vim"
}

7. Reports and summaries with aureport

While ausearch returns individual events, aureport aggregates them into overviews and statistics, ideal for regular reports rather than case-by-case investigation. aureport -au shows a summary of all authentication attempts with success and failure, aureport -f shows all file access events grouped by file, aureport -k lists all keys in use with their hit counts, a good first look at which rule fires most often.

For a weekly security report, aureport --summary combined with a time window via -ts/-te is a good fit, providing numbers on logins, file changes, process executions and anomalies in compact form, suitable for cron-driven reporting by mail. Both tools default to reading /var/log/audit/audit.log, but with -if they can also be pointed at archived, rotated log files, for example when reconstructing an incident from several weeks earlier.

8. The performance cost of aggressive audit rules

Every additional audit rule costs CPU time, because the kernel has to evaluate the filter conditions and write an event to the ring buffer, in addition to the actual operation, for every matching syscall. Watches on individual files like /etc/passwd are practically free, because they trigger rarely. Syscall-wide rules, such as an unfiltered watch on open or read on heavily used directories, can generate noticeable load, especially on I/O-intensive systems like database servers or Magento web servers with many concurrent PHP-FPM workers.

The -b parameter (backlog limit) in auditd.conf determines how many events are allowed to wait in the kernel buffer before they are dropped or the system blocks according to the failure_flag. With failure=2 (panic), an overflowing buffer can, in the worst case, cause syscalls to block until auditd has capacity again, a real risk for production systems under load. The recommendation: keep rules as tightly scoped as possible using -F filters (auid, exe, path) instead of watching entire syscall classes indiscriminately, and size the backlog generously with -b 8192 or higher rather than risking failure=2 in production.

9. auditd compared to standard logging

Standard logging via syslog or journald and kernel-level auditing via auditd solve different problems and are not mutually exclusive, they should run in parallel. The following overview shows where auditd delivers real added value compared to pure application logging.

In practice this means: syslog and journald stay responsible for application events and debugging, while auditd takes over security-relevant, tamper-resistant traceability at the kernel level. Combining both, and additionally shipping audit logs to a separate log system via audisp-remote or rsyslog forwarding, also prevents an attacker with root privileges on the compromised host from fully erasing their own tracks.

10. Summary

Linux audit logging with auditd solves a problem that standard logging structurally cannot cover: traceability at the kernel level, regardless of whether an application logs anything itself. Watches like -w /etc/passwd -p wa -k identity capture changes to sensitive files, syscall rules on execve, setuid and setresuid capture process execution and privilege escalation. Rules under /etc/audit/rules.d/ combined with augenrules --load make the configuration reboot-proof, and ausearch and aureport provide the tools for targeted case investigation and aggregated reporting.

The decisive point is the balance between capture depth and performance: blanket, unfiltered syscall rules on heavily used systems generate noticeable load and, in the worst case, risk blocking syscalls once the backlog fills up. Precise -F filters, a generously sized backlog limit and a clear separation between a few critical file watches and targeted syscall rules make auditd practical even on production Magento and PHP servers, without uncontrollably increasing system load.

11. FAQ: Linux Audit Logging with auditd