udev Rules: Automatically Handling Device Events

1. How udev processes kernel device events

Whenever the kernel detects a device, whether at boot or when a USB drive is plugged in later, it sends a so called uevent over the netlink socket to userspace. udev, an integral part of systemd for several years now, receives that event, enriches it with additional information from sysfs, and decides how to react based on its rules.

This separation between kernel and userspace is deliberate: the kernel itself has no concept of arbitrary device naming, permissions for specific user groups, or automatically starting a script. udev handles all of that in userspace, which keeps configuration flexible and changeable without a kernel restart, while the kernel itself stays lean and generic.

# Watch uevents in real time while a device is plugged in or removed
udevadm monitor --udev --property

2. Rule syntax: match keys and assign keys

Every udev rule consists of a combination of match keys, written with a double equals sign ==, which determine whether a rule applies at all, and assign keys with a single equals sign =, which set an action or property. Typical match keys are KERNEL for the kernel assigned device name, SUBSYSTEM for the device class such as block or net, and ATTR{...} for attributes from sysfs.

Assign keys such as SYMLINK+=, OWNER=, MODE= or RUN+= determine what happens to the device once all match conditions of a rule are satisfied. The plus sign on SYMLINK+= and RUN+= matters: it appends to existing values instead of overwriting them, so multiple rules can independently contribute additional symlinks or actions.

3. Creating custom rule files in /etc/udev/rules.d

Distributions ship default rules in /usr/lib/udev/rules.d/, which can be overwritten by a package update. Custom rules therefore belong exclusively in /etc/udev/rules.d/, which takes precedence over the default rules and stays untouched by package updates. The file name follows a convention of a two digit priority number followed by a descriptive name, for example 70-persistent-storage.rules.

The priority determines the processing order: lower numbers are evaluated first. Rules in the 60 to 69 range are traditionally reserved for persistent device naming, while rules starting at 90 typically trigger actions such as starting scripts or services after basic properties have already been set.

# Create a custom rule file, numbered for processing order
sudo nano /etc/udev/rules.d/70-persistent-storage.rules

# After changes: reload rules without restarting the server
sudo udevadm control --reload-rules
sudo udevadm trigger

4. Practical example: persistent device names for storage

Classic kernel names like /dev/sdb are not guaranteed to stay stable, because the order in which devices are detected at boot can vary, for example when an external disk is detected slightly later than expected. For storage in production servers, especially with RAID or LVM setups, a stable, hardware bound name is therefore essential, one that does not shift with boot order.

A custom rule can read the disk's serial number from sysfs and turn it into an additional, guaranteed stable symlink. In addition, udev already ships default symlinks under /dev/disk/by-id/, /dev/disk/by-uuid/ and /dev/disk/by-path/, which are sufficient for most cases and should be used in /etc/fstab instead of the raw /dev/sdX name.

# /etc/udev/rules.d/70-persistent-storage.rules
# Create a custom, stable symlink based on the serial number
SUBSYSTEM=="block", KERNEL=="sd?", ENV{ID_SERIAL}=="WD-WCC4N1234567", \
  SYMLINK+="disk/backup-volume"

# List symlinks already generated automatically by udev
ls -la /dev/disk/by-id/

5. Testing and debugging rules with udevadm

Before a new rule goes live, udevadm info shows every available property of an already present device that could serve as a match key in a rule. This is the first step with any new rule, because it is far more reliable to read actually existing attributes than to guess them.

For simulating the actual rule run, udevadm test replays rule processing for a specific device and logs in detail which rules would apply and which actions would fire, without actually changing the system. Only after a successful test run should udevadm trigger actually apply the rules to running devices.

# Show all properties of a device usable as match keys
udevadm info --query=all --name=/dev/sdb

# Simulate rule processing without changing the system
udevadm test /sys/class/block/sdb

# Actually apply rules to already present devices
udevadm trigger --subsystem-match=block

6. RUN+= actions and common timing pitfalls

The assign key RUN+= launches an external program once a rule matches. A common beginner mistake is trying to start a long running process directly through it: udev rules must be processed extremely fast, and a blocking call inside RUN+= delays processing of all subsequent uevents, in the worst case system wide.

The correct approach for longer actions is to launch a standalone systemd transient service via RUN+="/usr/bin/systemd-run ...", running outside the udev processing pipeline. That keeps the actual udev rule short and fast, while the real work runs asynchronously in a separate process.

# Wrong: blocks udev processing
# RUN+="/usr/local/bin/backup-new-disk.sh"

# Right: launches an independent, non blocking service
RUN+="/usr/bin/systemd-run --no-block /usr/local/bin/backup-new-disk.sh"

7. How udev and systemd device units work together

systemd automatically creates a so called device unit for every device recognized by udev, provided it is marked with the sysfs property SYSTEMD_WANTS or a matching rule. Other systemd units can then reference this device unit in their Requires= or After= directive and only start once the corresponding device is actually available.

This pattern is especially relevant for storage: a mount unit for an external backup volume can be configured with Requires=dev-disk-by\x2did-... so that systemd automatically waits until the device has been recognized and named by udev, instead of failing at boot because the device is not yet ready.

8. Practical example: setting persistent names for network interfaces

Servers with multiple network interfaces suffer from the same underlying problem as storage devices: the kernel assigned name such as eth0 or eth1 depends on detection order at boot and can suddenly change after a kernel update or when an additional card is added. Modern distributions therefore default to predictable names such as enp3s0, derived from the card's physical PCI position, which stay stable as long as the hardware is not physically moved.

For cases where even these predictable names are not enough, for example when two identical cards need to be clearly named by their role on the network, a custom udev rule allows an even more targeted mapping based on the MAC address. That way the interface for the internal storage network can reliably be named storage0, regardless of whatever name the kernel itself would have assigned.

# /etc/udev/rules.d/71-net-storage-interface.rules
# Uniquely name a network interface based on its MAC address
SUBSYSTEM=="net", ACTION=="add", ATTR{address}=="00:1a:2b:3c:4d:5e", \
  NAME="storage0"

# Check the current mapping of names to MAC addresses
ip -o link show

9. Best practices and debugging in a running system

Every custom rule should start with match keys as specific as possible, to avoid unexpected matches against other devices. An overly broad match such as just SUBSYSTEM=="block" without further restriction potentially matches every disk in the system and can unintentionally override existing rules or create conflicts.

For live debugging, journalctl -u systemd-udevd provides the udev daemon's own log output, while udevadm monitor shows in real time which uevents are actually arriving. This combination makes it visible whether a problem actually lies in a custom rule, or whether the expected uevent is simply never fired by the kernel in the first place.

10. Summary

11. FAQ: udev Rules