ZFS on Linux: Fundamentals for Servers With High Data Integrity Requirements

1. Why ZFS matters for high data integrity requirements

Classic combinations of RAID controller, LVM, and ext4 or XFS trust that the underlying hardware works flawlessly. Silent data corruption, so called bit rot, is usually only noticed once an application is already working with corrupted data, because none of the involved layers actually validates file content against a checksum.

ZFS was built from the ground up to solve exactly this problem. Every block gets a checksum on write, which gets recomputed and compared on every read. If it does not match and redundancy exists through mirroring or RAIDZ, ZFS automatically repairs the corrupted block from an intact copy, a process called self healing that runs in the background without any manual intervention.

2. Architecture: pools, vdevs, and datasets

A ZFS pool combines several physical storage devices into one shared storage resource, replacing both the classic RAID controller and the volume manager. Within a pool, one or more vdevs, virtual devices, form the actual redundancy layer, for example as a mirror of two disks or as a RAIDZ group with distributed parity.

Any number of datasets can be created on a pool, each with its own properties such as compression, quota, or access permissions, without reserving a fixed size for any of them. A dataset behaves like an ordinary directory to applications, but shares the entire free space of the pool with every other dataset, quite similar to Btrfs subvolumes.

3. Installation via OpenZFS

Due to license incompatibility between ZFS's CDDL and the Linux kernel's GPL, ZFS is not part of the mainline kernel and must be installed as a separate kernel module through the OpenZFS project. On Debian and Ubuntu based distributions, installation usually goes through DKMS, which automatically rebuilds the module against the current kernel on every kernel update.

After installation and either a reboot or manually loading the kernel module, the command line tools zpool and zfs become available, handling the entire remaining lifecycle of pools and datasets without ever requiring a system restart for individual operations.

# Install OpenZFS on Debian/Ubuntu via DKMS
apt update
apt install -y zfsutils-linux zfs-dkms

# Load the kernel module manually and check its version
modprobe zfs
zfs version

4. Practical example: creating a pool

A pool is created with zpool create and requires at least one vdev type as an argument. For production servers needing redundancy, RAIDZ1 is the common choice with three to five disks, while a mirror of two disks makes more sense for smaller setups with better random I/O performance, since RAIDZ inherently scales worse than a plain mirror for small, random writes.

It is important to consistently use stable device identifiers from /dev/disk/by-id instead of the classic /dev/sdX naming, since the latter can change between reboots, and a pool referencing the wrong devices may, in the worst case, no longer be importable.

# Create a RAIDZ1 pool from three disks, using stable IDs
zpool create datapool raidz1 \
  /dev/disk/by-id/ata-DISK1 \
  /dev/disk/by-id/ata-DISK2 \
  /dev/disk/by-id/ata-DISK3

# Check pool status and redundancy state
zpool status datapool

5. Practical example: a dataset for database storage

For database workloads, a dedicated dataset with adjusted properties is worth setting up instead of relying on ZFS defaults. Recordsize should match the database's typical page size, commonly 16 kilobytes for MySQL with InnoDB, so ZFS does not read and write larger blocks than the database actually requests.

Compression should generally stay enabled, since the modern lz4 algorithm compresses and decompresses so fast that it actually gains performance in most cases instead of costing any, because less physical data has to be read off disk. The sync parameter, on the other hand, controls behavior for synchronous writes and should not be set to disabled for databases without a very good reason, since transactions could otherwise be lost in the event of a crash.

# Dataset for the MySQL data directory with adjusted properties
zfs create -o recordsize=16K -o compression=lz4 -o atime=off \
  datapool/mysql

# Check current properties of a dataset
zfs get recordsize,compression,atime datapool/mysql

6. Checksums, self healing, and scrub in detail

Every block gets a fletcher4 checksum by default, stored in ZFS's Merkle tree like metadata structure, so not just payload data but the metadata structure itself is protected against corruption. If a checksum mismatches on read and a redundant copy exists through mirroring or RAIDZ parity, ZFS transparently serves the correct version and rewrites the corrupted block automatically.

A regular scrub proactively reads every block in the pool and compares it against its checksum, instead of waiting for a random read access to accidentally uncover a fault. For production servers, a weekly scrub is recommended, ideally via a cron job or the bundled systemd timer unit, since it catches faults early while enough redundancy for a repair still exists.

# Trigger a scrub manually and watch progress
zpool scrub datapool
zpool status datapool

# Enable a recurring scrub via systemd timer, if available
systemctl enable --now zfs-scrub-weekly@datapool.timer

7. Comparison to Btrfs and LVM with ext4

Btrfs shares similar core principles with ZFS through checksums and copy on write, but its parity based RAID levels such as RAID5 and RAID6 are still considered less mature and are avoided by many administrators for that reason, while ZFS RAIDZ has been production stable for many years. Conversely, Btrfs is already part of the mainline kernel and needs no separate kernel module, which simplifies installation considerably.

LVM combined with ext4 or XFS remains the pragmatic choice when checksums and self healing are not a hard requirement, since both filesystems have been proven for decades, are well documented, and are compatible with virtually every tool. Where data integrity is a central requirement, however, such as financial data or critical production databases, ZFS currently offers the most mature combination of checksums, self healing, and flexible volume management in a single layer.

8. ARC cache and RAM requirements in server operation

ZFS uses the Adaptive Replacement Cache, or ARC, by default, which reserves a substantial portion of available memory for frequently read blocks, making ZFS appear noticeably more RAM hungry compared to ext4 or XFS. On database servers that already manage their own buffer pool, such as the InnoDB buffer pool, the ARC competes directly with it for the same memory, which is why an explicit upper limit should be set via the zfs_arc_max parameter.

As a rule of thumb, the ARC should be sized so that enough memory remains available for the application, the operating system, and the database buffer pool, typically a quarter to at most half of total available memory, depending on how memory hungry the actual application is.

# Permanently cap the ARC at 8 GiB
echo "options zfs zfs_arc_max=8589934592" > /etc/modprobe.d/zfs.conf
update-initramfs -u

# Check current ARC size at runtime
arc_summary | grep -A2 "ARC size"

9. Best practices and pitfalls in daily operation

Snapshots on ZFS are just as cheap as on Btrfs and should be taken before every risky change, but they also start costing disk space over time as the original and the snapshot diverge. A pool should never be filled all the way to capacity, since ZFS's copy on write architecture noticeably loses performance once it reaches roughly ninety percent full, as less and less contiguous free space remains available for new blocks.

Anyone who started with ZFS before 2021 remembers the limitation that an existing RAIDZ vdev could not later be extended with additional disks. Since OpenZFS 2.13, that is exactly what the RAIDZ expansion feature allows, albeit with a few caveats regarding the actual space distribution after expansion that should be carefully reviewed before relying on it in production.

10. Summary

11. FAQ: ZFS on Linux