Planning a Kernel Version Upgrade: LTS, Rollback and Compatibility

1. Why a kernel version upgrade needs careful planning

A kernel version upgrade touches the layer that every process, every driver and every filesystem is built on top of. Unlike a userspace package update, a failed kernel upgrade can leave a server unable to boot at all after the next restart, because a driver is missing or a kernel module is no longer compatible. Anyone running production systems never plans a kernel version upgrade spontaneously, but treats it as its own small change process with a baseline snapshot, a compatibility check and a rollback plan.

The good news is that Linux distributions now offer robust mechanisms to carry out a kernel version upgrade with low risk. GRUB keeps old kernel entries in the boot menu, package managers install new kernels alongside the existing one, and DKMS automatically rebuilds third party modules for every new kernel. Still, it remains the administrator's responsibility to actively use these mechanisms instead of blindly trusting an automatic update. The sections below walk through the full process, from choosing a version to a verified rollback.

2. LTS versus mainline: choosing the right kernel line

Before a kernel version upgrade is due, it has to be clear which kernel branch even makes sense. The Long Term Support kernel (LTS) receives security and bug fixes for several years without fundamentally changing its feature set. For production servers running Magento, databases or other critical services, an LTS kernel is almost always the right choice, because stability matters more than the newest driver for brand new hardware. Mainline kernels bring more recent drivers and performance improvements, but also a higher risk of regressions that nobody wants to discover on a production server.

Distributions represent this choice differently. Ubuntu LTS maintains one main kernel for the lifetime of the release, but also offers newer kernel versions for older LTS releases through the Hardware Enablement stack (HWE). Debian stays with a very conservative kernel version per stable release. RHEL and Rocky Linux maintain their own heavily patched kernel with a long update guarantee. Anyone planning a kernel version upgrade should first decide whether to stay within the current distribution line or whether a jump to a newer HWE or mainline version is required, for example because of new NVMe controllers or network cards.

3. Documenting the current kernel state

Every kernel version upgrade starts with a baseline snapshot of the running system. Without a documented starting point, it is impossible to judge later whether a problem actually stems from the new kernel or already existed before. This includes the exact kernel version, the installed kernel packages, loaded modules and the currently active kernel parameters from the boot command line.

# Document current kernel state before any upgrade
uname -a
uname -r                          # exact running kernel version

# Installed kernel packages (Debian/Ubuntu)
dpkg -l | grep -E '^ii\s+linux-(image|headers)'

# Installed kernel packages (RHEL/Rocky family)
rpm -qa | grep -E '^kernel(-core|-devel)?-'

# Currently loaded modules as a baseline snapshot
lsmod > /root/kernel-upgrade-baseline-lsmod.txt

# Kernel command line as passed by the bootloader
cat /proc/cmdline

# Save everything into one dated report for the change record
{
  echo "=== Kernel Upgrade Baseline $(date -Iseconds) ==="
  uname -a
  cat /proc/cmdline
} > /root/kernel-upgrade-baseline.txt

This step feels trivial but is the most important safety net for a kernel version upgrade. If a network interface carries a different name after the upgrade, or a storage driver goes missing, comparing against the baseline immediately shows what changed. Without this reference, troubleshooting starts from zero, and that costs the most time during a night maintenance window.

4. Compatibility checks: drivers, modules and DKMS

The most critical part of a kernel version upgrade concerns kernel modules that are not part of the kernel tree but are rebuilt through Dynamic Kernel Module Support (DKMS). Typical examples are proprietary graphics drivers, some RAID controller drivers or VirtualBox kernel modules. DKMS remembers which kernel versions a module has already been built for, and automatically attempts a rebuild on a new kernel. If that build fails, for example because an internal kernel API changed, the new kernel still boots, but the affected device stops working.

Before every kernel version upgrade it therefore pays off to check the DKMS status and any manually installed kernel modules that are not maintained through the package manager. Drivers that produce warnings about outdated symbols in the kernel log are also a warning sign. For cloud and virtualized servers it is additionally worth checking whether the virtio drivers and the cloud init mechanism remain compatible with the new kernel version, since otherwise network or storage devices may not be detected after the reboot.

# List all DKMS-managed modules and their build status per kernel
dkms status

# Check which kernel versions a specific module was built for
dkms status -m nvidia

# Inspect a module's metadata before trusting it with a new kernel
modinfo -F vermagic virtualbox
# Output looks like: 6.1.0-18-amd64 SMP mod_unload modversions

# Search the kernel log for module or symbol warnings after boot
dmesg | grep -iE 'unknown symbol|disagrees about version|tainted'

# List modules that are not shipped by the distribution kernel package
comm -23 <(lsmod | awk 'NR>1{print $1}' | sort) \
         <(find /lib/modules/$(uname -r)/kernel -name '*.ko*' \
             -printf '%f\n' | sed 's/\.ko.*//' | sort)

5. The upgrade process on Debian/Ubuntu and the RHEL family

The actual installation step of a kernel version upgrade is unspectacular on modern distributions, because package managers install new kernel packages alongside the existing ones instead of overwriting them. On Debian and Ubuntu, apt install linux-image-generic or a specific HWE metapackage installs the new kernel without immediately removing the old one. On RHEL, Rocky Linux and AlmaLinux, dnf takes on the same task using the installonly_limit setting and keeps several kernel versions available in parallel by default.

It is important to carry out the kernel version upgrade during a maintenance window, because the new kernel only becomes active after a reboot. A plain apt upgrade without a reboot leaves the server running on the old kernel even though dpkg already reports a new kernel as installed. Live patching solutions such as Ubuntu Livepatch or kpatch for RHEL close this gap for urgent security fixes, but do not permanently replace a full version upgrade.

# Debian/Ubuntu: install the newest kernel metapackage
sudo apt update
sudo apt install --install-recommends linux-generic-hwe-24.04

# Verify which kernel packages are now on disk (old one is kept)
dpkg -l | grep linux-image

# RHEL/Rocky family: install newest kernel, keep old ones
sudo dnf install kernel kernel-devel kernel-headers
grep installonly_limit /etc/dnf/dnf.conf   # e.g. installonly_limit=3

# Confirm the new kernel is registered in the bootloader
sudo update-grub                 # Debian/Ubuntu
sudo grub2-mkconfig -o /boot/grub2/grub.cfg   # RHEL family

# Reboot into a maintenance window, not ad-hoc
sudo systemctl reboot

6. GRUB and rollback: keeping the old kernel as a fallback

The most important safety mechanism for every kernel version upgrade is not removing the previously running kernel right away. GRUB lists all installed kernel versions under "Advanced options" in the boot menu by default. If the server does not boot cleanly after the upgrade, or a critical driver is missing, the old kernel can be selected manually in the boot menu or set as the default for exactly one boot using grub-reboot, without changing the permanent GRUB configuration.

For unattended servers without physical console access, this rollback capability matters even more, since manual intervention in the boot menu over remote consoles such as IPMI or a cloud console is possible but slow. An automated boot test that specifically checks after the kernel version upgrade whether critical services start, and automatically falls back to the previous kernel on failure, significantly reduces downtime.

# List all kernel entries known to GRUB, with their menu index
awk -F\' '/menuentry / {print $2}' /boot/grub/grub.cfg

# Boot into the previous kernel exactly once, without changing defaults
sudo grub-reboot "Advanced options for Ubuntu>Ubuntu, with Linux 6.1.0-17"
sudo systemctl reboot

# After confirming stability, keep the new kernel as default,
# or roll back permanently by removing the broken package
sudo apt remove --purge linux-image-6.5.0-99-generic
sudo update-grub

# GRUB_DEFAULT=saved lets grub-reboot control the next boot only
grep -E 'GRUB_DEFAULT|GRUB_SAVEDEFAULT' /etc/default/grub

7. Kernel modules and third party drivers after the upgrade

Right after the first boot with the new kernel, a targeted check of the kernel modules is mandatory, not optional. A kernel version upgrade only counts as complete once every module that was loaded before is active again and no errors show up in the kernel log. Comparing against the baseline saved in section three immediately shows whether a module that was loaded before the upgrade is now missing.

Especially with storage and network drivers, a missing module can cause a RAID array to go undetected or a network card to stay offline, without the server refusing to boot at all. That is why the post-check of a kernel version upgrade always includes a look at ip link, lsblk and the output of systemctl --failed to find services that could not start due to missing hardware detection.

8. Testing strategy before the production rollout

A kernel version upgrade should never happen on the production system first. A staging server with an identical hardware configuration, or at least a virtual machine with the same kernel modules, uncovers most compatibility issues before they affect live operations. The test plan should include, beyond a mere boot success, functional checks for the critical services, for example whether PHP-FPM, the database and the web server are actually reachable again after the reboot.

Automated health checks right after the restart make a kernel version upgrade verifiable instead of relying on a gut feeling that "it seems to work". A simple script that checks the most important endpoints, mount points and network interfaces shortly after boot delivers a clear go or no go for the upgrade within seconds.

#!/usr/bin/env bash
# post-kernel-upgrade-check.sh — run right after the first reboot
set -euo pipefail

echo "== Kernel version =="
uname -r

echo "== Failed systemd units =="
systemctl --failed --no-legend || true

echo "== Network interfaces up =="
ip -brief link show

echo "== Mounted filesystems =="
mount | grep -E ' / | /var | /home '

echo "== Critical services responding =="
for svc in php8.4-fpm nginx mariadb; do
  systemctl is-active --quiet "$svc" && echo "OK: $svc" || echo "FAIL: $svc"
done

echo "== Kernel log errors since boot =="
dmesg --level=err,crit,alert,emerg | tail -20

9. Kernel upgrade strategies compared

There are several valid approaches to a kernel version upgrade that differ significantly in risk, effort and freshness. The choice depends on how critical the server is and how much control exists over maintenance windows.

In practice, most operators combine several rows of this table: an LTS kernel as the base, live patching for urgent security fixes between planned maintenance windows, and a full kernel version upgrade only two to four times a year with a complete test run. This combination keeps the risk small without waiting months for important security updates.

10. Summary

A safe kernel version upgrade is not a single command, but a small five step process: a baseline snapshot of the running system, a compatibility check of DKMS modules and drivers, installing the new kernel alongside the old one, a controlled reboot with a GRUB rollback option and a final verification of all critical services. Anyone who follows this sequence significantly reduces the risk of a server that fails to boot or ends up functionally limited.

The choice between LTS, HWE and mainline kernel additionally determines how often a kernel version upgrade becomes necessary at all and how high the respective risk turns out to be. For most production servers running PHP and database workloads, an LTS kernel combined with occasional live patching for security fixes is the most robust combination, complemented by a cleanly tested, full upgrade a few times a year.

11. FAQ: Planning a Kernel Version Upgrade