keepalived and VRRP Basics: High Availability with a Virtual IP

1. What VRRP solves and what keepalived stands for

keepalived is a Linux implementation of the Virtual Router Redundancy Protocol, or VRRP, originally designed for redundant routers and used just as well for highly available load balancers, database servers and reverse proxies for years. The basic problem keepalived solves is simple: a single server serving a critical IP address is a single point of failure. If it goes down, the service becomes unreachable, no matter how well the application itself is engineered.

The idea behind VRRP is that multiple servers share a so called virtual IP address that is never bound to a fixed network card but is dynamically assigned to whichever node currently acts as master. If the master fails, a backup node takes over the virtual IP within a few seconds, without clients needing to change any configuration. keepalived is available as a package on practically every Linux distribution and requires no special hardware, making it one of the most cost effective solutions for high availability in the Linux world.

In production environments, keepalived is often placed in front of a load balancer pair such as HAProxy or nginx, to make the load balancers themselves redundant as well. Without this layer, the load balancer itself would again be a single point of failure, no matter how many backend servers sit behind it. The following sections explain step by step how VRRP works internally, how keepalived is configured and which pitfalls appear in practice.

2. How it works: master, backup and the virtual IP

VRRP organizes a group of routers or servers into what is called a VRRP instance group. Every instance has a unique Virtual Router ID between 1 and 255 that all nodes in the same group must share. Within this group, a master is elected by priority value, the default range is 0 to 255, with the highest value winning. The master sends multicast packets at regular intervals, one second by default, to the address 224.0.0.18 to signal its presence.

If these advertisement packets stop arriving for a defined period, the backup nodes automatically transition into the master state and take over the virtual IP address using Gratuitous ARP. This ARP packet informs every device in the local network segment that the MAC address behind the virtual IP has changed, redirecting traffic to the new master without any DNS change or manual intervention. This exact mechanism is what makes keepalived so attractive: failover happens at the network layer, transparent to any application talking to the virtual IP.

It is important to understand that VRRP only manages the IP address, not the underlying service. A server can appear perfectly healthy as a VRRP master while the actual web server or database on that node has already crashed. That is exactly why keepalived is almost always combined with health checks that verify the real service state instead of only the network connectivity of keepalived itself.

3. Installation and basic configuration

Installing keepalived is a one line package command on most distributions. After installation, the central configuration file lives at /etc/keepalived/keepalived.conf and follows its own brace based syntax reminiscent of nginx configuration files. The most important block is vrrp_instance, where the interface, Virtual Router ID, priority and the virtual IP address are defined.

One central configuration point is the state state MASTER or state BACKUP, which is only a starting value and can always be overridden by the actual priority negotiation. The authentication field auth_pass prevents foreign VRRP packets in the same network segment from disturbing your own group, though it is not a cryptographic protection but rather a safeguard against accidental ID collisions.

# Install keepalived on Debian/Ubuntu
sudo apt update && sudo apt install -y keepalived

# Install keepalived on RHEL/AlmaLinux
sudo dnf install -y keepalived

# Enable IP forwarding and non-local binding (needed on some kernels
# so the service can bind before the VIP is actually assigned)
sudo tee -a /etc/sysctl.conf <<'EOF'
net.ipv4.ip_nonlocal_bind = 1
EOF
sudo sysctl -p

# Enable and start the service after config is in place
sudo systemctl enable --now keepalived
sudo systemctl status keepalived

# /etc/keepalived/keepalived.conf on node1 (intended master)
global_defs {
    router_id node1
    enable_script_security
    script_user root
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 150
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass s3cr3tPass
    }
    virtual_ipaddress {
        192.168.10.100/24
    }
}

On the second node, the same configuration is copied with an identical Virtual Router ID, identical auth_pass and the same virtual IP, but with a lower priority and the state BACKUP. This symmetry is essential: if the Virtual Router ID differs between nodes, they form two separate VRRP groups and neither will ever take over from the other.

4. Health checks with vrrp_script

So that keepalived checks not only network reachability but also the real application state, the vrrp_script block is used. It holds an arbitrary script or a single shell command that is executed at a fixed interval. If the script returns a non zero exit code, keepalived lowers the node's priority by the amount configured in weight, allowing another node with a higher effective priority to automatically become master.

A common example is checking whether a local HAProxy or nginx process is running and responding on the expected port. Instead of only checking whether the process exists, an actual HTTP or TCP request is more meaningful, because a process can well be running yet hung or have lost all its worker threads. It is important not to choose health check intervals too aggressively, since overly short intervals can trigger unnecessary failovers during brief load spikes.

vrrp_script check_haproxy {
    script "/usr/bin/pgrep haproxy"
    interval 2      # check every 2 seconds
    weight -50      # subtract 50 from priority on failure
    fall 3          # require 3 consecutive failures before acting
    rise 2          # require 2 consecutive successes to recover
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 150
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass s3cr3tPass
    }
    virtual_ipaddress {
        192.168.10.100/24
    }
    track_script {
        check_haproxy
    }
}

5. A complete two node setup

In practice, keepalived is mostly run in a pair of two nodes, working either as an active/passive database setup or as a redundant load balancer pair in front of multiple application servers. In the active/passive model, the backup node only takes over the virtual IP when the master actually fails, while day to day all traffic flows exclusively through the master. This is conceptually simpler to operate than an active/active setup with two virtual IPs, where both nodes serve traffic simultaneously and act as each other's backup.

For an active/active setup, each node defines two vrrp_instance blocks with different Virtual Router IDs, with each node being master for one instance and backup for the other. This configuration slightly doubles the configuration effort but utilizes both servers during normal operation instead of leaving one node idle. For database failover with MySQL or PostgreSQL, the simpler active/passive model is usually preferable, since parallel write access to two database instances would require additional replication logic.

An important practical note: the virtual IP address must reside in the same Layer 2 network segment as the physical interfaces of the participating nodes, since Gratuitous ARP does not work across router boundaries. In cloud environments with their own network overlays such as AWS or Azure, classic VRRP therefore often does not work out of the box and requires additional adjustments such as rewriting routing tables via script.

6. Priority, preemption and avoiding split brain

Priority values decide which node normally becomes master, but an often overlooked detail is the behavior after a failed node recovers. By default, preempt is enabled, meaning as soon as the original master with higher priority returns, it immediately takes back the virtual IP. This sounds intuitive at first, but can lead to unnecessary additional failovers in production environments, for instance when a node is briefly unstable after a restart.

With the option nopreempt, the currently active master stays active even if a node with higher priority comes back online, until the current master itself fails. This setting reduces the number of transitions and is the safer choice in many production setups. Another, more subtle problem is split brain: if the network connection between the nodes is interrupted while both nodes themselves keep running, both can simultaneously believe they are master, and both claim the same virtual IP.

Split brain situations almost always arise from network partitioning, not from keepalived itself. A reliable safeguard is a second, independent communication path between the nodes, for example a dedicated heartbeat connection or an additional network interface, so that a failure of the primary network segment does not immediately lead to conflicting master decisions. Using unicast_peer instead of multicast can also help improve the reliability of VRRP communication in environments with unreliable multicast routing.

7. Notify scripts for automation

A plain IP switch is not enough in many scenarios, because the new master requires additional actions, such as starting a service, sending a notification, or updating a DNS route. That is exactly what keepalived notify scripts are for, executed automatically on every state transition. The three relevant hooks are called notify_master, notify_backup and notify_fault, invoked with the new state as a parameter.

In practice, notify scripts are frequently used to automatically start a dependent service on transition to master that stays idle on the backup node, such as a cache warmup process or switching a replication role in a database. Also common is sending a Slack or email notification to the operations team, so a failover does not go unnoticed even if it completed technically smoothly.

#!/usr/bin/env bash
# /etc/keepalived/notify.sh — invoked automatically on state transitions
set -euo pipefail

STATE="$1"     # MASTER, BACKUP or FAULT
INSTANCE="$2"  # VRRP instance name
PRIORITY="$3"  # current priority value

case "$STATE" in
  MASTER)
    logger -t keepalived "Node became MASTER for $INSTANCE"
    systemctl start haproxy
    curl -s -X POST -H 'Content-Type: application/json' \
      -d "{\"text\":\"Failover: this node is now MASTER for $INSTANCE\"}" \
      "https://hooks.example.com/notify" || true
    ;;
  BACKUP)
    logger -t keepalived "Node became BACKUP for $INSTANCE"
    ;;
  FAULT)
    logger -t keepalived "Node entered FAULT state for $INSTANCE"
    systemctl stop haproxy || true
    ;;
esac

8. Monitoring and troubleshooting

The first step for any keepalived problem is a look at the system logs, since keepalived logs every state transition, every priority change and every health check failure. On systemd systems, journalctl -u keepalived -f provides a live stream of every event and is usually the quickest diagnostic method for unexpected failovers. The current VRRP state of a node can also be read directly with ip addr show, since the virtual IP only appears as an additional address on the interface of the current master.

A common practical problem is that multicast packets get blocked by firewalls or cloud security groups, causing both nodes to independently become master without knowing about each other. A tcpdump capture on port 112 or the VRRP protocol number quickly shows whether advertisement packets even arrive at the other node. Tools such as arping additionally help verify whether Gratuitous ARP actually reaches the network segment after a failover.

# Live-follow keepalived state transitions
journalctl -u keepalived -f

# Show current VRRP state and priority from the running config
sudo cat /var/run/keepalived.vrrp.state 2>/dev/null || \
  sudo cat /var/log/keepalived.log | tail -50

# Check whether the virtual IP is currently bound on this node
ip addr show eth0 | grep 192.168.10.100

# Capture VRRP advertisement packets (protocol 112) for troubleshooting
sudo tcpdump -i eth0 -n vrrp

# Verify multicast connectivity between nodes
sudo arping -I eth0 192.168.10.100

9. keepalived compared to alternatives

keepalived is not the only way to implement high availability on Linux servers. Depending on requirements for cluster size, resource management and complexity, more suitable alternatives exist, covered in detail in the next section about Pacemaker and Corosync. The following table lays out the key differences.

For most small to medium setups with two or three nodes, keepalived is the most pragmatic choice, because it only handles IP assignment and is therefore much simpler to understand and debug than full blown cluster managers. Once multiple different resources need to be coordinated together, for example a filesystem, a database service and a virtual IP all at once, keepalived reaches conceptual limits and a tool like Pacemaker becomes more sensible.

10. Summary

keepalived and VRRP solve a fundamental availability problem with comparatively little configuration effort: a virtual IP address that automatically switches between multiple nodes as soon as the current master fails. The combination of priority negotiation, Gratuitous ARP and health checks via vrrp_script ensures failover kicks in for both network and application problems, instead of relying only on the pure reachability of keepalived itself.

Anyone running keepalived in production should pay particular attention to nopreempt, sensible health check intervals and a second communication path between nodes, to avoid unnecessary failovers and split brain situations. Notify scripts extend a plain IP takeover with automation, such as starting dependent services or sending notifications. For simple two to three node scenarios, keepalived remains the most pragmatic solution before more complex cluster managers like Pacemaker even need to be considered.

11. FAQ: keepalived and VRRP Basics