Using Docker Healthchecks the Right Way Instead of Just Opening Ports

1. The problem with port checks alone

The most common misconception in Docker deployments is that a container which has opened its port is operationally ready. In reality, applications can bind a TCP port long before their database connection is established, configuration files are loaded, or internal caches are warmed up. Anyone who resolves dependencies between containers only through depends_on without condition: service_healthy starts downstream services against an upstream that is not actually finished, resulting in cascading startup failures.

The Docker Healthcheck is the official answer to this problem. Instead of checking externally whether a port is open, Docker runs a defined command inside the container itself at a fixed interval. If that command returns exit code 0, the container is considered healthy. If it returns exit code 1, it is considered unhealthy. Exit code 2 is reserved for future use and should not be used. This simple mechanism forms the foundation for resilient, self-healing container infrastructure.

The absence of a Healthcheck means the container remains permanently in the health: starting state, something many teams dismiss as harmless but which has serious consequences in orchestration scenarios. Orchestrators such as Docker Swarm actively use healthcheck status to control deployments and automatically replace unhealthy containers.

2. The HEALTHCHECK directive in the Dockerfile

The HEALTHCHECK directive belongs directly in the Dockerfile, making it part of the image rather than the runtime configuration. That is a crucial distinction: anchoring the Healthcheck in the image ensures that every instance of the container, whether local, in staging, or in production, runs the same health test. Adding one later in docker-compose.yml is possible and overrides the image-level Healthcheck, but is not recommended for reusable base images.

The syntax is compact: HEALTHCHECK [OPTIONS] CMD command. The CMD can be a shell command (CMD curl -f http://localhost/health) or a JSON array (CMD ["curl", "-f", "http://localhost/health"]). The JSON form is preferable since it avoids shell injection and does not require a shell inside the container. HEALTHCHECK NONE can be used to explicitly disable a Healthcheck defined in a parent image, useful for test images or images that run as a sidecar without their own HTTP endpoint.

# Dockerfile: HEALTHCHECK for a PHP-FPM/Nginx web service
FROM php:8.4-fpm-alpine

# Install curl for health probes (keep layer small)
RUN apk add --no-cache curl

COPY ./app /var/www/html
COPY ./docker/php-fpm.conf /usr/local/etc/php-fpm.d/www.conf

# HEALTHCHECK: test every 30s, timeout after 5s, 3 retries before unhealthy
# startPeriod gives the container 60s grace time during startup
HEALTHCHECK --interval=30s --timeout=5s --retries=3 --start-period=60s \
  CMD curl -fsS http://localhost/health || exit 1

EXPOSE 9000
CMD ["php-fpm"]

A common mistake is omitting the -f flag on curl. Without -f, curl returns exit code 0 even if the HTTP status code is 500, so the Healthcheck reports the container as healthy even though the application is throwing errors. The -s flag suppresses the progress bar, while -S still shows an error message when something fails. This -fsS combination is the recommended pattern for HTTP-based Healthchecks.

3. Understanding interval, timeout, retries and start period

The four parameters of the HEALTHCHECK directive need to work together so a container is neither flagged unhealthy too early nor stays in a degraded state for too long. --interval defines how often the test runs; the default of 30 seconds suits most web services, while critical services should reduce it to 10 to 15 seconds. --timeout limits how long the test command is allowed to run; if the test exceeds the timeout, it counts as failed.

--retries defines how many consecutive failures are needed before the container is marked unhealthy. With the default of 3 and an interval of 30 seconds, 90 seconds pass from the first failure to the unhealthy status. The most important, and most commonly forgotten, parameter is --start-period: it gives the container a grace period during which failed checks do not count toward the retry total. A Java service that needs 45 seconds to start should get at least --start-period=60s, otherwise it gets marked unhealthy before it has even finished starting.

4. Useful healthcheck commands beyond curl

Not every container has an HTTP endpoint, and curl is not always available in the image. For lean Alpine-based images, the built-in wget with wget -qO- http://localhost/health is an alternative. Even leaner is a plain TCP check using the nc tool: nc -z localhost 3306 checks whether a port is listening without any HTTP overhead. For PHP-FPM without an HTTP frontend, there is the cgi-fcgi tool, which can fetch a status directly over the FastCGI protocol.

For applications with no network interface at all, such as worker processes or cron jobs, a file-based Healthcheck is a good fit: the process regularly writes a timestamp to a temporary file, and the Healthcheck verifies that this file exists and that the timestamp is not too old. This pattern detects hung worker processes that are still running but no longer doing any work, a state that would be completely invisible to port checks. Another option for critical services is a dedicated health endpoint in the application code that tests internal subsystems and returns detail about the current state.

# Various HEALTHCHECK patterns for different container types

# --- HTTP service with wget (no curl needed) ---
HEALTHCHECK --interval=15s --timeout=3s --retries=3 --start-period=30s \
  CMD wget -qO- http://localhost:8080/health || exit 1

# --- TCP port check (no HTTP, just connectivity) ---
HEALTHCHECK --interval=20s --timeout=5s --retries=3 \
  CMD nc -z localhost 3306 || exit 1

# --- Worker/cron: heartbeat file must exist and be fresh (< 120s old) ---
HEALTHCHECK --interval=60s --timeout=5s --retries=2 --start-period=10s \
  CMD test -f /tmp/worker.heartbeat && \
      test $(( $(date +%s) - $(stat -c %Y /tmp/worker.heartbeat) )) -lt 120 \
      || exit 1

# --- PHP-FPM status via cgi-fcgi (no HTTP server needed) ---
HEALTHCHECK --interval=30s --timeout=5s --retries=3 --start-period=20s \
  CMD SCRIPT_NAME=/status SCRIPT_FILENAME=/status REQUEST_METHOD=GET \
      cgi-fcgi -bind -connect 127.0.0.1:9000 | grep -q "^pool:" || exit 1

5. Docker Compose: depends_on with condition

Docker Compose has supported the condition property under depends_on since version 2.1, which, combined with Healthchecks, enforces real startup ordering. Without condition: service_healthy, Compose only waits for the container to have started, not for it to be ready. The result is race conditions at startup: the application tries to access a database that has opened its port but is not yet accepting connections because its initialization process is still running.

The three available conditions are service_started (default behavior: container is running), service_healthy (container is in healthy status according to its Healthcheck), and service_completed_successfully (container has exited with code 0, for init containers). This combination lets you model a real startup sequence: the database runs and is ready, then the migration script completes, then the application starts. Every dependency is secured by the Healthcheck of the preceding container.

# docker-compose.yml: startup sequencing with healthcheck conditions
services:
  db:
    image: mysql:8.4
    environment:
      MYSQL_ROOT_PASSWORD: secret
      MYSQL_DATABASE: shop
    # Healthcheck: MySQL ready to accept connections
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost", "-u", "root", "-psecret"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 30s

  migrate:
    image: myapp:latest
    command: php bin/magento setup:upgrade
    # Wait until db is healthy before running migrations
    depends_on:
      db:
        condition: service_healthy
    # Migration is a one-shot container, no healthcheck needed

  app:
    image: myapp:latest
    ports:
      - "8080:80"
    # Wait for db healthy AND migration completed
    depends_on:
      db:
        condition: service_healthy
      migrate:
        condition: service_completed_successfully
    healthcheck:
      test: ["CMD", "curl", "-fsS", "http://localhost/health"]
      interval: 30s
      timeout: 5s
      retries: 3
      start_period: 60s

6. Healthchecks for databases and message brokers

Databases are the most common dependencies in container setups, and they have specific requirements for their Healthcheck. MySQL and MariaDB offer mysqladmin ping as a built-in command that directly tests the server's readiness to accept connections. PostgreSQL has pg_isready, which checks whether the server is ready to accept connections. Both commands are considerably more reliable than a plain TCP check because they speak the database protocol and confirm a genuinely operational service.

Redis offers redis-cli ping, which confirms readiness with the response PONG. RabbitMQ has a built-in management interface whose /api/healthchecks/node endpoint returns cluster status. For Kafka there is no equally direct tool baked into the image, but a Python or shell script that runs a producer test can serve as a Healthcheck. The shared principle across all message broker healthchecks: do not just test whether the port is open, test whether the protocol responds.

7. Healthchecks in Docker Swarm and restart policies

In Docker Swarm, Healthchecks show their full value: Swarm continuously monitors the status of every container in a service. When a container becomes unhealthy, Swarm marks the task as failed and starts a new instance, on a different node if one is available. This automation is at the core of self-healing in Swarm deployments. Without Healthchecks, a degraded container stays in the routing pool and keeps accepting requests it cannot process correctly.

Restart policies complement Healthchecks at the individual container level. restart: unless-stopped in Compose or --restart on-failure:3 in Swarm define what happens after a container crash. The difference from healthcheck-based replacement: restart policies kick in on process crashes (non-zero exit code), while Healthchecks also detect hung processes that have not crashed but are no longer doing any meaningful work. Together, both mechanisms form a resilient self-healing layer.

8. Reading and debugging healthcheck status

A container's current Healthcheck status is fully accessible through docker inspect. The JSON path .State.Health contains the current status (healthy, unhealthy, starting), the last exit code, the output of the most recent test, and the last five test results with timestamps. This history is invaluable when debugging: it shows when a container became unstable, what the healthcheck output was, and how long the degraded state has persisted.

A common debugging issue: the Healthcheck fails, but the command works fine when run manually. The cause is often that the Healthcheck runs as the root user while the application runs as a different user, and firewall rules or iptables configuration produce different access rights. Another frequent problem: DNS resolution behaves differently inside the Healthcheck than inside the application, because the healthcheck process has a different network namespace view. Using docker exec as the same user that runs the healthcheck process is the most reliable way to reproduce the issue.

# Inspect healthcheck state: full details
docker inspect --format='{{json .State.Health}}' my-container | jq .

# Quick status check
docker inspect --format='{{.State.Health.Status}}' my-container

# Watch health status in real time (updates every 5s)
watch -n 5 "docker inspect --format='{{.State.Health.Status}} - {{(index .State.Health.Log 0).Output}}' my-container"

# List all containers with their health status
docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"

# Manually run the healthcheck command to debug failures
# (runs as same user as the healthcheck process)
docker exec my-container curl -fsS http://localhost/health

# Show last 5 healthcheck results with timestamps
docker inspect my-container | jq '.[0].State.Health.Log[] | {Start, ExitCode, Output}'

9. Healthcheck strategies compared

Different check strategies suit different container types. Choosing the right Healthcheck approach has a direct impact on detection speed and overhead.

Combining a protocol-native check for databases with an HTTP check for web services covers most real-world cases. For worker containers without a network interface, the heartbeat file approach is the only method that also catches logically hung processes. Checking only for process existence is not a real Healthcheck at all: a crashed and restarted process stuck in a startup loop passes that test indefinitely.

10. Summary

Docker Healthchecks solve a fundamental problem in container orchestration: distinguishing between a container that is running and one that is actually operationally ready. The HEALTHCHECK directive in the Dockerfile anchors the test in the image and ensures it runs in every environment. The parameters --interval, --timeout, --retries, and --start-period need to match the service's startup time and behavior, and --start-period in particular is frequently forgotten, leading to false unhealthy classifications.

Docker Compose with depends_on: condition: service_healthy uses the Healthcheck status to enforce real startup ordering and eliminate race conditions. In Docker Swarm, Healthchecks form the basis for automatically replacing degraded containers. Debugging through docker inspect .State.Health provides the last five test results with timestamps and output, enough information to pinpoint problems quickly.

11. FAQ: Docker Healthchecks