Docker Container Startup Order and Wait-For Patterns Done Right

1. The Race Condition Problem at Container Startup

A race condition at container startup is the state where a service tries to reach a dependency that is not ready yet. The most common scenario: a PHP application starts and tries to open a MySQL connection on the first request. MySQL starts more slowly because, on its first run, it executes initialization scripts, creates database files and waits for its port to become free. The PHP application gets a connection-refused error and fails, not because MySQL is fundamentally unreachable, but because MySQL simply was not ready at that moment.

The problem is not limited to database connections. Redis must be ready before a cache warmer fills it. A message queue service must be ready before workers consume messages. An API gateway must be ready before downstream services forward traffic. In every one of these cases there is a timing dependency between services that Docker Compose has to represent. Naive depends_on without further configuration only answers the question of when a container is started, not when the service behind it can actually process requests. The gap between container start and service readiness is the core of the startup order problem.

This problem is often underestimated because it does not show up in development: on a local machine with warm caches, MySQL starts fast enough that the race condition never surfaces. It becomes visible in CI environments with fresh volumes, in staging environments with a larger database, or after an unplanned restart under load. The correct fix is not making services start faster, but robust wait-for patterns that wait for actual service readiness.

2. depends_on: Container Order vs. Service Readiness

The plain depends_on in Docker Compose specifies the order in which containers are started. depends_on: [db, redis] means: start web only after the db and redis containers have been started. It says nothing about whether MySQL accepts connections immediately after starting, or whether Redis has finished initializing. A MySQL container is "started" after a few milliseconds; the MySQL process itself may not be ready to accept connections for another 10 to 30 seconds. That gap is the startup race condition window.

Compose v2 extends depends_on with the condition field, which supports three values: service_started (the old behavior: the container is running), service_healthy (waits until the health check reports healthy) and service_completed_successfully (waits for exit code 0, useful for init containers). With condition: service_healthy, the startup order problem is delegated to the health check layer: the health check must be configured correctly and actually verify service readiness, not just that the process is running. This shifts complexity from the depends_on block into the HEALTHCHECK block, which is where it belongs.

# docker-compose.yml - depends_on with health-check conditions
# Compose v2 format: condition-based startup ordering
services:
  db:
    image: mysql:8.4
    environment:
      MYSQL_ROOT_PASSWORD: secret
      MYSQL_DATABASE: app
    # Health check: only healthy when MySQL accepts real connections
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost", "-u", "root", "-psecret"]
      interval: 5s
      timeout: 5s
      retries: 10
      start_period: 30s   # Grace period for initial MySQL setup

  redis:
    image: redis:7-alpine
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 5
      start_period: 10s

  web:
    image: registry.example.com/app:latest
    depends_on:
      db:
        condition: service_healthy    # Wait until MySQL health check passes
        restart: true                 # Restart web if db container restarts
      redis:
        condition: service_healthy
    environment:
      DB_HOST: db
      REDIS_HOST: redis

  migrations:
    image: registry.example.com/app:latest
    command: ["php", "bin/console", "doctrine:migrations:migrate", "--no-interaction"]
    depends_on:
      db:
        condition: service_healthy
    restart: "no"   # Run once and exit

  web-after-migrations:
    image: registry.example.com/app:latest
    depends_on:
      migrations:
        condition: service_completed_successfully   # Wait for exit code 0
      redis:
        condition: service_healthy

3. Health-Check-Based Dependencies with condition

The health check is the key mechanism behind condition: service_healthy. A poorly configured health check makes the condition useless: it reports healthy before the service is actually ready. For MySQL, the minimal reliable health check is mysqladmin ping, which opens a connection and expects a response. A better health check also runs an SQL query: mysql -u root -psecret -e "SELECT 1". That confirms not just that the process is running, but that the database has been initialized and accepts queries.

The start_period field in the health check matters a lot for startup order: it defines an initial window during which health-check failures are not counted as failures and do not put the container into the "unhealthy" state. For MySQL with database initialization, 30 seconds is a realistic value. After start_period ends, failures count against the retries limit. This timing setup must match the actual startup behavior of the dependency: too short a start_period produces false unhealthy signals, too long a one delays detection of real failures.

An important point for depends_on with condition: service_healthy: if the dependency never reaches the healthy state (because MySQL, for example, was misconfigured), the waiting service stays stuck in its startup state indefinitely. That is actually the desired behavior for startup order, better waiting forever than starting with a broken database connection, but it does require working monitoring that detects and alerts on stuck services.

4. wait-for-it and dockerize: External Waiting Tools

wait-for-it is a shell script that waits for TCP port availability and then runs a command. It is used as an entrypoint wrapper: ENTRYPOINT ["wait-for-it", "db:3306", "--timeout=60", "--", "php-fpm"]. As soon as port 3306 is reachable on the db host, PHP-FPM is started. This pattern has one significant downside: TCP port availability is not proof of service readiness. MySQL accepts TCP connections on port 3306 before initialization is complete, and then returns a "too many connections" or "system table missing" error. wait-for-it would still proceed and start PHP-FPM, which then faces a database that is not fully initialized.

dockerize is a more comprehensive tool that can additionally check HTTP endpoints, run template rendering and monitor several dependencies at once. For applications with HTTP-based health endpoints, dockerize -wait http://db:8080/health -timeout 60s is more reliable than a TCP port check. Both tools solve the wait-for pattern at the container entrypoint level, without requiring the application itself to implement retry logic. They are useful for services that lack their own retry logic (legacy applications, third-party images) or for quick setups in development environments.

# wait-for-db.sh - Robust wait-for pattern with actual connection test
# More reliable than wait-for-it (checks real connectivity, not just TCP port)
#!/bin/sh
set -e

HOST="${DB_HOST:-db}"
PORT="${DB_PORT:-3306}"
USER="${DB_USER:-root}"
PASSWORD="${DB_PASSWORD:-secret}"
MAX_ATTEMPTS="${WAIT_TIMEOUT:-60}"
INTERVAL=2

echo "[WAIT] Waiting for MySQL at ${HOST}:${PORT}..."

attempt=0
until mysqladmin ping -h "${HOST}" -P "${PORT}" -u "${USER}" -p"${PASSWORD}" \
    --connect-timeout=5 --silent 2>/dev/null; do
  attempt=$((attempt + 1))
  if [ "${attempt}" -ge "${MAX_ATTEMPTS}" ]; then
    echo "[ERROR] MySQL not ready after $((attempt * INTERVAL))s, aborting"
    exit 1
  fi
  echo "[WAIT] Attempt ${attempt}/${MAX_ATTEMPTS}, retrying in ${INTERVAL}s..."
  sleep "${INTERVAL}"
done

echo "[OK] MySQL is ready, starting application"
exec "$@"

5. Init Container Pattern: Separating Prerequisites

The init container pattern separates one-time setup tasks from the main application into dedicated containers that must fully complete before the main service starts. Typical init containers: running database migrations, generating configuration files from vault secrets, setting file permissions, or validating schemas. In Docker Compose this is implemented with condition: service_completed_successfully: the main service waits not just for the start, but for the successful completion (exit code 0) of the init container.

The init container pattern solves a problem that neither depends_on nor wait-for-it can solve on their own: it guarantees that idempotent prerequisites have run exactly once and to completion before the service starts. A migration container that brings the database up to the current schema must be guaranteed to finish before PHP-FPM accepts its first request. Setting restart: "no" on the migration container prevents it from running again if the Compose stack restarts; the next stack start recognizes, via the Compose exit code, that the init container already succeeded.

6. Application-Side Retry Logic: The Most Reliable Approach

The most reliable solution to the startup order problem is application-side retry logic: the application itself retries the database connection repeatedly at startup before giving up. This mirrors how professional applications in distributed systems handle transient failures. A PHP application that receives a PDO exception at startup should not treat it as an immediate fatal error, but should retry with exponential backoff up to a configurable timeout. This behavior makes the application resilient against restart scenarios, temporary database outages and deployment race conditions alike.

For PHP specifically: in many frameworks, PDO connections are opened on the first database access, not at startup. That means the startup race condition often only shows up on the first request, not at container start. An explicit connection test with retry logic at startup is therefore a valuable step in the entrypoint script. The pattern: open a test connection at container start, sleep and retry on failure, abort with a meaningful error message after a timeout. The container then fails cleanly and Kubernetes or Compose can restart it, giving the database fresh waiting time.

# entrypoint.sh - Application entrypoint with retry logic for all dependencies
#!/bin/sh
set -e

# Retry function: exponential backoff with max attempts
wait_for_service() {
  local name="$1"
  local check_cmd="$2"
  local max_attempts="${3:-30}"
  local attempt=0
  local wait_time=2

  echo "[WAIT] Checking ${name}..."
  until eval "${check_cmd}" 2>/dev/null; do
    attempt=$((attempt + 1))
    if [ "${attempt}" -ge "${max_attempts}" ]; then
      echo "[ERROR] ${name} not available after ${attempt} attempts, failing"
      exit 1
    fi
    echo "[WAIT] ${name} not ready (attempt ${attempt}/${max_attempts}), retrying in ${wait_time}s..."
    sleep "${wait_time}"
    # Exponential backoff: double wait time up to 30s
    wait_time=$(( wait_time < 30 ? wait_time * 2 : 30 ))
  done
  echo "[OK] ${name} is ready"
}

# Check MySQL: real connection test, not just TCP port
wait_for_service "MySQL" \
  "mysqladmin ping -h ${DB_HOST} -u ${DB_USER} -p${DB_PASSWORD} --silent" \
  30

# Check Redis: PING command
wait_for_service "Redis" \
  "redis-cli -h ${REDIS_HOST} -p ${REDIS_PORT:-6379} ping | grep -q PONG" \
  15

# Optional: check Elasticsearch
# wait_for_service "Elasticsearch" \
#   "curl -sf http://${ES_HOST}:9200/_cluster/health | grep -q '\"status\":\"green\"'"

echo "[OK] All dependencies ready, starting application"
exec "$@"

7. Reliably Detecting Database Readiness

MySQL and PostgreSQL have different signals for "ready". For MySQL, mysqladmin ping is the minimal test, but it can occasionally report a false positive while initialization is still running. A safer test for MySQL: run a SELECT query against a system table that is only available once initialization has completed. For PostgreSQL, pg_isready -U user -d dbname is the recommended readiness check: it verifies not just connectivity, but also whether the specified database exists and accepts connections. These specific database readiness checks are more reliable than generic TCP port checks.

A critical point that is often overlooked: database readiness at startup is not the same as sustained availability. After a successful health check, a database can still become overloaded, crash, or drop off the network. Application-side retry logic for database errors is therefore necessary even after a successful start. Startup order guarantees a clean start; resilience patterns in the application guarantee correct behavior beyond that start. Both are necessary, and neither replaces the other.

8. Debugging and Diagnosing Startup Problems

Startup race conditions are hard to debug because they are timing-dependent and behave differently under load than in a quiet development environment. The first step: docker compose logs --follow shows all service logs with timestamps. The order of log messages shows which service started when and when it produced errors. The pattern "Application connecting to DB failed" shortly after "MySQL starting" is the classic race condition signal.

The second diagnostic tool: docker inspect <container-id> shows the health-check status and the results of the most recent health-check runs. For a container stuck in the "starting" state due to a failing health check, docker inspect --format='{{json .State.Health}}' shows the latest failures and their timestamps. This helps distinguish whether the health check is fundamentally misconfigured (wrong command, wrong user) or whether the service simply was not finished initializing yet (too short a start_period). Logging from the entrypoint waiting script with timestamps is valuable too: it shows how long the wait lasted and whether the database eventually became reachable.

9. Wait-For Patterns Compared Head-to-Head

Every wait-for pattern has its place depending on the requirements. The right choice depends on control over the application, the environment and the reliability requirement.

For new projects, the recommendation is: depends_on with condition: service_healthy as the base pattern, complemented by application-side retry logic for resilience after startup, and init containers for database migrations. These three patterns complement each other and together fully solve the startup order problem, without external tooling like wait-for-it that requires yet another binary in the image.

10. Summary

The container startup order problem in Docker is not a side issue, it is one of the most common causes of flaky startups in multi-service stacks. The solution has several layers: depends_on with condition: service_healthy coordinates start order at the Compose level and waits for actual service readiness. Health checks that verify real connectivity rather than just port availability are the foundation of this mechanism. Init containers isolate one-time tasks such as database migrations. Application-side retry logic provides resilience beyond the initial start.

The most important takeaway: wait-for patterns are not a workaround, they are a necessary part of any production-ready container configuration. The right pattern to choose depends on how much control you have over the images: health check plus depends_on for your own images, entrypoint waiting scripts for third-party images, and application-side retry logic as a robust foundation for every service. Combined, these three approaches produce a multi-service stack that starts reliably, regardless of the order in which services come up.

11. FAQ: Container Startup Order and Wait-For Patterns