Detecting and Systematically Debugging Container Restart Loops

1. What a Restart Loop Actually Means

A restart loop happens when a container runs with a restart policy such as always, on-failure, or unless-stopped, but the main process exits with an error immediately after starting. Docker enforces that restart policy strictly and starts the container again according to the configuration, which lets it crash again with the same error, repeating the cycle indefinitely.

In this state, the container status typically flips constantly between Restarting and briefly Up, which is easy to observe with docker ps. Unlike Kubernetes, plain Docker has no explicit CrashLoopBackOff status with exponential backoff, but the Docker Engine internally increases the wait time between restart attempts over time to limit system load.

# Spot a restart loop from the status column
docker ps -a --filter "name=webshop_app"
# STATUS: Restarting (1) 3 seconds ago

# Show the number of restarts so far
docker inspect webshop_app --format '{{.RestartCount}}'

2. Step 1: Capture Logs Before the Last Crash

The single most important first step is to capture the container's log output before another restart overwrites the relevant error messages in the terminal. The plain docker logs command only shows the current run, while combining the --since option with a redirect into a file lets you permanently save the relevant excerpt instead of losing it in a scrolling terminal.

Especially valuable is the fact that Docker keeps the logs of a container's previous run available under the same container ID, as long as the container has not been fully removed, effectively giving you previous-run behavior without any extra flag. A simple redirect into a file also prevents output from getting lost in the terminal buffer during a very fast crash cycle.

# Save all logs so far with timestamps
docker logs -t webshop_app > /tmp/webshop_app_crash.log 2>&1

# Only the last 200 lines before the current restart
docker logs -t --tail 200 webshop_app

# Capture live while the next crash happens
docker logs -f webshop_app | tee /tmp/webshop_live.log

3. Step 2: Temporarily Disable the Restart Policy

As long as the restart policy stays active, Docker automatically restarts the container after every crash, which makes targeted debugging harder, since the container may already be restarting again while you are still analyzing the previous failure. The most reliable next step is therefore to temporarily set the restart policy to no, so the container stays in a stopped state after a crash instead of immediately restarting.

With the container stopped, you can then read the last exit code, which often already provides a decisive clue: exit code 1 usually points to a general application error, exit code 137 to a kernel-enforced SIGKILL, frequently triggered by an out-of-memory event, and exit code 143 to a regular SIGTERM that the application failed to handle cleanly within its grace period.

# Temporarily disable the restart policy
docker update --restart=no webshop_app

# Check the exit code of the last run
docker inspect webshop_app --format '{{.State.ExitCode}}'

# After analysis: restore the policy
docker update --restart=unless-stopped webshop_app

4. Common Cause 1: Missing or Wrong Environment Variable

One of the most common causes of restart loops is a missing, empty, or misnamed environment variable that the application strictly depends on at startup, such as database credentials or an API secret. Many application frameworks deliberately abort startup with a clear error when a required variable is missing, which shows up as a fast, repeated crash sequence, especially once the actual error message gets buried in a flooded log.

Comparing the environment variables actually set inside the container against what the application expects, for instance via a .env.example file, usually uncovers such problems within seconds. Particularly tricky are silent typos in a variable name, which the application does not recognize as missing but simply ignores, falling back to a default value that only causes an error later on.

# Check the environment variables actually set in the container
docker exec webshop_app env | sort

# Compare against the expected configuration
docker exec webshop_app env | sort > /tmp/actual.env
sort .env.example > /tmp/expected.env
diff /tmp/expected.env /tmp/actual.env

5. Common Cause 2: Database Not Ready Yet

Another classic pattern occurs in multi-container setups where an application tries to connect to a database immediately on startup, while the database itself is still initializing. The database container already reports Up, but does not yet accept connections, since internal initialization steps such as creating system tables are still running, which can take several seconds for MySQL and PostgreSQL on the very first start.

The application then receives a connection error and exits, after which the restart policy starts it again, often faster than the database finishes initializing, leading to several failed attempts in a row. The reliable fix is a healthcheck on the database container combined with depends_on and the service_healthy condition in docker-compose, instead of relying on plain process uptime.

# docker-compose.yml: correct dependency on a healthcheck
services:
  db:
    image: mysql:8.0
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
      interval: 5s
      timeout: 3s
      retries: 10
  app:
    depends_on:
      db:
        condition: service_healthy

6. Common Cause 3: An Overly Aggressive Healthcheck

Paradoxically, an overly strict healthcheck can itself trigger a restart loop, when combined with an external orchestrator or restart script configured to actively restart a container on a failed health status. A too-short interval combined with an application that needs slightly longer to start than expected, for instance due to a cache warmup, causes the container to be marked unhealthy while it is still in a normal startup phase.

The start_period healthcheck option is exactly the right tool for this: it defines a grace period during which failed healthchecks are not yet counted as failures, giving the application time for its actual initialization. If this option is forgotten or set too tight, an otherwise working container is falsely flagged as persistently unhealthy and gets unnecessarily restarted by higher-level systems.

# Healthcheck with a sufficient grace period
docker run -d \
  --health-cmd="curl -f http://localhost/health || exit 1" \
  --health-interval=10s \
  --health-timeout=3s \
  --health-retries=3 \
  --health-start-period=45s \
  webshop_app

7. Memory Problems: When the Kernel Kills the Container Itself

If a container was started with a fixed memory limit and the application exceeds it, the Linux kernel's out-of-memory killer steps in and forcibly terminates the main process with SIGKILL. This case reliably shows up as exit code 137, along with a kernel warning that appears outside the container in the host system log, while the container itself often cannot emit a meaningful error message anymore, since the process is terminated without warning.

To confirm this cause, it is worth checking the OOMKilled field in the Docker inspect output, which explicitly indicates whether the last crash was triggered by the out-of-memory killer. The fix lies either in raising the memory limit, or, more often the right call, in analyzing and fixing an actual memory leak in the application itself.

# Check whether the last crash was caused by an OOM kill
docker inspect webshop_app --format '{{.State.OOMKilled}}'

# Compare the current memory limit against actual usage
docker inspect webshop_app --format '{{.HostConfig.Memory}}'
docker stats webshop_app --no-stream

8. Preventing Restart Loops Instead of Debugging Them

Most of the causes described here can already be defused before the first crash, by writing applications so that a missing required configuration produces a clear, immediately recognizable error message instead of failing cryptically. An equally useful pattern is a built-in retry mechanism with a bounded number of attempts for external dependencies such as database connections, instead of terminating the entire process on the very first connection failure.

In Compose setups, it is also worth deliberately avoiding an overly aggressive restart policy during active development, for instance by temporarily using on-failure:3 instead of always, so a broken container stops after a few attempts and leaves room for analysis, instead of running indefinitely and flooding the logs with identical error messages.

# Bounded restart policy during development
docker run -d --restart=on-failure:3 webshop_app

# Equivalent in docker-compose.yml
# restart: on-failure:3

9. The Systematic Debugging Flow at a Glance

Instead of checking causes in random order for every restart loop, a fixed sequence pays off: first capture the logs, then disable the restart policy, then read the exit code, then narrow down between environment variables, dependencies, healthcheck configuration, and memory limits based on that exit code. This flow can be completed entirely within a few minutes and covers the vast majority of cases seen in practice.

The table below maps the most common exit codes and symptoms to their most likely causes, to steer troubleshooting from the very first observation instead of starting from zero for every new incident.

10. Summary

11. FAQ: Debugging Restart Loops: The Essentials at a Glance