Using Docker Compose Profiles for Dev, Test and CI

1. The problem with multiple compose files

The classic pattern for handling multiple environments in Docker Compose looks like this: docker-compose.yml as the base, docker-compose.dev.yml for development and docker-compose.ci.yml for CI, combined via docker compose -f docker-compose.yml -f docker-compose.dev.yml up. This pattern works, but it has a fundamental drawback: every change to a service has to be manually synced across all the files. A new volume mount, a changed environment variable, or a new health check interval: in practice, each of these changes often only makes it into the main file and gets forgotten in the override files.

Docker Compose Profiles, introduced with Compose v2, solve this problem in a fundamentally different way: instead of defining services in separate files, they get tagged with a profiles key inside a single file. Services with a profile only start when that profile is activated via --profile or the COMPOSE_PROFILES environment variable. Services without a profile always start. The result is a single, complete compose.yml that works for every environment, with no sync overhead and no merge conflicts.

2. Docker Compose Profiles: fundamentals and syntax

The syntax for Docker Compose Profiles is simple: the profiles key inside a service block takes a list of profile names as strings. A service can belong to multiple profiles. Services without a profiles key belong to the default profile and always start, regardless of which profiles are active. This is the key concept: core services like the database and web server are always active, while optional services like Mailhog, seed containers or monitoring exporters are only tied to a profile.

Profiles are activated with the --profile flag on the docker compose up command: docker compose --profile dev up starts every service without a profile plus every service that carries the dev profile. Multiple profiles can be active at once: docker compose --profile dev --profile debug up. Alternatively, you can set the COMPOSE_PROFILES=dev,debug environment variable in a .env file or in the shell, and you no longer need the flag on the command itself. This makes Docker Compose Profiles ideal for automated environments where environment variables are set per context.

# compose.yml, single file for all environments using Docker Compose Profiles

services:
  # Core services: always started (no profiles key)
  app:
    image: my-php-app:${APP_VERSION:-latest}
    depends_on:
      db:
        condition: service_healthy
    environment:
      - APP_ENV=${APP_ENV:-production}

  db:
    image: mysql:8.4
    environment:
      MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
      MYSQL_DATABASE: ${MYSQL_DATABASE:-app}
    volumes:
      - db-data:/var/lib/mysql
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
      interval: 10s
      retries: 5

  # Dev-only: email testing tool, started only with --profile dev
  mailhog:
    image: mailhog/mailhog:latest
    profiles: [dev]
    ports:
      - "8025:8025"

  # Dev-only: database admin UI
  adminer:
    image: adminer:latest
    profiles: [dev]
    ports:
      - "8080:8080"

volumes:
  db-data:

3. The dev profile: Mailhog, Adminer and debug tools

The dev profile in Docker Compose Profiles is meant for tools that only make sense in local development and are not needed in CI or production. The typical candidates are: Mailhog or Mailpit for intercepting emails (instead of configuring a real SMTP server), Adminer or phpMyAdmin for database administration, Redis Commander or RedisInsight for debugging Redis contents, and possibly a local S3 replacement such as MinIO or LocalStack for AWS S3 integrations. All of these services cost unnecessary resources in production and CI and increase the attack surface.

One important detail about Docker Compose Profiles for the dev profile: if a profiled service exposes a port, it is not started in CI and does not occupy that port. This avoids port conflicts on CI runners, which often run several parallel pipeline jobs. There is also no reason to start the Mailhog container in CI when tests are not supposed to send emails anyway. The dev profile can also contain volumes with source code mounts that are not mounted in CI, a common performance win in CI pipelines.

4. The test profile: seed containers and fixtures

The test profile in Docker Compose Profiles is meant for helper services that are only needed to run the tests. The classic use case is a seed container that populates the database with test data before the tests start. With service_completed_successfully as the condition in depends_on, the test runner can wait for the seed container to finish successfully before it starts itself.

Another typical example for Docker Compose Profiles with the test profile: a dedicated test database that is freshly initialized on every test run. Instead of the production database (which runs in its own service), a separate MySQL instance with a different port and a different volume is started, only when the test profile is active. The tests run against this isolated test database and leave no state behind in the development or production database. After the test run, the test database can be removed together with its data using docker compose --profile test down -v.

# Test and CI profile additions to compose.yml

  # Test profile: isolated test database (separate from dev DB)
  db-test:
    image: mysql:8.4
    profiles: [test, ci]
    environment:
      MYSQL_ROOT_PASSWORD: test
      MYSQL_DATABASE: app_test
    tmpfs:
      # Use tmpfs for test DB: faster I/O, no persistence needed
      - /var/lib/mysql
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
      interval: 5s
      retries: 10

  # Test profile: seed container that runs once and exits
  db-seed:
    image: my-php-app:${APP_VERSION:-latest}
    profiles: [test, ci]
    command: ["php", "bin/console", "doctrine:fixtures:load", "--no-interaction"]
    depends_on:
      db-test:
        condition: service_healthy
    environment:
      DATABASE_URL: mysql://root:test@db-test/app_test

  # Test runner: waits for seed to complete before running
  test-runner:
    image: my-php-app:${APP_VERSION:-latest}
    profiles: [test, ci]
    command: ["vendor/bin/phpunit", "--testdox"]
    depends_on:
      db-seed:
        condition: service_completed_successfully
    environment:
      DATABASE_URL: mysql://root:test@db-test/app_test

5. The CI profile: lean services without dev overhead

The ci profile in Docker Compose Profiles has a different set of requirements than the dev profile: CI environments often need services that local development does not, and vice versa. In CI, source code volume mounts (live reload) are pointless, since the code is already baked into the image. Debug tools like Adminer are not needed. But a dedicated test reporting service or an integration with a coverage database could be useful in the CI profile.

An important use case for Docker Compose Profiles in CI is controlling differences in service configuration without override files. In CI, the application could start with a different APP_ENV value without needing a separate compose file for that. With profiles and the COMPOSE_PROFILES environment variable set in the CI pipeline, the profile is activated automatically. The pipeline needs no special compose commands: plain docker compose up -d is enough, and the right configuration is steered through the environment variable.

6. Combining depends_on conditions with profiles

The combination of Docker Compose Profiles and depends_on conditions enables complex, environment-dependent service dependency graphs. Since Compose v2, depends_on knows three conditions: service_started (the container is running, no health check wait), service_healthy (the health check is green), and service_completed_successfully (the container exited with code 0). The last condition is especially valuable for migration and seed containers in test and CI profiles.

One important limitation with Docker Compose Profiles and depends_on: if service B waits on service A, and service A belongs to a profile that is not active, then starting service B fails. This is a common source of errors when profiled services are used as dependencies of services without a profile. The solution: core services (without a profile) may only depend on other core services. Services with a profile can depend on core services or on other services in the same profile.

# Activate profiles via environment variable (ideal for CI)
# In .env.ci file:
COMPOSE_PROFILES=ci
APP_ENV=testing
MYSQL_ROOT_PASSWORD=ci-secret
MYSQL_DATABASE=app_test

# In CI pipeline (GitHub Actions / GitLab CI):
# Copy the CI env file and start with profiles
cp .env.ci .env
docker compose up -d

# Or explicitly with --profile flag:
docker compose --profile ci up -d

# Wait for test runner to complete and capture exit code
docker compose --profile ci run --rm test-runner
TEST_EXIT_CODE=$?

# Cleanup after tests (remove test volumes too)
docker compose --profile ci down --volumes

exit $TEST_EXIT_CODE

# Run only the dev profile (local development)
COMPOSE_PROFILES=dev docker compose up -d
# Or:
docker compose --profile dev up -d

7. COMPOSE_PROFILES and per-environment .env files

The most elegant way to activate Docker Compose Profiles per environment is to combine COMPOSE_PROFILES in a .env file with an environment-specific .env.* file that gets loaded before the compose command runs. Compose automatically loads a .env file from the current directory. If you create separate .env.dev, .env.ci and .env.test files and activate them with cp .env.dev .env or symlinks, you end up with clear, traceable configuration management.

For local development, a small helper script that sets the right .env file and starts compose is handy. Anyone using Docker Compose Profiles in a team should check the .env.* files (without secrets) into the repository and add the .env file to .gitignore. That way every developer starts from the same baseline and can switch environments with a single command. Secrets are managed separately through actual environment variables or Docker secrets.

8. Using profiles in GitHub Actions and GitLab CI

In GitHub Actions, Docker Compose Profiles are activated most cleanly through the COMPOSE_PROFILES environment variable. In the workflow YAML you set it in the env block of the job or step: COMPOSE_PROFILES: ci. Docker Compose reads this variable automatically, so docker compose up -d works without an explicit --profile flag. That keeps the workflow readable and the compose command itself unchanged: only the environment variable decides which services start.

In GitLab CI the pattern is similar: the COMPOSE_PROFILES variable is set in the job's variables section. Different pipeline stages can use different values for the variable: no profiles in the build stage, ci,test in the test stage. This makes Docker Compose Profiles a powerful tool for multi-stage CI/CD pipelines, without needing to maintain separate compose files. A common benefit in practice: CI runners need no source code volume mounts that would slow down the build, since the dev profile with those mounts simply is never activated in CI.

9. Comparing profile strategies

There are several approaches to managing compose configuration across different environments. Docker Compose Profiles are one of several options and have clear strengths and limits.

In practice, the recommended approach is to combine both: Docker Compose Profiles for selecting services (Mailhog only in dev, seed only in CI) and a small override file for real configuration differences (different ports in CI, different volume mounts in dev). This combination gives you the flexibility of both approaches without their drawbacks.

10. Summary

Docker Compose Profiles are the tool of choice whenever you want to activate services selectively for different environments without maintaining multiple compose files. Services without a profiles key always start, profiled services only start when the profile is activated via --profile or COMPOSE_PROFILES. The dev profile bundles tools like Mailhog and Adminer that are not needed in CI or production. The test profile bundles seed containers and test databases. The CI profile can activate CI-specific services and a leaner configuration.

The combination of Docker Compose Profiles for service selection and .env.* files for per-environment configuration values is the complete pattern for real projects. depends_on with service_completed_successfully enables seed containers that must finish before the test runner starts. The most common antipattern with Docker Compose Profiles is making a core service (without a profile) depend on a profiled service, which causes failures whenever that profile is not active.

11. FAQ: Docker Compose Profiles for Dev, Test and CI