PHPUnit Test Reports and JUnit XML in CI, Output Done Right
AI generated
Tags PHPUnit
@test
assert
PHPUnit · JUnit XML · CI/CD · GitHub Actions · GitLab CI
PHPUnit Test Reports and JUnit XML in CI, Output Done Right
from --log-junit to the HTML coverage report

Anyone who treats PHPUnit results as nothing more than terminal output is throwing away most of the value that automated tests provide. JUnit XML, HTML coverage, Teamcity output, and structured test duration reports make test results analyzable for CI systems, teams, and monitoring, well beyond a single build run.

18 min read JUnit XML · HTML coverage · GitHub Actions · GitLab CI · Jenkins PHPUnit 10/11 · PHP 8.2-8.4

Table of Contents

  1. 1. Why test reports are more than terminal output
  2. 2. JUnit XML: format, configuration, and pitfalls
  3. 3. Coverage reports: HTML, Clover, and Cobertura
  4. 4. Integration with GitHub Actions
  5. 5. Integration with GitLab CI and Jenkins
  6. 6. Test duration reports and identifying slow tests
  7. 7. Configuring artifacts and retention correctly
  8. 8. Common mistakes in CI output
  9. 9. Output formats compared
  10. 10. Summary
  11. 11. FAQ

1. Why test reports are more than terminal output

PHPUnit's terminal output is useful for a developer running tests locally: it is green, red, or yellow, shows failing tests with a stack trace, and prints a summary at the end. For a CI pipeline, a dashboard, or a team report, terminal output is practically worthless. It is not machine readable, not persistable, and not comparable across build runs.

A structured test report solves this problem. JUnit XML is the de facto standard format for test results in CI systems, because it is natively read and visualized as a trend graph by GitHub Actions, GitLab CI, Jenkins, CircleCI, Azure DevOps, and nearly every other CI platform. An HTML coverage report gives developers a clickable overview of which lines of code are covered by tests. A test duration report shows which tests dominate the build time. Together, these reports turn automated tests into a lasting quality asset for the team, not just a gating mechanism in deployment.

This article shows how to correctly configure PHPUnit reports for various CI systems, what pitfalls arise with JUnit XML in PHP projects, and how to integrate coverage reports into pipelines with minimal overhead.

2. JUnit XML: format, configuration, and pitfalls

The JUnit XML format is, in practice, an unofficial standard: there is no official specification, but nearly all CI systems expect the same basic schema with <testsuites>, <testsuite>, and <testcase> elements. PHPUnit produces valid JUnit XML via the --log-junit parameter or via the phpunit.xml configuration. The configuration in phpunit.xml is preferable because it is versioned, reproducible, and independent of the CI script.

A common pitfall: PHPUnit produces JUnit XML with UTF-8 encoding, but some CI parsers do not tolerate XML special characters in test names. Test class names with colons, angle brackets, or special characters in data provider arguments can cause the parser to fail reading the file or to display incorrect test names. The solution is to keep data provider test names short and alphanumeric, or to name them explicitly. PHPUnit 10+ sanitizes test names more aggressively before writing the XML file than earlier versions, which has improved compatibility.


<?xml version="1.0" encoding="UTF-8"?>
<!-- phpunit.xml - PHPUnit configuration with JUnit XML and Coverage -->
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
         bootstrap="vendor/autoload.php"
         cacheDirectory=".phpunit.cache"
         executionOrder="depends,defects"
         requireCoverageMetadata="false"
         beStrictAboutCoverageMetadata="false">

  <testsuites>
    <testsuite name="Unit">
      <directory>tests/Unit</directory>
    </testsuite>
    <testsuite name="Integration">
      <directory>tests/Integration</directory>
    </testsuite>
  </testsuites>

  <!-- JUnit XML for CI test result reporting -->
  <logging>
    <junit outputFile="reports/junit.xml"/>
    <testdox-text outputFile="reports/testdox.txt"/>
  </logging>

  <!-- Coverage report - requires Xdebug or PCOV -->
  <coverage>
    <report>
      <html outputDirectory="reports/coverage/html" lowUpperBound="50" highLowerBound="90"/>
      <clover outputFile="reports/coverage/clover.xml"/>
      <cobertura outputFile="reports/coverage/cobertura.xml"/>
      <text outputFile="reports/coverage/coverage.txt" showOnlySummary="true"/>
    </report>
    <include>
      <directory suffix=".php">src</directory>
    </include>
    <exclude>
      <directory>src/DataFixtures</directory>
      <file>src/Kernel.php</file>
    </exclude>
  </coverage>

</phpunit>

This configuration produces four different reports in the reports/ directory after every test run. This directory is typically not checked into the repository, but uploaded as a CI artifact. Important: the directory must exist before PHPUnit is invoked, otherwise PHPUnit fails with a fatal error when writing the file. A mkdir -p reports/coverage in the CI script before the PHPUnit call prevents this error.

3. Coverage reports: HTML, Clover, and Cobertura

PHPUnit supports several coverage output formats, each optimized for a different purpose. HTML is the only format directly readable by humans: it produces a navigable view of all classes and methods with color-coded highlighting of covered and uncovered lines. HTML coverage is well suited for developers who want to check locally which lines of their new function are not yet covered by tests.

Clover XML is the preferred machine format for code coverage services such as Coveralls, Codecov, and SonarQube. It contains counters for lines, methods, classes, and branches. Cobertura XML is the format used by GitLab CI and Azure DevOps for built-in coverage visualization. Text coverage is the most compact summary and is suited for direct printing to CI logs. The choice of format depends on which tools exist in the stack, typically two or three formats are produced in parallel to serve different recipients.

Coverage reports are computationally expensive: Xdebug in coverage mode can extend the test runtime by a factor of five to ten. The solution is to produce coverage reports only in a separate CI job that does not block the critical build path. Unit tests without coverage run quickly on every commit; the coverage job runs only on the main branch or overnight.

4. Integration with GitHub Actions

GitHub Actions has introduced native support for test summaries via the $GITHUB_STEP_SUMMARY mechanism in recent years. In addition, there are marketplace actions that read JUnit XML and output it as a pull request comment or action summary. The simplest and lowest-maintenance solution is the combination of a direct --log-junit and the official actions/upload-artifact for persistence.


# .github/workflows/tests.yml
name: PHPUnit Tests

on:
  push:
    branches: [main, develop]
  pull_request:

jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup PHP
        uses: shivammathur/setup-php@v2
        with:
          php-version: '8.4'
          extensions: mbstring, xml, ctype, dom
          coverage: none   # no coverage = faster

      - name: Install dependencies
        run: composer install --no-interaction --prefer-dist

      - name: Create reports directory
        run: mkdir -p reports/coverage

      - name: Run Unit Tests
        run: vendor/bin/phpunit --testsuite Unit --log-junit reports/junit.xml

      - name: Upload test results
        uses: actions/upload-artifact@v4
        if: always()   # upload even on failure
        with:
          name: phpunit-results
          path: reports/junit.xml
          retention-days: 30

  coverage:
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    steps:
      - uses: actions/checkout@v4

      - name: Setup PHP with Xdebug
        uses: shivammathur/setup-php@v2
        with:
          php-version: '8.4'
          extensions: mbstring, xml, xdebug
          coverage: xdebug

      - name: Install dependencies
        run: composer install --no-interaction --prefer-dist

      - name: Run tests with coverage
        run: |
          mkdir -p reports/coverage
          XDEBUG_MODE=coverage vendor/bin/phpunit \
            --coverage-clover reports/coverage/clover.xml \
            --coverage-html reports/coverage/html

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-report
          path: reports/coverage/
          retention-days: 14

The key detail in this workflow is if: always() on the upload of the JUnit file. Without this condition, the artifact is not uploaded when PHPUnit exits with an error code, which is precisely when the results are most useful. GitHub Actions treats an upload step without if: always() as skipped when a preceding step failed. The JUnit report must be uploaded even (and especially) when tests fail, so that CI dashboards can analyze failure patterns.

5. Integration with GitLab CI and Jenkins

GitLab CI has native support for JUnit XML via the reports: junit keyword in the artifact configuration. GitLab automatically reads the file and shows the test results in the merge request interface, including a comparison against the target branch. For Cobertura coverage, GitLab shows inline in the diff which lines are covered by tests, a very effective tool for code reviews.


# .gitlab-ci.yml
stages:
  - test
  - coverage

unit-tests:
  stage: test
  image: php:8.4-cli
  before_script:
    - apt-get update -qq && apt-get install -y -qq git unzip
    - curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
    - composer install --no-interaction --prefer-dist
    - mkdir -p reports/coverage
  script:
    - vendor/bin/phpunit --testsuite Unit --log-junit reports/junit.xml
  artifacts:
    when: always     # crucial: upload even on failure
    reports:
      junit: reports/junit.xml
    paths:
      - reports/junit.xml
    expire_in: 30 days

coverage-report:
  stage: coverage
  image: php:8.4-cli
  before_script:
    - pecl install xdebug
    - docker-php-ext-enable xdebug
    - composer install --no-interaction --prefer-dist
    - mkdir -p reports/coverage
  script:
    - XDEBUG_MODE=coverage vendor/bin/phpunit
        --coverage-cobertura reports/coverage/cobertura.xml
        --coverage-html reports/coverage/html
        --coverage-text
  coverage: '/^\s*Lines:\s*\d+.\d+\%/'  # GitLab extracts this for badge
  artifacts:
    reports:
      coverage_report:
        coverage_format: cobertura
        path: reports/coverage/cobertura.xml
    paths:
      - reports/coverage/
    expire_in: 7 days
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

The regular expression after coverage: extracts the coverage percentage from PHPUnit's text output and shows it as a badge in the GitLab repository interface. Jenkins uses the JUnit plugin to process JUnit XML: after the PHPUnit step, junit 'reports/junit.xml' is called in the Jenkinsfile pipeline. Jenkins then automatically records the test trend across all builds.

6. Test duration reports and identifying slow tests

Long test runtimes are a common problem in PHP projects that remains invisible without measurement. PHPUnit offers the --log-events-verbose-text and --report-useless-tests parameters to give hints about problematic tests. Test durations can be evaluated more systematically via JUnit XML: the format contains a time attribute in seconds on every <testcase> element, which can be evaluated with simple shell tools or Python.

The --order-by=duration flag (PHPUnit 10+) runs the slowest tests first and makes it immediately visible in the terminal output which tests dominate the overall runtime. A test that takes three seconds is almost always a sign of an architectural problem in a unit test suite: real database connections, unmocked HTTP requests, or poorly mocked services. The solution does not lie in the test configuration but in the test design itself.

7. Configuring artifacts and retention correctly

The retention period for test artifacts is a trade-off between storage cost and analysis depth. JUnit XML files are small (a few kilobytes to a few megabytes) and can easily be kept for 90 days to enable trend analysis over several months. HTML coverage reports are significantly larger (10-100 MB for larger projects) and should be kept for a shorter period, typically 7-14 days.

For long-term trend analysis, it is worth exporting the core data from JUnit XML into a time series database. Simple shell scripts can extract the number of tests, the failure rate, and the total duration from JUnit XML and write them into InfluxDB or Prometheus. Grafana dashboards then visualize the development of these metrics over weeks and months, a considerably more powerful tool than the built-in trend chart of most CI systems.

CI system JUnit XML Coverage format Configuration
GitHub Actions upload-artifact + marketplace actions Clover via Codecov action if: always() on upload
GitLab CI Native via reports: junit Cobertura native when: always in artifacts
Jenkins JUnit plugin (junit step) Cobertura plugin Jenkinsfile post-always block
CircleCI store_test_results Clover via external service path in test_results_path
Azure DevOps PublishTestResults task Cobertura native testResultsFiles: '**/junit.xml'

8. Common mistakes in CI output

The most common mistake: the reports/ directory does not exist when PHPUnit tries to write the JUnit file. PHPUnit does not automatically create missing directories and instead aborts with a fatal error. A mkdir -p reports/coverage before the PHPUnit call in the CI script is mandatory. The second common mistake: coverage reports are enabled in every build job, which slows down the entire pipeline significantly. Coverage should be limited to a separate, optional job.

A third mistake concerns the XDEBUG_MODE environment variable. Without XDEBUG_MODE=coverage, Xdebug 3.x produces no coverage data, even if Xdebug is installed and loaded. PHPUnit issues a warning in this case, but the coverage files remain empty. The fourth mistake: JUnit XML is not found in the CI system because the path is wrong relative to the working directory. Absolute paths or an explicit cd before the PHPUnit call prevent this problem.

9. Output formats compared

PHPUnit offers various output formats, each optimized for a different recipient. Choosing the right format for the respective context reduces complexity and improves how well the test results can be evaluated within the team.

10. Summary

Structured PHPUnit test reports turn automated tests from a gating mechanism into a lasting quality instrument. JUnit XML is the universal format for CI systems and is natively supported by GitHub Actions, GitLab CI, Jenkins, and all other relevant platforms. The configuration belongs in phpunit.xml, not in CI scripts, so it stays versioned, reproducible, and independent of the CI system.

Coverage reports slow down tests considerably and should be moved into separate, non-blocking jobs. The if: always() or when: always pattern on artifact uploads is mandatory for test results, since the file is needed precisely when tests fail. Long-term trend analysis comes from exporting core data from JUnit XML into time series databases and visualizing it in Grafana dashboards.

PHPUnit Test Reports and JUnit XML, the essentials at a glance

JUnit XML configuration

Configure in phpunit.xml under <logging>. Run mkdir -p reports/ before the PHPUnit call. Always set if: always() on the CI artifact upload.

Coverage in a separate job

Coverage reports cost runtime. Separate job, main branch only. Do not forget XDEBUG_MODE=coverage. Clover for external services, Cobertura for GitLab/Azure.

CI system specifics

GitLab: reports: junit native. GitHub: upload-artifact + if: always(). Jenkins: JUnit plugin. All: test artifact paths before going to production.

Analyzing test duration

--order-by=duration shows slow tests first. Evaluate the time attribute in JUnit XML programmatically. Slow unit tests point to architectural problems.

11. FAQ: PHPUnit test reports and JUnit XML in CI

1Why does PHPUnit fail when writing the JUnit XML file?
PHPUnit does not automatically create missing directories. A mkdir -p reports/coverage before the PHPUnit call in the CI script is mandatory. Without the directory, PHPUnit aborts with a fatal error when writing the file.
2Why does GitHub Actions show no test results?
GitHub Actions has no native JUnit display without marketplace actions. Upload the artifact with upload-artifact. For PR display, use additional actions such as dorny/test-reporter.
3Why is the coverage file empty even though Xdebug is installed?
Xdebug 3.x requires XDEBUG_MODE=coverage as an environment variable. Without this variable, coverage is disabled even if Xdebug is loaded.
4Which coverage format for GitLab CI?
Cobertura XML for the native GitLab coverage visualization in the merge request. Additionally Clover XML for external services such as Coveralls or Codecov. HTML for local developers.
5How do I extract the coverage percentage for a GitLab badge?
With coverage: '/^\s*Lines:\s*\d+.\d+\%/' in .gitlab-ci.yml. GitLab automatically extracts the first match from the job output.
6Why move coverage into a separate CI job?
Xdebug in coverage mode slows down the runtime by a factor of five to ten. Separate jobs with their own trigger condition keep the critical build path fast.
7How do I find the slowest tests?
PHPUnit 10+ supports --order-by=duration. Alternatively evaluate and sort the time attribute from JUnit XML with a shell script.
8What does if: always() mean for GitHub Actions?
Without if: always(), GitHub Actions skips the upload step on failures. But JUnit XML is needed precisely then. if: always() ensures the file is always uploaded.
9How long should test artifacts be retained?
JUnit XML: 30-90 days. HTML coverage: 7-14 days. For long-term trends, export core data into a time series database instead of retaining large artifacts.
10Can PHPUnit output JUnit XML and coverage at the same time?
Yes. In phpunit.xml, several formats can be configured simultaneously under <logging> and <coverage>. PHPUnit produces all configured outputs in a single run.