PHPUnit and Composer Audit: Detecting Vulnerabilities in Test Dependencies

1. Why dev dependencies often get overlooked

In many security processes, the focus lands almost exclusively on require dependencies, the packages actually shipped to production. That makes sense at first glance, since only that code ultimately runs on the live server and is directly reachable from the outside. Test and dev dependencies under require-dev, such as PHPUnit itself, mocking libraries, coverage tools, or static analysis tools, mostly fall through the cracks entirely under this view.

This blind spot is dangerous because dev dependencies are far from harmless just because they do not run in production. They execute on every composer install, on developer machines and in every CI pipeline, often with the very same privileges as the actual build process, including access to environment variables, SSH keys for deployments, and sometimes even production credentials accidentally available in the CI environment.

2. Composer audit: how it works and how to run it

Since Composer 2.4, the built in composer audit command checks the composer.lock file against a security database, by default the FriendsOfPHP Security Advisories database. The command checks every package in the lock file, regardless of whether it is declared under require or require-dev, unless explicitly filtered. That means a plain invocation with no extra options already covers test and dev packages too.

The command outputs, for every vulnerability found, the affected package, the installed version, the CVE identifier, and a short advisory title. It matters to evaluate the command's return value in scripts: composer audit returns a non zero exit code when vulnerabilities are found, which can be used directly as a failure condition in a CI pipeline.

# Full audit including require-dev (default behavior)
composer audit

# Check production dependencies only (exclude require-dev)
composer audit --no-dev

# Machine readable output for further processing in a CI script
composer audit --format=json > audit-report.json

3. Why vulnerabilities in test tooling are dangerous anyway

The thought 'PHPUnit only runs locally and in the pipeline, not on the live server' underestimates two attack vectors. First, PHPUnit runs with all the privileges of the executing process, and in many CI setups that process has access to deployment secrets, database credentials for test environments resembling production data, or even cloud provider credentials for automated deployments. A compromised test dependency capable of code execution can exfiltrate these secrets without production code ever being touched.

Second, test and analysis tools in particular are a frequent target of supply chain attacks, because compared to large frameworks they receive less attention from security researchers while still being installed as a dev dependency in thousands of projects. One known case involved a compromised package that executed malicious code via a post install script at installation time, long before a single test ever ran. Attacks like that hit every developer who runs composer install, regardless of whether a test is ever actually started.

4. Integrating composer audit into the CI pipeline

A composer audit step should sit early in the pipeline, ideally before the actual PHPUnit run, so that a found vulnerability stops the build before potentially compromised code ever executes. For Magento projects, it makes sense to run the audit both against the root composer.json and, where present, against separate composer.json files of individual modules, provided they declare their own dependencies.

A pragmatic approach is to define the audit step as its own, fast CI job, independent of the longer test run, so it remains separately visible even when the PHPUnit run fails for unrelated reasons. This makes it clear whether a build failed due to a test failure or due to a security vulnerability, which matters for prioritizing the fix.

# .gitlab-ci.yml (excerpt)
composer-audit:
  stage: security
  script:
    - composer audit --format=json --no-interaction | tee audit-report.json
  artifacts:
    paths:
      - audit-report.json
    when: always
  allow_failure: false

5. Tiered handling by severity

Not every reported vulnerability justifies an immediate build failure. composer audit's JSON output also includes the severity of each advisory, allowing a differentiated policy: vulnerabilities with high or critical severity block the merge immediately, while low or medium severity ones initially only raise a warning and get turned into a ticket to be resolved within a defined deadline.

This tiering prevents teams from ignoring composer audit entirely after repeated false alarms over minor issues, or lazily disarming the check with allow_failure: true. A realistic policy tiered by risk noticeably increases team buy in compared to a rigid all or nothing rule.

<?php
// tools/audit-policy.php
declare(strict_types=1);

$report = json_decode((string) file_get_contents(__DIR__ . '/../audit-report.json'), true);
$blockingSeverities = ['critical', 'high'];
$blocked = false;

foreach ($report['advisories'] ?? [] as $package => $advisories) {
    foreach ($advisories as $advisory) {
        $severity = $advisory['severity'] ?? 'unknown';
        $isDev = str_contains($package, 'phpunit') || str_contains($package, 'mockery');

        if (in_array($severity, $blockingSeverities, true)) {
            fwrite(STDERR, sprintf(
                "BLOCKING: %s (%s, dev package: %s)\n",
                $package, $severity, $isDev ? 'yes' : 'no'
            ));
            $blocked = true;
        } else {
            fwrite(STDOUT, sprintf("WARNING: %s (%s)\n", $package, $severity));
        }
    }
}

exit($blocked ? 1 : 0);

6. The risk of post install scripts and plugins

Beyond known CVEs, Composer itself carries a structural risk: packages can execute arbitrary PHP code at install time via the scripts section in their composer.json, and Composer plugins can even hook into the installation process itself. A malicious or compromised test utility package can therefore execute code on a plain composer install alone, entirely independent of whether it is ever actually referenced in a test.

Since version 2.2, Composer offers the allow-plugins configuration option, which explicitly defines which packages are permitted to run Composer plugin code. For a deliberate security strategy, it pays to keep this list deliberately short and only allow new entries after briefly reviewing the respective package, instead of blanket approving everything Composer suggests on first install.

{
    "config": {
        "allow-plugins": {
            "composer/installers": true,
            "phpstan/extension-installer": true,
            "dealerdirect/phpcodesniffer-composer-installer": false
        }
    }
}

7. Automated updates for test dependencies

A single composer audit run only protects against vulnerabilities known at the time of the check, not against ones discovered later. In addition, a tool like Renovate or Dependabot should be set up to automatically open pull requests for outdated dependencies, including test dependencies under require-dev. It matters to configure these tools not just for require, but explicitly for require-dev as well, since some default configurations deprioritize or entirely skip dev dependencies.

A sensible compromise is auto merging high priority security updates for dev dependencies as long as the CI pipeline stays green, while pure feature updates with no security relevance get reviewed manually. That keeps maintenance overhead manageable without leaving known vulnerabilities open unnecessarily long.

8. Continuous monitoring beyond individual CI runs

A composer audit step in the CI pipeline only ever checks the state at the moment of a specific merge or pull request. If a new vulnerability in an already installed package becomes known afterward, it only surfaces at the next commit that triggers some CI pipeline, which can take weeks during quiet project phases. An additional, scheduled job that runs daily or weekly against the current composer.lock, independent of code changes, closes exactly this gap.

Such a scheduled job should not just turn the pipeline red on a finding, it should actively trigger a notification, for example to a Slack channel or by email to the responsible team, since nobody manually checks an unused, purely informational pipeline every day. Only this active alerting ensures a newly discovered vulnerability in an already installed test dependency gets noticed promptly, even when no one happens to be working on the affected module at that moment.

9. Special considerations in a Magento project context

Magento projects carry an especially broad attack surface due to the large number of third party modules and their own separate composer.json dependencies. Many Magento extensions declare their own test dependencies or developer tools, rarely reviewed by anyone outside the original development team. A project wide composer audit run covers this entire transitive dependency chain, while an isolated check of only your own app/code modules leaves blind spots.

For Mark Shust Docker setups or similar development environments, it is advisable to establish composer audit as part of the bin/composer wrapper workflow, for example as a standalone command run against the full lock file before every composer update, so new vulnerabilities are caught immediately instead of being discovered only weeks later at the next scheduled security review.

10. Summary

11. FAQ: Composer Audit for Dev Deps: The Essentials at a Glance