Risky Tests in PHPUnit: Spotting the Warning Signs and Fixing Them

1. What PHPUnit means by risky

A test in PHPUnit is normally considered passed if no exception is thrown and all assertions hold. On top of that binary result, PHPUnit has for many versions offered a third category: risky. A risky test is not formally a failure, but PHPUnit does not trust it. The most common reasons are a test with not a single assertion, a test that produces output while it runs, or a test that changes global state without restoring it.

The distinction from a classic failure matters: a risky test can still count as successful under default configuration, but shows up in the summary with its own counter. Anyone skimming PHPUnit's output easily misses these warnings because the overall suite still reports green. That is exactly what makes risky tests dangerous: they creep into a codebase unnoticed and, over time, grow into a real trust problem, because nobody knows anymore which tests actually verify anything.

2. Tests without assertions: the most common case

The classic risky test is a test method that runs code but never calls a single assert method. This usually happens out of convenience: a developer calls a method, sees in the IDE that no exception is thrown, and considers the test done. Formally, though, this test has verified nothing. It would stay green even if the tested method suddenly returned something completely different, as long as it does not throw.

PHPUnit detects this case automatically and marks the method as risky with a note that no assertions were performed. The fix is usually simple: add a concrete expectation, for example assertSame for a return value or assertInstanceOf for the type of object produced. For tests that are meant to check only that no exception occurs, expectNotToPerformAssertions is the better tool. It documents the intent explicitly and suppresses the risky warning without adding a meaningless dummy assertion.

final class InvoiceExporterTest extends \PHPUnit\Framework\TestCase
{
    // Risky: no assertion, this test verifies essentially nothing.
    public function testExportRunsWithoutError(): void
    {
        $exporter = new InvoiceExporter();
        $exporter->export(new Invoice('INV-1001'));
    }

    // Correct: a concrete expectation about the result.
    public function testExportReturnsGeneratedFilePath(): void
    {
        $exporter = new InvoiceExporter();
        $path = $exporter->export(new Invoice('INV-1001'));

        self::assertStringEndsWith('INV-1001.pdf', $path);
    }

    // If "no exception" really is the whole intent: make it explicit.
    public function testExportDoesNotThrowForEmptyInvoice(): void
    {
        $exporter = new InvoiceExporter();
        $exporter->export(new Invoice(''));

        $this->expectNotToPerformAssertions();
    }
}

3. Unexpected output during the test run

A second common reason for the risky flag is output a test produces while running: a forgotten var_dump, a debugging echo, or a library that writes warnings directly to standard output. PHPUnit expects tests to run silently and to communicate their results only through assertions and exceptions. Any output that bypasses this channel counts as a violation of that principle, even when the content itself is harmless.

In practice, this output usually comes from debug leftovers forgotten before a commit, or from legacy code that writes directly to STDOUT instead of using a logger. The fix is either to remove the debug code, or, if the tested method is meant to produce output, to capture that output deliberately and assert on it. expectOutputString is the right tool for that: the expected output becomes part of the assertion instead of remaining a side effect that PHPUnit flags as a warning sign.

final class ReportPrinterTest extends \PHPUnit\Framework\TestCase
{
    public function testPrintSummaryOutputsExpectedText(): void
    {
        $printer = new ReportPrinter();

        // The output becomes part of the assertion, not a side effect.
        $this->expectOutputString("Summary: 3 items processed\n");

        $printer->printSummary(3);
    }
}

final class ReportPrinter
{
    public function printSummary(int $count): void
    {
        echo "Summary: {$count} items processed\n";
    }
}

4. Unexpected changes to global state

The third major category covers tests that change global state, such as static class properties, superglobal arrays like $_SERVER or $_ENV, or singleton instances, and never reset that state. Under strict configuration, PHPUnit can detect when global variables change between the start and end of a test, and flags the test as risky because it could potentially affect other, later-running tests.

Such tests are especially tricky because their misbehavior often only becomes visible in combination with other tests: test A changes a static property, test B unknowingly relies on its original value and fails, but only if A ran first. The sustainable fix is to reset any changed global state in tearDown, and, where possible, to avoid global state altogether by injecting dependencies explicitly instead of pulling them implicitly from singletons or superglobals.

final class FeatureFlagTest extends \PHPUnit\Framework\TestCase
{
    private ?string $originalEnvValue;

    protected function setUp(): void
    {
        $this->originalEnvValue = $_ENV['FEATURE_NEW_CHECKOUT'] ?? null;
    }

    protected function tearDown(): void
    {
        // Reset global state back to exactly its original value.
        if ($this->originalEnvValue === null) {
            unset($_ENV['FEATURE_NEW_CHECKOUT']);
        } else {
            $_ENV['FEATURE_NEW_CHECKOUT'] = $this->originalEnvValue;
        }
    }

    public function testFeatureIsEnabledWhenFlagIsSet(): void
    {
        $_ENV['FEATURE_NEW_CHECKOUT'] = '1';

        self::assertTrue(FeatureFlag::isEnabled('FEATURE_NEW_CHECKOUT'));
    }
}

5. Controlling risky detection via phpunit.xml

How strictly PHPUnit detects risky tests can be configured in phpunit.xml. Options such as beStrictAboutTestsThatDoNotTestAnything, beStrictAboutOutputDuringTests, and beStrictAboutChangesToGlobalState each independently control whether the corresponding category is even checked. In many older projects these switches are not enabled at all, so risky tests can accumulate unnoticed while nobody on the team realizes it.

For new projects it is worth enabling all three switches from day one, so risky behavior surfaces immediately instead of coming back to bite the team months later. For existing projects with a lot of legacy code, a gradual approach makes more sense: enable detection first, document the current count of risky tests as a baseline, and then reduce it continuously instead of trying to fix every violation at once, which would quickly block a large project.

<!-- phpunit.xml -->
<phpunit bootstrap="vendor/autoload.php"
         beStrictAboutTestsThatDoNotTestAnything="true"
         beStrictAboutOutputDuringTests="true"
         beStrictAboutChangesToGlobalState="true"
         beStrictAboutTodoAnnotatedTests="true">
    <testsuites>
        <testsuite name="Unit">
            <directory>tests/Unit</directory>
        </testsuite>
    </testsuites>
</phpunit>

6. Making risky tests visible in the CI pipeline

Strict configuration alone is not enough if nobody reads the output carefully. Many CI pipelines only check PHPUnit's exit code, and risky tests do not automatically cause a nonzero exit code by default. A team can accumulate risky tests for months without the pipeline ever turning red, because formally all tests still pass.

To prevent that, it is worth parsing PHPUnit's JUnit XML report in the pipeline and counting risky-flagged tests explicitly. If that count exceeds a defined threshold, or increases compared to the previous run, the pipeline can fail deliberately. That turns a silent warning into a hard gate that stops the problem from spreading unnoticed, while still leaving enough room to reduce existing legacy issues gradually.

7. Work systematically instead of suppressing symptoms

Anyone facing a large number of risky tests should not try to fix the warnings one by one in arbitrary order, but start with an inventory. A useful first step is to count the three categories separately: how many tests have no assertion, how many produce output, how many change global state. This breakdown often already reveals where the biggest structural problems lie, for instance if a particular module systematically relies on static singletons.

After that, it pays to start with the category that is easiest and lowest risk to fix, usually tests without assertions, since those typically just need a missing expectation added. Global state problems are often the most expensive, because they reflect deeper architectural issues such as missing dependency injection. A team that follows this order sees quick early wins and stays motivated to tackle the harder cases as well.

8. Common pitfalls when fixing risky tests

A common mistake is making a risky warning disappear by adding a trivial assertion like assertTrue(true), without the test actually verifying anything as a result. That formally clears the warning but continues to hide the fact that the test carries no real meaning. Anyone who takes this shortcut only postpones the actual problem instead of solving it, and makes it harder for future developers to tell real tests from fake ones.

A second pitfall is reaching for @runInSeparateProcess as a quick fix for global state problems. That can suppress the symptoms, since every test runs in a fresh process, but it costs significant runtime and does not fix the underlying architectural issue. It is better to replace global state with injectable dependencies. Process isolation should remain the exception for genuinely unavoidable cases, not the default answer to every risky warning about global state.

9. Team guidelines for risky tests

For the ideas in this article to actually work day to day, a team needs clear, written guidelines: when are all three strict switches mandatory for new projects, how is legacy debt handled in existing projects, and who is responsible when the number of risky tests in CI increases. Without such an agreement, fixing risky tests stays a matter of individual judgment, and the count tends to grow rather than shrink over time.

A proven approach is to raise risky tests explicitly in code review whenever a pull request introduces new warnings, and to discuss the overall count regularly, for example monthly, as a team. The table below summarizes the three categories, their typical causes, and the matching fix, as a quick reference for daily work.

10. Summary

11. FAQ: Risky Tests in PHPUnit: The Essentials at a Glance