Property-Based Testing as a Complement to Example-Based Tests

1. Example-based tests vs. property-based tests: the core difference

A classic PHPUnit test for a price calculation picks one specific input, say 100 euros with a 10 percent discount, and checks that exactly 90 euros comes out. That is an example-based test: it documents exactly one input/output pair and fails as soon as behavior changes for that one case. Its strength lies in readability, anyone on the team immediately understands what behavior is expected for what input without having to read the implementation.

Property-based tests shift the question from "what comes out for this specific input" to "what property must hold for every valid input". Instead of hard-coding 100 euros and 10 percent, the test describes a rule such as the result is never negative or the result never exceeds the original price. A library like eris/eris then automatically generates hundreds of random price and discount combinations and checks the rule against every single one.

Neither testing style excludes the other. An example-based test documents expected behavior for the most important business case and works as a readable specification, while a property test systematically searches the gaps between the hand-picked examples. In a healthy test suite, both sit side by side, not as competitors, but as two different tools answering two different questions.

2. How a property-based testing library works

A property-based testing library like Eris follows a fixed pattern: for each declared generator, a value range is described first, for example an integer between 0 and 1,000,000 for a price in cents, or a percentage between 0 and 100. By default the library generates 100 random combinations of those values on every test run and executes the check, passed in as a then() callback, against each combination.

If the check fails for one of the generated combinations, Eris stops immediately, marks the test as failed, and starts the shrinking process described in detail in section 7. If the test passes for all generated values, the property is considered confirmed, though only for the range that was actually searched, not as a mathematical proof for every conceivable input. The number of iterations can be increased via limitTo() when a critical calculation path needs particularly thorough coverage.

Wiring this into an existing Magento project is low effort: Eris builds on top of PHPUnit and installs via Composer as a dev dependency, property tests run in the same vendor/bin/phpunit invocation as classic tests, and they integrate into existing CI pipelines through the normal PHPUnit test suite configuration, without needing a second test framework running in parallel.

{
  "require-dev": {
    "eris/eris": "^0.14.0",
    "phpunit/phpunit": "^10.5"
  }
}

3. What makes a good property

Not every statement about a function makes a good property. A good property describes an invariant that must hold for every valid input, regardless of the specific values. Typical examples from an e-commerce context: a price calculation never returns a negative value, sorting a product list is idempotent, sorting the same list a second time no longer changes the order, and encoding a value followed by decoding it returns exactly the original value, a classic encode/decode round-trip pattern.

Another useful pattern is monotonicity: adding an item to a cart must never decrease the cart total, no matter which item at what price gets added. Properties like these can often be derived directly from the business specification without knowing the concrete implementation, which makes property tests more resilient to refactoring than example-based tests, which frequently and unintentionally test implementation details.

Things get harder for functions whose correct result cannot simply be expressed as a rule, for example a complex ranking algorithm for search results. Here a comparison against a deliberately slow but obviously correct reference implementation, so-called model-based testing, often helps, or you settle for weaker but still valuable properties like "the number of results does not change when sorting".

4. Example-based test: a discount calculation with PHPUnit

The running example throughout this article is a DiscountCalculator class, the kind that might power a checkout discount promotion in a Magento project. The method applyDiscount(float $price, float $discountPercent): float takes a gross price and a discount percentage and returns the reduced price. A classic PHPUnit test checks a handful of hand-picked cases for it: 10 percent off 100 euros yields 90 euros, 0 percent leaves the price unchanged, 100 percent reduces it to 0.

These three cases cover the obvious scenarios and simultaneously document the expected behavior for new team members. What they do not cover is the entire space of possible combinations: what happens with a discount of 150 percent, with a negative price caused by a faulty upstream calculation step, or with a price carrying many decimal places due to a currency conversion? That gap is what the property test in the next section closes.

<?php
declare(strict_types=1);

namespace Mironsoft\Pricing\Test\Unit\Model;

use Mironsoft\Pricing\Model\DiscountCalculator;
use PHPUnit\Framework\TestCase;

/**
 * Example-based test: fixed, hand-picked inputs and expected outputs.
 */
final class DiscountCalculatorTest extends TestCase
{
    private DiscountCalculator $calculator;

    protected function setUp(): void
    {
        $this->calculator = new DiscountCalculator();
    }

    public function testTenPercentDiscountOnHundredEuros(): void
    {
        // Single hand-picked input/output pair
        $result = $this->calculator->applyDiscount(100.0, 10.0);

        $this->assertSame(90.0, $result);
    }

    public function testZeroPercentDiscountReturnsOriginalPrice(): void
    {
        $result = $this->calculator->applyDiscount(49.99, 0.0);

        $this->assertSame(49.99, $result);
    }

    public function testHundredPercentDiscountReturnsZero(): void
    {
        $result = $this->calculator->applyDiscount(75.0, 100.0);

        $this->assertSame(0.0, $result);
    }
}

5. The same discount calculation as a property with Eris

The property test for the same DiscountCalculator class no longer spells out concrete numbers, it states the invariant directly: the result of applyDiscount() is never less than 0 and never greater than the original price. Using Generator\choose(), the test defines the value range for price and discount percentage, Eris takes care of actually generating the test data and runs the check 100 times by default with different, random combinations.

In practice, the decisive difference from the example-based test almost always shows up in the same spot: as soon as the generator produces a discount percentage above 100, say 104 percent, the unguarded implementation returns a negative price, a bug that three hand-picked examples would never have found, because nobody thinks to test a discount above 100 percent on purpose. That exact class of bug, plausible but untested value ranges, is the real strength of property-based testing.

<?php
declare(strict_types=1);

namespace Mironsoft\Pricing\Test\Unit\Model;

use Eris\Generator;
use Eris\TestTrait;
use Mironsoft\Pricing\Model\DiscountCalculator;
use PHPUnit\Framework\TestCase;

/**
 * Property-based test: the same discount logic checked as a general invariant.
 */
final class DiscountCalculatorPropertyTest extends TestCase
{
    use TestTrait;

    private DiscountCalculator $calculator;

    protected function setUp(): void
    {
        $this->calculator = new DiscountCalculator();
    }

    public function testDiscountNeverProducesANegativeResult(): void
    {
        $this
            ->forAll(
                Generator\choose(0, 1000000),   // price in cents, wide range
                Generator\choose(0, 100)        // discount percent
            )
            ->then(function (int $priceCents, int $discountPercent): void {
                $result = $this->calculator->applyDiscount(
                    $priceCents / 100,
                    (float) $discountPercent
                );

                $this->assertGreaterThanOrEqual(0.0, $result);
                $this->assertLessThanOrEqual($priceCents / 100, $result);
            });
    }
}

6. How property-based testing automatically finds edge cases

Generators in Eris and comparable libraries are deliberately built to not produce uniformly distributed random values, but to deliberately seek out edge cases more often: 0, negative numbers, very large numbers near the integer limit, empty arrays and strings, and unicode characters including multi-byte characters and emoji. For a cart calculation, that means the generator will automatically produce an empty cart, an item priced at 0, a quantity of minus 1, and a product name containing Japanese characters, all without a developer ever having to write those cases down explicitly.

This automatic edge-case search hits exactly the spot where hand-written example-based tests systematically have blind spots: developers typically test the cases that were on their mind while writing the code, not the cases they forgot to think about. A property test for the discount calculation reliably discovers that a price of exactly 0.005 euros can round to a negative cent amount, an edge case simply never considered during implementation and one that would never show up in any manually written test case.

Importantly, these edge cases aren't found by pure chance, the generators statistically over-produce known trouble spots such as zero values, sign flips, and type-conversion boundaries far more often than uniform randomness would. That significantly raises the hit rate compared to true randomness and makes even 100 iterations per property surprisingly effective in practice.

7. Shrinking: from a failure to a minimal counterexample

When Eris finds a failing combination, the library does not simply report the originally generated, often unwieldy random values, say a price of 5834.21 euros with a 104 percent discount. Instead it automatically starts a shrinking process: Eris systematically tries smaller, simpler variants of the failing input, checks whether the property still fails for them, and repeats the procedure until no smaller failing value can be found.

What remains at the end is a minimal counterexample, in the example from section 5 the failure shrinks down to a price of one cent with a 101 percent discount. This reduction is not a cosmetic detail, it is the actual practical payoff of property-based testing: a developer who had to debug a five-digit random failure would waste time figuring out which numbers are even relevant. A minimal counterexample of one cent and 101 percent points straight at the bug, the discount was never capped at 100 percent.

In the terminal output, Eris logs every shrinking step, so it stays traceable how the library got from the original random value to the minimal case. That builds trust in the reduced example and saves the manual work of verifying that the simplified case really triggers the same bug as the original one.

$ vendor/bin/phpunit --filter testDiscountNeverProducesANegativeResult

1) Mironsoft\Pricing\Test\Unit\Model\DiscountCalculatorPropertyTest::testDiscountNeverProducesANegativeResult
Failed asserting that -0.01 is greater than or equal to 0.

Eris\Listener\ExceptionListener:
  Falsifiable after 47 tries, shrinking...
  Original counterexample: [priceCents => 583421, discountPercent => 104]
  Shrinking step 1: [priceCents => 100, discountPercent => 104]
  Shrinking step 2: [priceCents => 1, discountPercent => 101]
  Minimal counterexample: [priceCents => 1, discountPercent => 101]

  The library reduced a six-digit random failure to a single-cent
  price with a 101 percent discount, revealing the actual bug:
  discountPercent was never capped at 100.

8. The JS/TS perspective: fast-check and CI integration

Since Cypress and Playwright projects are written entirely in JavaScript or TypeScript, it's worth looking at fast-check, the JS/TS counterpart to Eris, for pure business logic on the frontend side, for example a client-side price preview or a form validation. The core idea is identical: fc.property() describes an invariant, generators like fc.float() or fc.integer() produce the test data, and fc.assert() runs the check 100 times by default, including automatic shrinking on failures.

For a Magento project running a Hyvä theme, that means in practice: server-side price, tax, and discount logic in PHP gets covered with Eris, while client-side calculations, say a live shipping cost preview in Alpine.js, can be tested with fast-check in Jest or Vitest, independently of the Cypress E2E tests that validate the full checkout flow in the browser. Both libraries integrate into existing CI pipelines without requiring a separate test-runner setup.

// test/discount.property.test.ts
import fc from 'fast-check';
import { applyDiscount } from '../src/discount';

describe('applyDiscount (property-based)', () => {
  it('never returns a negative result', () => {
    fc.assert(
      fc.property(
        fc.float({ min: 0, max: 1_000_000, noNaN: true }),
        fc.integer({ min: 0, max: 100 }),
        (price, discountPercent) => {
          const result = applyDiscount(price, discountPercent);
          return result >= 0 && result <= price;
        }
      ),
      { numRuns: 500 }
    );
  });
});

9. Where property-based testing fits in the testing strategy

Property-based testing is not a replacement for E2E tests with Cypress or Playwright, it is a targeted complement for a specific slice of the testing pyramid: pure, deterministic business logic without side effects, such as price, tax, and discount calculations, sorting and filtering functions, or serialization. E2E tests validate that a real user in a browser can actually add an item to the cart and reach checkout, including UI state, network calls, and third-party integrations, questions a property can never answer because it never starts a browser.

The sensible combination in practice looks like this: a single Cypress test confirms the end-to-end checkout flow for a realistic cart, while dozens of property test iterations in the background cover the underlying discount calculation across the entire input space. This division of labor plays to the strengths of both approaches: fast, thorough coverage of calculation logic at the unit level and realistic validation of the user flow at the E2E level, without either testing style having to take over the other's job.

10. Summary

Property-based testing complements classic example-based tests by shifting the question from concrete input/output pairs to general invariants. Instead of a handful of hand-picked values, a library like eris/eris for PHP or fast-check for JavaScript automatically generates hundreds of random inputs and checks whether a rule such as the result is never negative holds for every one of them. Good properties describe invariants such as non-negativity, idempotence of sorting functions, or encode/decode round-trips, formulated from the business specification rather than from the concrete implementation.

The practical payoff shows up in two places: automatically discovered edge cases like zero values, extreme numbers, or unicode strings that regularly slip through hand-written tests, and shrinking, which automatically reduces every failure to a minimal, immediately understandable counterexample. Property-based testing replaces neither unit nor E2E tests, it deliberately complements both for pure business logic such as price, tax, and discount calculations, while validating real user flows remains the job of Cypress or Playwright tests.

11. FAQ: Property-Based Testing