Contract Testing for REST APIs with OpenAPI

1. What contract testing is, and what it isn't

Contract testing checks whether an API implementation conforms to a defined contract. In the context of OpenAPI, that contract is the specification file: every response the API returns must match the schema defined there. Every endpoint declared in the spec must exist. Every required parameter must be accepted. Every status code must have the correct body structure.

What contract testing is not: it's no replacement for unit tests, and no replacement for end-to-end tests. Contract tests check the interface, not the business logic. Whether the API returns the correct products for a product search is a business-logic question that unit tests must answer. Whether the API returns a response with the correct structure per the specification for that same product search is a contract-testing question. This separation matters so contract tests don't get overloaded with too many responsibilities, which would make them expensive to maintain.

A second important distinction: provider-side contract tests (spec-driven) validate whether the backend honors its own contract. Consumer-driven contract tests validate whether the backend meets the requirements of a specific consumer (frontend, mobile app, another service). Both approaches solve different problems and can be combined. For most teams working with OpenAPI, the provider-side approach is the easier entry point.

2. Spec-driven contract testing with Schemathesis

Schemathesis is the most capable tool for spec-driven contract testing. It reads an OpenAPI specification and automatically generates test cases through property-based testing: for every endpoint, valid and invalid inputs are generated, the API is called, and the response is validated against the defined schema. This surfaces edge cases that manually written tests rarely cover: what happens with an integer value of MAX_INT? What about an empty string in a required field? What about Unicode characters in a URL parameter?

Schemathesis supports stateful test modes, in which it automatically builds test sequences from the API links and dependencies between endpoints: a user is created first, then fetched, then updated, then deleted. These sequences exercise the full lifecycle of a resource without a test author having to write the sequence by hand. The --checks flag controls which checks are enabled: not_a_server_error verifies that no 5xx errors occur; response_schema_conformance validates every response against the schema; content_type_conformance checks the Content-Type header.

# Install Schemathesis
pip install schemathesis

# Basic spec-driven contract test against running API
schemathesis run api/openapi.yaml \
  --url http://localhost:8080 \
  --checks all \
  --report .reports/schemathesis.html

# Stateful testing: follow API links between operations
schemathesis run api/openapi.yaml \
  --url http://localhost:8080 \
  --stateful=links \
  --checks not_a_server_error,response_schema_conformance

# Filter specific endpoints for focused testing
schemathesis run api/openapi.yaml \
  --url http://localhost:8080 \
  --endpoint "/users/{id}" \
  --method GET,PUT \
  --checks all

# Output JUnit XML for CI reporting
schemathesis run api/openapi.yaml \
  --url http://localhost:8080 \
  --checks all \
  --junit-xml .reports/schemathesis.xml

# With authentication (Bearer Token)
schemathesis run api/openapi.yaml \
  --url http://localhost:8080 \
  --checks all \
  --header "Authorization: Bearer $API_TOKEN"

# Reproduce a specific failing case (from schemathesis output)
schemathesis replay .schemathesis/case-abc123.yaml \
  --url http://localhost:8080

3. Dredd: example-based contract tests from the spec

Dredd is another tool for OpenAPI-based contract testing, with a different approach: it runs every request example defined in the specification against the real API and validates the response. Instead of property-based test generation like Schemathesis, Dredd uses the explicit examples from the spec, which means the quality of the Dredd tests depends directly on the quality of the examples in the spec. That makes Dredd ideal for teams who maintain their spec with complete examples and want to run exactly those examples as tests.

Dredd supports hooks in JavaScript and Python for setup and teardown logic between tests. That makes it possible to fetch authenticated tokens, create test data in the database, and clean up after the test. A hook can also dynamically validate responses that deviate from the static examples, for instance when IDs and timestamps are dynamic. Dredd is simpler to configure than Schemathesis and produces exactly one test case per example in the spec, which makes the test output predictable.

4. PHPUnit-based contract tests in Symfony

For PHP projects, PHPUnit-based contract tests are a natural complement to Schemathesis and Dredd. They run in the same test environment as all other tests, can access fixtures and database state, and are executed with the same CI tooling. The approach: an abstract contract test class loads the OpenAPI specification, instantiates a JSON schema validator, and provides assertion methods that validate every response against the corresponding schema. Concrete tests inherit from this class and call API endpoints.

In Symfony, PHPUnit contract tests integrate ideally with WebTestCase: tests create a Symfony client, issue requests, and validate responses at both the business-logic level and the schema level. That means the same test case checks: did the API return the correct data (business-logic assertion) and does the response structure match the spec (contract assertion). This combination prevents business-logic tests from implicitly drifting away from the spec.

<?php
// tests/Contract/AbstractContractTestCase.php
declare(strict_types=1);

namespace App\Tests\Contract;

use JsonSchema\Validator;
use Symfony\Bundle\FrameworkBundle\Test\WebTestCase;
use Symfony\Component\Yaml\Yaml;

/**
 * Base class for OpenAPI contract tests.
 * Validates that API responses conform to the OpenAPI specification schema.
 */
abstract class AbstractContractTestCase extends WebTestCase
{
    private static array $spec = [];
    private Validator $validator;

    protected function setUp(): void
    {
        parent::setUp();
        $this->validator = new Validator();

        if (empty(self::$spec)) {
            $specPath   = dirname(__DIR__, 2) . '/api/openapi.yaml';
            self::$spec = Yaml::parseFile($specPath);
        }
    }

    /**
     * Assert that the given response data conforms to the named schema.
     */
    protected function assertMatchesSchema(mixed $data, string $schemaName): void
    {
        $schema   = self::$spec['components']['schemas'][$schemaName]
            ?? throw new \LogicException("Schema '$schemaName' not found in OpenAPI spec");
        $jsonData = json_decode(json_encode($data, JSON_THROW_ON_ERROR));
        $jsonSchema = json_decode(json_encode($schema, JSON_THROW_ON_ERROR));

        $this->validator->validate($jsonData, $jsonSchema);

        if (!$this->validator->isValid()) {
            $errors = array_map(
                fn($e) => sprintf('[%s] %s', $e['property'], $e['message']),
                $this->validator->getErrors()
            );
            $this->fail(
                "Response does not match schema '$schemaName':\n" . implode("\n", $errors)
            );
        }
    }

    /**
     * Assert that the response status code and schema match the spec definition.
     */
    protected function assertApiResponse(
        int $expectedStatus,
        string $schemaName,
        string $responseBody,
    ): void {
        $data = json_decode($responseBody, true, 512, JSON_THROW_ON_ERROR);
        $this->assertMatchesSchema($data, $schemaName);
    }
}

<?php
// tests/Contract/UserContractTest.php
declare(strict_types=1);

namespace App\Tests\Contract;

use App\Tests\Contract\AbstractContractTestCase;

/**
 * Contract tests for the /users endpoint.
 * Validates that responses conform to the OpenAPI specification.
 */
final class UserContractTest extends AbstractContractTestCase
{
    public function testGetUserMatchesSchema(): void
    {
        $client = static::createClient();
        $client->request(
            'GET',
            '/users/1',
            [],
            [],
            ['HTTP_AUTHORIZATION' => 'Bearer ' . $this->getTestToken()]
        );

        $response = $client->getResponse();

        self::assertSame(200, $response->getStatusCode());
        self::assertResponseHeaderSame('content-type', 'application/json');

        // Business-logic assertion
        $data = json_decode($response->getContent(), true);
        self::assertSame(1, $data['id']);
        self::assertNotEmpty($data['email']);

        // Contract assertion: response must match UserResponse schema
        $this->assertApiResponse(200, 'UserResponse', $response->getContent());
    }

    public function testGetNonExistentUserReturns404WithSchema(): void
    {
        $client = static::createClient();
        $client->request('GET', '/users/99999', [], [], [
            'HTTP_AUTHORIZATION' => 'Bearer ' . $this->getTestToken(),
        ]);

        $response = $client->getResponse();

        self::assertSame(404, $response->getStatusCode());

        // Contract assertion: 404 response must match ErrorResponse schema
        $this->assertApiResponse(404, 'ErrorResponse', $response->getContent());
    }

    private function getTestToken(): string
    {
        return 'test-token'; // Or generate from test JWT factory
    }
}

5. Response validation as middleware in Symfony

A particularly elegant form of contract testing is automatic response validation as a Symfony event listener in the test environment. The listener is registered only for the test environment and validates every outgoing response against the OpenAPI specification, without individual tests having to include explicit validation calls. Every test that calls the API automatically performs a contract check. That means a business-logic test simultaneously surfaces contract violations, without the test itself containing any spec-validation code.

The downside of this approach is higher complexity: the event listener must know the current request path, find the corresponding schema in the spec, and validate the response. That requires a JSON schema resolver that resolves $ref references in the spec. The league/openapi-psr7-validator library provides this functionality in full and can be integrated as middleware or as an event listener in Symfony. The setup effort pays for itself quickly if contract compliance is meant to be checked implicitly across many tests.

6. Consumer-driven contracts with Pact

Consumer-driven contracts reverse the direction of the test: it's not the provider (backend) that defines the contract, but the consumer (frontend, mobile app). The consumer writes Pact tests in its own test environment that define which requests it sends to which endpoint and what response structure it expects. Pact generates a pact file from these tests, which the provider then verifies against its implementation. That ensures backend changes that would break the frontend get caught, before they reach production.

Pact is especially valuable in microservice architectures, where many services depend on one another and changes to one service can break others. For monolithic PHP applications with a single frontend client, Pact is often more effort than necessary, spec-driven contract tests with Schemathesis or Dredd combined with PHPUnit contract tests are usually enough. The choice between consumer-driven and spec-driven contract testing depends on the system architecture and the number of consumers.

7. Integrating contract tests into the CI pipeline

Integrating contract tests into the CI pipeline is essential to making sure they actually get used. The typical workflow: Schemathesis and PHPUnit contract tests run on every pull request as their own job, in parallel with unit tests and integration tests. On failure, they block the merge. The signal is clear: a spec deviation is treated exactly like a unit-test failure and must be fixed before the code can be merged.

For the CI workflow, setting up the test database matters: contract tests need realistic test data. A database seeding strategy with fixed seed data, the same database structure on every CI run, ensures contract tests produce deterministic results. Schemathesis generates random test cases, which can lead to different test cases on repeated runs. Schemathesis's --seed flag pins the random generator for reproducible runs in the CI pipeline.

# .github/workflows/contract-tests.yml
name: Contract Tests

on: [push, pull_request]

jobs:
  phpunit-contract-tests:
    runs-on: ubuntu-latest
    services:
      mysql:
        image: mysql:8.0
        env:
          MYSQL_DATABASE: api_test
          MYSQL_ROOT_PASSWORD: test
        ports: ["3306:3306"]
        options: --health-cmd="mysqladmin ping" --health-interval=5s

    steps:
      - uses: actions/checkout@v4
      - uses: shivammathur/setup-php@v2
        with: { php-version: '8.4', extensions: pdo_mysql }
      - run: composer install --no-dev
      - run: php bin/console doctrine:migrations:migrate --no-interaction --env=test
      - run: php bin/console doctrine:fixtures:load --no-interaction --env=test
      - run: vendor/bin/phpunit tests/Contract/ --log-junit reports/phpunit-contract.xml

  schemathesis-contract-tests:
    runs-on: ubuntu-latest
    needs: phpunit-contract-tests
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v4
        with: { python-version: '3.12' }
      - run: pip install schemathesis
      - name: Start API server (from previous job or Docker)
        run: |
          # Start the API server with test data loaded
          php -S localhost:8080 public/index.php &
          sleep 3
      - name: Run Schemathesis contract tests
        run: |
          schemathesis run api/openapi.yaml \
            --url http://localhost:8080 \
            --checks not_a_server_error,response_schema_conformance,content_type_conformance \
            --seed 42 \
            --junit-xml reports/schemathesis.xml \
            --header "Authorization: Bearer ${{ secrets.TEST_API_TOKEN }}"
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: contract-test-reports
          path: reports/

8. Comparison: Schemathesis vs. Dredd vs. PHPUnit contract tests

All three approaches check the alignment between API implementation and specification, but with different strengths and weaknesses. The choice depends on the desired test depth, the quality of the spec, and the team's stack.

The recommended combination: Schemathesis for broad, automatic schema validation of all endpoints in the CI pipeline; PHPUnit contract tests for critical endpoints where business logic and schema conformance need to be checked together. Dredd as a complement when the spec contains very complete examples and the test output should be visible for each example individually.

9. Summary

Contract testing for REST APIs automatically ensures that backend implementation and OpenAPI specification stay consistent. Schemathesis uses property-based testing to automatically generate hundreds of test cases from the spec and surfaces edge cases that wouldn't be written by hand. Dredd runs every example from the spec against the real API and is ideal when the spec is maintained with complete examples. PHPUnit contract tests combine business-logic assertions with schema validation in a single test run.

Integrating this into the CI pipeline as a merge gate ensures contract violations never reach production. An OpenAPI specification protected by automated contract tests is a reliable foundation for mock servers, TypeScript client generation, and API design reviews, because everyone knows it accurately reflects reality.

10. FAQ: Contract Testing for REST APIs