Detecting API Breaking Changes Automatically in CI with OpenAPI Diff Tools

1. Why silent breaking changes are a recurring problem

A breaking change in a public API rarely happens out of malice, it usually slips in through carelessness: a required field gets renamed, an enum value gets removed, a response field silently changes type from string to integer, and all of that happens inside a pull request that was only supposed to fix an internal bug. Without automated checking, the team often only finds out once an external API consumer reports that their client suddenly crashes with a parsing error.

Manual code review alone is not enough here, because an API contract break is frequently hidden in a single, unremarkable diff line pair that a human reviewer can easily miss among dozens of other changes. This is exactly where OpenAPI diff tools come in: they compare the specification of two API versions automatically and reliably, flagging every change that could be incompatible for existing consumers before it is even merged.

2. Linting with Spectral versus contract comparison with diff tools

Spectral and diff tools like oasdiff get confused for each other in practice, even though they solve fundamentally different problems. Spectral is a linter: it checks a single OpenAPI specification against a set of style rules, for example whether every endpoint has a description, whether property names consistently use camelCase, or whether every response code is documented. Spectral has no notion of a previous version of the API, it evaluates only the current state in isolation.

A diff tool like oasdiff or openapi-diff pursues a different goal: it compares two concrete versions of the same specification, typically the state on the target branch against the state in a feature branch, and identifies semantic differences between them. A perfectly lint-compliant schema can still contain a serious breaking change, for instance when a previously optional field suddenly becomes required, because that is stylistically flawless but contractually incompatible. Linting and diffing complement each other, they do not substitute for one another.

3. oasdiff in detail: how it works and what it outputs

Before a diff tool can compare anything, it needs two concrete OpenAPI files: one for the current state of the target branch and one for the state in the pull request. In a Symfony application that generates its API documentation from attributes via NelmioApiDocBundle, this export can be automated through a dedicated console command that runs in the CI pipeline right before the actual diff step.

oasdiff itself is a command-line tool that takes two OpenAPI documents (YAML or JSON) as arguments and outputs a structured list of changes, categorized by severity: info, warning, and breaking. The categorization follows OpenAPI semantics directly, oasdiff knows for example that adding a new optional response field is harmless, while removing an existing field or tightening a validation rule gets classified as breaking.

<?php
declare(strict_types=1);

namespace App\Command;

use Nelmio\ApiDocBundle\ApiDocGenerator;
use Symfony\Component\Console\Attribute\AsCommand;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
use Symfony\Component\Yaml\Yaml;

/**
 * Exports the current OpenAPI specification as a YAML file
 * so it can be compared against the target branch state in the CI pipeline.
 */
#[AsCommand(name: 'api:openapi:export')]
final class ExportOpenApiSpecCommand extends Command
{
    public function __construct(
        private readonly ApiDocGenerator $apiDocGenerator,
    ) {
        parent::__construct();
    }

    protected function execute(InputInterface $input, OutputInterface $output): int
    {
        $spec = $this->apiDocGenerator->generate();

        file_put_contents(
            'var/openapi/current.yaml',
            Yaml::dump($spec->toArray(), 6),
        );

        $output->writeln('OpenAPI specification exported to var/openapi/current.yaml.');

        return Command::SUCCESS;
    }
}

4. openapi-diff as an alternative: differences from oasdiff

openapi-diff (originally released by OpenAPITools as a Java-based tool) follows the same basic idea as oasdiff, but differs in implementation and ecosystem. While oasdiff, being a Go binary, starts up quickly and integrates easily into Docker images or GitLab CI jobs, openapi-diff runs on the JVM, which in a pure PHP/Symfony environment means an additional runtime dependency that has to be maintained separately.

In terms of content, both tools deliver comparable results when detecting classic breaking changes such as removed endpoints, changed required fields, or tightened enum constraints. The practical difference lies more in output formatting and integration maturity: oasdiff ships with a machine-readable JSON format out of the box that can be parsed directly in a CI script, while openapi-diff is primarily optimized for HTML report output and needs more customization for an automated gate check.

5. A CI gate that blocks PRs on unannounced breaking changes

A CI gate for API contract breaks works on a simple principle: the pipeline job fetches the OpenAPI specification of the target branch (usually main or develop), generates the specification of the current feature branch, runs oasdiff against both, and fails the job with a non-zero exit code as soon as at least one change is classified as breaking. The pull request technically cannot be merged as long as this job is configured as a required check.

What matters for team acceptance is that the error message in the CI log is specific and understandable, not just 'breaking change detected', but naming the exact property, the affected endpoint, and the kind of change. Only then can a developer judge within a few seconds whether the change was genuinely unintentional or deliberate and needs to go through the whitelist mechanism described below.

openapi-breaking-check:
  stage: test
  image: tufin/oasdiff:latest
  script:
    - git show origin/main:openapi/spec.yaml > /tmp/base.yaml
    - oasdiff breaking /tmp/base.yaml openapi/spec.yaml
      --fail-on ERR
      --exclude-ops-with-extension "x-breaking-allowed"
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'

6. What counts as a breaking change and what does not

Not every schema change automatically counts as a breaking change, and an overly aggressive diff tool configuration quickly causes alert fatigue in the team. Clearly breaking changes include: removing an endpoint or an HTTP method, adding a new required field to a request body, removing a response field that consumers might rely on, and tightening validation rules such as minimum or maximum lengths.

Not breaking, on the other hand, are adding a new optional field, adding a new endpoint, loosening a validation rule (say, raising a maximum length instead of lowering it), or adding a new additional enum value in a response, provided the client is implemented to tolerate unknown enum values. This distinction follows the principle that a change is breaking when it breaks an existing, correctly implemented client, regardless of whether the change itself was reasonable or intentional.

7. A whitelist mechanism for deliberately accepted breaking changes

In practice there are legitimate cases where a breaking change is deliberately wanted, for example when finally removing a field that has been marked deprecated for a long time as part of an announced major version. A rigid CI gate that blocks every breaking change without exception would prevent such planned changes too, and would tempt the team to just disable the check when in doubt, which defeats its actual purpose.

The workable approach is a whitelist mechanism: a developer who deliberately wants to introduce a breaking change explicitly registers the affected change in an exception list (for instance through a schema extension like x-breaking-allowed or a separate configuration file) and has to provide a short justification in the pull request explaining why the change is necessary and announced. The CI gate recognizes the exception and lets the pipeline turn green, but the change stays visible and documented in the review history instead of slipping through silently.

8. Fitting it into the team workflow: PR justification and review

To keep the whitelist mechanism from turning into a convenient way to bypass the gate, every exception should be tied to a mandatory justification in the pull request workflow and, ideally, to a second reviewer's sign-off. A simple pattern is a pull request template with a dedicated 'Breaking Changes' section that must be filled in whenever the CI gate reports an exception, and that gets explicitly countersigned by the API owner or a designated reviewer team.

This process does not make breaking changes impossible, it makes them visible and deliberate instead of letting them slip through by accident. For external API consumers it is also worth documenting accepted breaking changes in a public changelog and announcing them with a reasonable transition period before the new major version actually goes live.

9. The limits of automated detection at a glance

Automated diff checking does not replace a full understanding of the contract, it reliably detects structural changes in the schema but not semantic behavior changes that never show up in the OpenAPI specification, such as a changed sort order in a response list or altered error handling on edge-case timeouts. The table below places the discussed tools and mechanisms side by side.

10. Summary

11. FAQ: OpenAPI Diff Tools: The Essentials at a Glance