Detect Breaking OpenAPI Changes in CI with a Zero-Dependency Python Diff

Breaking an API usually doesn't happen on purpose. Someone renames a field in a shared User schema, makes an optional request field required, or drops an old endpoint. The OpenAPI file changes by a few lines, the PR looks harmless, and a mobile client that's three versions behind starts failing. A plain text diff of openapi.json can't tell you which changes break clients. A semantic diff can.

SpecDiff is a single-file Python tool (standard library only) that compares two OpenAPI JSON documents and sorts every difference into BREAKING, WARNING or INFO. It knows direction: removing a field from a response breaks clients, while making a request field required breaks callers. With --fail-on-breaking, it exits with code 2 so CI can block the merge. Everything below was run locally.

Two versions of a spec

openapi.main.json is a small Users API with GET /users/{id} (returns #/components/schemas/User), POST /users (accepts NewUser) and DELETE /users/{id}/avatar.

The PR version, openapi.pr.json, does four things:

1. removes email from the User component schema;
2. makes name required in NewUser;
3. marks DELETE /users/{id}/avatar as deprecated;
4. adds GET /users.

Running it

python3 specdiff.py -b openapi.main.json -t openapi.pr.json --no-color --fail-on-breaking
echo "exit $?"

[SpecDiff Contract Verification Report]

Summary: 2 BREAKING | 1 WARNINGS | 1 INFO

[BREAKING] Property -> paths./users/{id}.GET.responses.200.application/json.schema.properties.email
         Property 'email' removed from response payload
[WARNING]  Deprecation -> paths./users/{id}/avatar.DELETE
         Operation DELETE /users/{id}/avatar has been marked as deprecated
[INFO]     Operation -> paths./users.GET
         HTTP Method GET added to /users
[BREAKING] Constraint -> paths./users.POST.requestBody.content.application/json.schema.required[name]
         Existing property 'name' marked as REQUIRED
exit 2

Both breaking changes were made inside components/schemas, not in the path definitions. SpecDiff follows the local $ref pointers (#/components/...) and reports each finding at the endpoint where clients will notice it, which is the useful place for a reviewer.

A purely additive change (a new optional avatar_url field and a new GET /users) passes:

Summary: 0 BREAKING | 0 WARNINGS | 2 INFO

[INFO]     Operation -> paths./users.GET
         HTTP Method GET added to /users
[INFO]     Property -> paths./users/{id}.GET.responses.200.application/json.schema.properties.avatar_url
         Property 'avatar_url' added (optional)
exit 0

What it checks

From the rules in the source:

Area	BREAKING	Non-breaking
Endpoints / methods	endpoint or method removed	endpoint or method added (INFO), operation deprecated (WARNING)
Parameters	parameter removed, new required parameter, optional → required	new optional parameter
Request body	body now required, media type removed, request property made required	media type added
Responses	status code removed, response media type removed, response property removed	new status code or media type, new optional property
Schemas	data type changed (string → integer)	—

Reports for PRs and tooling

-f markdown writes a report you can post as a PR comment:

**Status**: ❌ CONTRACT BREACH DETECTED

| Metric | Count |
| :--- | :--- |
| **Breaking Changes** | 2 |
| **Warnings / Deprecations** | 1 |
| **Informational** | 1 |

### 🚨 Breaking Changes
| Category | Target Path | Issue Description |
| :--- | :--- | :--- |
| `Constraint` | `paths./users.POST.requestBody...required[name]` | Existing property 'name' marked as REQUIRED |
| `Property` | `paths./users/{id}.GET.responses.200...properties.email` | Property 'email' removed from response payload |

-f json gives a summary plus one object per finding, with the old and new value:

{
  "summary": {"total_diffs": 4, "breaking": 2, "warning": 1, "info": 1},
  "divergences": [
    {"severity": "BREAKING", "category": "Property",
     "path": "paths./users/{id}.GET.responses.200.application/json.schema.properties.email",
     "message": "Property 'email' removed from response payload",
     "old_value": "email", "new_value": null}
  ]
}

In CI

Compare the spec on the PR branch with the one on main:

- name: API contract check
  run: |
    git fetch origin main --depth=1
    git show origin/main:openapi.json > /tmp/openapi.main.json
    python3 tools/specdiff.py -b /tmp/openapi.main.json -t openapi.json \
      --fail-on-breaking --no-color

If there's a breaking change the step fails with exit 2. Without --fail-on-breaking the report is informational and the exit code is 0. When a breaking change is intentional (a new major version), skip the check for that PR or compare against the new version's baseline.

Try it without your own spec

python3 specdiff.py --demo

The demo runs the engine against two built-in sample specs, so you can see the output format first.

Limitations

• JSON only. For YAML specs, convert first, e.g. yq -o=json openapi.yaml > openapi.json.
• Only local references (#/components/...) are resolved. Specs split over several files need bundling into one first (most OpenAPI tooling has a "bundle" command).
• oneOf / anyOf / allOf compositions and enum value changes aren't analysed.
• Constraint changes like maxLength, pattern or minimum aren't classified.
• It reports changes. It doesn't check that your implementation matches the spec.

FAQ

OpenAPI 3.0 or 3.1? It reads the standard paths / components structure that both share. Swagger 2.0 (definitions) isn't the target.

Dependencies? None. Python 3 and the standard library.

Can I hide colours in logs? Yes, --no-color.

---

SpecDiff is a one-time purchase that includes the source and README: get SpecDiff ($19). More details are on the SpecDiff product page.