How I Built SchemaWatch: A CLI Tool That Catches Breaking API Changes Before Production

The Problem I Kept Running Into

It was a regular sprint. Backend team updated the API. Frontend team wasn't notified. The field user.id changed from integer to string. Nobody caught it in code review.

Production broke at 3AM.

Sound familiar?

I kept running into this exact problem — not because anyone was careless, but because there was no simple, automated way to catch breaking API changes before they shipped.

Existing tools like oasdiff are powerful, but they felt heavy for what I needed. I just wanted one command that would tell me: "Hey, this change will break your clients."

So I built SchemaWatch.

---

What SchemaWatch Does

SchemaWatch compares two OpenAPI schema files and detects breaking changes — classified by severity.

pip install schemawatch
python -m schemawatch.cli old.yaml new.yaml

That's it. No config files. No setup. Just two YAML files and one command.

The output looks like this:

====================================
🚨 SchemaWatch Report
====================================

Breaking changes detected: 5

── CRITICAL (2)
🔴 Endpoint removed: /orders
🔴 Method removed: GET /users

── WARNING (3)
🟡 Response field removed: User.email
🟡 Field type changed: User.id integer -> string
🟡 Field became required: User.id

------------------------------------
Summary:
- Total changes: 5
- Critical: 2
- Warning:  3
- Info:     0
------------------------------------

---

Why Severity Levels Matter

One thing I noticed with other tools: they treat all changes equally. But a removed description field is not the same as a removed endpoint.

SchemaWatch classifies changes into three levels:

Severity	Examples
🔴 Critical	Endpoint removed, HTTP method removed, schema removed
🟡 Warning	Field removed, type changed, field became required
🔵 Info	Enum changed, array item type changed

In a CI/CD pipeline, this means you can decide: "Fail the build only on Critical changes, warn on everything else."

---

CI/CD Integration

SchemaWatch is designed to live inside your pipeline:

# .github/workflows/schemawatch.yml
- name: Check for breaking API changes
  run: python -m schemawatch.cli openapi_old.yaml openapi.yaml

• Exit code 1 → breaking changes detected ❌ — build fails
• Exit code 0 → no breaking changes ✅ — build passes

---

How I Built It

The core logic is straightforward:

1. Parser — load and validate both YAML files
2. Diff Engine — compare paths, methods, schemas, fields recursively
3. Severity Classifier — tag each change as critical/warning/info
4. CLI — format and display output (text, JSON, or Markdown)

The trickiest part was recursive nested object comparison. OpenAPI schemas can have deeply nested properties, and a type change three levels deep is just as breaking as one at the top level.

def compare_properties(schema_name, old_props, new_props, path=""):
    for field in old_fields & new_fields:
        # Recursive check for nested objects
        if old_field.get("type") == "object" and new_field.get("type") == "object":
            changes.extend(
                compare_properties(schema_name, old_nested, new_nested, path=f"{field}.")
            )

---

What's Next

SchemaWatch is actively developed. On the roadmap:

• [ ] Request body change detection
• [ ] Response status code comparison
• [ ] Custom GitHub Action (Marketplace)
• [ ] .schemawatch-ignore file for intentional breaking changes

---

Try It

pip install schemawatch
python -m schemawatch.cli old.yaml new.yaml

GitHub: https://github.com/CemCelik79/schemawatch

If you've ever been burned by an undocumented API change, give it a try. And if you have feedback — especially on what breaking change patterns SchemaWatch doesn't catch yet — I'd love to hear it in the comments.

---

Built with Python, colorama, and a lot of 3AM production incidents as motivation.