sunnydachs

Posted on Sep 19

Your OpenAPI spec is a contract. Nobody is checking it. I built a CLI that detects spec↔code drift with zero dependencies.

#python #api #openapi #opensource

"The generated client keeps calling DELETE /items/{id}."
"The server returns 405 Method Not Allowed."

An OpenAPI spec is not just documentation. SDKs, frontend types, API docs, mock servers — everything is generated from the spec. When the implementation drifts from it, everything generated starts lying quietly.

This is about oas-drift, a CLI I built that detects that drift — with zero dependencies (Python 3.11+ standard library only, no LLM).

https://github.com/sunnydachs/oas-drift

What it does

You give it an OpenAPI JSON spec and a Python codebase. It finds three classes of drift:

SPEC ONLY — defined in the spec, but no matching route in code
CODE ONLY — route exists in code, but is not in the spec
METHOD MISMATCH — path matches, but the HTTP method differs

# scan the current directory against a spec (read-only)
oas-drift --spec openapi.json

# scan a specific root, machine-readable output
oas-drift --spec openapi.json ./src --json

Here is a real run against a deliberately-drifted demo app:

oas-drift — scanned src
  spec: 5 endpoint(s) | code: 4 route(s)

/health  ➕ CODE ONLY
    route implemented (GET) but not defined in spec — src/app.py:21
/items/{id}  ⚠️ METHOD MISMATCH
    /items/{id}: in spec but not implemented: DELETE; implemented but not in spec: POST — src/app.py:17
/users/{id}  ⚠️ METHOD MISMATCH
    /users/{id}: in spec but not implemented: DELETE — src/app.py:13
/admin/stats  ⬜ SPEC ONLY
    defined in spec (GET) but no matching route in codebase
/items  ⬜ SPEC ONLY
    defined in spec (GET) but no matching route in codebase

summary: {"code_only": 1, "method_mismatch": 2, "spec_only": 2} | ok: 2

"The spec is the contract" only works if someone checks both sides. Code review sees the diff against the last commit — not against a spec written three months ago.

Why it's a detector, not a gate

This was the core design decision. Most CI-facing drift tools fail the build when they find anything. oas-drift's exit code is 0 either way. Three reasons:

Adoption survives. A tool that fails your build on day one gets removed on day two. A detector stays.
Drift has priorities. A /health endpoint missing from the spec is usually fine. A METHOD MISMATCH on a payment route is not. Which drift fails the build is policy — the tool shouldn't decide that for you.
Fully deterministic. It parses sources with Python's ast module — never imports, never executes, never writes. Same input → same report, always.

If you do want to fail on specific statuses, wire it into CI with --json and jq — the report is machine-readable by design.

Same principles as the sibling tools I shipped this month — doc-drift (README↔code) and plan-drift (tracking plan↔code). Deterministic work deserves deterministic tools.

The rule: paths match literally

The detail I obsessed over: path parameters and router prefixes. The rule:

/users/{id} in the spec matches /users/{id} in code — and only that.
Router prefixes are NOT resolved. A router with prefix="/items" serving /{id} does not match a spec's /items/{id}.

I validated this against a real, widely-used codebase — the backend of FastAPI's official full-stack-fastapi-template (25 files, 14 paths, 23 routes detected). Scanning it with a spec written in prefixed paths produces exactly the false-positive pair this rule predicts:

/items/{id}  ⬜ SPEC ONLY
    defined in spec (DELETE, GET, PUT) but no matching route in codebase
/{id}  ➕ CODE ONLY
    route implemented (DELETE, GET, PUT) but not defined in spec — backend/app/api/routes/items.py:48, ...

Two names for the same route. oas-drift doesn't guess — it reports what literally exists. If your project uses router prefixes, normalize the spec side first (the surest source is the /openapi.json your app actually serves).

And the important detection works: flipping the implemented POST /login/access-token to PUT in a test spec gets reported precisely:

/login/access-token  ⚠️ METHOD MISMATCH
    /login/access-token: in spec but not implemented: PUT; implemented but not in spec: POST — backend/app/api/routes/login.py:23

Testing against real-world data

15 pure-function tests (no network, no fixtures on disk), all passing. Runtime: 0.04s.
e2e on a real install path: wheel build → install → actual drift detection, verified.
A real codebase: FastAPI's official full-stack-fastapi-template backend — the deliberately planted METHOD MISMATCH was caught, and the router-prefix false-positive pair was something I discovered and documented.

Honest limitations

JSON only (OpenAPI 3.x JSON; YAML is a future item).
Literal path matching — normalizing router prefixes is on you.
f-string paths are invisible — @app.get(f"/users/{id}") isn't extracted; string literals only.
Routes and methods only — request/response schema comparison is a future item.

These are documented in the README. The current version prioritizes minimal, honest detection over coverage.

Wrap-up

Specs drift the moment the code moves and nobody updates them. A contract is only a contract if something checks it — oas-drift is that checking part, built read-only, fully deterministic, and dependency-free.