- Book: System Design Pocket Guide: Interviews
- Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
Your mobile app reads total_cents from GET /orders/{id}. After this branch lands, that field is gone. The agent renamed it to totalCents "for consistency", regenerated openapi.yaml and updated the handler and its tests, so the suite is green.
The app lives in another repository, so the server's tests can't fail on its behalf. And the agent rewrote those tests along with the code. Nothing in this repo knows the client exists.
The spec does, though. On main, total_cents is a required field of the response. On the task branch it no longer exists. Comparing those two contracts is a job for a tool, and oasdiff is a command-line tool built for it. This post wires it in as a check the agent's work has to pass before it lands, and shows how to hand the failures back to the agent.
The change that passes every test
Here is the relevant part of an OpenAPI spec on main. Order is the 200 response of GET /orders/{id}, and RefundRequest is the request body of POST /orders/{id}/refund:
# openapi.yaml on main (excerpt)
components:
schemas:
Order:
type: object
required: [id, status, total_cents]
properties:
id:
type: string
status:
type: string
enum: [pending, paid, refunded]
total_cents:
type: integer
customer_note:
type: string
RefundRequest:
type: object
required: [reason]
properties:
reason:
type: string
enum: [duplicate, fraud, other]
And here is the same excerpt after a task that asked for "clean up the order schema and add a disputed status":
# openapi.yaml on the task branch (excerpt)
components:
schemas:
Order:
type: object
required: [id, status, totalCents]
properties:
id:
type: string
status:
type: string
enum: [pending, paid, refunded, disputed]
totalCents:
type: integer
customerNote:
type: string
RefundRequest:
type: object
required: [reason]
properties:
reason:
type: string
enum: [duplicate, fraud]
Four edits. Two renames, one new response enum value, and other quietly dropped from the refund reasons because the agent found no code path that sends it. Every one of them can pass the server's own tests.
Compare the task branch to main with oasdiff
The oasdiff README lists Homebrew, go install, Docker and prebuilt binaries. The latest release is v1.35.0, from October 28, 2026.
brew install oasdiff
# or
go install github.com/oasdiff/oasdiff@latest
You don't need to export the old spec by hand. oasdiff accepts <git-ref>:<path> as an input and runs git show for you (docs). Run it from inside the repo, with the base on the left and the file on disk on the right:
oasdiff breaking main:openapi.yaml openapi.yaml
The right side is the working-tree file, so the agent's uncommitted edits are checked too. If you'd rather have the base spec as a plain file, git show writes it out:
dir=$(mktemp -d)
git show main:openapi.yaml > "$dir/base.yaml"
oasdiff breaking "$dir/base.yaml" openapi.yaml
Prefer the native syntax when your spec is split into several files. oasdiff resolves relative $refs against the spec's path, and a copy in a temp folder has no neighbours to resolve against.
The server's tests check the server. The spec is the only place the client's side of the deal is written down.
ERR, WARN, and the rename it stays quiet about
oasdiff's breaking-changes doc defines three levels. ERR is a definite breaking change. WARN is a potential breaking change whose effect the definition can't determine. INFO is a non-breaking change. oasdiff breaking reports only ERR and WARN.
I haven't run oasdiff on this spec. Here is how the four edits map to the levels in oasdiff's rule definitions and its check catalog:
| Edit | Check | Level |
|---|---|---|
total_cents removed (required) |
response-required-property-removed |
ERR |
disputed added to response enum |
response-property-enum-value-added |
ERR |
other removed from request enum |
request-property-enum-value-removed |
ERR |
customer_note removed (optional) |
response-optional-property-removed |
INFO |
The enum row surprises people. Adding a value to a response enum feels additive, but the catalog's note is blunt: "The server may now return a value the previous contract excluded," and a client written against the old list may not handle it. If the list is meant to grow, the catalog says to declare it with x-extensible-enum.
The last row is the one to fix in your setup. Since oasdiff v1.30.0, removing an optional response property is INFO, on the grounds that "a conforming client already tolerates the element's absence." So breaking won't mention the customer_note rename at all. If your clients read that field and would show a blank where a note used to be, raise the level. A severity file has one check ID and a level per line, and the allowed levels are err, warn, info and none:
response-optional-property-removed err
Save it as oasdiff-levels.txt, commit it, and pass --severity-levels oasdiff-levels.txt.
WARN is for changes whose effect the definition can't pin down. The doc's examples are a change to an object parameter whose serialization isn't specified, and a change inside an allOf branch when you don't pass --flatten-allof. Don't fail the build on those. Read them during review.
Turn it into a pre-land check
--fail-on ERR makes oasdiff exit with code 1 when it finds any ERR change. That is all a check needs. Put it in a script so you, your CI and your agents all run the same thing:
#!/usr/bin/env bash
# scripts/api-compat.sh
set -euo pipefail
cd "$(git rev-parse --show-toplevel)"
base_ref=${API_BASE_REF:-main}
base=$(git merge-base "$base_ref" HEAD)
oasdiff breaking \
--fail-on ERR \
--severity-levels oasdiff-levels.txt \
"$base:openapi.yaml" openapi.yaml
Why git merge-base and not main? If another task landed a new endpoint on main after this branch was cut, the branch doesn't have it yet, and comparing against the tip of main would report that endpoint as removed. The merge base compares the spec as it was when the task started with the spec now, so the report covers only what this task changed. After a rebase onto main, the two are the same commit. oasdiff accepts a commit SHA as the ref. If your tasks start from a branch other than main, set API_BASE_REF to that branch.
The same script runs in CI, with two changes to the job. A pull request checkout has no local main branch, only origin/main, so set API_BASE_REF=origin/main in the job. And the git-revision doc says to check out with fetch-depth: 0, because a shallow clone lacks the history the syntax needs. The oasdiff GitHub Action is there if you'd rather not script it.
Hand the report back to the agent
A failing check is only half the job. The agent that made the change should undo the damage, because it knows why it made each edit.
In Claude Code, run the script in shell mode by prefixing it with !:
! ./scripts/api-compat.sh
Shell mode adds the command and its output to the conversation, and the docs say Claude responds to the output automatically. Follow with what "fixed" means, because "make the check pass" also invites the agent to edit oasdiff-levels.txt:
Restore backward compatibility. Keep total_cents
and customer_note next to the new names and mark
the old ones deprecated: true. Put "other" back
in the refund reason enum. Remove "disputed" from
the response enum for now. Do not edit
oasdiff-levels.txt. Re-run the check.
deprecated is a standard field on an OpenAPI 3.0 Schema Object: it "SHOULD be transitioned out of usage." Clients get a release to move, and the old names come out later as a deliberate breaking change instead of a side effect.
With any agent that runs shell commands, you can also add one line to the instructions file it reads: run ./scripts/api-compat.sh before saying a task is done.
Running it per task in Hermes IDE
I build Hermes IDE, so weigh this section with that in mind. Everything above works without it. What Hermes adds is a clean base-versus-head pair for every task, and a place for your own checks to decide when the task is done.
Press ⌘N (Ctrl+Shift+N on Windows and Linux), describe the task and pick the agent. The task starts on its own hermes/ branch in its own worktree, with the task as the agent's first prompt. Worktrees share the branch refs under refs/ (git docs), so main and the merge base resolve inside the task's folder the same way they do in your main checkout.
The ⌘N New task sheet: Claude Code on orders-api, in a new hermes/ worktree, with the "Hermes will run" preview of the claude command.
Make ./scripts/api-compat.sh one of those checks. The per-repository setup is in the worktree recipes doc. If the check fails, hand the report back to the agent with the prompt above.
When the check is green, open the Review Desk (⌘G, Ctrl+Shift+G on Windows and Linux) and read the work by turn. The turn that touched openapi.yaml is the one to read slowly, along with any WARN lines from the report. A line comment goes back to the agent that wrote that line.
The Review Desk by turn: two turns of a Claude Code task in orders-api, turn 1's diff of src/db.ts, and a commit message drafted from the turns.
Then the Land step commits on the branch, opens a pull request or merges locally into the branch the task started from, and can be undone. Hermes doesn't know what OpenAPI is. Your script decides what "compatible" means. Hermes is free and source-available under BSL 1.1. It runs on macOS, Windows and Linux, and is not affiliated with Nous Research, Anthropic, OpenAI or any agent vendor.
If this was useful
In a system design interview, the API you sketch early on is the contract every other box depends on, and a good follow-up question is how it changes without breaking the clients you can't see. The System Design Pocket Guide: Interviews walks through 15 real designs step by step, trade-offs included. If you want the check from this post running per task, download Hermes IDE, and if it catches a rename for you, star it on GitHub.
Top comments (0)