Stripe is the gold standard for API stability. Date-pinned versions, compatibility layers, migration guides — the works. And yet: since 2024 they ship breaking changes twice a year, on purpose, via named releases (acacia, basil, clover, dahlia).
If the best API in the world breaks you twice a year, what are the other 20 APIs you depend on doing?
I built a tool to answer that concretely. It diffs OpenAPI specs, classifies every change as breaking or additive, scans your codebase for the exact call sites affected, and generates a patch — gated on your own test suite passing. Then I ran it on Stripe's real published specs. Here's what happened.
The diff (real data)
Source: Stripe's actual spec3.json from their public openapi repo, at two release tags (v2250 → v2349), about 8MB each.
| Total changes | 679 |
| Breaking | 16 |
| Additive | 663 |
| Documented paths compared | 200+ |
The breaking set includes a genuinely interesting one:
card.iin was removed — with no replacement. The issuer identification number field that BIN-routing logic depends on (token.card.iin.slice(0, 6) → "route Visa to the primary processor") simply stops existing. No card.iin2. Your code compiles fine. It dies at runtime, on live traffic.
(Also in the set: the entire response shape of Terminal's cancel_action endpoint changed — 13 fields gone at once.)
Detection is the easy part
There are good tools that watch API surfaces and alert you. Alerting is necessary and completely insufficient: an alert that says "something broke" still leaves an engineer reading a changelog, grepping a repo, rewriting call sites, and hoping.
So the tool does the whole loop:
- Watch — diff the specs (deterministic, no LLM needed for classification)
- Scan — map each breaking change to your actual call sites (file:line)
- Remediate — codemod the affected lines
- Verify — run YOUR test suite against a simulated post-change server. The patch only ships if your tests pass.
The proof
I wrote a consumer integration the way real code gets written — token.card.iin for BIN routing — and pointed it at a mock server shaped like the NEW spec:
Tests BEFORE patch: 1 passed, 2 failed ← TypeError on live patterns
Tests AFTER patch: 3 passed, 0 failed
The patch it generated replaced direct .iin access with a null-guard shim and — this is the honest part — flagged it for human review, because Stripe removed the field with no replacement and choosing a new data source is a business decision, not a code decision. The tool prevents the crash; it doesn't pretend to make your product decisions.
Fun detail: on the first run, the test gate caught a bug in my own codemod (an over-broad regex mangled the file). The gate refused to pass it. That's the entire trust model in one anecdote: nothing merges without your CI saying yes.
What's open source
The spec differ and scanner are MIT-licensed: https://github.com/Xenrai/patchline — point it at any two OpenAPI specs:
patchline diff old_spec.json new_spec.json
patchline scan --repo . --report diff.json
The remediation engine (codemods + the migration corpus + CI gating) is the commercial layer, and there's a free watcher that alerts you when APIs you use ship breaking changes.
What I learned
- The long tail of API change is additive (663 of 679) — signal-to-noise is the real problem, which is why per-codebase impact analysis matters more than changelogs
- Field removals with no replacement are the nastiest class: silent, runtime-only, and unfixable by automation alone
- "We read the changelogs" is this generation's "we have backups" — the diff found things the changelog never mentioned
If you maintain more than a handful of integrations, run the differ on your stack and tell me what it finds. Issues and PRs welcome.
Top comments (1)
API diffs are most useful when they separate noise from contract risk. “679 changes” sounds scary, but the important layer is which changes alter generated types, validation behavior, retry assumptions, or runtime compatibility. That is where teams need alerts, not just a changelog.