The latest articles on DEV Community by Abhinav Sharma (@abhinavs1920). https://dev.to/abhinavs1920 https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4172137%2Fdc1ff21b-5b74-4a19-8d11-32bcd407f1e0.png https://dev.to/abhinavs1920 en Abhinav Sharma Fri, 09 Oct 2026 06:23:43 +0000 https://dev.to/abhinavs1920/i-built-13-labelled-openapi-breaking-change-cases-and-oasdiff-missed-2-of-them-1jmp https://dev.to/abhinavs1920/i-built-13-labelled-openapi-breaking-change-cases-and-oasdiff-missed-2-of-them-1jmp <p>I'm working on Impact Gate, a warn-only PR check that tells you which consumers an API change can affect. Before claiming anything about it, I wanted a public set of cases anyone can run. So I wrote one: github.com/impact-gate/impact-gate-benchmarks.</p> <p>What it is: 13 before/after OpenAPI specs on one small Orders API, each with a label (breaking or not) and two made-up consumers, one of which is affected or neither is. 10 breaking changes, 3 controls (a new optional field, a new endpoint, a renamed path parameter).</p> <p>What I ran: oasdiff v1.33.0 with --fail-on ERR. It caught 8, missed 2, raised no false alarms. The misses: removing an optional response field, and adding an API key requirement to an operation. oasdiff reports both as info, which is a defensible choice for a spec diff. But a consumer that reads that field, or calls without a key, breaks all the same. That gap between spec-level breaking and consumer-level breaking is what I want to measure.</p> <p>Limits, plainly: the cases are synthetic, I wrote the labels myself, and 13 cases proves nothing about real-world accuracy. Next: real-project cases and automated consumer scoring. If you disagree with a label, open an issue, I'd like to be wrong in public. To score your own tool: TOOL_CMD="your-tool" bash scripts/run.sh.</p> openapi api opensource testing