DEV Community: Vojta Holeš

I Added AI Auto-Fix to My Breaking Change Detector — Here's How /fix Works

Vojta Holeš — Fri, 05 Jun 2026 13:59:55 +0000

Last week I shipped a feature I've been wanting for months: type /fix in a GitHub PR comment and get a fix PR generated automatically.

Context
BreakShield CI is a GitHub App I built that detects breaking API changes in PRs using AST analysis. It catches removed fields, changed types, deleted endpoints — the stuff that slips through code review and breaks production.

But detection without a fix is just a fancy error message. So I added /fix.

How /fix works
When you comment /fix on a PR that has breaking changes:

BreakShield reacts with 👀 (acknowledged)
Fetches the affected file from the PR head
Sends it to your configured AI provider with:
The file content
What field/type was removed or changed
The before/after schema
AI generates the fixed file
BreakShield creates a new branch, commits the fix, opens a PR
Reacts with 🚀 (done)
Multi-provider support
You pick the AI:

Provider Free? Best model
Gemini ✅ 2.5 Flash
OpenAI ❌ GPT-5.4 mini
Claude ❌ Haiku 4.5
Groq ✅ Llama 3.3 70B
Perplexity ❌ Sonar Pro
Configure in the dashboard. BYOK — your key, your model, your choice.

Why not just use AI for detection too?
Because LLMs hallucinate. AST analysis doesn't.

Detection needs to be precise. 92% confidence means "I found this in the syntax tree, it's definitely gone." Not "I think maybe something changed."

AI is great for generating fixes. Terrible for detection where false positives erode trust.

Try it
Install: https://github.com/apps/breakshield-ci
Add an API key in settings (Groq is free)
Open a PR that removes a field from a TypeScript interface
Wait for BreakShield's comment
Reply with /fix
Watch the magic
Source: https://github.com/vojtisprime11/BreakShield

Free and open-source GitHub App for TypeScript teams. Detects breaking API changes in PRs using AST analysis and offers AI auto-fix with /fix command. Just shipped multi-provider support (Gemini, OpenAI, Claude, Groq). Would love feedback from anyone worki

Vojta Holeš — Thu, 04 Jun 2026 19:41:31 +0000

Vojta Holeš

Jun 4

You Shouldn't Need a Production Incident to Find Breaking API Changes

#github #typescript #devtools #api

3 min read

You Shouldn't Need a Production Incident to Find Breaking API Changes

Vojta Holeš — Thu, 04 Jun 2026 19:39:35 +0000

You open Slack on Monday morning. There's a thread with 43 replies. Someone merged a PR on Friday. A field got removed from an API response. Three services are broken. Customers are affected. Nobody caught it in review.

It didn't have to happen.

The Problem Nobody Talks About
Code review catches logic bugs, style issues, missing tests. But it almost never catches API contract violations.

A required field disappears from a response type
A parameter type changes from string to number
An endpoint gets deleted
A new required field appears in a request body
These are invisible in a diff. The TypeScript compiler won't warn you. Your tests pass because they mock the old contract. ESLint doesn't care.

And then production cares. A lot.

What If Your CI Caught It First?
I built BreakShield CI — a GitHub App that analyzes every PR for breaking API changes.

Not with an LLM. Not with vibes. With AST analysis.

It parses your TypeScript code, diffs the abstract syntax tree between base and head, and flags exactly what changed, where it's used, and how dangerous it is.

What It Detects
Change type Example
Removed field UserResponse.email deleted
Changed type id: number → id: string
Removed endpoint GET /api/users/:id gone
Added required field New organizationId required in request
Changed return type Function returns Promise instead of Promise
Removed interface PaymentMethod type deleted entirely
Every finding comes with:

Confidence score (80%+ = AST-verified, no guessing)
Consumer search — which files in your repo actually use that field
Risk level — CRITICAL / HIGH / MEDIUM / LOW / SAFE
Before/after — what it was, what it is now
AI Auto-Fix. In the PR.
Detection is only half the story. You also need a fix.

BreakShield CI now supports AI-generated fixes — directly from your PR.

Type /fix as a comment in your PR. BreakShield:

Reads the affected file
Sends it to your AI provider with context about what broke
Generates the fix
Opens a new PR with the corrected code
You review. You merge. Done.

Supported Providers
Provider Models Free tier?
Google Gemini 3.5 Flash, 2.5 Pro, 2.5 Flash ✅
OpenAI GPT-5.5, GPT-5.4, GPT-5.4 mini ✅
Anthropic Claude Opus 4.8, Sonnet 4.6, Haiku 4.5

Groq GPT-OSS 120B, Llama 3.3 70B, Qwen3 32B ✅
Perplexity Sonar Deep Research, Sonar Pro ✅
BYOK — bring your own key. You pick the provider and model in settings.

How It Works Under the Hood
No magic. No "AI-powered detection." Here's what actually happens:

PR opened/updated
↓
GitHub webhook fires
↓
Fetch changed files (base vs head)
↓
Parse both versions with ts-morph (TypeScript AST)
↓
Diff exported interfaces, types, functions, endpoints
↓
For each breaking change → search repo for consumers
↓
Calculate risk score → post PR comment + check run
The detection is deterministic. Same code → same result. Every time.

AI is only involved when you explicitly ask for a fix.

What the PR Comment Looks Like
When BreakShield finds something, you get a comment like this:

🟠 HIGH — 1 breaking change · 3 files with verified usages

UserResponse.name
Change removed field
File

user.ts
Confidence 92% · AST-verified
Before name: string
💡 Auto-fix available — comment /fix src/types/user.ts to generate a fix PR.

Zero Config Setup
Install the GitHub App: github.com/apps/breakshield-ci
Open a PR with API changes
BreakShield analyzes automatically
That's it. No YAML. No config files. No CI pipeline changes.

For AI auto-fix, go to the dashboard and add your API key: → breakshield-ci.vercel.app

Why Not Just Use TypeScript?
TypeScript catches type errors within a project. It doesn't catch:

Removing a field that external services depend on
Changing a response type that a mobile app consumes
Deleting an endpoint that 12 microservices call
TypeScript sees your repo. BreakShield sees your API contract.

Why Not an LLM for Detection?
LLMs hallucinate. They miss things. They have different results on different runs.

AST analysis is:

Deterministic — same input, same output
Fast — 500ms-2s per PR, not 30s
Precise — 92-95% confidence, not "probably breaking"
Explainable — exact file, line, usage type
LLMs are great for generating fixes. Terrible for detection where precision matters.

Try It
→ Install: github.com/apps/breakshield-ci → Dashboard: breakshield-ci.vercel.app

Open a PR. Remove a field from an interface. Watch it get caught.

Then type /fix and watch it get fixed.

Just published: the 3am production incident that made me build BreakShield CI. If you've ever debugged a breaking change at 3am — this one's for you 🛡️

Vojta Holeš — Tue, 02 Jun 2026 18:56:34 +0000

Vojta Holeš

Jun 2

The 3am Production Incident That Made Me Build BreakShield CI

#typescript #github #webdev #devops

2 min read

The 3am Production Incident That Made Me Build BreakShield CI

Vojta Holeš — Tue, 02 Jun 2026 18:55:34 +0000

It was 3am when my phone started ringing.

Our API was down. Half the app was broken. Users were angry. The on-call engineer was panicking.

After two hours of debugging we found it. Someone had removed the email field from UserResponse. Three other services were using it. Nobody caught it in code review. Nobody even knew those services existed.

We fixed it. Deployed at 5am. Wrote the post-mortem. Added it to the "lessons learned" doc that nobody reads.

Three months later it happened again.

The problem nobody talks about

Every team I've worked with has the same issue.

You have a TypeScript interface. It's used in 6 places across 4 different files. You remove one field. You open a PR. Your teammates review it. Nobody catches it because nobody knows about those 6 places.

The tests pass. The build passes. You merge.

And then production breaks.

The scary part? This isn't a skill issue. Senior engineers with 10 years of experience make this mistake. It's a tooling problem. Code review wasn't designed to catch this.

What I built

I spent a weekend building BreakShield CI — a GitHub App that does one thing:

Catches breaking API changes before you merge.

When you open a pull request, it automatically:

Parses your TypeScript interfaces and OpenAPI specs with a full AST
Finds every place in your codebase that uses the changed API
Shows you exactly which file and which line will break
Posts a report directly in your PR
Blocks merge if the risk is HIGH or CRITICAL

No config. No CLI. No YAML. Install once, works on every PR forever.

What it looks like in practice

You remove email from UserResponse. BreakShield CI posts this in your PR:

🔴 HIGH RISK — 1 breaking change

UserResponse.email — removed field File: src/types/user.ts Confidence: 94% — AST-verified

2 consumers affected:

src/components/UserCard.tsx:23 return ${user.name} <${user.email}>

src/pages/profile.tsx:41 const { email, name } = user

Your teammate sees this. They don't need to know the codebase. They don't need to grep for usages. The information is right there.

Why AST and not regex

Most tools use regex to find breaking changes. Regex lies.

If your interface has a field called emailAddress and you search for email, regex will find it. AST won't — because AST actually understands your code.

BreakShield CI uses ts-morph to parse a full TypeScript AST. It knows the difference between a property access, a destructuring, and a type annotation. Every finding is verified at the compiler level.

If there's no evidence — there's no warning. No noise, no false positives.

The 3am call I didn't get

Last week a teammate removed a field from a shared interface. BreakShield CI caught it. The PR comment listed 4 consumer files. We updated them before merging.

I didn't get a 3am call.

That's the whole point.

Try it

BreakShield CI is free during beta. It takes 30 seconds to install.

👉 breakshield-ci.vercel.app

If you've ever had a 3am production incident caused by a breaking change — you know exactly why I built this.

Just launched BreakShield CI — a free GitHub App that catches breaking API changes in PRs using AST analysis. Would love feedback from the TypeScript community!

Vojta Holeš — Tue, 02 Jun 2026 15:57:39 +0000

Vojta Holeš

Jun 2

I Built a GitHub App That Catches Breaking API Changes Before They Ship

#showdev #api #github #typescript

2 min read

I Built a GitHub App That Catches Breaking API Changes Before They Ship

Vojta Holeš — Tue, 02 Jun 2026 15:55:11 +0000

Last week I spent 10 hours debugging a production issue. Someone had removed a field from a TypeScript interface. Three other files were using it. Nobody caught it in code review because nobody knew those files existed.

That's when I decided to build BreakShield CI.

What it does

BreakShield CI is a GitHub App that automatically analyzes every pull request for breaking API changes. It runs in the background — no config, no CLI, no YAML.

When it finds something, it posts a comment directly in your PR:

Which interface or endpoint changed
What exactly changed (removed field, type change, optional → required)
Which files and lines in your codebase actually use the changed API
A risk level: SAFE / LOW / MEDIUM / HIGH / CRITICAL

HIGH and CRITICAL risk PRs get a failing Check Run that blocks merge.

How it works technically

The interesting part is the analysis engine.

Most tools use regex to find breaking changes. BreakShield CI uses a full TypeScript AST via ts-morph. This means it actually understands your code — not just searches for strings.

For every changed .ts file in a PR it:

Parses both the before and after version into an in-memory AST
Extracts all exported interfaces, type aliases, and functions
Diffs them to find breaking changes
Searches the codebase for files that use the changed API
AST-verifies each match — direct access, destructuring, type annotations
Scores each finding by confidence (0–100)

For OpenAPI specs it uses @apidevtools/swagger-parser to dereference all $ref pointers and diffs paths, parameters, request bodies, and response schemas.

What it detects

Breaking (blocks merge on HIGH/CRITICAL):

Removed field from interface
Changed field type
Optional → required
Removed endpoint
Removed function parameter
Added required field to request body

Safe (informational only):

Added optional field
New endpoint added

The confidence scoring

Every finding needs evidence. No evidence = no warning.

Evidence is scored by usage type:

obj.field direct access → 90%
const { field } = obj destructuring → 80%
: UserResponse type annotation → 80%
GitHub Search hit without AST verification → 35%

Low confidence noise is filtered automatically.

Try it

BreakShield CI is free during beta. Install it in 30 seconds:

👉 breakshield-ci.vercel.app

Would love feedback from anyone working with TypeScript APIs or microservices.
`