DEV Community

vibecodiq
vibecodiq

Posted on • Originally published at vibecodiq.com

Regression Fear in AI-Generated Codebases — Why Every PR Feels Like a Gamble

#testing #architecture #ai #softwareengineering

"Every PR is a gamble. We just don't know what it'll break."

If your team has stopped merging on Fridays — or Thursdays — you're experiencing regression fear. Not a psychological problem. A structural one.

This post explains the mechanism, how to measure it, and the fix.

The Mechanism: Unbounded Blast Radius

In a well-structured codebase, a change in module A affects module A and its direct dependents. The blast radius is bounded and predictable.

In most AI-generated codebases past month 3, the blast radius is unbounded:

BLAST RADIUS MAP

  Change: update pricing display

  PREDICTED impact:        ACTUAL impact:
  pricing/ui.tsx           pricing/ui.tsx
                           payments/checkout.ts
                           users/dashboard.tsx
                           notifications/email.ts

  predicted: 1 file        actual: 4 files
  blast radius: unbounded
Enter fullscreen mode Exit fullscreen mode

The mechanism: AI generates code that works by taking the shortest path. That shortest path often creates implicit dependencies — shared state, cross-module imports, business logic in unexpected places. Each implicit dependency extends the blast radius of every future change.

Detection: Three Measurements

1. Regression Ratio 

# Total commits in last 30 days
TOTAL=$(git log --oneline --since="30 days ago" | wc -l)

# Fix/revert commits
FIXES=$(git log --oneline --since="30 days ago" \
  --grep="fix\|revert\|hotfix\|broken\|rollback" | wc -l)

echo "Regression ratio: $(( FIXES * 100 / TOTAL ))%"
Enter fullscreen mode Exit fullscreen mode

What the number means:

  • 0-10%: Normal. Minor regressions caught early.
  • 10-20%: Warning. Coupling is growing.
  • 20%+: Critical. One in five commits fixes something a previous commit broke.

2. Cursed Files 

git log --since="90 days ago" --pretty=format: --name-only \
  | sort | uniq -c | sort -rn | head -10
Enter fullscreen mode Exit fullscreen mode

These are the files that appear in the most commits. In AI-generated codebases, the top 3 files are usually:

  • The largest files (500+ lines, multiple domains)
  • The files where regressions originate
  • The files nobody wants to touch

3. Merge Velocity Trend 

# Commits per week for last 12 weeks
for i in $(seq 0 11); do
  START=$(date -d "$((i+1)) weeks ago" +%Y-%m-%d)
  END=$(date -d "$i weeks ago" +%Y-%m-%d)
  COUNT=$(git log --oneline --since="$START" --until="$END" | wc -l)
  echo "Week -$i: $COUNT commits"
done
Enter fullscreen mode Exit fullscreen mode

A declining trend means the team is merging less frequently — not because they're doing less work, but because each merge is riskier.

Why "More QA" Doesn't Fix This

QA catches regressions after they happen. It does not prevent the architectural condition that causes them.

The regression cycle:

  1. Developer ships change. Passes unit tests.
  2. QA finds regression in unrelated area. Files bug.
  3. Developer context-switches back. Debugs for hours.
  4. Fix introduces new side effect. Cycle repeats.

Adding more QA people makes the cycle faster — not shorter. The root cause remains: unbounded blast radius.

The Cost

A team of three developers at $150/hour loaded:

  • 30% of features cause regressions = 2.4/month
  • 6 hours average to find, diagnose, fix each
  • $2,160/month = $25,920/year in regression costs alone

Plus velocity decay:

Month 3:  8 features/month (productive)
Month 6:  5 features/month (fear + overhead)
Month 9:  3 features/month (regression management dominates)
Enter fullscreen mode Exit fullscreen mode

The Structural Fix: Bounded Blast Radius

Step 1: Identify Cursed Files

Run the cursed files detection above. The top 3 files are your decomposition targets.

Step 2: Map Implicit Dependencies

For each cursed file, trace what depends on it:

# Find all files that import from the cursed file
grep -rln "from.*cursedFile\|require.*cursedFile" \
  --include="*.ts" --include="*.tsx" src/
Enter fullscreen mode Exit fullscreen mode

Step 3: Decompose by Domain

Split cursed files by business domain. Each new file has one responsibility and explicit imports.

Step 4: Enforce One-Directional Dependencies 

# Verify no circular dependencies exist
npx madge --circular --extensions ts,tsx src/
Enter fullscreen mode Exit fullscreen mode

After decomposition, the dependency graph should be a tree, not a web.

Step 5: Add Regression Tests for Boundaries

For each module boundary, add a test that verifies the contract:

// Test that pricing module doesn't depend on notification module
import { getPrice } from '@/modules/pricing';

test('getPrice returns number without side effects', () => {
  const result = getPrice({ plan: 'pro', period: 'monthly' });
  expect(typeof result).toBe('number');
  // No notifications sent, no user state changed
});
Enter fullscreen mode Exit fullscreen mode

Step 6: CI/CD Enforcement

Block merges that:

  • Introduce circular dependencies
  • Exceed file size thresholds
  • Miss test coverage on boundary contracts

The Result

After structural enforcement, the blast radius of every change is bounded and predictable. QA becomes verification — confirming bounded changes work as expected — not discovery of unrelated breakage.

Regression fear disappears. Not because the team is braver. Because the architecture makes cross-module leakage structurally impossible.

This is part of the AI Chaos series — a structural analysis of failure patterns in AI-generated codebases. Based on ASA (Atomic Slice Architecture) — an open architecture standard for AI-generated software.

Resources

  • ASA Standard — the open specification
  • GitHub — source, examples, documentation
  • Vibecodiq — structural diagnostics for AI-built apps

Top comments (0)