DEV Community

Cover image for I Built an AI Code Review Pipeline — One Command for Review + Auto-Fix + Tests + Report
Conor
Conor

Posted on

I Built an AI Code Review Pipeline — One Command for Review + Auto-Fix + Tests + Report

#ai #codereview #opensource #devtools

I Built an AI Code Review Pipeline — One Command for Review + Auto-Fix + Tests + Report

Zero dependencies, 6 AI providers, runs with npx. Quality control for the Vibe Coding era.

The Problem: Who Reviews AI-Generated Code?

AI-assisted coding tools (Cursor, Copilot, Windsurf) have made us 3-5x faster. But they've also created a new problem:

AI writes fast, but who ensures quality?

You let Cursor write a component. It runs. But:

  • Any XSS vulnerabilities?
  • Edge cases handled? Null checks?
  • Types all any?
  • Error handling silently swallowing exceptions?

Manual code review? If you're a solo dev, a small team, or just busy shipping — there's no time.

So I built ai-review-pipeline — let AI review AI-generated code.

npx ai-review-pipeline
Enter fullscreen mode Exit fullscreen mode

One command: Review → Test generation → HTML report. Fully automated.

What It Does

Default Mode: Review + Tests + Report (Read-Only) 

npx ai-rp --file src/ --full
Enter fullscreen mode Exit fullscreen mode

Pipeline:

① AI Code Review (score + issue list + fix suggestions)
       ↓
② AI Test Case Generation (functional / adversarial / edge cases)
       ↓
③ HTML Visual Report
       ↓
④ Has 🔴 issues → exit(1) blocks CI; all green → exit(0) passes
Enter fullscreen mode Exit fullscreen mode

Doesn't touch your code. Review only, report only.

Terminal output in action:

![Terminal screenshot — AI Review real-time output]

--fix Mode: Automated Fix Loop 

npx ai-rp --fix --file src/views/Home.vue --full
Enter fullscreen mode Exit fullscreen mode 
① Review → finds 3 🔴 issues
       ↓
② AI auto-fix (quality issues only, never touches business logic)
       ↓
③ Re-review → 1 🔴 remaining
       ↓
④ Fix again → re-review → 0 🔴 ✅
       ↓
⑤ Test case generation
       ↓
⑥ HTML report
       ↓
⑦ Auto git commit
Enter fullscreen mode Exit fullscreen mode

Keeps fixing until clean, up to N rounds (default 5). If still not clean after N rounds? It won't get stuck — generates tests and report anyway, then exit(1) to tell CI "still has issues."

Auto-generated HTML report:

![HTML Report — score, severity levels, fix suggestions at a glance]

Get Started in 30 Seconds 

# 1. Set an API key (pick any AI provider)
echo 'OPENAI_API_KEY=sk-xxx' >> .env.local

# 2. Run
npx ai-review-pipeline --file src/ --full
Enter fullscreen mode Exit fullscreen mode

That's it. No npm install. No config file. No account signup.

6 AI Providers, Auto-Detected 

OPENAI_API_KEY=sk-xxx              # → OpenAI
DEEPSEEK_API_KEY=sk-xxx            # → DeepSeek
ANTHROPIC_API_KEY=sk-ant-xxx       # → Claude
DASHSCOPE_API_KEY=sk-xxx           # → Qwen (Alibaba)
GEMINI_API_KEY=xxx                 # → Google Gemini
AI_REVIEW_PROVIDER=ollama          # → Local Ollama (free, private)
Enter fullscreen mode Exit fullscreen mode
Provider Default Model Best For
OpenAI gpt-4o-mini Stable, great ecosystem
DeepSeek deepseek-chat Cheapest, fast
Claude claude-sonnet-4 Strongest code understanding
Gemini gemini-2.0-flash Generous free tier
Ollama qwen2.5-coder Fully local, zero cost

Don't want to send code to the cloud? Use Ollama:

ollama pull qwen2.5-coder
echo 'AI_REVIEW_PROVIDER=ollama' >> .env.local
npx ai-rp --file src/ --full
Enter fullscreen mode Exit fullscreen mode

What It Reviews

Issues are categorized by severity:

🔴 Critical (Blocks Merge)

Logic errors, security vulnerabilities (XSS, injection, credential leaks), data risks (race conditions, precision errors)

🟡 Warning (Should Fix)

Unhandled edge cases (null, undefined, timeout), type issues (any, unsafe casts), missing error handling

🟢 Info (Improve Later)

Code duplication, unclear naming, performance hints

Test Case Generation

Not just review — it also generates three types of test cases:

Type Covers
✅ Functional CRUD, state flows, component rendering
⚔️ Adversarial XSS injection, SQL injection, overflow
🔲 Edge Cases Null, 0, negative, MAX_SAFE_INTEGER

Output includes descriptions and runnable test code (auto-detects your stack: Vitest / Jest / pytest / Go testing).

CI/CD Integration

GitHub Actions 

- name: AI Code Review
  run: npx ai-review-pipeline --json
  env:
    OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
Enter fullscreen mode Exit fullscreen mode

Git Hook (lefthook) 

# lefthook.yml
pre-push:
  commands:
    ai-review:
      run: npx ai-rp --fix --max-rounds 3
Enter fullscreen mode Exit fullscreen mode

Auto-review before push. Blocks if issues remain.

Key Design Decisions

Unified Pipeline, Not Separate Commands

In v2, I had three commands: review, fix, test. After using it daily, I realized:

90% of the time you want "review + tests + report" in one shot.

v3 unified everything into one pipeline. --fix is an enhancement flag. Old commands kept as aliases for backward compatibility.

Safety Mechanisms

Auto-fix has a safety valve: the fixed file can't be less than 50% of the original. Prevents AI from "fixing" by deleting half your code.

Zero Dependencies

The entire tool has 0 required dependencies. Node.js 18+ built-in fetch for API calls, child_process for git, fs for files. The only optional peer dependency is https-proxy-agent for proxy setups.

Exit Codes for CI

Scenario Exit Code Meaning
Review passed 0 All clear
Has 🔴 issues 1 Block merge
--fix success 0 Fixed + committed
--fix failed 1 Still has issues

Even when --fix fails, tests and report are still generated — the report is for humans, not for passing CI.

How It Compares

Feature ai-review-pipeline CodeRabbit SonarQube
Setup npx, zero config SaaS, signup Deploy server
Dependencies 0 — Java runtime
Auto-fix ✅ Multi-round ❌ ❌
Test gen ✅ 3 types ❌ ❌
AI models 6 providers + local Fixed Rule engine
Data privacy Ollama = fully local Code → cloud Self-hosted ok
Cost Pay per token (pennies) $15/mo+ Community = free

The key difference: this is a CLI tool, not a SaaS. Your code goes directly to the AI API — no middleman server.

Project Config 

npx ai-rp init
Enter fullscreen mode Exit fullscreen mode

Generates .ai-pipeline.json — commit to git, share with team:

{
  "review": {
    "threshold": 95,
    "maxRounds": 5,
    "customRules": [
      "No any types allowed",
      "API keys must not be hardcoded",
      "All API calls must have error handling"
    ]
  },
  "test": {
    "stack": "auto",
    "maxCases": 8
  }
}
Enter fullscreen mode Exit fullscreen mode

customRules is the highlight — write your team's standards, AI enforces them on every review.

Quick Reference 

# Default: review + tests + report (read-only)
ai-rp
ai-rp --file src/views/Home.vue --full
ai-rp --file src/ --full

# Fix mode: review + auto-fix + tests + report
ai-rp --fix
ai-rp --fix --file src/a.vue --full --max-rounds 3

# Standalone test generation
ai-rp test --file src/utils.ts

# Initialize config
ai-rp init
Enter fullscreen mode Exit fullscreen mode

Try It 

npx ai-review-pipeline --file src/ --full
Enter fullscreen mode Exit fullscreen mode

3 minutes to give your codebase a health check.

If you found this useful, check out the repo and drop a ⭐:

👉 GitHub - ai-review-pipeline
📦 npm - ai-review-pipeline

Got questions or feedback? Drop a comment below — I read every one.

Top comments (0)