How AI Pull Requests Slip Past Code Review

AI-generated pull requests look plausible at first glance. They pass CI, use the right style, and reference the correct APIs. But they miss the design context that only a human who has lived in the codebase possesses. The result is a flood of technically correct but architecturally incoherent changes that consume reviewer time and erode trust.

In this article you will learn:

• How to spot the telltale patterns of AI-authored code in a PR
• A GitHub Actions workflow that flags suspicious submissions before they reach a human
• The tradeoffs of moving off GitHub to a forge like Codeberg
• A lightweight contribution policy that preserves the social contract of open source

Detect the Patterns That CI Misses

AI code tends to be locally consistent but globally disconnected. It reimplements helpers that already exist, introduces unnecessary abstractions, and leaves comments that describe what the code does instead of why it exists. These signals are invisible to a linter but obvious to a maintainer who knows the codebase.

## .github/workflows/ai-pr-detect.yml

name: AI PR Detection
on:
  pull_request:
    types: [opened, synchronize, reopened]
jobs:
  heuristic-check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - name: Run heuristic scanner
        run: |
          python3 << 'EOF'
          import subprocess, sys, re

          # Get diff against base branch
          base = subprocess.check_output(['git', 'merge-base', 'origin/main', 'HEAD'], text=True).strip()
          diff = subprocess.check_output(['git', 'diff', base, '--', '*.zig', '*.c', '*.h'], text=True)

          flags = []
          # Pattern: verbose comments explaining obvious code
          if re.search(r'//\s*(This function|The following|We create|Initialize)', diff):
              flags.append('explanatory comments on trivial code')
          # Pattern: reimplementing stdlib helpers
          if re.search(r'fn\s+(alloc|free|memcpy|strlen)\s*\(', diff):
              flags.append('possible stdlib reimplementation')
          # Pattern: excessive new files in a single PR
          new_files = len(re.findall(r'^\+\+\+ b/', diff, re.MULTILINE))
          if new_files > 8:
              flags.append(f'{new_files} new files in one PR')

          if flags:
              print('::warning::AI-like patterns detected:', '; '.join(flags))
              sys.exit(1)
          EOF

This workflow runs a fast heuristic scan on every PR. It does not block merges; it surfaces a warning so reviewers know where to look first. The patterns are intentionally conservative -- they catch the most common AI tells without false-positiving legitimate contributors.

Harden the Review Checklist

Automation catches the obvious. A shared checklist catches the rest. Add these items to your PULL_REQUEST_TEMPLATE.md so every contributor sees them and every reviewer verifies them.

## Reviewer Checklist

- [ ] Changes follow existing architectural patterns (no new paradigms introduced)
- [ ] No duplicate helpers -- searched codebase for existing equivalents
- [ ] Comments explain *why*, not *what* ("why" = design intent, tradeoffs, constraints)
- [ ] Tests cover failure paths, not just happy paths
- [ ] PR description links to an issue or design doc (no "drive-by" features)

The checklist shifts the burden from "prove this is human" to "prove this fits the project." Legitimate contributors appreciate the clarity; drive-by AI submissions rarely pass.

Weigh the Platform Migration

Zig moved to Codeberg because GitHub's incentive structure favors engagement metrics over maintainer health. Codeberg (a Gitea instance) offers no AI copilot, no algorithmic PR surfacing, and a nonprofit governance model. The tradeoffs are real:

Factor	GitHub	Codeberg
Discoverability	High (search, stars, trending)	Low (manual discovery)
CI/CD Ecosystem	Actions, massive marketplace	Woodpecker, Drone, self-hosted
AI Tooling Integration	Copilot, code scanning, bots	None built-in
Governance	Corporate (Microsoft)	Nonprofit association
Migration Effort	N/A	Medium (issues, wiki, CI rewrite)

If your project relies on drive-by contributions from casual users, GitHub's reach matters. If your project depends on a small cohort of trusted committers, Codeberg's quieter environment reduces noise. Zig chose the latter. Your calculus may differ.

Write a Policy That Survives Contact

A ban on AI contributions is unenforceable without a social layer. Zig's policy works because it is paired with a culture: contributors are known, discussions happen in public, and the maintainer team has authority to reject without appeal. Codify that culture.

## CONTRIBUTING.md (excerpt)

## AI-Generated Code

We do not accept pull requests authored primarily by AI coding assistants.
This includes but is not limited to GitHub Copilot, Cursor, Claude Code, and
local LLM integrations.

**Why:** AI code lacks the design context that makes our codebase maintainable.
It introduces subtle inconsistencies that cost reviewer hours to unwind.

**How we verify:**
- PRs flagged by our heuristic scanner receive extra scrutiny.
- Contributors may be asked to explain design decisions in their own words.
- Repeated submissions showing AI patterns will be closed without merge.

**Exception:** Using AI as a *reference* (e.g., "how do I write this Zig pattern?")
is fine. The submitted code must be your own understanding.

The policy is specific, explains the reasoning, and defines the exception. It gives maintainers a concrete basis for rejection without accusing anyone of bad faith.

Preserve the Social Contract

Open source runs on reciprocity: I review your patch, you review mine. AI submissions break this loop because there is no human on the other side to reciprocate. The fix is not purely technical -- it is social.

• Require a signed-off-by line (git commit -s) so every commit has a human name attached.
• Hold regular triage meetings where maintainers discuss incoming PRs together; shared context makes AI patterns obvious.
• Onboard new contributors with a first-issue mentor, not a bot.

These practices existed before AI. They matter more now.

Key Takeaways

• Heuristic scanning catches the low-effort AI PRs before they waste reviewer time.
• A shared checklist shifts review from "is this human?" to "does this fit?"
• Platform choice is a tradeoff between reach and signal-to-noise; choose deliberately.
• A written policy with clear reasoning and an exception clause survives scrutiny.
• The social layer -- mentorship, triage, signed commits -- is the real defense.

Source

Andrew Kelley Interview: Why He Built Zig, Banned AI Contributions, and Moved Zig off GitHub -- Added working detection workflow, review checklist template, platform tradeoff table, and a reusable contribution policy with enforcement guidance.

Source

This article builds on Andrew Kelley Interview: Why He Built Zig, Banned AI Contributions, and Moved Zig off GitHub, adding implementation detail and tradeoffs for practitioners.

Support this work

These write-ups are researched and published with no paywall, sponsor, or tracking. If one saved you an afternoon, a small tip keeps them coming.

USDT, USDC or USDD · TRC-20 (Tron)

TFTNsfyomKrnUutRjBTGVULp19ByW29KbY