A Three-Layer PR Filter for AI-Generated Code: Patterns, Semantics, and a Free Sandbox

AI assistants now draft whole pull requests faster than you can read them. The uncomfortable part is that these PRs look perfectly formatted, properly named, and confidently explained — which is exactly why they deserve a different inspection pipeline than human contributions.

This article walks through a three-layer filter for AI-generated code: a mechanical pattern scan, a semantic risk review, and an evidence check. Along the way, you will see how to run the entire pipeline without touching your local machine by using MonkeyCode's free models and free server option, which I'll introduce when the pipeline needs them. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Layer 1: Mechanical Pattern Scan

The first layer catches the embarrassingly obvious problems that slip into generated diffs: bare exceptions, hardcoded secrets, debug prints, and leftover TODOs. None of these need a language model; they need a grep-like script with a non-zero exit code for CI.

Here is a self-contained Python script that scans a unified diff and reports the exact lines that match a suspicious pattern list:

#!/usr/bin/env python3
import sys, re

SUSPICIOUS = [
    (r'\bTODO\b', 'unresolved TODO'),
    (r'\bFIXME\b', 'unresolved FIXME'),
    (r'except\s*:', 'bare except'),
    (r'except\s+Exception\s*:', 'broad exception'),
    (r'password\s*=\s*[\'"][^\'"]+', 'hardcoded password'),
    (r'api[_-]?key\s*=\s*[\'"][^\'"]+', 'hardcoded API key'),
    (r'secret\s*=\s*[\'"][^\'"]+', 'hardcoded secret'),
    (r'\bprint\(', 'debug print'),
    (r'\bassert\s+True\b', 'meaningless assert'),
    (r'time\.sleep\(', 'sleep in production path'),
    (r'\beval\(', 'eval used'),
    (r'\bexec\(', 'exec used'),
]

def scan(diff_text: str) -> list:
    findings = []
    for lineno, raw in enumerate(diff_text.splitlines(), 1):
        if not raw.startswith('+') or raw.startswith('+++'):
            continue
        code = raw[1:]
        for pattern, label in SUSPICIOUS:
            if re.search(pattern, code):
                findings.append((lineno, code.strip()[:120], label))
    return findings

if __name__ == '__main__':
    diff = sys.stdin.read()
    issues = scan(diff)
    if issues:
        for line, code, label in issues:
            print(f'{line}: {code}  [{label}]')
        sys.exit(1)
    print('No common red flags found.')

Wire it into your pre-merge pipeline like this:

git diff origin/main...HEAD | python3 pr_audit.py

Why start with a pattern scan? It is deterministic, fast, and immediately catches the habits agents inherit from training data — like swallowing exceptions or logging secrets. It is also easy to extend: add your org's forbidden imports, dangerous function names, or configuration patterns to the list.

Layer 2: Semantic Risk Review

Patterns cannot tell you whether a loop exits correctly or whether a refactor changed business logic. That requires a model with context. This is where MonkeyCode's free models come in: instead of burning your local GPU or paying per token, you can send a trimmed diff plus a focused question to a free model and get back a ranked risk list.

A practical prompt template looks like this:

You are a senior code reviewer. Analyze the diff below.
Rank the three riskiest changes, from highest to lowest.
For each, give a one-sentence reason and the line number.
Respond in plain text.

<diff>
...
</diff>

The output is not authoritative; it is a prioritization aid. Paste the model's answer into your review thread, then verify each claim against the actual code.

How Free Models Matter Here

• They make semantic review cheap enough to run on every PR, not just the scary ones.
• They keep your code off paid endpoints if you are prototyping.
• They let you phrase the same question to two different models and compare answers — a poor man's ensemble check.

The catch: free models may have smaller context windows and lower rate limits. Keep the diff under a few hundred lines, or split it into logical chunks before sending.

Layer 3: Evidence Check

The final layer asks a simple question: did this PR actually prove its behavior? Most AI-generated PRs include tests, but those tests often verify the happy path only. Build a table like this for every PR you review:

Evidence Type	Question to Answer	Minimum Requirement
Unit tests	Do critical branches return expected values?	At least one test per changed branch
Integration tests	Do DB/API calls work in the real flow?	One successful run against a test DB or mock
Regression tests	Did previous behavior stay intact?	Existing suite passes
Boundary inputs	What happens on empty, huge, or malformed input?	One negative test per public function changed

The table turns "add more tests" into a concrete checklist. If the PR fails the evidence check, send it back with the exact missing row.

Putting It All Together on a Free Server

Running all three layers locally can strain a laptop, especially when you are simultaneously running an IDE, Docker, and a browser full of Stack Overflow tabs. MonkeyCode's free server option gives you a remote workspace where you can clone the repo, run the pattern script, call the free models, and execute the test suite — all without consuming local memory.

A minimal remote workflow looks like this:

# On the free server
gh repo clone your-org/your-repo
cd your-repo
git fetch origin main
ct=`git diff origin/main...HEAD`
echo "$ct" | python3 pr_audit.py || echo "pattern issues found"
# Then send the diff to a free model via MonkeyCode's API or CLI

The free server is not a production deployment; treat it as a disposable sandbox. If you need heavier compute, you can still start there and move the exact same commands to your local machine — the pipeline is environment-agnostic.

Limitations and Who Should Not Use This

This three-layer filter is not a formal verification tool. The pattern scan produces false positives; a CLI tool that legitimately prints output will trigger the print( rule. Free models can hallucinate line numbers or miss context-specific risks. The evidence table is a guideline, not a guarantee of correctness.

Teams in regulated industries — healthcare, fintech, aerospace — should treat this pipeline as a first pass, not the final authority. If your PR touches authentication, payment logic, or hardware control, you still need human review from someone who understands the domain, plus possibly formal methods. This approach is also overkill for tiny mechanical PRs; run only the pattern scan there and save the semantic layer for meaningful changes.

The sweet spot: a mid-sized feature branch generated by an AI assistant, where the risk is real but the scope is manageable. That is where three layers of filtering save the most review time while keeping quality high.

Bottom Line

AI-generated PRs deserve a repeatable filter, not vibes. Start with a deterministic pattern scan, add a free-model semantic pass to prioritize risks, and finish with an explicit evidence checklist. Run everything on a free server if your laptop is already at its limit. That combination turns PR review from a guessing game into a pipeline.

If you want to try the whole flow without spending a cent, MonkeyCode's free models and free server are a practical way to start — clone a repo, scan a diff, and see where the output stands. That is the cheapest audit you can run today.