A Zero-Cost Pre-Merge Gate for AI-Generated Changes: Fixtures, Diff Tests, and a Throwaway Server

AI-assisted coding tends to mix small improvements with quiet regressions. A generated refactor may rename a function, change a return type, or hard-code an assumption that only breaks in a timezone edge case that no one thought to re-run. The problem is not the model writing code; it is that the full CI suite often runs after the change is already sitting in a pull request, and the reviewer is left staring at a diff instead of a regression signal.

A cheaper gate is to run only the tests that are affected by the diff, in a clean environment, against the set of regression cases you collected from previous bugs. This is a pre-merge gate, not a replacement for the full suite. The full suite can stay slow; the gate just needs to be fast enough to catch the obvious failures before human review starts.

The workflow below assumes two availability claims from MonkeyCode's free tier: free model access for failure triage and a free server for a disposable runner. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Those claims are account-level availability, not benchmark numbers; I am not assuming specific model names, quotas, hardware, or duration.

The useful part of this article stands without MonkeyCode: collect fixtures, run focused tests, isolate the run.

Start with a regression contract, not a giant fixture file

The first step is to preserve the exact condition that failed in a real bug. A unit test that documents a bug is more durable than a Slack message or a comment on a closed PR.

For example, suppose a billing module once produced the wrong renewal date for a negative daylight-saving-time offset. Instead of leaving that in memory, create a standalone regression case:

# tests/regressions/gh_1024_next_renewal_dst.py
from app.billing import next_renewal

def test_negative_dst_offset():
    assert next_renewal("2026-11-01T00:30:00+00:00", "monthly") == "2026-12-01T00:00:00+00:00"

Name the file after the bug ID and the function or behavior it covers. That naming rule is what makes the diff gate practical later: the runner can match changed functions to regression files without needing a full type-aware parser.

If you cannot write a unit test because the behavior lives behind a UI or a network call, write a small executable fixture that takes JSON input and returns a comparable JSON result. The shape does not matter as much as the fact that the failure can be replayed in seconds.

Run only the tests that touch the diff

The second step is a best-effort Python script that uses two signals from git diff: changed file paths and changed function or class names. It then filters pytest --collect-only output to the test nodes that look related.

This is a heuristic, not a replacement for coverage analysis. It will miss tests that call a changed function through indirection, decorators, or dynamic imports. It is still useful because it turns a 12-minute suite into a 20-second check for most focused refactors.

#!/usr/bin/env python3
"""diff_gate.py: run the pytest nodes likely affected by a git diff.

This is intentionally heuristic. It is not a production-grade coverage tool.
"""
import argparse
import re
import subprocess
import sys

def changed_files(base, candidate):
    out = subprocess.check_output(
        ["git", "diff", "--name-only", f"{base}...{candidate}"]
    )
    return [line for line in out.decode().splitlines() if line.strip()]

def changed_symbols(base, candidate):
    out = subprocess.check_output(
        ["git", "diff", "-U0", f"{base}...{candidate}"]
    )
    diff = out.decode(errors="replace")
    functions = set(re.findall(r"^[-+]\s*(?:async\s+)?def\s+(\w+)", diff, re.M))
    classes = set(re.findall(r"^[-+]\s*class\s+(\w+)", diff, re.M))
    return functions | classes

def select_tests(paths, symbols):
    listing = subprocess.check_output(
        ["pytest", "--collect-only", "-q", "--no-header", "--no-summary"]
    )
    selected = []
    for line in listing.decode().splitlines():
        if "::" not in line:
            continue
        if any(path_fragment in line for path_fragment in paths):
            selected.append(line.split()[0])
            continue
        if any(symbol in line for symbol in symbols):
            selected.append(line.split()[0])
    return selected

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--base", default="main")
    parser.add_argument("--candidate", required=True)
    args = parser.parse_args()

    paths = changed_files(args.base, args.candidate)
    symbols = changed_symbols(args.base, args.candidate)
    tests = select_tests(paths, symbols)

    if not tests:
        print("No focused tests matched. Run the full suite as a fallback.")
        return 1

    print(f"Running {len(tests)} focused tests")
    result = subprocess.run(["pytest", *tests], check=False)
    return result.returncode

if __name__ == "__main__":
    sys.exit(main())

The three-dot form base...candidate tells Git to diff from the merge base, which usually matches what a PR would merge. If your team uses a different branching model, replace it with a two-dot diff or an explicit merge-base command.

Move the run to a throwaway environment

The third step is to avoid running this gate on a machine that already has uncommitted files, stale environment variables, or a venv with old dependencies. A clean runner gives you a second chance at catching "works on my machine" failures.

The free server option fits here as a disposable runner, not as a permanent evaluation host. A minimal shell wrapper is enough:

#!/usr/bin/env bash
set -euo pipefail

REPO_URL="$1"
CANDIDATE_BRANCH="$2"

git clone --depth=50 "$REPO_URL" gate-run
cd gate-run
git fetch origin "$CANDIDATE_BRANCH"
pip install -r requirements.txt
python diff_gate.py --base origin/main --candidate "$CANDIDATE_BRANCH"

Treat the cloned directory as disposable. If the gate fails, fix the PR locally and re-run; do not accumulate state on the runner. The point is isolation, not continuity.

Add model triage only where a diff is not enough

When a focused test fails, the free model access becomes useful for a narrow task: turning a traceback and the relevant diff into a candidate regression test or a possible root-cause explanation. This is triage, not judgment.

The call shape below is pseudocode. Different model providers expose different request schemas, so it should not be copied blindly. The important constraint is that you send the failure context into the prompt and keep the model's output outside the commit until a human reads it.

# Pseudocode: adapt to the schema of the model endpoint you are using.
def triage_failure(test_id, diff, traceback):
    prompt = (
        "A focused test failed in a pre-merge gate.\n"
        f"Test: {test_id}\n\nDiff:\n{diff}\n\nTraceback:\n{traceback}\n\n"
        "Suggest: 1) the likely root cause, 2) a minimal regression test. "
        "Do not change behavior speculatively."
    )
    return call_model_endpoint(prompt)

I would not use this step to auto-approve or auto-fix a PR. A model can produce a plausible test that passes while asserting the wrong behavior, so every suggested regression test needs the same review as human-written code.

Decision table: when this gate is worth using

Codebase shape	Gate result	Notes
Small Python service with regression tests	Useful	Fast diff match already exists; clean runner adds isolation.
Python monorepo with heavy build dependencies	Partial	Diff filtering works, but setup may dominate the run.
Compiled service with no Python tests	Not applicable	The script filters pytest nodes, not arbitrary test frameworks.
No test suite or no stable merge branch	Not applicable	The gate cannot find anything to run.
High-sensitivity data with strict compliance rules	Avoid free server	Keep the gate local or in a compliant runner.

Limitations

• Diff-based symbol extraction is brittle. It can miss methods called through getattr, plugins, or dynamic dispatch, and it can over-match when a changed word appears in unrelated test names.
• The gate only catches regressions that already have a test or fixture. A novel bug introduced by an AI-generated change can still pass.
• Model triage can hallucinate a clean-sounding explanation. It should never be the final decision maker.
• Free-tier runner capacity may be limited or network-isolated; this is a pre-merge signal, not a continuous evaluation pipeline.
• The full CI suite still needs to run before merge. The gate reduces review noise; it does not certify correctness.

A reasonable first step is to apply the fixture gate to one module that has caused recurring regressions, without adding model triage. The diff-aware runner and clean environment are enough to show whether the approach helps with your review loop.