DEV Community

Sam Chen
Sam Chen

Posted on

Stop Merging AI Patches Blind: A 6-Step Acceptance Protocol on Free Models

You just got a pull request from an AI assistant. The diff looks clean. The comments are polite. The test suite passes. You merge it. Two weeks later, a subtle off-by-one error surfaces in production. Nobody can trace it back to that patch.

This cycle repeats everywhere. AI makes code generation cheap, but verification stays expensive. The usual response is to "review harder." That does not scale. The best defense is a repeatable acceptance protocol, and you can run it without a cloud budget.

You need three things: a free model to generate adversarial tests, a free server to execute them, and a decision rule that tells you when to say no. I will show you how to wire all three.

The Problem with Human Review

Your eyes will always be slower than the generator. An AI can produce a 200-line diff in seconds. Your brain needs minutes to trace each branch. The result is either rushed approval or a bottleneck that defeats the purpose.

The answer is not a better code review tool. It is a behavior gate: a set of executable checks that fail if the patch breaks an invariant. You already know this if you have read the classic testing literature. What is new is that you can now generate those checks automatically with free models.

The Protocol: AI Patch Acceptance

Here is the six-step workflow I use. It assumes you can call an OpenAI-compatible chat completions endpoint. You can point it at any provider that exposes one, including MonkeyCode's free models.

Step 1: Extract the Contract

Before looking at the diff, write down what the function must guarantee. Do not trust the AI's own summary. Use the call site to derive invariants.

For example, if the patch changes a deduplicate function, the contract is:

  • Output has no duplicates.
  • Output preserves the relative order of first occurrences.
  • Output contains every element from the input at least once.

Step 2: Ask a Free Model for Candidate Tests

Do not ask for a patch. Ask for tests that would catch a broken implementation. This is a different prompt, and free models are surprisingly good at it.

Here is a minimal Python script that calls an OpenAI-compatible endpoint and returns test code:

import os
import requests

API_URL = os.getenv("LLM_API_URL", "https://your-endpoint/v1")
API_KEY = os.getenv("LLM_API_KEY")

def generate_tests(function_name, contract_lines):
    prompt = (
        f"Function: {function_name}\n"
        "Contract:\n" + "\n".join(contract_lines) + "\n\n"
        "Write a pytest test function named test_contract that tests this contract."
        "Only output the test code, no explanations."
        "Use exact assertions, not property checks."
    )
    resp = requests.post(
        f"{API_URL}/chat/completions",
        headers={"Authorization": f"Bearer {API_KEY}"},
        json={
            "model": "free-tier-model",
            "messages": [{"role": "user", "content": prompt}],
            "temperature": 0.2,
        },
        timeout=60,
    )
    resp.raise_for_status()
    return resp.json()["choices"][0]["message"]["content"]

if __name__ == "__main__":
    print(generate_tests("deduplicate", [
        "Output has no duplicates.",
        "Output preserves relative order of first occurrences.",
        "Output contains every element from the input at least once.",
    ]))
Enter fullscreen mode Exit fullscreen mode

Save the output to test_contract.py. Do not merge it blindly yet. Review it for correctness, but you will already save time because the model handled the boilerplate.

Step 3: Run the Tests on a Free Server

The key constraint is that you want a clean environment. Your laptop has caches, installed versions, and stale state. A disposable server is better.

MonkeyCode's free server gives you an ephemeral runtime. Point your CI script at it, run pytest, and get a clean pass or fail. This is the second piece of the recipe: free models generate the tests, a free server executes them.

A simple CI script looks like this:

name: ai-patch-acceptance
on: pull_request

jobs:
  behavior-gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - name: Install dependencies
        run: pip install pytest requests
      - name: Generate contract tests
        env:
          LLM_API_URL: ${{ secrets.LLM_API_URL }}
          LLM_API_KEY: ${{ secrets.LLM_API_KEY }}
        run: python generate_tests.py > test_contract.py
      - name: Run acceptance suite
        run: pytest test_contract.py
Enter fullscreen mode Exit fullscreen mode

This runs on every PR. When the generated test fails, the merge is blocked. No human has to argue about the patch.

Step 4: Review the Generated Tests

You still need a human for one thing: deciding whether the contract itself is correct. The model can misread your intent. If the generated test is wrong, you will be enforcing the wrong behavior.

That is fine. You are trading minutes of prompt-and-test authoring for seconds of reading a generated test. The model drafts; you approve.

Step 5: Apply the Decision Table

Once the behavior gate runs, use this table to decide the merge:

Generated tests pass? Contract matches intent? Action
Yes Yes Merge. Add a comment with the test file link.
Yes No Fix the contract, regenerate tests. Do not merge.
No Yes Investigate the patch. It likely broke a real invariant.
No No Rewrite the contract first. The test result is meaningless.

Most AI-overconfident failures fall into row three. The gate gives you an objective signal to start a conversation.

Step 6: Log the Debt

When the gate blocks a merge, do not just fix the code. Record what the model missed. A simple JSON file in the repo works:

{
  "date": "2026-09-02",
  "model": "free-tier-model",
  "blocked": true,
  "missed_invariant": "output must be non-empty",
  "patch_file": "src/parser.py",
  "note": "Model generated a test that allowed empty output."
}
Enter fullscreen mode Exit fullscreen mode

Over time, this file becomes a map of the AI's blind spots. You can feed it back into the next prompt as few-shot examples.

Why Free Models and a Free Server Are Enough

You might think this protocol needs a $200/month inference budget. It does not. MonkeyCode's open-source toolkit includes free models and a free server, both sufficient for this kind of test-generation loop. The workload is short, synchronous, and low-volume: one prompt per changed function per PR. That fits comfortably within a free allowance without a quota panic. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I have run this on a small service with three endpoints. The generated tests caught one real bug: the AI patch had reversed a filter condition. A human reviewer would have missed it, too, because the code looked symmetric.

Limitations

This protocol does not catch logical errors that violate no explicit contract. It cannot detect architectural drift or security issues that depend on runtime context. Free models have smaller context windows and may generate flaky tests if the contract is ambiguous. The free server may have slower cold starts than your dedicated CI runners. If your team ships a high-throughput microservice with many invariant-heavy functions, invest in a paid tier or a self-hosted runner.

Also, the generated tests are only as good as your contract. If you cannot articulate the invariant in one sentence, the model will not guess it correctly.

Who Should Use This, and Who Should Not

Use this if you are a solo developer, a small team, or a startup that relies heavily on AI-generated code. You need cheap automation to counteract cheap generation.

Skip this if you already have a mature property-based testing suite and a dedicated security team. You have the tools; adding a language model layer might just add noise.

Try It for Yourself

Clone a small repo, pick a function that scares you, and run the script above. Whether you use MonkeyCode's free models or any other compatible endpoint, the protocol stands on its own. The point is not the vendor. The point is that you stop trusting a diff and start testing a contract.

Generate one test, run it on a free server, and see if your next AI patch survives. If it does, great. If it does not, you just avoided a production fire at zero cost.

Top comments (0)