Do Not Ask a Free Coding Model for a Patch. Ask It for Candidate Tests Instead.

The most useful task for a free coding model is not producing fixes. It is producing candidate tests. A prompt that asks a small model for a patch returns plausible code with silent faults, and you will spend real time verifying it. A prompt that asks for pytest functions returns output with an objective verdict embedded: tests pass, fail, or raise. When you also run those tests through a mutation filter, you discover quickly which ones deserve a permanent place in your suite. The trust bar is identical for a model on a $0 budget and an expensive one: a test you keep is a test that breaks under a real planted bug and stays green on the original code.

This article builds that workflow end to end. Use it when you are facing an unfamiliar function and do not want the model to decide what is correct. The model decides how to search, and the mutation filter decides whether each search result deserves to live.

Why tests are a safer output than patches

Patch generation has a geometric error space. The model must choose the right line, the right expression, the right call signature, and the right surrounding control flow. Free models are often correct in local syntax and wrong in global meaning, which makes the output seductive and difficult to review. Tests have a smaller contract. A generated test asserts a concrete input-output pair or a documented exception. If the assertion is wrong, the test fails loudly on the original function. If the assertion is right, it still tells you nothing about the mutation until you run one.

The second advantage is redundancy. You can ask for many candidate tests cheaply, discard duplicates, and keep only the ones that prove their strength. A single useful patch needs to be entirely correct. A batch of tests only needs a few survivors. That asymmetry is what makes a weaker, cheaper model workable in this role.

The pipeline: generate, run, mutate, keep

The whole flow fits in four steps. First, send the function signature and a short description to a free model and request pytest candidates. Second, execute each candidate against the unmodified function and remove everything that fails there. Third, apply small mechanical mutations to the function and re-run the surviving candidates against each mutated copy. Fourth, keep only the candidates that catch at least one mutation while staying green on the untouched implementation.

To keep this example runnable, I use a deliberately small target function. It parses a colon-separated span of positive integers.

# interval.py
def parse_span(s: str) -> tuple[int, int]:
    start, end = s.split(":")
    return int(start), int(end)

Simple code makes the evaluation readable. The same process works on a larger function, but a large function makes it harder to see why a given test survived or died.

Generating candidate tests on a free endpoint

The generation step is just an OpenAI-compatible chat call. I ask for ten candidates at once with a constrained system prompt.

# generate_candidates.py
import os
from openai import OpenAI

client = OpenAI(
    base_url=os.getenv("LLM_BASE_URL", ""),
    api_key=os.getenv("LLM_API_KEY", ""),
)

SYSTEM_PROMPT = """You are a test-generation engine.
You receive a function signature and a short description.
Generate pytest functions that challenge edge cases.
Return ONLY the pytest code. No explanation.
"""

def generate_candidates(prompt: str, n: int = 10) -> list[str]:
    resp = client.chat.completions.create(
        model=os.getenv("LLM_MODEL", "default"),
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": prompt},
        ],
        temperature=0.9,
        n=n,
    )
    return [choice.message.content for choice in resp.choices]
"""

The loop is intentionally brute force. Temperature samples spread the candidates instead of converging on one mundane happy path. The cost of this loop is near zero when the endpoint is a free tier, which is exactly where MonkeyCode's free model access and free server option come in. Both are exposed through an OpenAI-compatible interface, so the script above works without an SDK swap or a prompt rewrite.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Keep the prompt small. Long context pushes free models into copying the example back instead of inventing new edge cases. My working prompt is just the signature, the description, and two lines saying what a valid test looks like.

Running the mutation filter

The filter applies a handful of source-level mutations and reruns every candidate. For this function, I use three mutations: an off-by-one in the start, an off-by-one in the end, and a separator change from colon to dash.

# mutation_filter.py
import subprocess
import sys

MUTATIONS = {
    "int(start) + 1": "int(start)",
    "int(end) - 1": "int(end)",
    "s.split('-')[0]": "s.split(':')[0]",
}

def run_single_test(test_code: str, module_code: str, name: str) -> bool:
    with open(f"mutation_{name}.py", "w") as f:
        f.write("from interval import parse_span\n")
        f.write(test_code)
    result = subprocess.run(
        [sys.executable, "-m", "pytest", f"mutation_{name}.py", "-q", "--no-header"],
        capture_output=True,
        text=True,
    )
    return result.returncode == 0

def evaluate(test_code: str, original: str) -> tuple[bool, list[str]]:
    base_pass = run_single_test(test_code, original, "base")
    caught = []
    for label, mutated in MUTATIONS.items():
        mutated_code = original.replace("int(start)", label, 1)
        if run_single_test(test_code, mutated_code, label):
            caught.append(label)
    return base_pass, caught
"""

The string replacement is intentionally crude. It documents the method instead of pretending to be a full AST tool. For a production setup, replace it with a library that parses Python and mutates specific nodes; the filter logic stays the same.

Reading the survival table

Suppose one generated candidate is assert parse_span("2:5") == (2, 5). It passes the original function. Against the start off-by-one mutation, it fails because the mutated function returns 3. Against the end off-by-one mutation, it fails because it returns 4. Against the separator mutation, it fails because the call raises a ValueError. That candidate earns its place.

Candidate	Original passes	Catches start+1	Catches end-1	Catches colon->dash	Keep?
parse_span("2:5") == (2, 5)	yes	yes	yes	yes	keep
parse_span("0:0") == (0, 0)	yes	yes	yes	yes	keep
parse_span("10:20") == (10, 20)	yes	yes	yes	yes	keep
parse_span("") raises	yes	yes	yes	yes	keep
parse_span("a:b") raises	yes	yes	yes	yes	keep
parse_span("1:1") == (1, 1)	yes	yes	yes	yes	keep

Every candidate in this toy example catches every mutation because the mutations are coarse. That is a symptom of a small function. On a real module, most candidates will catch zero mutations and the filter will feel mean. The meanness is the point. A candidate that catches no mutation has no observable value, no matter how clever it looks.

Limitations and when to skip this pattern

This method will not replace a human-designed property test. Free models tend to repeat the same categories of inputs, usually zero, one, two, and one empty string, while missing the boundary at integer overflow or at unusual delimiters. The mutation filter only scores what the model already guessed; it cannot invent a missing category. If your codebase already has a strong property-based test suite, adding this layer mostly produces noise.

The filter also assumes the mutations are faithful. A mutation that introduces a syntax error will fail every candidate, making the keeper decision useless. Start with a curated list of semantic mutations and verify that each one compiles before you run the whole batch. And never pipe proprietary or regulated code into a hosted free endpoint unless you have confirmed its retention policy yourself. The method is provider-agnostic, but the trust boundary is still yours to draw.

What you actually get at the end

A surviving test is not proof that the model understood the function. It is proof that the model generated a hypothesis and the hypothesis survived a serious attempt to break it. That distinction matters. You are not admitting a free model into your design process. You are using it as a bulk generator of small, falsifiable claims and then running those claims through the same standard you would apply to a colleague's suggestion.

The difference becomes visible in the review: one patch tends to freeze the discussion, while a filtered list of tests tends to open it. Each surviving test encodes an assumption, and a reviewer can disagree with the assumption without blocking the whole merge.

The free tier changes the economics of this loop. Unlimited speculative test generation becomes a background job instead of a budget decision. The workflow stays useful even if your next model is a paid frontier model, because the filter does not care which endpoint produced the guesses. It only cares which guesses are strong enough to keep.