My Free-Server Loop for Reviewing AI Refactors (and the First Thing It Broke)

I keep catching myself doing the same AI review loop by hand. Ask a model to refactor a function, read the diff, run the tests, notice the patch does not apply, ask again, run the tests again. The model is not the slow part. I am the slow part, because I never wrote the loop down.

So I made it boring.

The point is not to trust the model more. The point is to create a cheap repeatable signal before I open a real PR. If a proposed refactor passes a baseline test suite, I will read it carefully. If it fails, I skip it and stop turning my evening into a vibes check.

A lot of AI coding advice right now feels like either 'trust the agent' or 'never trust the agent.' Both are feelings. I wanted a tiny referee instead.

The free-tier setup

The two MonkeyCode pieces I used for this experiment are the free model access and the free server option. (Disclosure: This article was prepared as part of MonkeyCode's product outreach. I am treating those availability claims as operator-supplied, not a permanent guarantee.) The model proposes a diff. The server does the unglamorous work: apply the patch, run tests, and report whether the repo still passes.

I am deliberately not calling this a benchmark. One refactor is one sample, and a green test run does not prove a change is correct. It only proves the change did not break the checks I already had.

The runner I copied

This is a proposal script, not a production harness. I used a generic OpenAI-style client because many low-cost and free model paths expose one. Replace the base URL, model name, and repo path with your own values.

import json
import os
import subprocess
import sys
from pathlib import Path

JOBS = Path('jobs.json')
PROJECT_DIR = Path('./repo')


def run_tests():
    result = subprocess.run(
        ['pytest', '-q', '--disable-warnings'],
        cwd=PROJECT_DIR,
        capture_output=True,
        text=True,
        timeout=120,
    )
    return result.returncode == 0, result.stdout + result.stderr


def ask_model(prompt):
    from openai import OpenAI

    client = OpenAI(
        base_url=os.environ['MODEL_BASE_URL'],
        api_key=os.environ['MODEL_API_KEY'],
    )
    response = client.chat.completions.create(
        model=os.environ.get('MODEL_NAME', 'free-model'),
        messages=[{'role': 'user', 'content': prompt}],
        temperature=0.2,
    )
    return response.choices[0].message.content


def main():
    baseline_ok, baseline_log = run_tests()
    print('baseline ok:', baseline_ok)
    if not baseline_ok:
        print(baseline_log)
        sys.exit(1)

    jobs = json.loads(JOBS.read_text())
    for job in jobs:
        proposed_diff = ask_model(job['prompt'])
        patch_path = Path('candidate.patch')
        patch_path.write_text(proposed_diff)

        check = subprocess.run(
            ['git', 'apply', '--check', str(patch_path)],
            cwd=PROJECT_DIR,
            capture_output=True,
            text=True,
        )
        if check.returncode != 0:
            print(job['id'], '-> SKIP: patch did not apply')
            continue

        subprocess.run(['git', 'apply', str(patch_path)], cwd=PROJECT_DIR, check=True)
        ok, log = run_tests()
        subprocess.run(['git', 'apply', '-R', str(patch_path)], cwd=PROJECT_DIR, check=True)

        print(job['id'], '->', 'PASS' if ok else 'FAIL')
        if not ok:
            print(log[-800:])


if __name__ == '__main__':
    main()

Sample jobs.json:

[
  {
    "id": "extract-validator",
    "prompt": "Return only a git patch for the validate_email function in src/validators.py. Replace the nested conditionals with a clearer guard clause. Do not change public behavior."
  }
]

The baseline check matters more than the model. If the repo does not pass before the model runs, the loop is meaningless. My first dry run failed because pytest discovered zero tests and exited successfully. That was not a model failure; it was a harness failure. Now I make the suite catch a deliberate failing test before I add any model work.

A routing table instead of a robot

Risk	Example change	Route	Why
Low	Variable rename, formatting	Free model plus local tests	Cheap and easy to revert
Medium	Multi-file behavior change	Free server runner plus targeted tests	The longer run can happen off my laptop
High	Auth, payment, or security-sensitive path	Human review first	A free model should not be the final gate

Where the free server actually helps

The free server option matters for the long-running part. I can leave the runner pointed at a small repo, schedule it against a small queue, and let it churn through low-risk mechanical patches while I do other work. The server does not make the model smarter. It removes the excuse I usually have for not running the loop.

Limitations and when to skip this

• A free server is not a CI system. It may not have an SLA, durable storage, or enough time for large suites.
• Free model quotas can change. Do not send proprietary or regulated code into this loop.
• Passing tests is a floor, not proof. A bad suite will happily bless a bad refactor.
• This is triage, not model evaluation. Do not rank models from one or two patches.

Who should not use this: anyone handling production secrets, compliance-heavy code, or repositories where a mistaken apply could be expensive. Start with a toy repo or a branch you can delete.

If you try this, start with one mechanical refactor and deliberately make the baseline test fail once. If your runner catches it, the loop is working. Then you can go back to arguing with the model about naming, but at least the boring part is already handled.