Build an AI Code Reviewer That Fits in a Free Tier (Step-by-Step, Harness First)

You don't need a paid plan to run an AI code reviewer. You need a modest token allowance, a free server, and — this is the part everyone skips — a way to prove the reviewer actually catches bugs. This is that tutorial: from zero to a deployed /review endpoint, with a verification step at every stage.

Every week, another AI code-reviewer discussion pops up on DEV. The demos look great. The tests are usually missing. That's a strange gap, because "AI as reviewer" is exactly the case where you can plant known defects and grade the output. No flaky benchmarks. No secret datasets. Just three diffs, each with a bug I put there on purpose.

Sidebar: I used MonkeyCode for the infrastructure so the whole thing stays free — the free model access covers the LLM calls, and the free server option hosts the bot. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The quota number below is what the operator told me, not a benchmark I ran; free tiers drift, so check the docs before you trust my arithmetic.

What We're Building

A two-piece system:

1. A harness that sends a crafted diff to the model, grades the reply against planted bugs, and prints a pass/fail table.
2. A server that exposes the same logic over HTTP, so a webhook can drop a real diff on it later.

Both pieces talk to the same chat-completions endpoint. Same prompt. Same temperature. If the harness passes, the server gets the same brain. If it fails, you saved yourself from wiring a useless reviewer into CI.

The Math: Can Free Tokens Carry It?

A typical review here costs roughly 3,000–4,000 tokens: the system prompt, one full diff, and a JSON reply. The free allowance I was told about is 10 million tokens. Do the division:

10,000,000 ÷ 3,500 ≈ 2,800 reviews

That's a few thousand PR reviews, not pocket change. The real constraint isn't the quota anyway — it's the harness. A bad reviewer burns tokens and still misses the bug. So we build the harness first.

Step 1: Credentials and a Smoke Test

Create the project and set two environment variables. I'm deliberately not inventing a CLI here — the endpoint and key live in the MonkeyCode dashboard, and dashboards change.

mkdir reviewer && cd reviewer
export MONKEYCODE_API_KEY="..."      # from the dashboard
export MONKEYCODE_BASE_URL="..."     # chat-completions base URL
export MONKEYCODE_MODEL="default"    # pick the model in the docs

Verification: before you write a single prompt, prove the connection works.

curl "$MONKEYCODE_BASE_URL/models" \
  -H "Authorization: Bearer $MONKEYCODE_API_KEY"

A JSON list means the key works. If you get a 401, stop there — no app you build will fix bad credentials.

Step 2: The Harness

This is the heart of the article, so let's be careful. I create three fake diffs. Each one has exactly one real defect: an off-by-one in a slice, a swallowed exception, unsanitized data leaking into a response.

# harness.py
import json
import os

import httpx

CASES = [
    {
        "name": "off-by-one",
        "diff": """-return title[:max_len]
+return title[:max_len - 1]""",
        "bugs": ["off-by-one"],
    },
    {
        "name": "swallowed-exception",
        "diff": """-    except DatabaseError:
-        user = None
+    except DatabaseError:
+        pass""",
        "bugs": ["swallows", "no logging"],
    },
    {
        "name": "raw-input",
        "diff": """-return {"count": len(items)}
+return {"count": len(items), "raw": items}""",
        "bugs": ["unsanitized", "exposed"],
    },
]

SYSTEM_PROMPT = (
    "You are a conservative code reviewer. "
    "Return JSON only: [{\"line\": int, \"issue\": str, \"severity\": \"low|medium|high\"}]. "
    "Do not invent issues."
)

def review_prompt(diff: str) -> str:
    return f"Review this diff:\n```
{% endraw %}
diff\n{diff}\n
{% raw %}
```"

def grade(reply: str, bugs: list[str]) -> list[str]:
    try:
        findings = json.loads(reply)
    except json.JSONDecodeError:
        return []
    text = json.dumps(findings).lower()
    return [b for b in bugs if b.lower() in text]

def main() -> None:
    for case in CASES:
        r = httpx.post(
            f"{os.getenv('MONKEYCODE_BASE_URL')}/chat/completions",
            headers={"Authorization": f"Bearer {os.getenv('MONKEYCODE_API_KEY')}"},
            json={
                "model": os.getenv("MONKEYCODE_MODEL", "default"),
                "messages": [
                    {"role": "system", "content": SYSTEM_PROMPT},
                    {"role": "user", "content": review_prompt(case["diff"])},
                ],
                "temperature": 0.2,
            },
            timeout=60,
        )
        reply = r.json()["choices"][0]["message"]["content"]
        detected = grade(reply, case["bugs"])
        print(f"{case['name']}: {'PASS' if detected else 'FAIL'} -> {detected}")

if __name__ == "__main__":
    main()

Run it:

python harness.py

Verification: you want three PASS lines. If the model misses a planted bug, don't tweak the grader to make it pass — tweak the prompt. That distinction is the whole craft of working with free models.

Note: grade() expects valid JSON. Real model output is messier; wrap it with a JSON-extraction regex before you trust this in production.

Step 3: The Server

The server wraps that same brain in a webhook-friendly route. FastAPI keeps it small.

# app.py
import os

import httpx
from fastapi import FastAPI

app = FastAPI()

async def call_model(diff: str) -> str:
    async with httpx.AsyncClient(timeout=60) as client:
        r = await client.post(
            f"{os.getenv('MONKEYCODE_BASE_URL')}/chat/completions",
            headers={"Authorization": f"Bearer {os.getenv('MONKEYCODE_API_KEY')}"},
            json={
                "model": os.getenv("MONKEYCODE_MODEL", "default"),
                "messages": [
                    {"role": "system", "content": SYSTEM_PROMPT},
                    {"role": "user", "content": f"Review this diff:\n{diff}"},
                ],
                "temperature": 0.2,
            },
        )
        return r.json()["choices"][0]["message"]["content"]

@app.post("/review")
async def review(payload: dict) -> dict:
    diff = payload.get("diff", "")
    if not diff:
        return {"status": "empty diff"}
    return {"findings": await call_model(diff)}

Verification, locally:

pip install fastapi uvicorn httpx
uvicorn app:app --port 8000

curl -s http://localhost:8000/review \
  -H "Content-Type: application/json" \
  -d '{"diff": "-return 1\n+return 1"}'

You should get back a JSON findings array. Empty diff? You get "status": "empty diff". That's the contract.

Step 4: Deploy to the Free Server

The free server option on MonkeyCode accepts a container. Here's the Dockerfile:

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY app.py ./
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

Exact deploy steps drift with the dashboard, so I won't invent a button name. The contract on your side is just two env vars and port 8000. Once it's up:

curl -s https://<your-app-host>/review \
  -H "Content-Type: application/json" \
  -d '{"diff": "-return title[:max_len]\n+return title[:max_len - 1]"}'

Verification: a 200 with a findings array means the bot is alive outside your laptop.

Step 5: Connect a Real Webhook

GitHub PRs are the obvious input. The diff_url approach keeps the server dumb:

1. Webhook event: pull_request, action opened or synchronize.
2. Server fetches payload["pull_request"]["diff_url"].
3. Sends the diff to the model, posts findings back via the checks API.

Two production warnings. First, verify X-Hub-Signature-256 before trusting any payload. Second, don't auto-merge on "AI approved" — treat the review as a suggestion channel, not a gate. The harness should one day run inside CI, not just on your laptop.

Limitations, Stated Out Loud

This setup is deliberately simple. That means:

• No secret handling: never send private source through a free endpoint unless you've read the terms. Check data-residency rules first.
• Latency is not a promise: shared free endpoints return in seconds, sometimes tens of seconds. This is a background reviewer, not a pre-commit gate.
• My three planted bugs are a start, not a suite. Grow CASES with every regression you find in your own diffs.
• JSON output is fragile. My grade() assumes valid JSON; real replies need a JSON-extraction step.

Who should not use this? Teams with strict data residency, codebases where determinism is a compliance requirement, and anyone who'll auto-post AI reviews to public repos without a human glance. Static analysis is still your friend for the boring, expensive, deterministic rules.

The Part I Want You to Steal

The bot is the bait here. The harness is the actual lesson: plant bugs, grade the reply, and only then wire the model into your workflow. Free servers and free tokens make the experiment cost nothing — a 10M-token allowance covers thousands of these reviews.

If you replicate this, run your three diffs, and get a pass rate — post it. The reviewer-discussion threads on DEV need measurements, and there's one missing human from them: you.