Hosted Free Models or a Free Server: Where an Agent's SQL Drafting Loop Should Run

Last month I watched an agent spend forty-one attempts drafting one reporting query, and not one of those attempts was slow enough to notice on its own. The loop only looked expensive when I plotted attempts against wall time and prompt tokens per retry. What surprised me was that the placement of the drafting step, not the model quality, explained most of the variance. That observation turned into an argument I keep having with a colleague, so this article states both sides properly and then writes down a rule for choosing.

The scene, and why placement matters

The task was ordinary: an agent reads a schema summary, drafts SQL, runs it read-only, reads the error, and revises. Each revision resends the growing conversation, so attempt twelve carries far more prompt tokens than attempt one. When the drafting step runs behind a shared hosted endpoint, retry latency and quota pressure are somebody else's scheduling problem. When it runs on a machine you control, latency is your problem but the retry storm is yours to shape.

I am not going to pretend I have a benchmark that settles this for everyone, because I have not run one against MonkeyCode's endpoints myself. What I can give you is a reproducible harness, the four signals it collects, and a decision rule that a team can argue about using its own numbers.

Position A: borrow the hosted free model layer

The first position is that an agent's drafting loop should default to a hosted free tier, because the cost of getting started is minutes rather than days. You authenticate, point the loop at an endpoint, and the schema summarization, prompt assembly, and retry logic stay exactly where they were. For a solo developer validating whether an agent can even reach a usable query, this is the shortest path to evidence.

A second argument is operational: nobody has to patch, restart, or monitor anything. If the endpoint degrades, the retry logic you already wrote absorbs it, and you can rerun the same task suite tomorrow. The counterargument, which I happen to hold, is that a shared free tier gives you no control over the retry ceiling, so a runaway loop is invisible until it stops working.

MonkeyCode is relevant here because it advertises free model access and a free server option, which is exactly the fork this article is about. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I am describing it as a user of the free tier, not as a benchmark authority, and I have treated the free token grant (the operator states ten million tokens at signup) and the free server as current-at-writing claims. Verify both on the project's own pages before you build a plan around them, because free-tier terms change.

Position B: own the drafting step on a free server

The second position is that the drafting step belongs on a machine you control, even if that machine arrives free. Once the model runs next to the database, the schema summary never leaves your boundary, and you can cap concurrency, retries, and context growth yourself. For regulated schemas or for loops that touch production metadata, that boundary is often the whole argument.

Ownership also makes the failure mode legible. When a retry storm starts, you see memory, queue depth, and per-request latency on a dashboard you built, not in someone else's status page. The cost is real, though: someone must own the image, the model weights, the restart policy, and the capacity ceiling.

The artifact: a placement harness you can actually run

Here is a proposed harness, not a validated benchmark. It writes one JSONL row per drafting attempt so that placement decisions rest on your data rather than on my opinions. The token counter is a cheap proxy and the backend is a dry-run echo, so nothing here depends on invented provider numbers.

# bench_placement.py — proposed harness; I have not executed this against MonkeyCode.
import argparse, hashlib, json, statistics, time
from dataclasses import asdict, dataclass

@dataclass
class Attempt:
    task_id: str
    placement: str      # "hosted-free" | "free-server"
    attempt_no: int
    prompt_tokens: int
    completion_tokens: int
    wall_ms: int
    ok: bool
    error_class: str | None
    sql_sha256: str | None

def estimate_tokens(text: str) -> int:
    # Proxy only. Swap in the tokenizer of whichever model you actually call.
    return max(1, len(text) // 4)

def draft_sql(task: str, history: list[str], dry_run: bool) -> tuple[str, str | None]:
    if dry_run:
        return f"SELECT 1 /* {task} */", None
    raise SystemExit("Wire your real backend here; keep the return contract.")

def run(tasks: list[str], placements: list[str], max_attempts: int, out: str) -> None:
    rows: list[Attempt] = []
    for placement in placements:
        for task in tasks:
            history: list[str] = []
            for n in range(1, max_attempts + 1):
                prompt = "\n".join(history + [task])
                t0 = time.perf_counter()
                try:
                    sql, err = draft_sql(task, history, dry_run=True)
                except Exception as exc:  # classify, never swallow silently
                    sql, err = "", type(exc).__name__
                wall_ms = int((time.perf_counter() - t0) * 1000)
                rows.append(Attempt(
                    task_id=task[:24], placement=placement, attempt_no=n,
                    prompt_tokens=estimate_tokens(prompt), completion_tokens=estimate_tokens(sql),
                    wall_ms=wall_ms, ok=err is None, error_class=err,
                    sql_sha256=hashlib.sha256(sql.encode()).hexdigest()[:12] if sql else None,
                ))
                if err is None:
                    break
                history.append(f"attempt {n} failed: {err}")
    with open(out, "w") as fh:
        for row in rows:
            fh.write(json.dumps(asdict(row)) + "\n")
    for placement in placements:
        lat = sorted(r.wall_ms for r in rows if r.placement == placement)
        if lat:
            p95 = lat[min(len(lat) - 1, int(0.95 * len(lat)))]
            print(f"{placement}: n={len(lat)} p50={statistics.median(lat)}ms p95={p95}ms")

if __name__ == "__main__":
    ap = argparse.ArgumentParser()
    ap.add_argument("--tasks", default="tasks.txt")
    ap.add_argument("--placements", default="hosted-free,free-server")
    ap.add_argument("--max-attempts", type=int, default=12)
    ap.add_argument("--out", default="placement.jsonl")
    args = ap.parse_args()
    with open(args.tasks) as fh:
        tasks = [line.strip() for line in fh if line.strip()]
    run(tasks, args.placements.split(","), args.max_attempts, args.out)

Steps to run the comparison

1. Write twenty tasks into tasks.txt, mixing trivial lookups with two or three joins, and keep the same file for both placements.
2. Wire your backend into draft_sql, keeping the return contract so the JSONL schema does not change between runs.
3. Run python bench_placement.py --placements hosted-free --out hosted.jsonl, then repeat with --placements free-server.
4. Compare attempts per task and p95 wall time, not average latency, because retry storms hide in the tail.
5. Gate every generated statement before execution, regardless of placement, using a read-only transaction with a timeout.

-- Cheap guard: never let a draft touch data until it has been planned read-only.
BEGIN;
SET LOCAL transaction_read_only = on;
SET LOCAL statement_timeout = '3s';
EXPLAIN (FORMAT JSON) /* generated statement goes here */;
ROLLBACK;

The signals that actually decide it

Four signals from the JSONL file carry most of the decision weight. Attempts per task tells you how much context resending you are paying for. The ratio of prompt to completion tokens tells you whether your loop is dominated by conversation history rather than by generation. Tail latency tells you how badly a shared tier handles your retry pattern. Sensitivity class of the schema tells you whether the boundary argument overrides every number above.

Signal	Hosted free tier wins when	Free server wins when
Attempts per task	Under roughly four	Above eight, and you can cap concurrency
Schema sensitivity	Public, fixture, or synthetic only	Internal or regulated metadata
Tail latency (p95)	You can tolerate queueing	You need a stable ceiling you control
Ops capacity	Nobody available to own a box	Someone owns restarts and capacity

Failure analysis on both sides

On the hosted side, the failure I see most is silent quota exhaustion mid-loop: the agent keeps retrying, the endpoint keeps refusing, and the loop reports a schema error. Instrument error classes separately or you will misread a rate limit as a bad join. The second is context bloat, where attempt fifteen carries so much history that the useful instruction is a rounding error.

On the server side, the failure is usually resource exhaustion rather than refusal. A warm model plus three concurrent agents can exceed memory, and the process dies without a useful message. Cold starts also distort early latency, so discard the first task in each run or you will compare startup cost with steady state. Neither placement fixes an incorrect query, and that is the failure people most want placement to solve.

The decision rule I would defend

Start every new agent loop on the hosted free tier, with the harness running, because evidence beats architecture debate. Move the drafting step to a free server once attempts per task cross eight, or once the schema becomes sensitive, or once tail latency is the constraint users complain about. Keep the harness in version control either way, since the numbers that justified the move are the numbers that will justify moving back.

a decision like this is only as good as the alternatives you actually tried, so if you want to test both placements on the same task suite, MonkeyCode's free model access and free server option give you two runnable configurations without a procurement conversation.

Limitations and who should skip this

This harness measures placement, not correctness, and the token counter is a proxy rather than a real tokenizer, so treat the numbers as relative. The free token grant and the free server are operator-stated availability claims that I have not independently benchmarked, and self-hosting on a free server still requires someone to own the image. Skip the debate entirely if you need deterministic sub-second p99 latency or if your compliance rules forbid third-party inference, because in both cases the decision was made before the benchmark ran. Also skip it if your agent has never produced a query a reviewer approved, since placement will not rescue a loop that has not yet proven it can work.