Temperature 0 or Five Samples? A Debate Over Reproducible Agent SQL Drafts

Consider a composite Tuesday that plays out in many teams: an agent re-drafts a query it produced the day before, a reviewer approves it a second time, and the execution plan quietly flips from an index seek to a full scan. Nobody edited the prompt, the schema diff came back empty, and the provider had announced no model change. Two engineers then lose an afternoon arguing about whether the drafting loop should be pinned to one deterministic output or deliberately sampled several times and filtered.

That argument deserves a decision rule instead of a taste preference, because the answer determines how you spend a free token allowance, what your CI can honestly assert, and what you are able to say at review time. Below are the two strongest positions I can defend, a reproducible harness for telling them apart on your own schema, and a rule for choosing between them. No provider benchmarks are claimed here; every number you act on should come from your own replay run.

1. Two credible positions, stated fairly

Position A — pin it. Send one call at temperature 0, store the exact SQL text plus a normalized fingerprint, and treat any later drift as a reviewable diff rather than as noise. The argument is economic and procedural: one call per draft keeps a free allowance stretched across far more scenarios, and a single artifact is easy to attach to a pull request, annotate, and re-check after a dialect migration.

The weakness is that temperature 0 is not a determinism guarantee. Provider-side batching, floating-point non-associativity in sampling, and silent model revisions all produce different outputs for identical inputs, so Position A still needs a fingerprint and a stored plan, not just a stored string.

Position B — sample it. Request k candidates, run static checks on each, and keep the survivor whose plan is cheapest under a stated cost model. The argument is coverage: dialect edge cases such as quoting rules, date arithmetic, and window frames show up as variance across samples, and a single greedy decode tends to hide them behind confident-looking syntax that parses but misbehaves.

The weakness is selection bias plus cost. Static checks cannot detect semantic wrongness, so filtering mostly re-ranks plausible candidates, and five samples against a free tier consume roughly five times the budget for one accepted draft.

What both sides already agree on. You need a normalized fingerprint, a captured plan, and an approval record. That agreement is the part worth implementing first, which is exactly what the harness below does.

2. A replayable draft harness in three steps

The goal is a script that takes a prompt, obtains candidates, records what the database actually does with them, and fails CI when behavior changes without approval. The examples below are illustrative and labeled as such: the code is runnable, but the pass criteria must be tuned to your schema.

Step 1 — Guard the budget before the next call

# draft_harness.py (illustrative)
from dataclasses import dataclass

@dataclass
class Budget:
    limit: int
    used: int = 0

    def charge(self, prompt: str, completion: str) -> None:
        # Rough guard only: approximate token count, not provider billing.
        estimate = (len(prompt) + len(completion)) // 4
        if self.used + estimate > self.limit:
            raise RuntimeError("draft budget exhausted; stop before the next call")
        self.used += estimate

Charge after every candidate, not after every prompt, otherwise a k=5 loop overshoots a free allowance by a factor of five on the first bad day. The guard is intentionally crude, and you should log the real reported usage alongside it.

Step 2 — Parse, restrict tables, and fingerprint

import hashlib, sqlglot

WRITE_NODES = (sqlglot.exp.Insert, sqlglot.exp.Update, sqlglot.exp.Delete)

def inspect(sql: str, dialect: str, allowed_tables: set[str]) -> dict:
    tree = sqlglot.parse_one(sql, read=dialect)
    tables = {t.name.lower() for t in tree.find_all(sqlglot.exp.Table)}
    return {
        "parses": True,
        "unknown_tables": sorted(tables - allowed_tables),
        "has_write": any(tree.find(node) for node in WRITE_NODES),
        "fingerprint": hashlib.sha256(tree.sql(dialect=dialect).encode()).hexdigest()[:12],
    }

Fingerprinting the normalized tree rather than the raw text absorbs whitespace and comment churn, so a changed fingerprint means a changed query shape. Reject any candidate with unknown tables before you ever execute it, and keep the dialect argument explicit so a Postgres draft is never silently validated as generic SQL.

Step 3 — Capture the plan and assert on change

import sqlite3

def plan(sql: str, db_path: str) -> list[str]:
    with sqlite3.connect(db_path) as conn:
        return [row[3] for row in conn.execute("EXPLAIN QUERY PLAN " + sql)]

Store (prompt_hash, fingerprint, plan, approved_by) in a small table checked into the repo. CI then replays every stored prompt, compares fingerprints first and plans second, and fails on any unapproved difference. Plans are engine-specific, so treat a SQLite plan as a smoke signal and re-capture on the target engine before you trust it.

The selection rule for Position B

def pick(candidates: list[dict]) -> dict:
    clean = [c for c in candidates if not c["unknown_tables"] and not c["has_write"]]
    if not clean:
        raise ValueError("no candidate passed static checks")
    return min(clean, key=lambda c: (len(c["plan"]), c["fingerprint"]))

Using plan length as a placeholder cost model is defensible only for small fixtures; swap in your engine's cost estimate or a row-count probe before this touches anything that matters.

3. What free compute changes, and what it does not

Free capacity moves the cost side of this debate and leaves correctness untouched. A k=5 loop that was previously unaffordable becomes affordable, while the need for fingerprints, plan capture, and human approval stays exactly where it was. The operator-supplied terms for this account describe MonkeyCode's free model access and a free server option, including a free tier of 10 million tokens; treat those as availability claims from the project's outreach rather than as measured throughput, and confirm current terms before you size a CI budget around them.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

A practical arrangement is to run the drafting and replay loop on the free server option so the harness has a stable place to live, and to spend the free model access on replayed prompts rather than on exploratory chat. Keep the harness provider-agnostic regardless: if the endpoint changes, your stored fingerprints and plans are still the source of truth.

4. Decision rule

Situation	Choose	Why
Regulated schema, monthly migrations, reviewer signs off	Position A: temperature 0 plus stored fingerprint	One artifact per prompt; drift becomes a visible diff
Schema changes weekly, multiple SQL dialects in play	Position B: k=5 with static filters	Variance surfaces quoting and date-arithmetic edge cases
Free allowance is the hard constraint	Position A, or k=3 with early exit	Candidate count is the dominant cost multiplier
Latency-sensitive interactive tooling	Position A	Five calls per keystroke-adjacent action is untenable
No staging replica and no approval process	Neither, yet	Without a replay target the harness asserts nothing

5. Limitations and who should skip this

This harness validates syntax, table access, and one engine's plan shape; it cannot tell you that a JOIN fans out rows or that a filter uses the wrong timezone. Teams without a staging replica, a fixture snapshot, or a reviewer who can approve a fingerprint should not build CI around fingerprints at all, because they will be freezing noise.

The approach also ages poorly against dialect churn: parsers lag real engines, so a query that fails to parse may still be correct, and your harness should record parse failures as warnings rather than hard blocks during upgrades. Finally, the budget guard is an estimate, and any team spending real money on drafts should log provider-reported usage next to it.

If you already run a replay suite, the cheapest next step is to take your five most-edited prompts, record fingerprints and plans today, and let CI tell you how often they actually drift before you commit to either position.