Meter a Free Model Route as a Read-Only Canary Before It Writes State

The current AI feature cycle has moved from "which model answers best" to "which tool is allowed to change state." Recent agent-tooling discussions keep landing on the same failure boundary: write access. A model can produce a bad summary in a sandbox and you lose nothing. The same model updating a ticket status, deleting a row, or publishing a reply creates cleanup work that can dwarf the token cost.

Start with budget arithmetic because it changes the canary design. Suppose a grounded RAG call uses 2,400 prompt tokens and 700 completion tokens. That is 3,100 tokens per call, so a 30M-token allowance is about 9,700 calls. If an agent path retries or re-plans an average of five calls per task, the same budget covers fewer than 2,000 tasks. A free server slot is not the scarce resource in that scenario; token burn and side effects are.

For the sandbox below I used MonkeyCode's free model access and free server option, which the operator describes as including a 30M-token allowance and a free server slot. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I treat those two availability claims as operator-supplied rather than a permanent quota or hardware guarantee.

Decide what is read-only before the model runs

The first gate is not a prompt. It is an HTTP method allowlist. Most writes arrive as POST, PATCH, PUT, or DELETE. A read-only canary should reject every one of those verbs in the default mode, while still allowing model-backed GET routes such as summarization, classification, or entity extraction. That keeps the model useful without allowing it to change application state.

Route intent	Default canary mode	What belongs in the ledger
Summarize a support thread	GET, read only	route, prompt tokens, completion tokens, timestamp
Classify or extract fields	GET, read only	route, token split, accuracy sample ID if available
Draft a reply but do not send	GET, read only	route, token split, draft length
Update a CRM status	blocked unless ALLOW_WRITE=1	separate write event with actor and review status
Delete or archive	never enabled on a free canary	destructive guard, not token-only

The table forces the decision to be explicit. If a route is not safe to call with GET, it should not run in the default canary deployment. If it is safe, it still needs to record token usage so the free allowance does not disappear silently.

The artifact: a read-only guard plus a token ledger

The reproduction is small: a FastAPI app, a SQLite ledger, and a guard that disallows write methods unless an environment variable is set.

import os
import sqlite3
import time
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse
import httpx

app = FastAPI()
DB = os.getenv("LEDGER_DB", "canary.db")
ALLOW_WRITE = os.getenv("ALLOW_WRITE") == "1"


def init_db():
    with sqlite3.connect(DB) as conn:
        conn.execute(
            """
            CREATE TABLE IF NOT EXISTS token_events (
                id INTEGER PRIMARY KEY,
                route TEXT NOT NULL,
                prompt_tokens INTEGER NOT NULL,
                completion_tokens INTEGER NOT NULL,
                created_at INTEGER NOT NULL
            )
            """
        )


@app.middleware("http")
async def read_only_guard(request: Request, call_next):
    if not ALLOW_WRITE and request.method not in ("GET", "HEAD", "OPTIONS"):
        return JSONResponse(
            {"error": "write_path_blocked", "mode": "read_only_canary"},
            status_code=403,
        )
    return await call_next(request)


async def call_model(route: str, messages: list[dict]) -> dict:
    base_url = os.getenv("MODEL_BASE_URL")
    if not base_url:
        raise RuntimeError("MODEL_BASE_URL is not set")

    resp = httpx.post(
        f"{base_url}/chat/completions",
        json={"messages": messages, "temperature": 0},
        timeout=30,
    )
    resp.raise_for_status()
    data = resp.json()

    usage = data.get("usage") or {}
    prompt_tokens = int(usage.get("prompt_tokens") or 0)
    completion_tokens = int(usage.get("completion_tokens") or 0)

    with sqlite3.connect(DB) as conn:
        conn.execute(
            "INSERT INTO token_events(route, prompt_tokens, completion_tokens, created_at) "
            "VALUES (?, ?, ?, ?)",
            (route, prompt_tokens, completion_tokens, int(time.time())),
        )

    return data


@app.get("/read/ticket-summary/{ticket_id}")
async def ticket_summary(ticket_id: str):
    response = await call_model(
        "summary",
        [
            {"role": "system", "content": "Summarize the ticket without taking an action."},
            {"role": "user", "content": f"Read ticket {ticket_id} and summarize only."},
        ],
    )
    return {"ticket_id": ticket_id, "summary": response["choices"][0]["message"]["content"]}

The example assumes the provider endpoint accepts an OpenAI-style messages body and returns a usage object. If your provider uses a different HTTP shape, replace call_model without changing the ledger. The ledger is the part that matters.

After a few calls, inspect where the allowance is going:

SELECT route,
       COUNT(*) AS calls,
       SUM(prompt_tokens) AS prompt_tokens,
       SUM(completion_tokens) AS completion_tokens,
       SUM(prompt_tokens + completion_tokens) AS total_tokens
FROM token_events
GROUP BY route
ORDER BY total_tokens DESC;

For a budget check, subtract the recorded total from the operator-supplied allowance rather than hardcoding the number as a permanent guarantee.

def remaining_budget(budget=30_000_000) -> int:
    with sqlite3.connect(DB) as conn:
        row = conn.execute(
            "SELECT COALESCE(SUM(prompt_tokens + completion_tokens), 0) FROM token_events"
        ).fetchone()
    return budget - row[0]

Test before you grant write access

A useful canary run should prove three things before ALLOW_WRITE=1 is ever set.

1. Default mode rejects writes. POST, PATCH, PUT, and DELETE should return 403 with write_path_blocked. If they do not, the method guard is not the first gate.
2. The ledger matches provider-reported usage. Run the same read-only request three times and compare prompt_tokens and completion_tokens. Wide variance means the route is nondeterministic, the provider is retrying silently, or the prompt is being expanded in ways you did not expect.
3. The remaining budget decreases by the expected amount. Check remaining_budget before and after a known request. The change should equal the reported usage minus any provider rounding.

The write path should be a separate deployment, not a flag flip in the canary environment. Keep the canary at ALLOW_WRITE=0, then create a second release with explicit write authorization and a different database or tenant.

Limits and who should not use this

The read-only guard only checks HTTP verbs. It cannot catch a GET route that has a hidden side effect in application code, so keep read-only routes pure. The ledger also depends on provider-reported token counts; a local tokenizer may disagree, and that difference is part of the evaluation.

This is also not a security boundary against prompt injection. A model that can read private data and return it to an aggressive prompt is still a data-exfiltration risk, even if the route cannot write. Treat the canary as a release gate, not as an access-control substitute.

Skip this approach if you already run an LLM gateway with budget alerts and audit logs, or if every write is already human-reviewed before execution. If I were building a new evaluation sandbox, however, I would use a disposable free server slot for exactly this read-only stage before asking for write permission.