DEV Community: kongkong

Pin the Schema Before a Free Model Can Write

kongkong — Thu, 24 Sep 2026 18:00:57 +0000

I keep walking a small summarizer from the browser into a real save, and the pretty paragraph is never the part that breaks. Someone pastes a customer note, the completion looks clean, and the save returns a 500 before any row changes. The handler is doing a naked JSON parse on a string that starts with a markdown fence. So did the model fail, or did we skip the contract that should have rejected that string?

A popular thread this week is really about whether short functions are enough to make behavior obvious. I buy the worry, but the summarizer handler was already short, and the save behavior was still opaque. Nobody could point to a rule for which completions were allowed to become a saved note. So is the fix another line of commentary above the parse, or a contract the test can fail?

I think the free model is the wrong thing to blame first, and the wrong thing to trust next. A cheaper completion is still an untrusted upstream, closer to a flaky webhook than to a function in your repo. If the shape can drift, the write path should fail closed before a row is touched. Would you let an unmarked HTTP client insert into orders just because the demo looked fluent?

Where the save actually breaks

The architectural choice I want is a versioned schema between the provider response and the database, owned by the API. The UI may render a preview, but that preview is not authority, and a second client can always skip it. I tried the other way in a prototype, repairing fences in the browser so the demo kept moving. That repair hid the drift, a scripted client posted the raw string, and the note column filled with backticks.

Here is the contract I would pin before any model swap, including a move onto free inference. The example below is a proposed fixture, not a trace from a production incident, and you should run it yourself. I am not claiming a benchmark, a latency number, or a saved dollar from this snippet. If the bounds feel strict, that is the opinion, because a loose parser is how drift becomes data.

import json
from pydantic import BaseModel, Field, ValidationError

FENCE = "`" * 3

class SummaryV1(BaseModel):
    schema_version: str = Field(pattern=r"^1$")
    summary: str = Field(min_length=1, max_length=500)
    confidence: float = Field(ge=0.0, le=1.0)

def parse_completion(raw: str) -> SummaryV1:
    text = raw.strip()
    if text.startswith(FENCE):
        raise ValueError("fenced_markdown")
    try:
        data = json.loads(text)
    except json.JSONDecodeError as exc:
        raise ValueError("invalid_json") from exc
    return SummaryV1.model_validate(data)

The fence check is deliberate, because a helpful model often wraps an object and still sounds correct to a reader. I would rather return 422 with a stable error code than spend another prompt trying to shave the backticks. Extra keys can be ignored by configuration, missing keys should reject, and a truncated object should never become a null summary. Have you ever watched a UI show saved while the column stored an empty string that passed a truthiness check?

Pin a version, then fail closed

The route then stays boring, which is the whole point of putting the contract in one place. It loads the note, calls the provider behind one seam, and persists only after the schema accepts the body. A rejected completion is stored as a failed attempt with the schema version, not as a successful note. If you skip that attempt record, the next on-call engineer sees only a 500 and a prompt that already vanished.

def save_summary(note: str, raw_completion: str, db) -> dict:
    try:
        parsed = parse_completion(raw_completion)
    except (ValueError, ValidationError) as exc:
        db.attempts.insert({
            "schema_version": "1",
            "status": "rejected",
            "error": str(exc),
        })
        return {"status": 422, "error": "contract_rejected"}
    db.notes.update_summary(note_id=note, summary=parsed.summary)
    return {"status": 200, "schema_version": parsed.schema_version}

That insert is pseudocode for the seam, not a claim that your ORM or your tables look like this. The idea travels to SQL or a document store, because rejection is data and success is a separate write. I want 422 when the shape is wrong, and 502 only when the provider failed before a body existed. Mixing those codes sends people into the wrong layer, which is how a parser bug gets filed as an outage.

You can prove the gate without a network call, and I would not skip that step to save an afternoon. A fixture with a fenced string should fail, a valid object should pass, and an empty summary should fail the length bound. I keep those three cases beside the route so a model change cannot land as a silent prompt edit. Why would you trust a live swap if the repo cannot already reject the last bad body you saw?

python -m pytest tests/test_summary_contract.py -q

def test_fenced_markdown_is_rejected():
    fence = "`" * 3
    payload = '{"schema_version":"1","summary":"ok","confidence":0.5}'
    raw = fence + "json\n" + payload + "\n" + fence
    try:
        parse_completion(raw)
    except ValueError as exc:
        assert str(exc) == "fenced_markdown"
    else:
        raise AssertionError("fence should fail closed")

Prove it locally before the free endpoint

Only after that local gate is green do I point the same parser at a live free endpoint. The credential stays in the server environment, and the browser never learns the provider base URL. A demo page that calls the model directly has already skipped the schema and exposed the key path. I would rather see a 401 from my own API than a fluent completion that never passed validation.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. The brief I was given describes MonkeyCode as an open-source project with free model access and a free server option. That pair is useful as a second environment for the same fixture, not as permission to persist. I am not stating a token allowance, a hardware size, a duration, or a model name here.

Those terms move quickly, and I have not re-checked a primary page before writing this draft. Read the current project terms yourself, and treat any number remembered from chat as stale until you do. If the free option is gone or narrowed, the local fixture still stands, which is why it lives in the repo. Would you drop the schema pin just because a posted allowance changed between two weekly deploys?

The browser sends the note to your API, your API holds the credential, and the free server is another base URL behind that seam. A session header on that curl is a stand-in, not a real token, and you should use your own auth cookie. I log the provider status beside the schema version so a 429 is not mistaken for a bad summary. If the body is prose apologizing for the format, quarantine it and stop, because a retry will not grow a schema.

curl -sS -X POST http://127.0.0.1:8000/summaries \
  -H 'content-type: application/json' \
  -H 'authorization: Bearer dev-session' \
  -d '{"note":"Customer asked to move delivery to Friday."}'

Point the server at the free option with environment variables, and keep the secret out of the repo. The host below is a placeholder, not a documented endpoint, so replace it only after you read the current project docs. I refuse to paste a live host or a sample key into a post that will be copied into terminals.

export PROVIDER_BASE_URL="https://example.invalid/free-server"
export PROVIDER_API_KEY="from-your-secret-store"

What usually fails next is rarely the tidy JSON object you practiced against while writing the prompt. A free pool can return 429, an empty content field, or a 200 whose body is an apology. The fixture should record the status and the first hundred characters, then stop without a blind retry. Retrying a 200 that failed the schema only burns the allowance and teaches the handler to be generous.

Limits I will not talk past

There is a maintainability tradeoff I accept on purpose, and it will annoy anyone chasing a magical demo. A strict schema will reject some completions a human would have repaired, so the first week looks less fluent. That cost is smaller than a column full of fences, and cheaper than a migration after three clients stored three shapes. Performance stays predictable because you parse a small object and you do not hold a transaction open across the provider call.

When version two arrives, I would add a second model and backfill stored notes, not reinterpret old JSON. Old records keep their schema version, so a reader can see which contract produced the summary. A flag in the component that swaps prompts without a migration is how you get two meanings in one column. Have you already found a text field that means different things depending on which week it was written?

This approach is a poor fit when the note holds secrets or regulated data you cannot send to an external host. It is a poor fit if you need a latency guarantee, because free capacity is not an SLA I will pretend to own. Teams that want the model to invent new fields every week should not pin version one, or they will fight their own gate. If the request has no authenticated user, do not add this write, since a public summarizer becomes an open proxy.

Before I call the change done, the fence fixture has to fail and then pass on a valid body. The empty summary has to be rejected, and a 422 has to stay distinct from a provider 502. The provider key has to be absent from the browser bundle, which you can check with a simple search. That is the reusable gate, and it does not depend on which model happened to answer the note.

I would ship that gate before I ever allow a free completion to update a customer note. The free server is only a second environment for the same test, not a reason to widen what you persist. Call, then validate, then write, in that order, even when the endpoint costs you nothing today. Which status came back first on your last bad save, a 422 from the gate or a 500 from a naked parse?

Add Model Spend Behind a Budget Row, Not a Browser Agent

kongkong — Wed, 23 Sep 2026 15:58:30 +0000

The support console fired a rewrite request at 4:11 p.m., and the widget treated a model response like a database grant. The browser held a provider key in local storage, posted a completion, then PATCHed the invoice with whatever paragraph came back. Did anyone ask whether that tenant still had inference budget for ticket rewrites this month? Did a second tab retry the same click and persist a second, slightly different version of the legal text?

I think the fashionable answer this week is wrong, and I will not hedge it. An agent that never leaves the browser is a demo of latency, not a delivery path for customer writes. The first layer that fails is not the model, because the model did return usable prose. The missing budget row should have refused the write, and nobody on the path asked it.

Free tokens are a subsidy, not a permission

Free inference is a prototype subsidy, and it is not permission to mutate invoices, comments, or stored schema. If your UI can call a model and your API can persist the result, you already gave write authority to a meter you do not own. Why do we keep wiring it the other way around, week after week, in otherwise serious codebases? The happy path looks cheap until a retry doubles spend and the bad paragraph is already in Postgres.

Can you roll that write back from a chat transcript after the toast has already gone green? I cannot, and I do not believe a streaming DOM node will save you either. The contract I want is boring on purpose, which is usually a sign it will survive contact with production. Every model call that can affect stored state must present a tenant, a purpose, and a budget row. If the row cannot cover the call, the handler returns 402 and nothing is written.

Follow the click until the handoff dies

Walk the request with me as a vertical slice, not as an architecture poster on a wiki. A human clicks rewrite, and the browser should post a command with ticket_id, intent, and an idempotency key. That client should not hold a provider key, and it should not pick a model name from a dropdown somebody added during a hack day. Where does this fail first when the spend envelope is not a real row?

It fails right after the model returns text, because nothing in the path is allowed to say no. Auth said the user could edit the ticket, and the model said here is a paragraph, so Postgres returned 200. The budget said nothing at all, because it was never asked as a locked row. That is not an artificial intelligence problem hiding in a vendor SDK. That is a missing constraint at the only layer that can refuse a write.

The budget row as a pasteable slice

I am labeling the next blocks as a reference implementation you can run locally, not as a souvenir from a named cluster. You should keep that label in your own repo until the 402 path is proven with a real status code. The unique constraint on the idempotency key is the whole point of the second table. Network retries must not mint a second spend event and a second invoice rewrite.

CREATE TABLE inference_budgets (
  id            uuid PRIMARY KEY,
  tenant_id     uuid NOT NULL,
  purpose       text NOT NULL,
  period_start  timestamptz NOT NULL,
  period_end    timestamptz NOT NULL,
  token_limit   integer NOT NULL,
  token_spent   integer NOT NULL DEFAULT 0,
  UNIQUE (tenant_id, purpose, period_start)
);

CREATE TABLE inference_spend_events (
  id               uuid PRIMARY KEY,
  budget_id        uuid NOT NULL REFERENCES inference_budgets(id),
  idempotency_key  text NOT NULL UNIQUE,
  provider         text NOT NULL,
  input_tokens     integer NOT NULL,
  output_tokens    integer NOT NULL,
  status           text NOT NULL,
  created_at       timestamptz NOT NULL DEFAULT now()
);

Have you watched a mobile client replay a POST because the toast never rendered on a flaky radio? The second write is almost always the one that lands in support later. A FastAPI handler can refuse the business write when a locked budget row cannot cover a conservative estimate. This sketch estimates input tokens plus a hard output cap, then reconciles after the provider responds, which is the actual seam.

# Reference implementation: prove the 402 locally. Not production gospel.
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel, Field

router = APIRouter()
ESTIMATED_OUTPUT_CAP = 800

class RewriteCommand(BaseModel):
    ticket_id: str
    intent: str = Field(pattern="^rewrite_friendly$")
    idempotency_key: str
    tenant_id: str

@router.post("/tickets/{ticket_id}/rewrite")
def rewrite_ticket(
    ticket_id: str,
    cmd: RewriteCommand,
    db=Depends(get_db),
    models=Depends(get_model_client),
):
    if cmd.ticket_id != ticket_id:
        raise HTTPException(409, "ticket_id mismatch")

    existing = db.fetch_spend_event(cmd.idempotency_key)
    if existing:
        return {"status": existing.status, "spend_id": existing.id, "replayed": True}

    budget = db.lock_budget(cmd.tenant_id, purpose="ticket_rewrite")
    if budget is None:
        raise HTTPException(403, "no budget row for purpose")

    estimate = estimate_input_tokens(ticket_id) + ESTIMATED_OUTPUT_CAP
    if budget.token_spent + estimate > budget.token_limit:
        raise HTTPException(402, "inference budget exhausted")

    reservation_id = db.reserve_tokens(budget.id, estimate, cmd.idempotency_key)
    try:
        result = models.complete(
            purpose="ticket_rewrite",
            prompt=load_ticket_prompt(ticket_id),
            max_tokens=ESTIMATED_OUTPUT_CAP,
        )
    except ProviderError as exc:
        db.release_reservation(reservation_id, reason=str(exc))
        raise HTTPException(502, "provider failed; budget released") from exc

    db.finalize_spend(
        reservation_id,
        provider=result.provider,
        input_tokens=result.input_tokens,
        output_tokens=result.output_tokens,
    )
    db.apply_ticket_rewrite(ticket_id, result.text, spend_id=reservation_id)
    return {"status": "applied", "spend_id": reservation_id}

Notice what the handler refuses to do, even when the process is running on a scratch box you stood up for a demo. It does not accept a model name from the client, and it does not stream tokens into a websocket that also applies SQL. The purpose string is the product contract, not a log field you add later for analytics. Why would playground_chat and ticket_rewrite share one pool when they do not share one blast radius?

Prove the 402 on a scratch server

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

A scratch environment is the right place to teach the ledger to say no, because that experiment should not ride a paid provider bill. MonkeyCode's free model access and free server option can host that slice while you are still proving the refusal path. Treat the open-source project as a sandbox for the contract, not as a substitute for the contract. If you cannot get a 402 from the free box, you will not get a real refusal in production either.

python -m venv .venv
source .venv/bin/activate
pip install fastapi uvicorn "psycopg[binary]"
uvicorn app:app --reload --port 8080

psql "$DATABASE_URL" -c "insert into inference_budgets
  (id, tenant_id, purpose, period_start, period_end, token_limit)
  values ('00000000-0000-0000-0000-000000000001',
          '11111111-1111-1111-1111-111111111111',
          'ticket_rewrite', now(), now() + interval '30 days', 200);"

curl -sS -X POST "$API/tickets/T-42/rewrite" \
  -H "content-type: application/json" \
  -H "authorization: Bearer $DEV_TOKEN" \
  -d '{"ticket_id":"T-42","intent":"rewrite_friendly",
       "idempotency_key":"rewrite-T-42-01",
       "tenant_id":"11111111-1111-1111-1111-111111111111"}'

The first call may apply if the estimate still fits inside that tiny envelope. The second call, with a new idempotency key, should return 402 and leave the ticket body untouched. Replaying the same key should return the original status and still not double-write the invoice. If your status codes do not split that way, the seam is theater, not a contract.

I also want a test that the browser never saw a provider key in headers or payloads. That is the difference between a tab-resident agent and a feature that survives a stolen laptop. The test is ugly on purpose, because pretty tests tend to skip the overdraw and only assert that JSON looks friendly.

def test_rewrite_does_not_overdraw_or_write(client, db):
    db.set_token_limit(tenant="t1", purpose="ticket_rewrite", limit=50)
    before = db.ticket_body("T-42")
    first = client.post("/tickets/T-42/rewrite", json=command("T-42", "k1"))
    assert first.status_code in {200, 402}
    second = client.post("/tickets/T-42/rewrite", json=command("T-42", "k2"))
    assert second.status_code == 402
    if first.status_code == 402:
        assert db.ticket_body("T-42") == before
    assert "sk-" not in str(second.headers)
    replay = client.post("/tickets/T-42/rewrite", json=command("T-42", "k1"))
    assert replay.json().get("replayed") in {True, None} or replay.status_code == 402

Is that test going to impress a design critique on a clean-code thread? No, and I do not need it to. Spend is a row, writes are a later row, and the model is a vendor behind a purpose. The browser is a client, not a wallet, and it should not get a vote on either row.

The failures that show up when you skip the row

The failure mode looks like this, and it is more common than a clever browser agent demo. A handler temporarily reads max_tokens from the client, and the client sends a larger cap after the first 402. A streaming UI applies a partial sentence when the socket dies at eighty percent, then applies again on resume. A free-server compose file lands in production with the budget check commented out because it made the pitch deck jitter.

The model layer looks healthy, and the persist layer looks healthy, so the dashboard stays green. The handoff is an if statement somebody deleted on a Friday so the demo would not blink. Does that sound like a model-quality incident to you? It is a permissions incident with extra tokens attached.

Performance testing of the API does not save you if every scenario still has tokens left in the envelope. A load test that never exhausts budget is a load test of the happy path, which you already knew worked. Can you show me the 402 p95, and can you show that ticket bodies did not change on those requests? If you cannot, you have not tested the contract you claim to have.

Who should not copy this

If your product is a local playground with no persistence, a budget row is ceremony, and you should not add it for the aesthetic. If you cannot name a tenant, a purpose, and a period, you are not ready to attach a ledger. A free model will only hide that gap until the first shared demo key leaks into a write path. Also skip this sketch if you need hard multi-region accounting today, because it assumes one database and a row lock.

That starting contract is not a billing platform, and I will not pretend otherwise. People who only need a read-only eval slice should stay on the read path until a human grants write authority. Do not use this article as cover for giving a chat widget schema rights. The whole point is the opposite move: prove refusal before you celebrate the prose.

Prove the click, the 402, the replay, and the absent provider key before you ship the rewrite. Keep model names on the server, keep purposes distinct, and keep the free server in the eval path until the ledger refuses correctly. Which layer handoff is least stable in your stack right now, budget to model or model to persist? Send a failure state or a response code, not a model name, because I care whether the write happened after the budget already said no.

Add Tool Mutations Behind a Receipt Row, Not a Chat Stream

kongkong — Tue, 22 Sep 2026 14:15:33 +0000

The click looks harmless enough on a Monday morning support board that already has too many open tickets. A lead presses Draft follow-up and file it, and the side panel immediately starts streaming unfinished sentences. Everyone watching the standup unclenches because tokens are moving and the demo gods finally appear satisfied. Then the ticket still has no follow-up, and the only souvenir is a chat bubble nobody can honestly replay.

Where did that request actually die, once the socket went quiet and the coffee got cold? It died in the first layer that treated a language model as if it were a database with manners. I will not soften that opinion just to make a nicer architecture diagram look complete. Tool calling is not a conversation you happened to JSON-encode on the way to the browser.

If a model may create a row, close a ticket, or enqueue mail, that mutation belongs on a receipt you can query after the socket drops. The stream is a progress channel for humans who need reassurance. It is not a ledger, and it is a terrible audit log when legal asks what actually happened. Have you ever tried explaining “the assistant said filed” to someone who only believes SELECT statements?

Teams skip the receipt because free inference makes the happy path feel cheap enough to ship. Someone stands up a scratch box, dumps a tools array into the request handler, and watches argument JSON appear like magic. I have watched that prototype survive into a staging URL because the chat UI looked finished. Does a streaming paragraph prove the follow-up exists, or does it only prove a tokenizer was billed?

When I need a disposable provider seam to prove the receipt path, I park completions on MonkeyCode. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The project is open source, and the operator-supplied free model access plus free server option is useful as a stand-in provider, not as the product you show customers. Delete the name tomorrow and the same contract should still point at any other completion endpoint without rewriting your mutations.

I want the route to pick the tool, and I want the model only to fill arguments against a frozen schema. Letting the model choose from an unbounded toolbox is how a summarize action becomes a delete action with extra confidence. Analogize it to a bank teller: the customer does not pick the vault protocol, they fill the slip, and the bank decides whether the drawer opens. Why would we give a statistical decoder more authority than a teller window?

Here is the working path I actually implement when the user action is “draft and file.” Create the receipt first in pending, stream only after that insert commits, run the allowlisted tool with a deadline, then mark applied or failed before the UI is allowed to say done. The browser may render tokens the whole time, but the source of truth is the row, not the last assistant sentence. If the worker dies, you still have a receipt id you can replay without asking the model to remember its own side effects.

-- proposed walkthrough: receipts are the ledger, not the chat table
CREATE TABLE tool_receipts (
  id              UUID PRIMARY KEY,
  user_id         UUID NOT NULL,
  idempotency_key TEXT NOT NULL,
  tool_name       TEXT NOT NULL CHECK (tool_name IN ('file_followup')),
  arguments_json  JSONB NOT NULL,
  status          TEXT NOT NULL CHECK (status IN ('pending','applied','failed')),
  http_status     INTEGER,
  error_code      TEXT,
  result_ref      TEXT,
  created_at      TIMESTAMPTZ NOT NULL DEFAULT now(),
  applied_at      TIMESTAMPTZ,
  UNIQUE (user_id, idempotency_key)
);

The API does not start by calling a provider. It starts by inserting that row under the caller’s auth context and a client-generated idempotency key, because retries are not optional on flaky browsers. Only then do you ask a model to fill ticket_id, summary, and assignee against a JSON schema the route already owns. If argument fill fails validation, the receipt goes to failed with 422, and the stream is allowed to narrate the failure without pretending a write occurred.

# proposed walkthrough — FastAPI-shaped, not a claimed production dump
from fastapi import APIRouter, Header, HTTPException
from pydantic import BaseModel, Field
import httpx, json, os, uuid

router = APIRouter()
ALLOWED = {"file_followup"}
FILL_SCHEMA = {
    "type": "object",
    "required": ["ticket_id", "summary", "assignee"],
    "additionalProperties": False,
    "properties": {
        "ticket_id": {"type": "string", "minLength": 8},
        "summary": {"type": "string", "minLength": 12, "maxLength": 500},
        "assignee": {"type": "string", "minLength": 3},
    },
}

class FileReq(BaseModel):
    ticket_id: str
    notes: str = Field(min_length=1)
    idempotency_key: str = Field(min_length=8)

async def complete_arguments(notes: str, ticket_id: str) -> dict:
    # Provider seam: swap base_url without touching mutation code.
    payload = {
        "messages": [{
            "role": "user",
            "content": f"Fill file_followup args for ticket {ticket_id}. Notes: {notes}"
        }],
        "response_format": {"type": "json_schema", "json_schema": FILL_SCHEMA},
    }
    async with httpx.AsyncClient(timeout=20.0) as client:
        r = await client.post(os.environ["COMPLETIONS_URL"], json=payload)
    r.raise_for_status()
    return json.loads(r.json()["choices"][0]["message"]["content"])

@router.post("/tickets/{ticket_id}/followups")
async def file_followup(
    ticket_id: str,
    body: FileReq,
    db,
    user_id: str,
    idempotency_key: str = Header(alias="Idempotency-Key"),
):
    if body.ticket_id != ticket_id:
        raise HTTPException(422, "ticket_id mismatch")
    receipt_id = uuid.uuid4()
    inserted = await db.fetchrow(
        """INSERT INTO tool_receipts
           (id, user_id, idempotency_key, tool_name, arguments_json, status)
           VALUES ($1,$2,$3,'file_followup','{}'::jsonb,'pending')
           ON CONFLICT (user_id, idempotency_key)
           DO UPDATE SET idempotency_key = EXCLUDED.idempotency_key
           RETURNING id, status, result_ref, http_status""",
        receipt_id, user_id, idempotency_key,
    )
    if inserted["status"] == "applied":
        return {"receipt_id": str(inserted["id"]), "status": "applied",
                "result_ref": inserted["result_ref"]}
    try:
        args = await complete_arguments(body.notes, ticket_id)
    except Exception:
        await db.execute(
            "UPDATE tool_receipts SET status='failed', http_status=502, error_code='fill_failed' WHERE id=$1",
            inserted["id"],
        )
        raise HTTPException(502, "argument fill failed")
    if args.get("ticket_id") != ticket_id:
        await db.execute(
            "UPDATE tool_receipts SET status='failed', http_status=422, error_code='arg_mismatch' WHERE id=$1",
            inserted["id"],
        )
        raise HTTPException(422, "model attempted to retarget ticket")
    # Mutation happens only after the receipt exists and args match the route.
    followup_id = await apply_followup(db, user_id, args)  # your real write
    await db.execute(
        """UPDATE tool_receipts
           SET status='applied', http_status=201, arguments_json=$2::jsonb,
               result_ref=$3, applied_at=now()
           WHERE id=$1""",
        inserted["id"], json.dumps(args), followup_id,
    )
    return {"receipt_id": str(inserted["id"]), "status": "applied",
            "result_ref": followup_id}

Notice what the handler refuses to do, even when the model sounds eager. It will not invent a second ticket id, it will not run an unnamed tool, and it will not report success while the receipt is still pending. The completions URL is an environment variable on purpose, so a free server and a paid provider are the same seam. Are you still tempted to stream the write confirmation before apply_followup returns, just because the sentence already looks right?

I also budget the tool-call loop like an API, not like a chatroom that can talk forever. Argument fill gets twenty seconds. The mutation gets its own transaction and its own timeout. If the model retries internally, that is the provider’s problem, not a license for my worker to loop until the free quota feels embarrassed. A stream that outlives the receipt deadline should end in failed with 504, even if leftover tokens keep arriving like late guests.

# proposed failure tests — run these before you celebrate the widget
import pytest

@pytest.mark.asyncio
async def test_duplicate_key_does_not_double_file(client, db, user):
    headers = {"Idempotency-Key": "follow-7f3a"}
    body = {"ticket_id": "TCK-1008", "notes": "Customer still blocked on SSO."}
    first = await client.post("/tickets/TCK-1008/followups", json=body, headers=headers)
    second = await client.post("/tickets/TCK-1008/followups", json=body, headers=headers)
    assert first.status_code in (200, 201)
    assert second.status_code == 200
    assert second.json()["result_ref"] == first.json()["result_ref"]
    count = await db.fetchval("SELECT count(*) FROM followups WHERE ticket_id='TCK-1008'")
    assert count == 1

@pytest.mark.asyncio
async def test_model_cannot_retarget_ticket(client, monkeypatch):
    async def lying_fill(notes, ticket_id):
        return {"ticket_id": "TCK-EVIL", "summary": "nope", "assignee": "ada"}
    monkeypatch.setattr("app.complete_arguments", lying_fill)
    r = await client.post(
        "/tickets/TCK-1008/followups",
        json={"ticket_id": "TCK-1008", "notes": "please file this"},
        headers={"Idempotency-Key": "follow-retarget"},
    )
    assert r.status_code == 422
    assert r.json()["detail"] == "model attempted to retarget ticket"

@pytest.mark.asyncio
async def test_fill_timeout_marks_receipt_failed(client, monkeypatch, db):
    async def hang(notes, ticket_id):
        raise httpx.TimeoutException("fill exceeded 20s")
    monkeypatch.setattr("app.complete_arguments", hang)
    r = await client.post(
        "/tickets/TCK-1008/followups",
        json={"ticket_id": "TCK-1008", "notes": "timeout path"},
        headers={"Idempotency-Key": "follow-timeout"},
    )
    assert r.status_code == 502
    status = await db.fetchval(
        "SELECT status FROM tool_receipts WHERE idempotency_key='follow-timeout'"
    )
    assert status == "failed"

Those tests are the article, more than the widget ever will be. A duplicate key that files twice is a production incident, not a cute retry. A model that retargets ticket_id is a confused intern with write access. A timeout that leaves pending forever is how support leads refresh themselves into a second mutation. If your stream already flushed pretty tokens, which status code do you still owe the receipt?

Production caveats sit in the boring places people skip when inference feels free. You still need auth on the route, because a public completions box is not a permission system. You still persist arguments, because “the model probably used the right ticket” is not an incident review. You still cap output size, because a five-hundred-character summary is a feature and a thirty-thousand-character poem is a storage bill. And you still hide provider keys on the server, even when the server itself was free to provision.

This approach is wrong for a few honest cases, and I would rather say that than stretch the pattern. Do not use a receipt row as a substitute for an agent that must explore an unbounded tool catalog in research settings. Do not use it for purely local CLIs where the human already stares at the diff and the only side effect is a file they can undo. Do not use it if your product cannot tolerate an extra write before the model speaks, because that extra write is the whole point. If you need the assistant to improvise tools at runtime, you are not filing follow-ups, you are building a different machine.

Reuse this in order when you wire the next action: insert receipt, fill arguments against a frozen schema, execute one allowlisted mutation, stamp applied or failed, then let the stream narrate the row. Skip a step and you are back to chatting with a database. I would rather a user see a blunt 422 than a lyrical paragraph that never landed. What is the least stable handoff in your stack right now, argument fill, tool execution, or receipt apply, and which response code do you return when the stream already flushed tokens?

If you need a throwaway box to hang that provider seam on while the receipt table is still ugly, MonkeyCode’s free server option is enough to exercise the contract. Keep the mutations on your side of the wall.

Deliver Completions With a Job Row, Not a Held Connection

kongkong — Mon, 21 Sep 2026 12:18:12 +0000

Last Tuesday I sat with a teammate who clicked Generate on a supposedly simple report screen. The spinner behaved like a prayer, nginx hit sixty seconds, and the gateway returned 504 while a model process kept talking to nobody. Did the user get a paragraph? No. Did a refresh spend another free inference call on the same prompt? Yes, immediately, because the POST had been the product.

I am done treating that architecture as a prototype shortcut. If a browser is holding a TCP connection while a model thinks, you do not have a feature. You have a costume. Free tokens do not change the physics of a reverse proxy, and a free server does not make a blocked request thread honest.

Why do we keep doing this? Because the UI looks finished when a button calls /generate and awaits JSON. AI-assisted scaffolding makes that lie cheaper to type. The first layer that fails is not the prompt. It is the handoff between the user's POST, the process that actually calls the model, and the row that should still exist after a timeout.

The opinion, without a shrug

A user-facing completion should be accepted as work, not performed inside the request that asked for it. Return 202 Accepted with a job identifier, persist the prompt under the caller's identity, and let a worker touch the provider. Poll or subscribe for the artifact. If that sounds heavy for a demo, the demo is already lying about production.

I do not mean you need a workflow engine on day one. I mean a job row with a status, an owner, a budget, and an idempotency key. Think of it as a coat-check ticket. The coat can still be lost, but at least you stop pretending the doorway is a closet.

People argue that free inference makes the queue optional. That is backwards. A constrained or cold server is exactly when the request thread will 504, and a free path is exactly when the client will retry without shame. If you cannot survive that, you cannot survive a billed provider either.

A vertical slice you can actually run

Label this as a proposed working slice, not a trophy benchmark. I want one authenticated action: POST /reports with a prompt, then GET /reports/{id} until the body exists. Permissions and persistence show up before the model does, or the rest is theater.

CREATE TABLE report_jobs (
  id            TEXT PRIMARY KEY,
  user_id       TEXT NOT NULL,
  idempotency_key TEXT NOT NULL,
  prompt        TEXT NOT NULL,
  status        TEXT NOT NULL CHECK (status IN ('queued','running','succeeded','failed')),
  http_status   INTEGER,
  result_text   TEXT,
  error_code    TEXT,
  token_estimate INTEGER NOT NULL,
  created_at    TEXT NOT NULL,
  updated_at    TEXT NOT NULL,
  UNIQUE (user_id, idempotency_key)
);

CREATE TABLE inference_budgets (
  user_id        TEXT PRIMARY KEY,
  daily_tokens   INTEGER NOT NULL,
  spent_tokens   INTEGER NOT NULL,
  day_utc        TEXT NOT NULL
);

The unique pair is the whole point. A double click, a React remount, or a browser retry must collide on the same row instead of inventing a second conversation with the model. If your table cannot say no, your provider will say yes twice.

# report_api.py — proposed slice, not a framework manifesto
import os, uuid, datetime as dt
from fastapi import FastAPI, Header, HTTPException
from pydantic import BaseModel, Field

app = FastAPI()
DAILY_TOKEN_CAP = int(os.environ["DAILY_TOKEN_CAP"])

class CreateReport(BaseModel):
    prompt: str = Field(min_length=1, max_length=8000)

def estimate_tokens(prompt: str) -> int:
    return max(1, len(prompt) // 4)

@app.post("/reports", status_code=202)
def enqueue_report(
    body: CreateReport,
    authorization: str = Header(...),
    idempotency_key: str = Header(..., alias="Idempotency-Key"),
):
    user_id = require_user(authorization)  # 401 if the bearer is junk
    tokens = estimate_tokens(body.prompt)
    existing = db.fetch_job(user_id, idempotency_key)
    if existing:
        return {"job_id": existing.id, "status": existing.status}

    if not db.reserve_budget(user_id, tokens, DAILY_TOKEN_CAP):
        raise HTTPException(status_code=429, detail={
            "error": "budget_exhausted",
            "retry": "tomorrow",
        })

    job_id = str(uuid.uuid4())
    now = dt.datetime.utcnow().isoformat() + "Z"
    db.insert_job(
        id=job_id, user_id=user_id, idempotency_key=idempotency_key,
        prompt=body.prompt, status="queued", token_estimate=tokens,
        created_at=now, updated_at=now,
    )
    broker.publish("report.jobs", job_id)
    return {"job_id": job_id, "status": "queued"}

@app.get("/reports/{job_id}")
def read_report(job_id: str, authorization: str = Header(...)):
    user_id = require_user(authorization)
    job = db.get_job(job_id)
    if job is None or job.user_id != user_id:
        raise HTTPException(status_code=404)
    payload = {"job_id": job.id, "status": job.status, "http_status": job.http_status}
    if job.status == "succeeded":
        payload["text"] = job.result_text
    if job.status == "failed":
        payload["error_code"] = job.error_code
    return payload

Notice what the handler refuses to do. It does not import an SDK. It does not stream tokens through the user's socket. It reserves a cheap local estimate against a daily envelope, then it leaves. If reservation feels pessimistic, good. A free path still creates paid side effects in logs, disks, and support, and I want those side effects named.

The worker is where the provider seam lives. Keep the URL and the key in the environment. Do not sprinkle them through route files like seasoning.

# report_worker.py — proposed slice
import os, time, httpx

PROVIDER_URL = os.environ["INFERENCE_URL"]
PROVIDER_KEY = os.environ["INFERENCE_KEY"]
TIMEOUT_S = float(os.environ.get("INFERENCE_TIMEOUT_S", "45"))

def handle(job_id: str) -> None:
    job = db.claim_job(job_id)  # queued -> running, skip if already terminal
    if job is None:
        return
    try:
        response = httpx.post(
            PROVIDER_URL,
            headers={"Authorization": f"Bearer {PROVIDER_KEY}"},
            json={"prompt": job.prompt},
            timeout=TIMEOUT_S,
        )
        if response.status_code >= 500:
            db.fail_job(job_id, error_code="provider_5xx", http_status=response.status_code)
            db.release_budget(job.user_id, job.token_estimate)  # only on hard failure
            return
        if response.status_code != 200:
            db.fail_job(job_id, error_code="provider_rejected", http_status=response.status_code)
            db.release_budget(job.user_id, job.token_estimate)
            return
        text = response.json()["text"]
        db.complete_job(job_id, text=text, http_status=200)
    except httpx.TimeoutException:
        db.fail_job(job_id, error_code="provider_timeout", http_status=504)
        db.release_budget(job.user_id, job.token_estimate)

That claim_job update is the difference between a queue and a wish. If two workers see the same message after a crash, only one should move queued to running. Completions are not naturally idempotent. Your table has to be.

Where a free path actually helps

I still want a constrained inference box in staging, because timeouts and cold processes show up there first. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode is an open-source project that currently offers free model access and a free server option, which is enough to point INFERENCE_URL at a real process without wiring a billed provider into the first vertical slice.

I will not pretend that free access is a capacity plan, a latency SLO, or a reason to skip auth. It is a seam. If the worker can fail closed against that seam, swapping the URL later is boring, and boring is the goal. If you cannot name the status code you expect when the free server is down, you are not ready to point a UI at it.

Use it to rehearse the ugly path, not to skip the job row. A free server that you await inside POST /reports is still a held connection. You just stopped paying the model vendor for the privilege of hanging nginx.

Cross-layer failures I actually test

I do not start with a golden prompt. I start with the user action and the first lie.

Replay the same Idempotency-Key twice and assert one row, one 202, and one provider call. Kill the worker after running and assert the client still reads a job, not a blank 500 from a vanished POST. Return a provider 503 and assert the job is failed with provider_5xx, budget released, and the UI not spinning forever. Wait past your gateway timeout on purpose and assert the user never held that wait on their socket.

# test_report_contract.py — proposed tests
def test_duplicate_key_does_not_double_spend(client, db, provider):
    headers = {
        "Authorization": "Bearer user-1",
        "Idempotency-Key": "click-77",
    }
    a = client.post("/reports", json={"prompt": "summarize the invoice"}, headers=headers)
    b = client.post("/reports", json={"prompt": "summarize the invoice"}, headers=headers)
    assert a.status_code == 202
    assert b.status_code == 202
    assert a.json()["job_id"] == b.json()["job_id"]
    assert provider.call_count == 0  # enqueue only
    worker.drain_one()
    assert provider.call_count == 1
    worker.drain_one()
    assert provider.call_count == 1

def test_reader_does_not_see_someone_elses_job(client, db):
    job_id = db.insert_owned(user_id="user-1", prompt="secret")
    r = client.get(f"/reports/{job_id}", headers={"Authorization": "Bearer user-2"})
    assert r.status_code == 404

If those tests feel like ceremony, ask what your spinner does on a 504 today. Ceremony is cheaper than two silent completions and a support thread that cannot find a job id.

Who should not take this advice

If you are writing a local CLI that you watch in a terminal, await the model. If you are running an overnight batch with no interactive client, a queue is still nice, but a blocked process is not a UX lie. This argument is for anything a human clicks, especially anything you will one day put behind a shared domain.

Also skip this if you cannot store the prompt. A job row that holds user text is a data store, with retention and access control attached. Moving secrets from a request body into a table is not an upgrade unless you meant to keep them.

What this does not fix

A 202 does not validate output. A budget envelope is not a legal policy. A free server can disappear without a deprecation essay, so keep the provider behind that environment variable and keep the client on job state. I am not offering model names, quotas, or duration claims, because those go stale while the contract stays useful.

The maintainability tradeoff is real. You now operate a worker, a broker, and a table. In return you get a place to put timeouts, retries, and permissions that is not the browser. I will take that trade every time the alternative is a spinner negotiating with nginx.

Reuse this when you cut the slice: authenticate first, persist the prompt, reserve a budget, return 202, claim the row, call one provider, and test the duplicate key until the call count stays at one. Then, and only then, let the UI poll. If a layer cannot explain its failure with a status code, it is not a layer yet.

Which handoff is least stable in your stack right now, the POST to the queue, the worker to the provider, or the poll back to the UI? Send me a concrete failure state or a response code, not a vibe. If you want a constrained box for the worker URL while you rehearse that path, MonkeyCode's free model access and free server option are sufficient to fail in staging instead of in front of a customer.

Build a Read-Only Eval Slice Before Giving Free Inference Write Authority

kongkong — Sun, 20 Sep 2026 10:35:22 +0000

Last Tuesday a teammate wired "draft a reply" onto the ticket page because inference had become free overnight. A support lead clicked that button on a live customer thread, watched the spinner stall, and then saw the draft publish. Was the model actually wrong, or did we skip the layer that should have kept that draft private? The first failure was not generation quality; it was a write path that treated a guess like a commit.

I keep arguing the same unpopular position on these prototypes, and I will argue it again here. Free inference is a gift for evaluation loops, but it is not permission to skip handoff tests. If your feature can mutate a ticket, an invoice note, or a user record, the model is not the product. The product is the contract between the button click, the provider, the store, and the identity of the actor.

Teams still celebrate a cheap completion the way they once celebrated a green unit test on a mapper. They paste a client call into a route handler, render the string, and call the vertical slice done. Have you noticed how often that slice dies at the first timeout, the first empty body, or the first retry? A contended rehearsal server will show you those failures earlier, which is useful only if you actually look.

So I refuse to give a completion write authority until a read-only eval slice can fail in public. The slice uses the same provider protocol the production route will use, minus the database mutation. Think of it like a dress rehearsal with the lighting board live and the curtain still down. You want the lights to flicker now, not during the matinee when customers are already in their seats.

When I need a cheap place to run that rehearsal, I point the same protocol at MonkeyCode's free model access and free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. That free path is enough if you want to drive this harness, and I still refuse to treat it as launch capacity. The application pins an envelope, a timeout, and a failure vocabulary that any later provider still has to speak.

Pin the envelope, not the vendor

Here is the working path I actually implement, labeled as a local pattern you can run without my private traffic. Start from the user action, which is draft a reply, and stop at the first layer that can lie. In this feature that layer is usually the HTTP client pretending a hung socket is an empty string. I would rather publish this boring contract than another screenshot of a model sounding confident.

# eval_contract.py
from __future__ import annotations

from dataclasses import dataclass
from typing import Protocol


@dataclass(frozen=True)
class CompletionEnvelope:
    request_id: str
    tenant_id: str
    actor_id: str
    purpose: str
    input_sha256: str
    timeout_ms: int


@dataclass(frozen=True)
class CompletionResult:
    request_id: str
    status: str  # ok | empty | timeout | unauthorized | unavailable
    text: str
    http_status: int


class CompletionProvider(Protocol):
    def complete(self, envelope: CompletionEnvelope, prompt: str) -> CompletionResult:
        """Return a typed failure, never a raised surprise, for eval and apply."""
        ...

Why freeze the envelope instead of letting each caller invent headers like a souvenir shop? Because retries, eval reruns, and later provider swaps need a stable identity for the same user action. If request_id and input_sha256 drift, your read-only eval is just another vibe-coded chat window. Would you accept a payment intent that quietly changed its amount between the authorize call and the capture call?

A read-only runner that cannot write

The eval runner loads a fixture, calls the provider, and records status without touching storage. It has no repository handle, which is the entire point of the type boundary. I want a sneaky insert to look ridiculous in review, not clever after an incident. Boring is how you stop a free prototype from becoming an accidental publisher.

# eval_runner.py
from hashlib import sha256
from uuid import uuid4

from eval_contract import CompletionEnvelope, CompletionProvider, CompletionResult


def build_envelope(tenant_id: str, actor_id: str, ticket_body: str) -> CompletionEnvelope:
    digest = sha256(ticket_body.encode("utf-8")).hexdigest()
    return CompletionEnvelope(
        request_id=str(uuid4()),
        tenant_id=tenant_id,
        actor_id=actor_id,
        purpose="ticket_reply_draft",
        input_sha256=digest,
        timeout_ms=8000,
    )


def run_read_only_eval(provider: CompletionProvider, ticket_body: str) -> CompletionResult:
    envelope = build_envelope("tenant_acme", "actor_support", ticket_body)
    prompt = f"Draft a support reply. Do not invent policy. Ticket:\n{ticket_body}"
    result = provider.complete(envelope, prompt)
    # No session.commit(). No ticket.messages.append(). Eval ends here.
    return result

If a colleague asks why this looks like a lot of ceremony for a text box, ask them where the note goes after a 504. Does the UI keep a ghost paragraph? Does the worker retry and post twice? The interesting work is the failure table you run against this runner before anyone wires INSERT.

Map sockets to statuses, not exceptions to hope

I also want an HTTP adapter that cannot raise its way past the eval door. Timeouts, transport errors, and empty JSON bodies should become named statuses the apply gate already understands. This adapter is a proposed local seam, not a vendor SDK tour, and it should stay replaceable without rewriting the ticket page.

# http_provider.py
import httpx
from eval_contract import CompletionEnvelope, CompletionResult


class HttpCompletionProvider:
    def __init__(self, base_url: str, token: str, timeout_s: float = 8.0):
        self.base_url = base_url.rstrip("/")
        self.token = token
        self.timeout_s = timeout_s

    def complete(self, envelope: CompletionEnvelope, prompt: str) -> CompletionResult:
        try:
            response = httpx.post(
                f"{self.base_url}/v1/complete",
                headers={"Authorization": f"Bearer {self.token}"},
                json={
                    "request_id": envelope.request_id,
                    "purpose": envelope.purpose,
                    "prompt": prompt,
                },
                timeout=self.timeout_s,
            )
        except httpx.TimeoutException:
            return CompletionResult(envelope.request_id, "timeout", "", 504)
        except httpx.TransportError:
            return CompletionResult(envelope.request_id, "unavailable", "", 503)

        if response.status_code in (401, 403):
            return CompletionResult(
                envelope.request_id, "unauthorized", "", response.status_code
            )
        if response.status_code >= 500:
            return CompletionResult(
                envelope.request_id, "unavailable", "", response.status_code
            )

        text = ""
        content_type = response.headers.get("content-type", "")
        if content_type.startswith("application/json"):
            payload = response.json()
            text = str(payload.get("text", ""))
        if not str(text).strip():
            return CompletionResult(envelope.request_id, "empty", text, response.status_code)
        return CompletionResult(envelope.request_id, "ok", text, response.status_code)

Point that class at whatever rehearsal base URL you already have, then keep the UI on the eval route until the tests below are green. Do not let the browser supply purpose, tenant_id, or timeout_ms, because those fields are authorization in slow motion. A stolen token with a writable purpose is not a model problem, and swapping vendors will not fix it.

Cross-layer failures, not model-card scores

I do not score eloquence in this slice, because pretty prose is the easiest lie in the stack. I score whether each layer tells the truth when the world gets ugly, which is the part demos skip. The tests below are meant to be executed locally; they are not architecture theater. If your only test is that the model sounded pretty, you are measuring the wrong instrument.

# test_eval_handoffs.py
from eval_contract import CompletionEnvelope, CompletionResult
from eval_runner import run_read_only_eval


class FakeProvider:
    def __init__(self, result: CompletionResult):
        self.result = result
        self.seen = None

    def complete(self, envelope: CompletionEnvelope, prompt: str) -> CompletionResult:
        self.seen = envelope
        return self.result


def test_timeout_does_not_look_like_success():
    provider = FakeProvider(CompletionResult("r1", "timeout", "", 504))
    result = run_read_only_eval(provider, "Refund arrived twice.")
    assert result.status == "timeout"
    assert result.text == ""
    assert result.http_status == 504


def test_empty_completion_is_a_named_failure():
    provider = FakeProvider(CompletionResult("r2", "empty", "   ", 200))
    result = run_read_only_eval(provider, "Need VAT invoice.")
    assert result.status == "empty"


def test_unauthorized_stops_before_any_apply_path():
    provider = FakeProvider(CompletionResult("r3", "unauthorized", "", 401))
    result = run_read_only_eval(provider, "Reset mailbox access.")
    assert result.status == "unauthorized"
    assert result.http_status == 401


def test_envelope_carries_tenant_and_purpose():
    provider = FakeProvider(CompletionResult("r4", "ok", "We can help.", 200))
    run_read_only_eval(provider, "Cannot login after SSO change.")
    assert provider.seen.tenant_id == "tenant_acme"
    assert provider.seen.purpose == "ticket_reply_draft"
    assert len(provider.seen.input_sha256) == 64

Run the file with a command you can paste into CI without decorating it for a blog screenshot.

python -m pytest test_eval_handoffs.py test_apply_gate.py -q

Notice what is missing on purpose. There is no leaderboard, no BLEU footnote, and no chat transcript taped to the pull request. Those artifacts do not tell you whether a 504 became a blank note on a customer timeline. Can a model-card screenshot explain a duplicate send after a retry? I have never seen one that could.

Apply is a second door, with a key

Only after the eval slice is green do I add an apply function, and it demands an idempotency key from the envelope. A rehearsal server that retries behind your back is not evil; an apply path that inserts twice is. The apply door stays locked unless status is ok and a reviewer, or a later automated gate, sets approved=True. That second door is the whole opinion: generation is cheap, mutation is not.

# apply_draft.py
from eval_contract import CompletionEnvelope, CompletionResult


class ApplyRejected(Exception):
    pass


def apply_draft(envelope, result, approved, insert_note) -> str:
    if result.status != "ok" or not result.text.strip():
        raise ApplyRejected(f"refusing apply for status={result.status}")
    if not approved:
        raise ApplyRejected("eval passed, write authority not granted")
    if envelope.purpose != "ticket_reply_draft":
        raise ApplyRejected("purpose mismatch")
    return insert_note(
        tenant_id=envelope.tenant_id,
        actor_id=envelope.actor_id,
        idempotency_key=envelope.request_id,
        body=result.text.strip(),
    )

The insert_note collaborator should hit a unique index on tenant_id plus idempotency_key, not a hopeful comment in the handler. I have watched retry storms from polite clients double-post because the route was probably fine. Was it fine? The second email to the customer answered that question faster than any dashboard.

A tiny persistence test belongs here too, because eval-green and apply-broken is how vibe-coded features ship. If you only test the model client, you will launch a very eloquent way to corrupt a thread.

# test_apply_gate.py
import pytest
from apply_draft import ApplyRejected, apply_draft
from eval_contract import CompletionEnvelope, CompletionResult


def envelope():
    return CompletionEnvelope("req-9", "t1", "u1", "ticket_reply_draft", "a" * 64, 8000)


def test_ok_but_unapproved_does_not_write():
    calls = []
    result = CompletionResult("req-9", "ok", "Thanks, we are looking.", 200)
    with pytest.raises(ApplyRejected):
        apply_draft(
            envelope(),
            result,
            approved=False,
            insert_note=lambda **k: calls.append(k),
        )
    assert calls == []


def test_timeout_never_reaches_insert():
    calls = []
    result = CompletionResult("req-9", "timeout", "", 504)
    with pytest.raises(ApplyRejected):
        apply_draft(
            envelope(),
            result,
            approved=True,
            insert_note=lambda **k: calls.append(k),
        )
    assert calls == []

What this does not buy you

This pattern will not make a weak draft sound like a senior agent, and it will not turn a rehearsal server into a capacity plan. It also will not replace authentication, tenant isolation, or retention rules you already owe the rest of the app. If you ship healthcare advice, money movement, or anything that must be right the first time, do not use a read-only eval as a substitute for a human in the loop.

I would not use this approach on a throwaway weekend toy that never stores state, because the ceremony is heavier than the risk. I also would not let a marketing site call the apply door from the browser with a user-supplied purpose string. The envelope is a server-side object, not a souvenir from the client. Production caveats still apply when the eval backend costs nothing at the register: timeouts bunch up, empty completions arrive with HTTP 200, and copied curl headers rot in silence.

Reuse this order before you celebrate a free path. Pin the envelope. Run the read-only eval. Prove timeout, empty, and 401. Lock apply behind approval plus idempotency. Only then aim a real UI at the write door.

Which layer handoff is least stable in your stack right now, and what concrete failure state or response code does it return when the draft should never have been stored?

Replay the Captured Request Before You Swap the Model

kongkong — Sat, 19 Sep 2026 08:50:55 +0000

The product manager clicked Generate weekly digest and save it, then watched a green toast appear. The network panel showed 201 Created, which should have meant the row existed for that tenant. Support still could not find the digest, because the write landed in a sandbox schema the app never reads. Was the model actually wrong here, or did we just celebrate the wrong layer again?

I do not think the current argument about smarter models is the production bottleneck most teams actually have. The first end-to-end user action already fails at a handoff the prompt never sees. Auth, provider, persistence, and audit each return a status, and we rarely replay all four. If you cannot replay yesterday's captured user request, swapping today's model is still theater.

That sounds harsh, and I mean it as an opinion rather than a gentle suggestion about process. A prototype that streams pretty tokens can hide a 401 on the save path until someone actually clicks persist. It can also hide a 409 when two workers finalize the same digest key after a retry. Why would swapping in a different completion model change any of those status contracts at all?

Here is the working path I want in the repo before anyone files a try another model ticket. Capture one real user request as a fixture, including tenant, idempotency key, and the intended side effect. Replay that fixture through a provider seam, not through a concrete vendor SDK call. If that replay is red, you are debugging the application rather than the prompt.

Think of the model like a guest chef walking into a restaurant you already operate every night. The guest can plate a beautiful special, but the ticket still has to survive the pass, the allergen check, and the kitchen printer. Swapping chefs does not fix a ticket printer that drops the table number on busy Fridays. Your AI feature is that same meal moving across named stations: authenticate, complete, persist, and audit.

I will keep this example small and label it as a proposed slice, not a customer case with invented timings. The user action is create a weekly digest for tenant T and persist it exactly once. The first failing layer in this design is persistence identity, because a 201 that writes the wrong schema is a lie. We should make that lie fail a test instead of failing in a support thread.

The fixture is the artifact, not the chat transcript, because transcripts do not pin tenant identity or idempotency. I want a JSON document that a test can load without talking to a network on the happy path. Proposed contents look like the block below, and they should live next to the test rather than in a prompt file.

{
  "action": "create_weekly_digest",
  "actor_id": "user_82",
  "tenant_id": "tenant_acme",
  "idempotency_key": "digest:tenant_acme:2026-W38",
  "prompt": "Summarize shipped work for the week and persist the digest.",
  "expect": {
    "auth": 204,
    "complete": 200,
    "persist": 201,
    "audit": 202
  }
}

Notice the expect map talks in status codes, not in model brand names or temperature folklore. The week identity sits inside the idempotency key so a retry cannot invent a second row. If your UI retries on a flaky connection, that key is the only thing standing between one digest and a folder of twins. Does your current demo even send that key, or does the browser just hope for luck?

The provider seam exists so a free completion path and a later billed path cannot leak into handlers. I do not want environment branches sprinkled across views like confetti after a noisy launch party. A tiny protocol keeps the handler honest, and the replay can inject a fake completer that never spends a token. Proposed code, not a benchmark of any hosted model, looks like the following sketch.

# proposed slice — illustrative, not measured in production
from typing import Protocol, TypedDict

class Completion(TypedDict):
    text: str
    finish: str

class Completer(Protocol):
    def complete(self, prompt: str, tenant_id: str) -> Completion: ...

class DigestRecord(TypedDict):
    tenant_id: str
    idempotency_key: str
    body: str

class DigestStore(Protocol):
    def create_once(self, record: DigestRecord) -> str: ...
    # returns "created" | "conflict" | "wrong_tenant"

The handler should be boring on purpose, because boring code is what a replay can actually accuse. Auth runs first and can return 401 or 403 before any completion is purchased from a provider. Persist maps store outcomes onto 201, 409, and 422, which is the layer that lied in the opening story. Audit is last and must not be able to change the persist result after the row exists.

def create_digest(req, authz, completer: Completer, store: DigestStore, audit) -> tuple[int, dict]:
    decision = authz.allow(req.actor_id, req.tenant_id, "digest:write")
    if decision == "unauthenticated":
        return 401, {"error": "unauthenticated"}
    if decision == "denied":
        return 403, {"error": "forbidden"}

    completion = completer.complete(req.prompt, req.tenant_id)
    if completion["finish"] != "stop":
        return 422, {"error": "incomplete_completion"}

    outcome = store.create_once({
        "tenant_id": req.tenant_id,
        "idempotency_key": req.idempotency_key,
        "body": completion["text"],
    })
    if outcome == "wrong_tenant":
        return 422, {"error": "tenant_mismatch"}
    if outcome == "conflict":
        return 409, {"error": "digest_exists", "key": req.idempotency_key}

    audit.emit("digest.created", req.tenant_id, req.idempotency_key)
    return 201, {"key": req.idempotency_key, "tenant_id": req.tenant_id}

Would I put streaming tokens into this handler on day one of the feature work? No, because streaming makes people stare at language while the persist path stays untested. Language is only the garnish on this plate, and the contract is the plate itself. Ship those statuses first, then add streaming only if the contract still holds under a retry.

The replay test is where the opinion becomes executable, and I want it to fail on the opening bug. A fake store that writes sandbox while the request says tenant_acme must not be allowed to return 201. A second call with the same key must return 409, not another green celebration toast. Proposed pytest below is the whole point of this article, not a decorative appendix.

# tests/test_digest_replay.py — proposed harness
import json
from pathlib import Path
from types import SimpleNamespace

FIXTURE = json.loads(Path("tests/fixtures/create_weekly_digest.json").read_text())

class FakeAuth:
    def __init__(self, decision): self.decision = decision
    def allow(self, actor_id, tenant_id, perm): return self.decision

class FakeCompleter:
    def complete(self, prompt, tenant_id):
        return {"text": f"digest for {tenant_id}", "finish": "stop"}

class FakeStore:
    def __init__(self, outcome): self.outcome = outcome
    def create_once(self, record):
        if record["tenant_id"] != "tenant_acme":
            return "wrong_tenant"
        return self.outcome

class FakeAudit:
    def __init__(self): self.events = []
    def emit(self, *args): self.events.append(args)

def replay(auth_decision, store_outcome):
    req = SimpleNamespace(**{k: FIXTURE[k] for k in ("actor_id", "tenant_id", "idempotency_key", "prompt")})
    return create_digest(req, FakeAuth(auth_decision), FakeCompleter(), FakeStore(store_outcome), FakeAudit())

def test_happy_path_matches_fixture_expect():
    status, body = replay("allow", "created")
    assert status == FIXTURE["expect"]["persist"]
    assert body["tenant_id"] == "tenant_acme"

def test_sandbox_write_is_not_a_201():
    status, body = replay("allow", "wrong_tenant")
    assert status == 422
    assert body["error"] == "tenant_mismatch"

def test_retry_does_not_duplicate_digest():
    status, _ = replay("allow", "conflict")
    assert status == 409

def test_save_is_blocked_when_session_is_cold():
    status, _ = replay("unauthenticated", "created")
    assert status == 401

Run those tests like a skeptic, not like a demo narrator looking for a round of applause.

pytest tests/test_digest_replay.py -v

When test_sandbox_write_is_not_a_201 fails, you have found the opening incident without paging support. When test_retry_does_not_duplicate_digest fails, your UI retry is about to double-file the week. Only after those assertions stay green would I discuss swapping the completer, because until then the model is the wrong suspect.

What usually fails along the way is not the fake completer, which is almost too easy to write quickly. The authz adapter silently uses a default tenant when the session cookie is stale, and every replay looks like a 201. The store uses INSERT without a unique constraint on the idempotency key, so the conflict path never exists in production. The audit call sits inside the persist transaction in staging and outside it in production, so a 201 can vanish after a crash.

Production caveats are mostly about honesty at the edges you cannot fake with a stub. A fixture cannot prove the real provider will finish cleanly under load, so keep a live canary against a non-prod tenant. Idempotency keys must be unique in the real table, not only in the in-memory stub, or the 409 test is fiction. Cross-region retries need the same key in the header, or you will mint twins the replay never saw.

Who should not bother with this approach yet, even if the demo looks convincing on a laptop? If you are hacking a local toy with no tenants, no persistence, and no user besides yourself, the harness is overhead. If your product cannot name the write permission yet, fix authorization before you capture fixtures, because a completer 200 is not a grant. If you need subsecond token streaming as the actual product, this slice will feel too strict, and that discomfort is the point.

Before I would merge the feature, I want a short proof I can reuse the following week without a demo.

Captured request names tenant, actor, and idempotency key.
Handler returns 401, 403, 409, 422, and 201 without a vendor SDK.
Sandbox writes cannot return 201, and retries cannot create a second row.
Non-prod canary replays the same expect map and records four status codes.

I keep seeing teams argue about model quality the way kitchens argue about salt while the ticket printer is on fire. Can you replay last week's user request today and get the same four status codes back from each layer? Tell me which layer handoff is least stable in your stack, and send a concrete failure state or response code. A model name is not an incident report, and I will not treat it as one.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode provides free models that can run this workflow. A free server option is enough to reproduce the setup.

A Free-Server Demo Is Not a Shipped Feature

kongkong — Fri, 18 Sep 2026 06:54:17 +0000

I watched a teammate click Export last month on a prototype that lived on a free server. The model had drafted the worker, the UI painted a green check, and Slack filled with celebration. Finance opened the same path from a real account twenty minutes later and caught a 401 before any file existed. Have you ever seen a demo succeed on a scratch box and the next hop die like that?

I do not think free models are the scandal, and I do not think free servers are either. The scandal is calling that happy path engineering evidence when it never crossed a handoff. The first layer that fails is almost never the model completion sitting in the chat log. It is identity, persistence, retry, and the boring contract that says which machine may write.

People are arguing this week about vibe coding versus real engineering, as if the insult lives in the autocomplete. I think the insult lives in the missing promotion test that nobody bothered to run. If the only proof you have is a chat session on a scratch box, you have a draft, not a feature. Why would a green button on an anonymous local user tell you anything about production?

Here is the opinion I will defend, without hedging, for the rest of this piece. A free environment is a workshop, and a workshop is allowed to stay messy while you learn. The moment you show that workshop to a user, you owe them a replay of one real action against auth, storage, and retries. Without that replay, you are performing confidence for the team, not delivering a vertical slice.

I started treating every free-server prototype as guilty until a small promotion probe actually passed. The probe is not a platform or a framework, and it should not grow into one. It is a recorded request, a tiny adapter, and three assertions that fail loudly on purpose. You freeze one user action as JSON and refuse to call the work shipped until the production-shaped path returns the same contract.

The action I use is deliberately boring: create an export job for the current billing period. The UI posts JSON, the API enqueues work, and the worker writes a row plus an object key. If any of those hops are fake on the free box, the probe should fail before a human does. If that sounds too small to count as engineering, that smallness is the entire argument I am making.

This is a proposed probe you can paste, not a benchmark and not a claim about any particular model. It does not score the draft. It asks whether the first user action still exists after you stop pretending the workshop user is real.

# probe/test_promotion_export.py
import json
import os
from pathlib import Path

import httpx

FROZEN = Path(__file__).parent / "frozen_export_request.json"
PROD_BASE = os.environ["PROD_SHAPED_BASE_URL"]
PROBE_TOKEN = os.environ["PROBE_BEARER_TOKEN"]

def load_frozen():
    return json.loads(FROZEN.read_text())

def test_export_job_survives_the_handoff():
    payload = load_frozen()
    headers = {
        "Authorization": f"Bearer {PROBE_TOKEN}",
        "Idempotency-Key": payload["idempotency_key"],
        "Content-Type": "application/json",
    }
    with httpx.Client(timeout=20.0) as client:
        first = client.post(f"{PROD_BASE}/v1/exports", json=payload, headers=headers)
        retry = client.post(f"{PROD_BASE}/v1/exports", json=payload, headers=headers)

    assert first.status_code in (200, 201), first.text
    body = first.json()
    assert body["status"] in {"queued", "succeeded"}
    assert body.get("export_id"), "export_id vanished after the auth handoff"
    assert "download_url" not in body or body["download_url"].startswith("https://")
    assert retry.status_code in (200, 201)
    assert retry.json()["export_id"] == body["export_id"], "retry minted a second job"

I freeze the request next to the test so the argument cannot drift into prompt folklore. The file is the user action, and the user action is the only evidence I will accept.

{
  "period": "2026-08",
  "format": "csv",
  "idempotency_key": "exp_2026-08_probe_01"
}

On the free box I still let a model draft the handler, because speed in the workshop is not a moral failure. I do not let that handler talk to production credentials from a chat window. I put a seam in front of identity, jobs, and object storage so the same function can run against a stub and against staging-shaped adapters. Have you noticed how many vibe-coded handlers hide a global USER_ID = "demo" right next to the business logic?

# app/exports/seam.py
from dataclasses import dataclass
from typing import Protocol

class Identity(Protocol):
    def user_id(self) -> str: ...
    def can_export(self, period: str) -> bool: ...

class Objects(Protocol):
    def put_csv(self, key: str, body: bytes) -> str: ...

class Jobs(Protocol):
    def create_or_get(self, user_id: str, period: str, idempotency_key: str) -> dict: ...

@dataclass
class ExportRequest:
    period: str
    format: str
    idempotency_key: str

def create_export(req: ExportRequest, identity: Identity, jobs: Jobs, objects: Objects) -> dict:
    if not identity.can_export(req.period):
        return {"error": "forbidden", "status": 403}
    job = jobs.create_or_get(identity.user_id(), req.period, req.idempotency_key)
    if job.get("status") == "queued":
        return {"export_id": job["id"], "status": "queued"}
    key = f"exports/{identity.user_id()}/{req.period}.csv"
    url = objects.put_csv(key, job["csv_bytes"])
    return {
        "export_id": job["id"],
        "status": "succeeded",
        "object_key": key,
        "download_url": url,
    }

Notice what this function refuses to do, because refusals are the actual architecture. It does not call a model, and it does not read leftover environment from a free server. It takes protocols, and the probe supplies the production-shaped implementations. When the workshop version authenticated as a hardcoded local user, the seam made that lie visible instead of painting another green check.

The workshop fake is useful, and it is also the first place I expect the story to cheat. I keep it ugly on purpose so nobody confuses it with staging.

# probe/fakes.py
class WorkshopIdentity:
    def user_id(self) -> str:
        return "demo"

    def can_export(self, period: str) -> bool:
        return True  # this lie is why the workshop demo always looks green

I run the workshop against in-memory fakes, then I point the same probe at a staging-shaped base URL. The command line is the promotion ceremony, not a screenshot in Slack, and not a chat transcript that says the worker looks right.

# workshop: prove the handler exists at all
PROBE_BEARER_TOKEN=workshop-local \
PROD_SHAPED_BASE_URL=http://127.0.0.1:8000 \
pytest probe/test_promotion_export.py -q

# staging-shaped: prove identity and retries survive the hop
PROBE_BEARER_TOKEN="$STAGING_PROBE_TOKEN" \
PROD_SHAPED_BASE_URL="$STAGING_BASE_URL" \
pytest probe/test_promotion_export.py -q

If the first command passes and the second command 401s, you do not have a product. You have a skit with good lighting. A missing Authorization header is the most common leak I see when a free-server habit travels intact into staging, and curl makes that failure impossible to narrate away.

curl -i -X POST "$STAGING_BASE_URL/v1/exports" \
  -H "Content-Type: application/json" \
  -d '{"period":"2026-08","format":"csv"}'
# expect 401 when the workshop habit of skipping Authorization leaks into staging

If both pytest commands pass but a double click creates two export rows, you still do not have a product. Idempotency is part of the user action, not a later optimization you will remember after the invoice file duplicates itself. Would you accept a payment API that minted a second charge because the browser retried? Then why accept it from an export job a model drafted on a free box?

This is where a free coding environment actually helps, if you keep it in its lane. I will draft handlers faster on a scratch machine with free model access than I will on a locked-down cluster, and that speed is legitimate workshop behavior. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode is an open-source project that currently offers free model access and a free server option, which is enough workshop to build the slice and the probe without pretending the workshop is staging. If you want a scratch place to assemble that vertical slice, it is one workshop you can try, then you should immediately aim the probe at something that looks like production.

I do not need the workshop to be eternal, large, or impressive on a slide. I need it to be disposable. The promotion probe is what I keep when the free box goes away, because the recorded request outlives the machine that first made it look easy. The model can rewrite the handler tomorrow. The frozen JSON and the 401 assertion should not rewrite themselves to match the new vibe.

Limitations matter, because this opinion is easy to oversell into a religion. A promotion probe does not evaluate model quality, latency, cost, or whether the CSV is even correct. It will not catch a biased export, a leaked tenant in a file, or a worker that dies after the HTTP 201. It assumes you already know the user action, which many early prototypes simply do not. If you are hacking a throwaway animation for yourself, you should not add this ceremony and then congratulate the process.

Anyone shipping regulated data through a scratch server should not use this as cover. Anyone who cannot name the user action in one sentence should not. Anyone hoping a free environment will replace staging, secrets management, or on-call should not, because a furnace for drafts is not a blast door. I will keep using free workshops. I will not let them testify in the shipping meeting.

Production caveats I keep repeating to myself when the demo looks too clean. Rotate the probe token independently from ordinary user sessions, or the probe becomes another shared password. Never let the workshop process hold production object-store credentials, even for a minute of convenience. Record the status codes you actually got, because it worked on my free box is not a response code you can grep later. If the model rewrites the handler, freeze the contract first and treat the diff as untrusted until the probe is green again.

I still want a short gate readers can paste, even though I dislike ritual lists dressed up as strategy. Call it a closed door, not a vibe, and keep it next to the frozen request.

promotion gate
[ ] one user action frozen as JSON
[ ] identity is a real bearer, not a demo user
[ ] create is idempotent under retry
[ ] persistence is a row plus an object key, not a chat blob
[ ] workshop credentials cannot write to production
[ ] probe fails closed on 401, 403, and duplicate ids

My position, again, without softening it for the current argument on social feeds. A free model can draft the worker by lunch, and that still does not decide whether a signed-in retry creates one job or two. The probe decides. Everything else is theater with a green button.

Which layer handoff is least stable in your app right now, and what status code does it actually return when a signed-in user retries the same click?

Add Schema Writes Behind an Apply Journal, Not a Chat Window

kongkong — Thu, 17 Sep 2026 04:58:51 +0000

Last Friday a product manager pinged me before standup and asked for a refund reason code. I watched a coding agent dump an ALTER TABLE statement into the pull request like it already owned production. Staging looked green, because that database sits alone with nobody racing the migration during lunch. Have you ever shipped a chat-generated schema change and felt your stomach drop in the deploy window?

The public argument this week pretends the crisis is whether models already code better than most working developers. That framing is a distraction from the layer that actually fails when a user request hits a live stack. I think vibe coding is fine for sketches, and calling those sketches engineering is how outages get a polite commit hash. The first broken handoff is not the model; it is schema authority leaking into a chat window.

I am taking a blunt position here, and I will not sand it down for comfort. A free coding loop is a prototype furnace, not a migration process, and treating those as the same job is how you burn a weekend. If the user action is “add this field before Friday traffic,” the first layer that fails is persistence under concurrency, not the quality of the generated SQL. Why do we keep measuring the chat when the journal is empty?

Picture the chat like a loading dock with no bills of lading. Anyone can wheel a crate onto the truck, and the driver still leaves on time. Your production database is that truck, and an unsourced ALTER is a crate with no seal, no destination, and no way to prove it was already applied. I want the model to propose a crate label. I do not want the model to start the engine.

The working path I keep reaching for is small and slightly boring, which is the point. Freeze a schema snapshot, ask the model only for a plan file, validate that plan against the snapshot, and apply through an idempotent journal with a human or pipeline as the only writer. The chat never receives database credentials, and the apply job never receives a prose paragraph. If that sounds slower than pasting SQL, ask yourself what you will replay at two in the morning.

Here is a proposed contract I would drop into a service repo before any model is allowed near write authority. It is illustrative, not a claim about a specific production outage, and you should treat the snippet as a starting shape rather than a blessed library.

# apply_journal.py — proposed shape, not a published package
from __future__ import annotations

import hashlib
import json
from dataclasses import dataclass
from datetime import datetime, timezone
from typing import Literal

PlanOp = Literal["add_column"]

@dataclass(frozen=True)
class ColumnPlan:
    table: str
    name: str
    sql_type: str
    nullable: bool

@dataclass(frozen=True)
class SchemaPlan:
    plan_id: str
    op: PlanOp
    column: ColumnPlan
    based_on_schema_sha: str

    def digest(self) -> str:
        payload = json.dumps({
            "plan_id": self.plan_id,
            "op": self.op,
            "table": self.column.table,
            "name": self.column.name,
            "sql_type": self.column.sql_type,
            "nullable": self.column.nullable,
            "based_on_schema_sha": self.based_on_schema_sha,
        }, sort_keys=True).encode()
        return hashlib.sha256(payload).hexdigest()

ALLOWED_TYPES = {"TEXT", "INTEGER", "BOOLEAN", "TIMESTAMPTZ"}
ALLOWED_TABLES = {"refunds", "refund_events"}

class PlanRejected(Exception):
    pass

def validate_plan(plan: SchemaPlan, live_schema_sha: str, existing_columns: set[str]) -> None:
    if plan.based_on_schema_sha != live_schema_sha:
        raise PlanRejected("plan_stale_schema")
    if plan.op != "add_column":
        raise PlanRejected("op_not_allowed")
    if plan.column.table not in ALLOWED_TABLES:
        raise PlanRejected("table_not_in_contract")
    if plan.column.sql_type not in ALLOWED_TYPES:
        raise PlanRejected("type_not_in_contract")
    if plan.column.name in existing_columns:
        return  # idempotent no-op, not an error
    if not plan.column.name.isidentifier() or plan.column.name.startswith("_"):
        raise PlanRejected("name_not_safe")

Notice what the model is not doing in that file. It is not choosing a table outside the allowlist, and it is not inventing a vendor type that your ORM cannot round-trip. It is not concatenating SQL from a paragraph that sounded confident in the chat. Would you let a contractor rewire your kitchen from a voice note, or would you make them mark the joists first?

The apply side is where teams usually get lazy, because the happy path looks like a single statement. I want a journal row that records the plan digest, the actor, and a terminal state the job can safely retry. Network retries will happen. Process crashes will happen. Chat windows will resubmit the same “yes, apply it” because somebody double-clicked. If your apply is not keyed by digest, you do not have a workflow. You have a vibe.

# journal_apply.py — proposed apply path
class JournalConflict(Exception):
    pass

TERMINAL = {"applied", "noop"}

def apply_add_column(conn, plan: SchemaPlan, actor: str) -> str:
    digest = plan.digest()
    existing = conn.fetch_journal(digest)
    if existing and existing["state"] in TERMINAL:
        return existing["state"]
    if existing and existing["state"] == "applying":
        raise JournalConflict("apply_in_flight")

    conn.insert_journal({
        "digest": digest,
        "plan_id": plan.plan_id,
        "actor": actor,
        "state": "applying",
        "started_at": datetime.now(timezone.utc).isoformat(),
    })
    cols = conn.list_columns(plan.column.table)
    if plan.column.name in cols:
        conn.finish_journal(digest, "noop")
        return "noop"

    null_sql = "NULL" if plan.column.nullable else "NOT NULL"
    # Identifier already validated; still parameterize nothing into names.
    conn.execute(
        f'ALTER TABLE {plan.column.table} '
        f'ADD COLUMN {plan.column.name} {plan.column.sql_type} {null_sql}'
    )
    conn.finish_journal(digest, "applied")
    return "applied"

That f-string still makes me itch, which is useful discomfort. Column names cannot be bound as values in most SQL dialects, so the real control is the validator, not a prettier driver call. If you skip validation and keep the f-string, you did not build a contract. You built a footgun with logging. Can you point to the exact function that would refuse a plan for refunds; DROP TABLE on a tired Friday?

I also want a cross-layer failure test that does not require a model in the loop, because the model is the noisiest part of the system. Freeze a schema SHA, submit a plan built on a different SHA, and assert the API answers 409 with plan_stale_schema. Submit the same digest twice and assert the second call returns noop without a second ALTER. Submit a second apply while the first is applying and assert 409 with apply_in_flight. If those three states are not tests, they will become incident comments.

# test_apply_journal.py — proposed pytest sketch
def test_stale_plan_is_rejected(client, live_schema_sha):
    plan = {
        "plan_id": "refund-reason-code-1",
        "op": "add_column",
        "column": {
            "table": "refunds",
            "name": "reason_code",
            "sql_type": "TEXT",
            "nullable": True,
        },
        "based_on_schema_sha": "deadbeef",
    }
    res = client.post("/schema/plans/validate", json=plan)
    assert res.status_code == 409
    assert res.json()["code"] == "plan_stale_schema"

def test_replay_does_not_alter_twice(conn, valid_plan):
    first = apply_add_column(conn, valid_plan, actor="pipeline")
    second = apply_add_column(conn, valid_plan, actor="pipeline")
    assert first in {"applied", "noop"}
    assert second == "noop"
    assert conn.list_columns("refunds").count("reason_code") == 1

Where does a free coding environment belong in this story, if it belongs at all? I will use one for the propose step on a scratch server, the same way I use a whiteboard before I touch a load-bearing wall. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode is an open-source project with free model access and a free server option, which is enough to generate plan files and run the validator without parking the model on the apply path. That is the whole relevant seam. The product does not become the journal, and the journal does not become a pitch.

I still would not let that free loop hold production credentials, and I would not let it mark a journal row applied. The cheap environment is for proposing and failing closed. The expensive environment is for the pipeline identity that already has change windows, backups, and a human who can say no. If your team cannot describe that split in one sentence, you are not productionizing a prototype. You are decorating a risk.

This approach has limits, and they are not subtle. Do not use it as a substitute for a real migrator if you already have expand-and-contract discipline, strong review, and lock-step deploys that you trust. Do not use it on regulated data if your journal would store payloads you are not allowed to retain. Do not use it if you need multi-statement migrations, check constraints, or backfills that touch existing rows, because this slice only adds nullable columns on an allowlist. People chasing a fully autonomous schema agent should stop here and keep the model read-only.

Cost and maintainability sit on the same hinge. A plan file is cheap to store and cheap to diff, while a chat transcript is expensive to audit and almost impossible to replay. A journal row gives you a unique digest you can show in an incident channel without quoting seven screens of model rambling. The tradeoff is obvious to me: you give up the feeling of speed so you can keep the feeling of sleep.

If you want a reusable pass before you grant write authority, walk the path once on a throwaway database. Snapshot the schema, emit a plan, reject a stale SHA, apply once, replay once, and only then wire a UI button that never talks to the model. Keep the button talking to the journal. If you try this on a scratch box, including a free MonkeyCode server if you need a sandbox, send me the failure state you actually got, not a vibe.

Which layer handoff is least stable in your stack when a “simple field” request shows up on Friday? I want a concrete response code or journal state, not a speech about how good the model felt that afternoon.

Add Completions Behind a Provider Contract, Not Across Your Codebase

kongkong — Wed, 16 Sep 2026 03:15:48 +0000

I keep reconstructing the same staging incident, and it always starts with one hopeful click. A receiving clerk taps Explain this short shipment and waits for an exception note that never lands. The model already answered in the console, yet the API never wrote a row. The handler had mixed authentication, JSON parsing, and a vendor SDK into a single breath.

People keep arguing whether AI already codes better than most developers, and I think that question is a distraction. A completion that cannot survive a provider swap, a 401, and an idempotent apply is not a product feature. It is a demo glued to a vendor, and demos rot the moment the network retries or the schema drifts. So I am taking a blunt position: freeze a provider contract before you let the model near production credentials.

Why does the click fail at persistence instead of at the prompt? Because the first layer that must tell the truth is not the model. It is the handoff between a structured plan and a permissioned apply. If that seam is missing, every retry becomes a second author, and every schema tweak becomes a silent 500. I would rather ship a read-only plan that looks boring than a write path that looks magical.

Think of the model as a contractor who drafts a work order, not as a night-shift DBA with root. The contractor can be swapped. The work order cannot be vague. Your application owns identity, idempotency, and the actual insert, which means a free model and a disposable server are useful only when they exercise that same contract. They are not a shortcut around auth, storage, or deployment.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I am using MonkeyCode here as a rehearsal bench with free model access and a free server option, not as a substitute for production ownership. The walkthrough below is a proposed vertical slice, not a claim about live traffic, quotas, or hardware. If a number is not in a test assertion, treat it as unlabeled.

Here is the end-to-end action I want frozen. A signed-in receiver posts a short-shipment event. The API asks the provider for a plan that matches a schema. Only after validation does a separate apply path insert one exception note keyed by a client idempotency token. If the provider coughs, times out, or returns poetry, the row is not created. That is the whole product, and the model is the least interesting part of it.

I start with the contract, not the SDK, because SDKs love to leak into route handlers. The request carries tenant identity, the receiving event identifier, and a frozen JSON schema version. The result is either a valid plan or a typed failure the UI can render without guessing. Notice there is no vendor client in this module. That absence is the architecture.

# proposed contract — not a vendor tutorial
from typing import Literal, Protocol
from pydantic import BaseModel, Field, conint, constr

class ShortShipmentEvent(BaseModel):
    tenant_id: constr(min_length=8, max_length=64)
    receiving_event_id: constr(min_length=8, max_length=64)
    sku: constr(min_length=1, max_length=64)
    expected_qty: conint(ge=1)
    received_qty: conint(ge=0)
    schema_version: Literal["exception_note.v1"] = "exception_note.v1"

class ExceptionNotePlan(BaseModel):
    summary: constr(min_length=12, max_length=240)
    likely_cause: Literal["carrier_short", "pick_error", "count_mismatch", "unknown"]
    follow_up: Literal["hold_dock", "message_vendor", "recount", "no_action"]
    confidence: conint(ge=0, le=100)

class ProviderFailure(BaseModel):
    code: Literal["timeout", "invalid_json", "schema_mismatch", "refused"]
    retryable: bool

class CompletionRequest(BaseModel):
    event: ShortShipmentEvent
    idempotency_key: constr(min_length=16, max_length=128)

class CompletionResult(BaseModel):
    status: Literal["planned", "failed"]
    plan: ExceptionNotePlan | None = None
    failure: ProviderFailure | None = None

class ModelProvider(Protocol):
    def complete(self, request: CompletionRequest) -> CompletionResult: ...

The apply path is deliberately dull, which is the point. Dull code is what you want touching Postgres after a model has been creative. I keep the provider call outside the transaction so a hung completion cannot hold a row lock. I also refuse to apply when the plan is missing, even if the HTTP client is impatient. Have you noticed how many vibe-coded handlers treat a 200 from a model as permission to write?

# proposed apply service
class ExceptionNote(BaseModel):
    tenant_id: str
    receiving_event_id: str
    idempotency_key: str
    plan: ExceptionNotePlan
    provider_name: str
    schema_version: str

class NoteStore(Protocol):
    def get_by_idempotency(self, tenant_id: str, key: str) -> ExceptionNote | None: ...
    def insert_if_absent(self, note: ExceptionNote) -> ExceptionNote: ...

class AuthContext(BaseModel):
    tenant_id: str
    actor_id: str
    can_write_exceptions: bool

def apply_exception_note(
    auth: AuthContext,
    request: CompletionRequest,
    result: CompletionResult,
    store: NoteStore,
    provider_name: str,
) -> tuple[int, dict]:
    if not auth.can_write_exceptions or auth.tenant_id != request.event.tenant_id:
        return 403, {"error": "forbidden"}
    existing = store.get_by_idempotency(auth.tenant_id, request.idempotency_key)
    if existing:
        return 200, existing.model_dump()
    if result.status != "planned" or result.plan is None:
        code = result.failure.code if result.failure else "invalid_json"
        http = 503 if result.failure and result.failure.retryable else 422
        return http, {"error": code}
    note = ExceptionNote(
        tenant_id=auth.tenant_id,
        receiving_event_id=request.event.receiving_event_id,
        idempotency_key=request.idempotency_key,
        plan=result.plan,
        provider_name=provider_name,
        schema_version=request.event.schema_version,
    )
    stored = store.insert_if_absent(note)
    return 201, stored.model_dump()

I rehearse that path on a disposable server before any production secret exists, because production is a terrible scratchpad. Point the same contract at a fake provider in unit tests, then at whatever free model access you have on a free server, and refuse to change the handler. If the free model returns looser prose than your paid one, that is a gift. Schema validation should fail loudly instead of the UI inventing a follow-up action.

# proposed tests — run these before any write credential exists
class FakeProvider:
    def __init__(self, result: CompletionResult):
        self.result = result
        self.calls = 0
    def complete(self, request: CompletionRequest) -> CompletionResult:
        self.calls += 1
        return self.result

class MemoryStore:
    def __init__(self):
        self.rows = {}
    def get_by_idempotency(self, tenant_id, key):
        return self.rows.get((tenant_id, key))
    def insert_if_absent(self, note):
        slot = (note.tenant_id, note.idempotency_key)
        if slot in self.rows:
            return self.rows[slot]
        self.rows[slot] = note
        return note

def test_retry_does_not_double_write():
    plan = ExceptionNotePlan(
        summary="Two cartons missing against the ASN for SKU A-19.",
        likely_cause="carrier_short",
        follow_up="message_vendor",
        confidence=74,
    )
    provider = FakeProvider(CompletionResult(status="planned", plan=plan))
    store = MemoryStore()
    auth = AuthContext(tenant_id="tenant_live", actor_id="recv_9", can_write_exceptions=True)
    req = CompletionRequest(
        event=ShortShipmentEvent(
            tenant_id="tenant_live",
            receiving_event_id="recv_evt_01",
            sku="A-19",
            expected_qty=12,
            received_qty=10,
        ),
        idempotency_key="idem_short_ship_01xx",
    )
    first = apply_exception_note(auth, req, provider.complete(req), store, "rehearsal")
    second = apply_exception_note(auth, req, provider.complete(req), store, "rehearsal")
    assert first[0] == 201
    assert second[0] == 200
    assert len(store.rows) == 1

def test_timeout_does_not_insert():
    failure = ProviderFailure(code="timeout", retryable=True)
    result = CompletionResult(status="failed", failure=failure)
    store = MemoryStore()
    auth = AuthContext(tenant_id="tenant_live", actor_id="recv_9", can_write_exceptions=True)
    req = CompletionRequest(
        event=ShortShipmentEvent(
            tenant_id="tenant_live",
            receiving_event_id="recv_evt_02",
            sku="B-04",
            expected_qty=8,
            received_qty=3,
        ),
        idempotency_key="idem_short_ship_02yy",
    )
    status, body = apply_exception_note(auth, req, result, store, "rehearsal")
    assert status == 503
    assert body["error"] == "timeout"
    assert store.rows == {}

On the rehearsal box I want HTTP proof, not a screenshot of a chat transcript. The commands below are the working path from setup to a visible result, assuming the API is listening locally after you deploy the same contract. Swap the host when the free server is up. Do not swap the headers, the schema version, or the idempotency key behavior.

# proposed rehearsal — same contract, disposable host
export HOST=http://127.0.0.1:8080
export TOKEN=staging-receiver-token

curl -sS -D - -o /tmp/plan.json \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: idem_short_ship_01xx" \
  -d '{
    "tenant_id": "tenant_live",
    "receiving_event_id": "recv_evt_01",
    "sku": "A-19",
    "expected_qty": 12,
    "received_qty": 10,
    "schema_version": "exception_note.v1"
  }' \
  "$HOST/v1/exception-notes:plan"

curl -sS -D - \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: idem_short_ship_01xx" \
  -d @/tmp/plan.json \
  "$HOST/v1/exception-notes:apply"

# cross-layer failures I actually care about
curl -sS -D - "$HOST/v1/exception-notes:apply"   # expect 401
curl -sS -D - -H "Authorization: Bearer $TOKEN" \
  -H "Idempotency-Key: idem_short_ship_01xx" \
  -d @/tmp/plan.json "$HOST/v1/exception-notes:apply"  # expect 200, not 201

The production caveats are not decorative. A free model will drift, a free server will vanish, and neither fact excuses a missing unique index on (tenant_id, idempotency_key). Provider names belong in the stored row so you can audit which seam produced a plan, but they must not leak into authorization. If you cannot replay a failing request against a fake provider, you cannot debug a customer incident without spending money and hoping the model rhymes again.

Who should not use this approach? Anyone who needs the model to execute SQL, move inventory, or send vendor email inside the completion itself. Anyone whose rehearsal data is real personal information, because a disposable server is still a server. Anyone who cannot freeze exception_note.v1 long enough to write the tests above. If your team is still shopping for a model that "just understands the warehouse," you are bargaining with prose when you needed a schema.

I also would not use a rehearsal box as a hidden production. No cron that bills customers. No shared admin token. No "we will add permissions later" comment sitting next to insert_if_absent. Later is how vibe-coded features become incident reports. The contract is the product, and the model is a plugin that has to fail closed.

Reusable checklist, kept short on purpose: freeze the schema before the prompt; keep the SDK behind ModelProvider; plan outside the transaction; apply only with auth and idempotency; map timeout to 503 without a row; replay the same key until you see 200. If any item feels optional, the feature is still a demo. Shipping the demo anyway is how teams confuse fluency with delivery.

If you want a cheap place to prove that seam, MonkeyCode's free model access and free server option are enough to run the rehearsal I sketched. They will not invent your unique index or your 403. That work stays yours, which is the whole argument.

Which layer handoff is least stable in your stack right now, and what status code does it emit when the provider times out after the UI has already shown success?

Your Coding Agent Is Only as Good as the Machine It Can Wreck

kongkong — Tue, 15 Sep 2026 03:05:45 +0000

The patch worked everywhere except on a clean machine

Last month an agent closed a failing test for me in about ninety seconds, and for a moment I felt genuinely clever about my tooling. Then I cloned the same branch into an empty container, and the suite died before it printed a single line of output. Was the model somehow slower or dumber on the second machine? No. The room was different, and the room had been doing a lot of quiet work for me.

The patch leaned on a virtualenv that existed only on my laptop, a Postgres container I had started three weeks earlier and forgotten about, and a .env file that never made it into version control. The agent answered my request precisely. I had simply made the request inside a room where all the furniture was already arranged, and portability was never part of what I asked for.

Nothing in that output was wrong, and nothing in it was portable either. That gap is what I want to argue about, because I think most teams are measuring the wrong component.

The position: grade the room, not the prompt

Almost every discussion about AI coding quality circles the same question — is the model good enough yet? I think that question has quietly become the less interesting one, because on the work I actually ship, the model is rarely the failing part. The failing part is the handoff: the moment work leaves a machine where everything was already warm and lands on a machine where nothing is.

If you accept that framing, the useful measurement becomes almost brutally simple. Hand the agent a clean machine, give it a repository URL and nothing else, and observe which step dies first. I call this a cold-clone survival test, and it takes an afternoon to build, which is roughly the same afternoon you would otherwise spend reading benchmark leaderboards.

And yes, this is an opinion, so let me say it plainly: most teams should stop shopping for a better model and start buying themselves a disposable machine. Model improvements lift every task a little. A reproducible environment fixes the specific task that keeps failing, and it keeps working after the next model release resets your intuition.

The harness I run before I trust any agent patch

The driver below runs inside a fresh container, records each step as a tab-separated row, and never short-circuits on failure — that last property matters, because a suite that stops at the first error hides how many things were actually broken. Note the --no-cache-dir, the throwaway virtualenv, and the deliberately tiny default smoke command.

#!/usr/bin/env bash
# coldstart.sh — run this ONLY inside a fresh, disposable container.
set -uo pipefail

: "${REPO_URL:?export REPO_URL=git@github.com:you/app.git}"
WORK="$(mktemp -d)"
REPORT="${REPORT:-$WORK/steps.tsv}"
STEP_TIMEOUT="${STEP_TIMEOUT:-900}"
SMOKE_CMD="${SMOKE_CMD:-.venv/bin/python -c 'import app; print(app.__name__)'}"

run_step() {
  local name="$1" cmd="$2" log="${WORK}/$1.log"
  timeout "$STEP_TIMEOUT" bash -lc "$cmd" >"$log" 2>&1
  printf '%s\t%s\t%s\n' "$name" "$?" "$log" >>"$REPORT"
}

run_step clone   "git clone --depth 1 '$REPO_URL' '$WORK/app'"
run_step install "cd '$WORK/app' && python3 -m venv .venv && .venv/bin/pip install --no-cache-dir -r requirements.txt"
run_step test    "cd '$WORK/app' && .venv/bin/python -m pytest -q"
run_step smoke   "cd '$WORK/app' && $SMOKE_CMD"

python3 "$(dirname "$0")/score.py" "$REPORT"

The scorer is fail-closed on purpose: a missing step is not a pass, it is a blocked run, and the distinction saves you from celebrating a suite that never executed.

#!/usr/bin/env python3
"""score.py — fail-closed verdict for a cold-clone survival run."""
import pathlib, sys

REQUIRED = ("clone", "install", "test", "smoke")
EXIT = {"PASS": 0, "FAIL": 1, "BLOCKED": 2}

def read_report(path):
    rows = {}
    for line in pathlib.Path(path).read_text().splitlines():
        step, code, log = line.split("\t")
        rows[step] = {"exit": int(code), "log": log}
    return rows

def main(path):
    rows = read_report(path)
    missing = [s for s in REQUIRED if s not in rows]
    if missing:
        print(f"BLOCKED  never ran: {', '.join(missing)}")
        return EXIT["BLOCKED"]
    failed = next((s for s in REQUIRED if rows[s]["exit"] != 0), None)
    if failed:
        tail = pathlib.Path(rows[failed]["log"]).read_text().splitlines()[-15:]
        print(f"FAIL     first broken step: {failed} (exit {rows[failed]['exit']})")
        print("\n".join(tail))
        return EXIT["FAIL"]
    print("PASS     clean clone reproduces the test suite")
    return EXIT["PASS"]

if __name__ == "__main__":
    sys.exit(main(sys.argv[1]))

Running it is one command, and the only thing you need locally is Docker and a read-only deploy key:

docker run --rm -it \
  -e REPO_URL=git@github.com:you/app.git \
  -e SMOKE_CMD=".venv/bin/python -m pytest -q tests/test_smoke.py" \
  -v "$PWD/harness:/harness:ro" \
  -v "$PWD/out:/out" \
  -e REPORT=/out/steps.tsv \
  python:3.12-slim bash /harness/coldstart.sh

The failures it surfaces are rarely the interesting-looking ones. It finds implicit global installs the lockfile never captured, services that something assumes are listening on localhost, shell variables exported so long ago that nobody remembers exporting them, and filename collisions that only appear when a case-insensitive laptop meets a case-sensitive container. My first several runs failed on step two, not step three, which is the whole point: the agent's code was fine, and the workspace contract around it was fiction.

The machine you are willing to destroy

Here is where the sandbox question stops being abstract. A cold-clone loop reruns the same three commands many times a day, and doing that on your own machine means contending with your IDE, your database, and your own muscle memory. What you want is a box you are happy to rm -rf, plus model calls that do not drain a personal budget while you iterate.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is an open-source project that advertises free model access and a free server option, which is exactly the shape of thing this workflow needs: an environment where the agent can be handed a repository and permission to break things. The operator currently describes a free token allowance on the order of ten million tokens, and I would verify that number on the project page before you build a habit around it, because promotional allowances and server lifetimes move. Treat the server as ephemeral by default: no secrets you cannot rotate, no customer data, no production database URL, and no assumption that yesterday's box is still there today. If you want somewhere to run the harness above without wrecking your own workstation, that is the one use I would genuinely recommend it for.

Before you hand anything over, write down the blast radius. This table is the part I would copy into your repo and argue about in review, because it is the cheapest alignment you will ever buy.

Capability	Disposable agent box	Your laptop / protected CI
Clone the repo with a read-only key	yes	yes
Install public dependencies, no cache	yes	yes
`rm -rf` the working tree freely	yes	no
Cloud credentials, prod DB URL, real customer rows	never	via secret manager, human-approved
Push to a protected branch or merge	no	no

What this test cannot tell you, and who should skip it

A green cold-clone run does not mean the code is correct. It means the environment is reproducible, which is a smaller and much more defensible claim. It will not catch load-dependent races, migration ordering problems, or a third-party API that fails one request in fifty, because a single sequential run simply does not produce those conditions. It also cannot tell you whether the agent solved the right problem, and I have shipped patches that passed every step while quietly implementing my misread of the ticket.

So treat a passing run as an admission ticket, not a merge approval. The protected branch still needs a human, the diff still needs a reader, and the smoke command you choose is a statement about what you actually care about — if it only imports a module, you have tested your import statements and nothing else.

Some teams should not adopt this at all. If your build genuinely needs a physical device farm or a licensed toolchain, a container cannot reproduce it, and you will burn days chasing a false negative. If you operate under a compliance regime that requires vetted hardware, a shared free box is the wrong place for that workload, no matter how convenient it looks on a Tuesday. And if you have not yet decided what the agent is allowed to touch, build the table before you build the harness, because the harness will happily execute whatever you accidentally authorized.

The question I would actually like answered

Which layer of your handoff breaks first when the environment changes? Is it the install, the test run, the smoke command, or the moment a service you assumed was up turns out to be a container somebody stopped last Friday? Paste the failing step and the exit code, or the HTTP status if it is a request rather than a command, and I will tell you where I would put the assertion.

Because the model is going to keep getting better whether or not any of us write a script this week. The machine underneath it, though, will keep being whatever we happened to leave lying around, and that is the part I have finally stopped trusting.

Build a Budgeted Envelope Before Giving the Agent Filing Authority

kongkong — Sun, 13 Sep 2026 19:13:15 +0000

The request looked harmless from the UI when a finance ops person clicked summarize and file. They waited through a spinner, watched a clean paragraph appear, and assumed the cabinet was done. Then the record landed in another tenant's workspace because the model call was wired like a chat box. Have you ever shipped a demo that survived the prompt and died on the first authenticated mutation?

I keep hearing this week that models already write code better than most working developers. That argument is a sideshow once your product has to file a record, charge a card, or notify a human. The first layer that fails is rarely syntax; it is tenant scope, idempotency, or a missing spend cap. Why would a stronger completion fix a handler that never even forwarded the workspace header downstream?

I am taking a blunt position, and I do not intend to sand it down for comfort. If your agent can mutate storage before you can name the envelope fields, you are decorating a prototype. You are not productionizing an AI feature, no matter how confident the summary sounds in staging. The rehearsal I want is boring: freeze the request shape, cap the spend, and prove the handoff on a throwaway host.

The click, and the layer that actually breaks

Picture the same click again as an HTTP POST with a cookie, a workspace id, and an invoice id. Your handler should refuse that call unless those three facts survive every hop into the model client. What happens in vibe-coded slices is a route that concatenates PDF text, fires a vendor SDK, and inserts the JSON. The summary can be gorgeous while the insert is still a cross-tenant write into the wrong cabinet.

I treat that POST as one vertical slice rather than an AI feature glued onto a database. The envelope is the contract for who is asking, which tenant, which artifact, and how many tokens remain. If any field is missing, the model never sees the bytes, and that refusal is the product. Does that feel slower than dropping an SDK into the route and hoping the prompt behaves?

Carry money through the port, not a model nickname

I do not want model names sprinkled through handlers like confetti after a too-happy demo day. I want one port that accepts an envelope and returns a draft the rest of the application can persist. The sketch below is a proposed template, not a measured service, so treat the token fields as local fixtures. You should copy the shape, not any claim about latency, quality, or capacity that I have not measured here.

# envelope.py — proposed template, not production metrics
from dataclasses import dataclass
from typing import Protocol

@dataclass(frozen=True)
class FilingEnvelope:
    workspace_id: str
    actor_id: str
    invoice_id: str
    idempotency_key: str
    max_output_tokens: int
    attempt: int

@dataclass(frozen=True)
class FilingDraft:
    summary: str
    provider: str
    usage_tokens: int

class ModelPort(Protocol):
    def draft_summary(self, envelope: FilingEnvelope, text: str) -> FilingDraft:
        ...

class BudgetExceeded(Exception):
    pass

class TenantMismatch(Exception):
    pass

The handler then becomes a gate instead of a prompt playground with a database side effect. Notice how the model sits downstream of auth, budget, and the idempotency key on every attempt. If the ledger says this key already committed, we return the stored draft and skip another paid call. That is the opinion in code: the agent proposes, and the envelope decides whether anyone may ask.

# handler.py — proposed FastAPI slice
from fastapi import APIRouter, Header, HTTPException
from .envelope import FilingEnvelope, BudgetExceeded, TenantMismatch

router = APIRouter()

@router.post("/workspaces/{workspace_id}/invoices/{invoice_id}/file")
def file_invoice(
    workspace_id: str,
    invoice_id: str,
    payload: dict,
    x_actor_id: str = Header(...),
    idempotency_key: str = Header(...),
):
    env = FilingEnvelope(
        workspace_id=workspace_id,
        actor_id=x_actor_id,
        invoice_id=invoice_id,
        idempotency_key=idempotency_key,
        max_output_tokens=800,
        attempt=payload.get("attempt", 1),
    )
    try:
        return filing_service.apply(env)
    except TenantMismatch:
        raise HTTPException(status_code=403, detail="workspace mismatch")
    except BudgetExceeded:
        raise HTTPException(status_code=402, detail="filing budget exhausted")

Would I let this route import a vendor SDK and talk to storage in the same function? Not anymore, because that is how tenant identifiers vanish and how retries double-file the same invoice. The service loads the invoice under the workspace, checks the ledger, and only then asks the port. If you cannot narrate those three steps without mentioning a model nickname, the slice is still a toy.

# service.py — proposed apply path
class FilingService:
    def __init__(self, invoices, ledger, filings, port: ModelPort):
        self.invoices = invoices
        self.ledger = ledger
        self.filings = filings
        self.port = port

    def apply(self, env: FilingEnvelope):
        invoice = self.invoices.get(env.workspace_id, env.invoice_id)
        if invoice is None:
            raise TenantMismatch()
        existing = self.filings.get_by_key(env.idempotency_key)
        if existing:
            return existing
        if self.ledger.remaining(env.workspace_id) < env.max_output_tokens:
            raise BudgetExceeded()
        draft = self.port.draft_summary(env, invoice.redacted_text())
        self.ledger.consume(env.workspace_id, draft.usage_tokens)
        return self.filings.commit(env, draft)

Rehearse the status codes on a host you can throw away

Here is the workflow I want teams to run before anyone debates which completion sounds smarter. Stand up a throwaway server that speaks the same envelope your production handler will speak next month. Point a cheap client at it, then replay one recorded invoice until you can predict the status codes. You are not hunting for a prettier paragraph; you are hunting for 403, 402, 409, and 201.

A free rehearsal lane matters here because production credentials do not belong in that first loop at all. MonkeyCode fits that lane with free model access and a free server option, which keeps the experiment off billing. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I will not invent quotas, hardware, model names, or promises of permanence, because the envelope is the artifact.

A local replay looks like the curl below, aimed at a throwaway origin rather than a shared staging box. Freeze a redacted invoice as a fixture, then send the same headers your production browser will send. If the so-called AI feature cannot survive that request without a live customer database, it is not a feature. Can you honestly say your current filing button would pass that bar on a host you are willing to delete?

# replay.sh — local fixture against a throwaway origin
export REHEARSAL_ORIGIN="http://127.0.0.1:8088"
curl -i -X POST "$REHEARSAL_ORIGIN/workspaces/ws_9/invoices/inv_17/file" \
  -H "content-type: application/json" \
  -H "x-actor-id: user_22" \
  -H "idempotency-key: file:ws_9:inv_17:v1" \
  --data '{"attempt":1}'

The test I care about is not a similarity score and it is not whether the summary feels on-brand. It is a cross-layer failure test that pins tenancy, money, and retries to real HTTP statuses. Wrong workspace must be 403, a dead budget must be 402 before the provider is touched, and the same key must not insert twice. If those three cases are flaky, arguing about model quality is like polishing the hood of a car with no brakes.

# test_filing_envelope.py — proposed pytest, fixtures only
def test_wrong_workspace_is_forbidden(client, invoice_in_ws9):
    response = client.post(
        "/workspaces/ws_other/invoices/inv_17/file",
        headers={"x-actor-id": "user_22", "idempotency-key": "k1"},
        json={"attempt": 1},
    )
    assert response.status_code == 403
    assert invoice_in_ws9.filed is False

def test_zero_budget_never_calls_provider(client, ledger, fake_port):
    ledger.set_remaining(workspace_id="ws_9", tokens=0)
    response = client.post(
        "/workspaces/ws_9/invoices/inv_17/file",
        headers={"x-actor-id": "user_22", "idempotency-key": "k2"},
        json={"attempt": 1},
    )
    assert response.status_code == 402
    assert fake_port.calls == []

def test_retry_same_key_does_not_double_file(client, filing_repo):
    headers = {
        "x-actor-id": "user_22",
        "idempotency-key": "file:ws_9:inv_17:v1",
    }
    first = client.post(
        "/workspaces/ws_9/invoices/inv_17/file",
        headers=headers,
        json={},
    )
    second = client.post(
        "/workspaces/ws_9/invoices/inv_17/file",
        headers=headers,
        json={},
    )
    assert first.status_code in {200, 201}
    assert second.status_code in {200, 201}
    assert filing_repo.count("inv_17") == 1

Think of the envelope like a night-deposit slot at a bank after the lobby has closed. The essay a teller might write about your cash does not matter if the slot opens on the wrong account. Why are so many teams still grading the essay while the slot is unlatched for every workspace? File the invoice through the slot, then argue about prose, not the other way around.

What failed when the envelope was missing

I used to let the prototype chat with a live PDF and then persist whatever looked good enough. The first gateway timeout retried the handler and filed the same invoice twice into the cabinet. The first borrowed staging cookie wrote a summary into a workspace that did not belong to that actor. The first cost spike was not intelligence either; it was an unbounded attempt counter dressed up as helpfulness.

None of those failures required a famous model, a huge context window, or a new agent framework. They required a route that had no envelope, and they showed up as ordinary status codes if you looked. Production caveats follow from that mess, and they are not optional once money or tenancy is in the path. Redact fixtures, isolate rehearsal secrets from billing, and treat the free host as a fuse you will pull.

Do not promote the slice until 403, 402, and 409 are as boring as the happy 201 path. Do not keep customer PDFs overnight on a box you advertised as disposable, because cheap then becomes an incident. Keep the ledger schema in your real migration path, since budget rows are application data rather than prompt fluff. Are you still tempted to skip the ledger because the demo only files one invoice on a good day?

Who should not bother with this

If you are training weights, this envelope will not help you, and you should stop reading for a lab setup. If your app has no tenants and no persistence, stay in a notebook and enjoy the chat window while it lasts. If legal needs a vendor review before any third-party host sees text, run the same tests against a local fake port. The contract is the artifact you keep; the rehearsal host is optional and should remain easy to burn.

You set up a port, an envelope, a ledger, and three tests that fail closed on purpose. You run the curl against a throwaway origin until those status codes stay deterministic across ordinary retries. Only then do you point the same handler at a paid provider hiding behind the identical port. That path from setup to result stays useful even if you remove every product name from this article.

So here is my ask, and I want a concrete failure rather than a feeling about the model. Which layer handoff is least stable in your filing slice, and which status code or body do you actually get? If you rehearse the envelope on a throwaway host, does 402 appear before 403 when both gates should trip? That order tells you whether money or tenancy is the weaker lock on the cabinet you thought you had sealed.

Build a Read-Only Ticket Slice Before Giving the Model Write Authority

kongkong — Sat, 12 Sep 2026 17:07:52 +0000

Last Tuesday a product manager asked for ticket notes, and the coding panel spat out a complete POST handler before lunch. The demo path looked perfect in a notebook session, because every request ran as a local superuser against an empty database. Then a real help-desk agent tried the same write, and the first failure was not the model. It was a missing migration, a missing role check, and a 500 that told the user nothing useful.

I keep seeing this pattern in the current agents-will-write-the-feature conversation, and I think the framing is backwards. Whether a model can emit plausible FastAPI is almost uninteresting now, because emitting a handler is the easy part. The real question is whether your application can refuse a confident write until a read-only slice has proven auth and persistence. If you cannot name the status code that should come back, you do not have a feature, only a vibe.

So here is the opinion I will defend with a working path instead of another architecture diagram. Do not let an AI coding loop open POST, PATCH, or schema apply until a GET slice exists that fails closed. Use a cheap model and a disposable server for that rehearsal if you want, but keep write authority behind a plan file. I will walk a ticket-notes change through that gate, including the commands I actually run.

The user action stays small on purpose, because a tiny request still crosses every production layer you care about. An authenticated agent opens a ticket, expects to read notes, and only later wants to append one. That request should die at the API with 401, 403, or 404 long before it dies inside generated SQL. When I skip this order, the model helpfully creates the table during the request, and production traffic learns about the lock the hard way.

I encode the request as an intent record, not as another chat transcript that nobody can hash. The file is boring JSON, which is exactly the point, because later jobs can refuse a mutated copy. write_allowed stays false on purpose, and that flag is the whole architecture in one field. The model may propose a handler, a migration, and even a test name, but it does not flip the flag.

{
  "intent_id": "intent_ticket_notes_01",
  "actor_role": "agent",
  "resource": "ticket",
  "resource_id": "TCK-1042",
  "action": "read_notes",
  "write_action": "add_note",
  "write_allowed": false
}

A human or a pipeline with a signed checklist flips write_allowed after the read-only slice returns a known matrix. I ask the model for a plan file, never for a patch against main, because patches hide the mutation switch. The contract stays tiny so a free model can fill it without inventing a second ORM. If the model returns enabled: true on the write route, I treat that plan as failed output.

{
  "plan_id": "plan_ticket_notes_01",
  "intent_id": "intent_ticket_notes_01",
  "migration": {
    "name": "add_ticket_notes",
    "sql": "ALTER TABLE tickets ADD COLUMN notes TEXT NOT NULL DEFAULT ''"
  },
  "read_route": {
    "method": "GET",
    "path": "/tickets/{ticket_id}/notes",
    "auth": "bearer",
    "roles": ["agent", "admin"]
  },
  "write_route": {
    "method": "POST",
    "path": "/tickets/{ticket_id}/notes",
    "auth": "bearer",
    "roles": ["agent"],
    "enabled": false
  }
}

Why would we trust a generator that cannot keep its own hands off the mutation switch in its own plan? I hash the file before anyone talks about code, because a rewritten plan is a different change. The commands below are the whole review surface for that step, and they belong in CI.

sha256sum plan_ticket_notes_01.json
python - <<'PY'
import json
plan = json.load(open("plan_ticket_notes_01.json"))
assert plan["intent_id"] == "intent_ticket_notes_01"
assert plan["write_route"]["enabled"] is False
print("plan_ok_write_disabled")
PY

The application code then implements GET only, and every miss fails closed on purpose. This is the slice I run on a disposable server before anyone talks about shipping POST. I collapse missing tickets and cross-org tickets into 404, because a 403 on another tenant identifier leaks existence. Can you see how the AI-wrote-a-complete-CRUD-module story skips that tenancy decision entirely?

# notes_read.py — worked example for the read-only slice
from fastapi import FastAPI, Header, HTTPException
from pydantic import BaseModel

app = FastAPI()
TICKETS = {"TCK-1042": {"org_id": "org_9", "notes": "Customer heard hold music."}}
ROLES = {
    "agent_token": ("agent", "org_9"),
    "admin_token": ("admin", "org_9"),
    "other_token": ("agent", "org_other"),
}

class NotesOut(BaseModel):
    ticket_id: str
    notes: str

def actor(authorization: str | None):
    if not authorization or not authorization.startswith("Bearer "):
        raise HTTPException(status_code=401, detail="missing_token")
    token = authorization.removeprefix("Bearer ").strip()
    if token not in ROLES:
        raise HTTPException(status_code=401, detail="bad_token")
    return ROLES[token]

@app.get("/tickets/{ticket_id}/notes", response_model=NotesOut)
def read_notes(ticket_id: str, authorization: str | None = Header(default=None)):
    role, org_id = actor(authorization)
    ticket = TICKETS.get(ticket_id)
    if ticket is None or ticket["org_id"] != org_id:
        raise HTTPException(status_code=404, detail="ticket_not_found")
    if role not in {"agent", "admin"}:
        raise HTTPException(status_code=403, detail="role_forbidden")
    return NotesOut(ticket_id=ticket_id, notes=ticket["notes"])

I have watched generated handlers return 403 for foreign ids with total confidence, then call the leak a security feature. The rehearsal host should be something you can delete, not a shared staging database that still holds yesterday's customer rows. I run the slice locally first, then on a free server when I want the same curl matrix to survive a cold machine. The four commands below are not setup theater; they are the merge gate.

uvicorn notes_read:app --host 127.0.0.1 --port 8088
curl -i http://127.0.0.1:8088/tickets/TCK-1042/notes
curl -i -H "Authorization: Bearer nope" http://127.0.0.1:8088/tickets/TCK-1042/notes
curl -i -H "Authorization: Bearer other_token" http://127.0.0.1:8088/tickets/TCK-1042/notes
curl -i -H "Authorization: Bearer agent_token" http://127.0.0.1:8088/tickets/TCK-1042/notes

Those four curls are the feature, not a prelude to the feature, and I will die on that hill. I want 401, 401, 404, and 200 in that order before I even ask the model for POST. If a free model helped draft the handler, this matrix is still the merge gate, because typing speed is not correctness. Write authority stays behind a second job that reads the plan checksum and only then mounts the router.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I use MonkeyCode as an open-source project with free model access and a free server option when that rehearsal loop needs to leave my laptop. I do not treat that host as a system of record, and I do not pretend a free server replaces SSO, backups, or a secrets manager. The value is repeating the same four curls against a clean process, not collecting a new production dependency.

# notes_write.py — present in the repo, mounted only after the plan flip
from fastapi import APIRouter, Header, HTTPException
from pydantic import BaseModel, Field

write_router = APIRouter()
SEEN: set[tuple[str, str]] = set()

class NoteIn(BaseModel):
    body: str = Field(min_length=1, max_length=2000)
    idempotency_key: str = Field(min_length=8, max_length=64)

def mount_write_routes(app, plan):
    if not plan["write_route"]["enabled"]:
        return
    app.include_router(write_router)

@write_router.post("/tickets/{ticket_id}/notes")
def add_note(ticket_id: str, payload: NoteIn, authorization: str | None = Header(default=None)):
    role, org_id = actor(authorization)
    if role != "agent":
        raise HTTPException(status_code=403, detail="role_forbidden")
    ticket = TICKETS.get(ticket_id)
    if ticket is None or ticket["org_id"] != org_id:
        raise HTTPException(status_code=404, detail="ticket_not_found")
    key = (ticket_id, payload.idempotency_key)
    if key in SEEN:
        return {"ticket_id": ticket_id, "notes": ticket["notes"], "replayed": True}
    ticket["notes"] = payload.body
    SEEN.add(key)
    return {"ticket_id": ticket_id, "notes": ticket["notes"], "replayed": False}

Idempotency is not optional once retries exist, and model-generated POST handlers forget that constantly in otherwise pretty code. Network clients retry, humans double click, and mobile apps replay a buffered submit when the radio wakes. If your apply path cannot survive that, the model did not finish the feature, no matter how clean the names look. I keep a pytest file that encodes the matrix so a future model run cannot simplify the 404 into a 403.

# test_notes_read.py
from fastapi.testclient import TestClient
from notes_read import app

client = TestClient(app)

def test_missing_token_is_401():
    response = client.get("/tickets/TCK-1042/notes")
    assert response.status_code == 401
    assert response.json()["detail"] == "missing_token"

def test_bad_token_is_401():
    response = client.get(
        "/tickets/TCK-1042/notes",
        headers={"Authorization": "Bearer nope"},
    )
    assert response.status_code == 401
    assert response.json()["detail"] == "bad_token"

def test_foreign_org_is_404():
    response = client.get(
        "/tickets/TCK-1042/notes",
        headers={"Authorization": "Bearer other_token"},
    )
    assert response.status_code == 404
    assert response.json()["detail"] == "ticket_not_found"

def test_owner_can_read():
    response = client.get(
        "/tickets/TCK-1042/notes",
        headers={"Authorization": "Bearer agent_token"},
    )
    assert response.status_code == 200
    assert response.json()["notes"] == "Customer heard hold music."

def test_write_route_absent_while_plan_disabled():
    response = client.post(
        "/tickets/TCK-1042/notes",
        json={"body": "hi", "idempotency_key": "abc12345"},
    )
    assert response.status_code in {404, 405}

pytest -q test_notes_read.py

What failed for me along the way is worth naming, because the failures were architectural, not cosmetic. I once let the model emit Alembic and auto-apply on startup, which is a cute demo until two replicas race the same lock. I once shipped GET and POST in the same pull request because empty SQLite returned green, and the first real tenant collision arrived as a 500. I once trusted a generated 403 for unknown ids, which is just an existence oracle with extra steps.

Production caveats pile up as soon as you leave the toy dictionary, and they should slow you down. A real notes column needs a reversible migration, a lock story for hot tickets, and an audit row the model must not invent ad hoc. Free servers are fine for proving status codes against a disposable schema, and they are the wrong place for customer tokens. If your world needs a signed vendor review, do not point production identity at a rehearsal host just because inference is free.

Who should not follow this path, even though I like it for full-stack teams shipping real apps? Skip it if you want the plan file to replace code review, because a JSON schema will not catch a destructive default. Skip it if your so-called read-only GET still executes a stored procedure with side effects, since the slice would be lying. Skip it if you cannot freeze the intent, because a chatty agent that renegotiates the resource on every turn will rot your tests in public.

Limitations of the example stay honest, and I will not dress them up as a platform. The in-memory map is not Postgres, and the bearer table is not your identity provider. The plan JSON does not handle partial applies, and I have not claimed latency or token-cost numbers because those claims would be theater without a measured run. The method is the gate, not the brand of host you used to exercise it.

I still like a free model for the plan draft and a free server for the curl matrix, because cheap rehearsal is the only rehearsal people actually run. If you want that loop on MonkeyCode, start with the GET slice and four status codes, not generated CRUD. Then tell me which layer handoff is least stable in your stack when the model sounds sure. Is it auth, migration apply, idempotency, or the moment someone flips write_allowed, and what status code did you actually see?

Here is a short checklist I reuse before I let any model-backed change gain write authority.

Freeze an intent with write_allowed false.
Accept a plan file and reject any plan that enables POST.
Ship GET until 401, 403, 404, and 200 stay stable.
Rehearse on a host you can delete without a change board.
Flip write only after idempotency tests pass on a retry.

That is the whole argument, and I do not think it is a close call. Models can type quickly, but your job is keeping them from writing until the read path already tells the truth. If the GET slice cannot fail closed, the POST handler is not a feature yet.