Pin the Schema Before a Free Model Can Write

I keep walking a small summarizer from the browser into a real save, and the pretty paragraph is never the part that breaks. Someone pastes a customer note, the completion looks clean, and the save returns a 500 before any row changes. The handler is doing a naked JSON parse on a string that starts with a markdown fence. So did the model fail, or did we skip the contract that should have rejected that string?

A popular thread this week is really about whether short functions are enough to make behavior obvious. I buy the worry, but the summarizer handler was already short, and the save behavior was still opaque. Nobody could point to a rule for which completions were allowed to become a saved note. So is the fix another line of commentary above the parse, or a contract the test can fail?

I think the free model is the wrong thing to blame first, and the wrong thing to trust next. A cheaper completion is still an untrusted upstream, closer to a flaky webhook than to a function in your repo. If the shape can drift, the write path should fail closed before a row is touched. Would you let an unmarked HTTP client insert into orders just because the demo looked fluent?

Where the save actually breaks

The architectural choice I want is a versioned schema between the provider response and the database, owned by the API. The UI may render a preview, but that preview is not authority, and a second client can always skip it. I tried the other way in a prototype, repairing fences in the browser so the demo kept moving. That repair hid the drift, a scripted client posted the raw string, and the note column filled with backticks.

Here is the contract I would pin before any model swap, including a move onto free inference. The example below is a proposed fixture, not a trace from a production incident, and you should run it yourself. I am not claiming a benchmark, a latency number, or a saved dollar from this snippet. If the bounds feel strict, that is the opinion, because a loose parser is how drift becomes data.

import json
from pydantic import BaseModel, Field, ValidationError

FENCE = "`" * 3

class SummaryV1(BaseModel):
    schema_version: str = Field(pattern=r"^1$")
    summary: str = Field(min_length=1, max_length=500)
    confidence: float = Field(ge=0.0, le=1.0)

def parse_completion(raw: str) -> SummaryV1:
    text = raw.strip()
    if text.startswith(FENCE):
        raise ValueError("fenced_markdown")
    try:
        data = json.loads(text)
    except json.JSONDecodeError as exc:
        raise ValueError("invalid_json") from exc
    return SummaryV1.model_validate(data)

The fence check is deliberate, because a helpful model often wraps an object and still sounds correct to a reader. I would rather return 422 with a stable error code than spend another prompt trying to shave the backticks. Extra keys can be ignored by configuration, missing keys should reject, and a truncated object should never become a null summary. Have you ever watched a UI show saved while the column stored an empty string that passed a truthiness check?

Pin a version, then fail closed

The route then stays boring, which is the whole point of putting the contract in one place. It loads the note, calls the provider behind one seam, and persists only after the schema accepts the body. A rejected completion is stored as a failed attempt with the schema version, not as a successful note. If you skip that attempt record, the next on-call engineer sees only a 500 and a prompt that already vanished.

def save_summary(note: str, raw_completion: str, db) -> dict:
    try:
        parsed = parse_completion(raw_completion)
    except (ValueError, ValidationError) as exc:
        db.attempts.insert({
            "schema_version": "1",
            "status": "rejected",
            "error": str(exc),
        })
        return {"status": 422, "error": "contract_rejected"}
    db.notes.update_summary(note_id=note, summary=parsed.summary)
    return {"status": 200, "schema_version": parsed.schema_version}

That insert is pseudocode for the seam, not a claim that your ORM or your tables look like this. The idea travels to SQL or a document store, because rejection is data and success is a separate write. I want 422 when the shape is wrong, and 502 only when the provider failed before a body existed. Mixing those codes sends people into the wrong layer, which is how a parser bug gets filed as an outage.

You can prove the gate without a network call, and I would not skip that step to save an afternoon. A fixture with a fenced string should fail, a valid object should pass, and an empty summary should fail the length bound. I keep those three cases beside the route so a model change cannot land as a silent prompt edit. Why would you trust a live swap if the repo cannot already reject the last bad body you saw?

python -m pytest tests/test_summary_contract.py -q

def test_fenced_markdown_is_rejected():
    fence = "`" * 3
    payload = '{"schema_version":"1","summary":"ok","confidence":0.5}'
    raw = fence + "json\n" + payload + "\n" + fence
    try:
        parse_completion(raw)
    except ValueError as exc:
        assert str(exc) == "fenced_markdown"
    else:
        raise AssertionError("fence should fail closed")

Prove it locally before the free endpoint

Only after that local gate is green do I point the same parser at a live free endpoint. The credential stays in the server environment, and the browser never learns the provider base URL. A demo page that calls the model directly has already skipped the schema and exposed the key path. I would rather see a 401 from my own API than a fluent completion that never passed validation.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. The brief I was given describes MonkeyCode as an open-source project with free model access and a free server option. That pair is useful as a second environment for the same fixture, not as permission to persist. I am not stating a token allowance, a hardware size, a duration, or a model name here.

Those terms move quickly, and I have not re-checked a primary page before writing this draft. Read the current project terms yourself, and treat any number remembered from chat as stale until you do. If the free option is gone or narrowed, the local fixture still stands, which is why it lives in the repo. Would you drop the schema pin just because a posted allowance changed between two weekly deploys?

The browser sends the note to your API, your API holds the credential, and the free server is another base URL behind that seam. A session header on that curl is a stand-in, not a real token, and you should use your own auth cookie. I log the provider status beside the schema version so a 429 is not mistaken for a bad summary. If the body is prose apologizing for the format, quarantine it and stop, because a retry will not grow a schema.

curl -sS -X POST http://127.0.0.1:8000/summaries \
  -H 'content-type: application/json' \
  -H 'authorization: Bearer dev-session' \
  -d '{"note":"Customer asked to move delivery to Friday."}'

Point the server at the free option with environment variables, and keep the secret out of the repo. The host below is a placeholder, not a documented endpoint, so replace it only after you read the current project docs. I refuse to paste a live host or a sample key into a post that will be copied into terminals.

export PROVIDER_BASE_URL="https://example.invalid/free-server"
export PROVIDER_API_KEY="from-your-secret-store"

What usually fails next is rarely the tidy JSON object you practiced against while writing the prompt. A free pool can return 429, an empty content field, or a 200 whose body is an apology. The fixture should record the status and the first hundred characters, then stop without a blind retry. Retrying a 200 that failed the schema only burns the allowance and teaches the handler to be generous.

Limits I will not talk past

There is a maintainability tradeoff I accept on purpose, and it will annoy anyone chasing a magical demo. A strict schema will reject some completions a human would have repaired, so the first week looks less fluent. That cost is smaller than a column full of fences, and cheaper than a migration after three clients stored three shapes. Performance stays predictable because you parse a small object and you do not hold a transaction open across the provider call.

When version two arrives, I would add a second model and backfill stored notes, not reinterpret old JSON. Old records keep their schema version, so a reader can see which contract produced the summary. A flag in the component that swaps prompts without a migration is how you get two meanings in one column. Have you already found a text field that means different things depending on which week it was written?

This approach is a poor fit when the note holds secrets or regulated data you cannot send to an external host. It is a poor fit if you need a latency guarantee, because free capacity is not an SLA I will pretend to own. Teams that want the model to invent new fields every week should not pin version one, or they will fight their own gate. If the request has no authenticated user, do not add this write, since a public summarizer becomes an open proxy.

Before I call the change done, the fence fixture has to fail and then pass on a valid body. The empty summary has to be rejected, and a 422 has to stay distinct from a provider 502. The provider key has to be absent from the browser bundle, which you can check with a simple search. That is the reusable gate, and it does not depend on which model happened to answer the note.

I would ship that gate before I ever allow a free completion to update a customer note. The free server is only a second environment for the same test, not a reason to widen what you persist. Call, then validate, then write, in that order, even when the endpoint costs you nothing today. Which status came back first on your last bad save, a 422 from the gate or a 500 from a naked parse?