The Changeover Workshop: A Schema Gate for Free Model Output

A rented room, eighteen laptops, one shared power strip. The exercise was an invoice extractor. It worked on three PDFs and then, on the fourth, returned a paragraph of cheerful prose where a total should have been. The student blamed the model. The model was fine. Nobody had written down what a finished answer looks like, so nothing could catch the moment the answer stopped being finished.

That is the whole premise of this workshop. Before students touch a free endpoint, they build a changeover gate: a contract that says what "done" means, a repair loop that gives the model one honest chance to fix itself, and an escalation path that fails loudly instead of quietly. The gate is boring. Boring is the point.

The session runs ninety minutes and produces one file students can rerun at home. MonkeyCode, which offers free model access and a free server option, is where most of the room sends its requests, because eighteen people cannot each need a credit card to learn a validation pattern. Those two availability claims come from the project operator; current limits belong on the project's own pages, not in a workshop handout.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The clock

Minutes	Segment	What students hold at the end
0–10	Framing: the fourth PDF	A one-sentence definition of a finished answer
10–25	Exercise 1: write the contract	A JSON Schema with five to seven fields, no more
25–45	Exercise 2: wire one call	A working ask() and a dry run that passes
45–65	Exercise 3: break it on purpose	Three mutations, three distinct error paths
65–80	Exercise 4: escalate and log	A shift report and a rejects.jsonl
80–90	Debrief: who should not use this	A named boundary for the technique

Exercise 1 is deliberately small. Students who write a schema for the whole invoice format spend the next hour debugging their own strictness instead of learning the gate. Five fields. invoice_id, currency, total_cents, line_items, and one item shape. Everything else is homework.

The artifact

One file, no framework, standard library plus jsonschema. The transport is left as an exercise on purpose, since request shapes drift and a workshop should not hard-code a shape it cannot verify on the day.

"""changeover.py — the workshop artifact.

Dry run, no network:  python changeover.py --dry-run --tasks 12
Live, after exercise 2: python changeover.py --tasks 12
"""

from __future__ import annotations

import json
import sys
import time
from pathlib import Path

import jsonschema

CONTRACT = {
    "type": "object",
    "required": ["invoice_id", "currency", "total_cents", "line_items"],
    "additionalProperties": False,
    "properties": {
        "invoice_id": {"type": "string", "minLength": 3},
        "currency": {"enum": ["USD", "EUR", "CNY"]},
        "total_cents": {"type": "integer", "minimum": 0},
        "line_items": {
            "type": "array",
            "minItems": 1,
            "items": {
                "type": "object",
                "required": ["sku", "qty", "unit_cents"],
                "additionalProperties": False,
                "properties": {
                    "sku": {"type": "string"},
                    "qty": {"type": "integer", "minimum": 1},
                    "unit_cents": {"type": "integer", "minimum": 0},
                },
            },
        },
    },
}

SYSTEM = (
    "You extract invoice data. Reply with one JSON object and nothing else. "
    "Money is integer cents. Never wrap the object in prose or code fences."
)

DRY_RUN = "--dry-run" in sys.argv
MAX_REPAIRS = 2

CANNED = [
    '{"invoice_id": "INV-1042", "currency": "USD", "total_cents": "12.00", '
    '"line_items": [{"sku": "A-1", "qty": 2, "unit_cents": 600}]}',
    '```

json\n{"invoice_id": "INV-1042", "currency": "USD", "total_cents": 1200, '
    '"line_items": [{"sku": "A-1", "qty": 2, "unit_cents": 600}]}\n

```',
]


def ask(prompt: str, attempt: int) -> str:
    if DRY_RUN:
        return CANNED[min(attempt, len(CANNED) - 1)]
    # Exercise 2: replace this with exactly one HTTP call.
    # Read the base URL, key and model name from the environment, send
    # SYSTEM + prompt, return the assistant text.
    raise SystemExit("live mode not wired yet: fill in ask() from the provider docs")


def strip_fence(text: str) -> str:
    t = text.strip()
    if t.startswith("```

"):
        t = t.split("\n", 1)[1] if "\n" in t else t
        t = t.rsplit("

```", 1)[0]
    return t.strip()


class GateFailure(RuntimeError):
    def __init__(self, reasons: list[str]) -> None:
        super().__init__("; ".join(reasons))
        self.reasons = reasons


def repair_note(reasons: list[str]) -> str:
    lines = ["", "Your previous reply was rejected by a JSON Schema validator."]
    lines += [f"- {r}" for r in reasons[-3:]]
    lines.append("Return the corrected object only. Integers, not strings.")
    return "\n".join(lines)


def extract(prompt: str) -> tuple[dict, int]:
    reasons: list[str] = []
    for attempt in range(MAX_REPAIRS + 1):
        text = ask(prompt + repair_note(reasons) if reasons else prompt, attempt)
        try:
            doc = json.loads(strip_fence(text))
            jsonschema.validate(doc, CONTRACT)
            return doc, attempt
        except json.JSONDecodeError as exc:
            reasons.append(f"not JSON: {exc.msg} at char {exc.pos}")
        except jsonschema.ValidationError as exc:
            path = "/".join(str(p) for p in exc.absolute_path) or "<root>"
            reasons.append(f"{path}: {exc.message}")
    raise GateFailure(reasons)


def tasks(count: int) -> list[str]:
    sample = "Invoice INV-1042, 2 x A-1 at 6.00, total 12.00 USD."
    return [sample] * count


def main() -> None:
    count = 12
    if "--tasks" in sys.argv:
        count = int(sys.argv[sys.argv.index("--tasks") + 1])
    shift = {"tasks": 0, "first_pass": 0, "repaired": 0, "escalated": 0}
    started = time.perf_counter()
    for prompt in tasks(count):
        shift["tasks"] += 1
        try:
            _, repairs = extract(prompt)
        except GateFailure as exc:
            shift["escalated"] += 1
            with Path("rejects.jsonl").open("a", encoding="utf-8") as fh:
                fh.write(json.dumps({"input": prompt, "reasons": exc.reasons}) + "\n")
            continue
        shift["first_pass" if repairs == 0 else "repaired"] += 1
    shift["seconds"] = round(time.perf_counter() - started, 2)
    print(json.dumps(shift))


if __name__ == "__main__":
    main()

The dry run prints {"tasks": 12, "first_pass": 0, "repaired": 12, "escalated": 0, "seconds": 0.0}. Students read that and immediately ask whether the model is bad. It is not a measurement. The canned replies are fixtures that prove the wiring, and the whole first-pass number is an artifact of the fixture list. Real numbers require real calls, and even then twelve tasks is a smoke test, not a benchmark.

Exercise 3: break it on purpose

Reading an error message is not the same as causing one. Students mutate a known-good document three ways and record which part of the contract fires.

python - <<'PY'
import json
import changeover as c
from jsonschema import ValidationError, validate

base = json.loads(c.strip_fence(c.CANNED[1]))
mutations = [
    ("drop currency", lambda d: d.pop("currency")),
    ("zero qty", lambda d: d["line_items"][0].__setitem__("qty", 0)),
    ("extra field", lambda d: d.__setitem__("tax_cents", 99)),
]
for name, mutate in mutations:
    doc = json.loads(json.dumps(base))
    mutate(doc)
    try:
        validate(doc, c.CONTRACT)
        print(f"{name:14} -> passed, which means the contract has a hole")
    except ValidationError as exc:
        path = "/".join(map(str, exc.absolute_path)) or "<root>"
        print(f"{name:14} -> {path}: {exc.message}")
PY

The third mutation is the teaching moment. additionalProperties: False turns a harmless extra field into a rejection, and students who left it out discover their gate was a suggestion rather than a gate. A schema with holes behaves like a turnstile with a gap at knee height: it looks like control and it is not.

Exercise 4: the shift report

Escalation is the part everyone skips. When the repair loop exhausts its budget, the gate raises, the input and the last three rejection reasons land in rejects.jsonl, and the loop keeps going. Nobody's batch dies because one document was hostile.

The rejects file is the deliverable worth keeping. Reading twenty of them tells a team more about its prompts than any percentage, because the reasons cluster: string money, missing currency, an item list that came back as a single object. Those clusters become contract edits or prompt edits, and the decision is made from evidence rather than from a hunch about which model is smart.

A word on what the free lane buys and does not buy. Free model access and the free server option make a room of eighteen viable, and the free server means no one spends the first hour installing anything. They do not buy throughput guarantees, and a workshop should never imply otherwise. Students who need a latency contract should read current terms before they design around them.

Who should not use this

Teams that need a guaranteed rate or a latency SLA should not put a free lane in the request path, gate or no gate. Anyone extracting values that move money, clear a person, or feed a regulator needs a human in the loop regardless of how green the shift report looks, because schema-valid is not the same as correct: total_cents: 1300 validates perfectly and can still be wrong.

The repair loop also costs calls. It doubles the worst case and triples it under MAX_REPAIRS = 2, which is why the budget is a named constant and not a hidden default. And anyone who cannot store raw rejected inputs should not run this at all, since the rejects file is where the learning lives.

Run the workshop once and the room leaves with three things: a contract they wrote, a failure they caused on purpose, and a file of rejects. The last one is the useful one. Check the project's own pages for what the free lane currently allows before you plan the next session around it.