Workshop: Split Generation From Execution With a Two-Clock Tool Lab in 80 Minutes

A single elapsed-time figure for an AI-backed tool call still hides two different systems. Classroom labs should pin a two-clock card before anyone debates latency, cost, or correctness claims. Generation time belongs to the model path, while execution time belongs to the tool server. Mixing those clocks produces ungradeable homework, because a lucky live session is not a contract.

This eighty-minute workshop freezes a tool-call shape, then records both clocks in one ledger. Students leave a small harness they can rerun later without depending on live model variance. The grade attaches to those files, not to a demo that happened to work once. Each file is small enough to diff in review, which keeps weekend reruns comparable across machines.

What this lab is for

Tool-calling tutorials often print one duration beside a JSON payload and then call the demo finished. That number cannot tell a student whether the model stalled, the server stalled, or the path between them stalled. A teaching lab needs a split that survives weekend reruns, so grades attach to artifacts instead of luck. Without that split, two honest students can submit opposite conclusions from the same flaky live minute.

You will leave with three files: a frozen shape card, a two-clock recorder, and a local tool server. Students can start the server from a single command and grade traces from a JSONL ledger. The lab does not require a paid model endpoint on the first graded pass through the recorder. The notes stay useful if every commercial product name is later stripped from the text.

Timing box

Use this timing box as the session contract, and cut discussion first if a block starts to overrun.

• 00–10 min: name the mixed-clock failure and read the two-clock card aloud
• 10–25 min: freeze the tool-call shape against a tiny schema and three fixtures
• 25–45 min: wire generation and execution clocks in a JSONL recorder
• 45–65 min: rerun the worked inventory example and grade the traces
• 65–80 min: walk the failure matrix and decide who should skip this lab

Keep a visible timer on the projector. If a section overruns, cut discussion rather than the recorder, because the artifact is the grade surface.

The two-clock card

Pin this JSON document before any student is allowed to hit a live generation endpoint. The card is the assignment contract for the lab, not a dashboard widget for later screenshots. Required keys, forbidden keys, and clock ownership stay frozen even when the model path changes. Reviewers should reject traces that add fields to this card without a written amendment in the ledger.

{
  "lab_id": "two-clock-tool-lab-2026-09-22",
  "tool_name": "inventory.lookup",
  "shape_freeze": {
    "required_keys": ["sku", "warehouse"],
    "forbidden_keys": ["prompt", "raw_user_text"],
    "sku_pattern": "^[A-Z]{3}-[0-9]{4}$",
    "warehouse_enum": ["east", "west"]
  },
  "clocks": {
    "t_generate_ms": {
      "owner": "model_path",
      "includes_network_to_model": true
    },
    "t_execute_ms": {
      "owner": "tool_server",
      "includes_network_to_server": true
    }
  },
  "pass_rule": "both clocks recorded unless shape fails; shape_freeze holds; execute status is 200 or 404"
}

The pass rule is deliberately narrow so stopwatch luck cannot rescue a broken tool-call shape. A fast generation that emits a forbidden key still fails, because the lab grades the contract. Status 404 is allowed because unknown SKUs are valid execution outcomes, not recorder crashes. Status 500 is not allowed, because the frozen map should not throw when the shape already passed.

Exercise 1 — freeze the shape (15 minutes)

Students often let the model invent extra fields such as reason, comment, or a copied prompt. Those fields leak homework text into the tool server and make two traces incomparable after class. Freeze the shape first, then allow generation, or the later clocks will measure the wrong object. The next block is lab code for teaching, not production middleware and not a security boundary.

# shape_freeze.py — lab-only contract check; not a security boundary
import json
import re
import sys

REQUIRED = {"sku", "warehouse"}
FORBIDDEN = {"prompt", "raw_user_text"}
SKU = re.compile(r"^[A-Z]{3}-[0-9]{4}$")
WAREHOUSES = {"east", "west"}


def freeze_ok(payload: dict) -> list[str]:
    errors = []
    keys = set(payload)
    missing = REQUIRED - keys
    extra_forbidden = keys & FORBIDDEN
    if missing:
        errors.append(f"missing:{sorted(missing)}")
    if extra_forbidden:
        errors.append(f"forbidden:{sorted(extra_forbidden)}")
    sku = payload.get("sku")
    if not isinstance(sku, str) or not SKU.match(sku):
        errors.append("sku_pattern")
    if payload.get("warehouse") not in WAREHOUSES:
        errors.append("warehouse_enum")
    return errors


if __name__ == "__main__":
    data = json.loads(sys.stdin.read())
    errs = freeze_ok(data)
    print(json.dumps({"ok": not errs, "errors": errs}))
    raise SystemExit(0 if not errs else 2)

Issue three fixtures before opening any model path, and refuse to continue until all three behave.

1. Fixture A is a valid WID-2044 lookup against the east warehouse and must print ok: true.
2. Fixture B leaks prompt text into the payload and must exit nonzero with a forbidden-key error.
3. Fixture C uses a lowercase sku and must fail the pattern check even when other keys look fine.

echo '{"sku":"WID-2044","warehouse":"east"}' | python shape_freeze.py
echo '{"sku":"WID-2044","warehouse":"east","prompt":"need stock"}' | python shape_freeze.py
echo '{"sku":"wid-2044","warehouse":"east"}' | python shape_freeze.py

Expected exits are 0, then 2, then 2. Students who continue after a green Fixture A alone have not frozen the contract.

Exercise 2 — record two clocks (20 minutes)

The recorder never stores a single elapsed_ms field as the grade for a student attempt. It stores two clocks and a shape verdict, then writes one JSON line per attempt into a ledger. A missing execute clock is valid when shape freeze fails, because the server must not see bad payloads. Students who print only the sum of both clocks fail the exercise even if both systems look healthy.

# recorder.py — generation clock and execution clock stay separate
from __future__ import annotations

import json
import time
import urllib.request
from pathlib import Path

from shape_freeze import freeze_ok

LEDGER = Path("two_clock_ledger.jsonl")


def timed(fn):
    start = time.perf_counter()
    result = fn()
    ms = int((time.perf_counter() - start) * 1000)
    return result, ms


def generate_tool_call(user_text: str) -> dict:
    # Lab stub: swap later for a live free-model call.
    # First graded rerun stays deterministic.
    if "inject-prompt" in user_text:
        return {
            "sku": "WID-2044",
            "warehouse": "east",
            "prompt": user_text,
        }
    if "widget" in user_text.lower():
        return {"sku": "WID-2044", "warehouse": "east"}
    return {"sku": "UNK-0000", "warehouse": "east"}


def execute_tool(payload: dict, base: str) -> dict:
    body = json.dumps(payload).encode()
    req = urllib.request.Request(
        base + "/inventory/lookup",
        data=body,
        headers={"Content-Type": "application/json"},
        method="POST",
    )
    with urllib.request.urlopen(req, timeout=3) as resp:
        return {"status": resp.status, "body": json.loads(resp.read().decode())}


def run_attempt(user_text: str, base: str) -> dict:
    payload, t_generate_ms = timed(lambda: generate_tool_call(user_text))
    shape_errors = freeze_ok(payload)
    if shape_errors:
        record = {
            "user_text": user_text,
            "payload": payload,
            "shape_errors": shape_errors,
            "t_generate_ms": t_generate_ms,
            "t_execute_ms": None,
            "pass": False,
        }
    else:
        result, t_execute_ms = timed(lambda: execute_tool(payload, base))
        record = {
            "user_text": user_text,
            "payload": payload,
            "shape_errors": [],
            "t_generate_ms": t_generate_ms,
            "t_execute_ms": t_execute_ms,
            "status": result["status"],
            "pass": result["status"] in (200, 404),
        }
    with LEDGER.open("a", encoding="utf-8") as handle:
        handle.write(json.dumps(record) + "\n")
    return record

The sum remains a convenience field they may compute later, never the source of truth for grading. Keep generate_tool_call replaceable so a live free-model path can land without rewriting the ledger schema. Timeouts stay at three seconds during class so a hung server does not consume the remaining workshop.

Recorder checks before the live swap

• Ledger lines must include t_generate_ms on every attempt, including shape failures.
• Ledger lines must set t_execute_ms to null when shape_errors is not empty.
• No graded line may introduce elapsed_ms as a replacement for the two clocks.
• pass may be true only for status 200 or 404 after a clean shape.

Worked example students can rerun (20 minutes)

Stand up a local inventory server that answers from a frozen map instead of a live catalog. The map is the oracle for this lab, so weekend reruns stay comparable across student machines. Unknown SKUs return 404, which still records t_execute_ms and can pass the narrow rule above. Do not add authentication, sleeps, or random stock, because those extras destroy rerun equality.

# lab_server.py — frozen inventory map, not a product catalog
from http.server import BaseHTTPRequestHandler, HTTPServer
import json

STOCK = {
    ("WID-2044", "east"): 12,
    ("WID-2044", "west"): 0,
}


class Handler(BaseHTTPRequestHandler):
    def do_POST(self):
        length = int(self.headers.get("Content-Length", "0"))
        payload = json.loads(self.rfile.read(length) or b"{}")
        key = (payload.get("sku"), payload.get("warehouse"))
        if key not in STOCK:
            self.send_response(404)
            body = {"error": "unknown_sku"}
        else:
            self.send_response(200)
            body = {"sku": key[0], "warehouse": key[1], "qty": STOCK[key]}
        raw = json.dumps(body).encode()
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(raw)))
        self.end_headers()
        self.wfile.write(raw)

    def log_message(self, fmt, *args):
        return


if __name__ == "__main__":
    HTTPServer(("127.0.0.1", 8765), Handler).serve_forever()

Start order for a clean rerun

1. python lab_server.py
2. echo '{"sku":"WID-2044","warehouse":"east"}' | python shape_freeze.py
3. python -c "from recorder import run_attempt; print(run_attempt('Need widget stock', 'http://127.0.0.1:8765'))"
4. python -c "from recorder import run_attempt; print(run_attempt('inject-prompt restock', 'http://127.0.0.1:8765'))"
5. python -c "from recorder import run_attempt; print(run_attempt('Need unknown item', 'http://127.0.0.1:8765'))"

Expected ledger behavior is part of the grade, not a hint students may skip after a green demo. The first attempt records both clocks and should pass if the stub emits WID-2044 for the widget prompt. The planted reject path must appear in the ledger with t_execute_ms set to null and pass false. The unknown-item attempt should pass with status 404, proving execution outcomes are not the same thing as recorder crashes.

When students later swap the stub for a live model path, the server and shape card stay frozen. That swap is the only moving part, which is the entire point of the two-clock split. If the live path is unavailable, the stub still produces a complete and reviewable homework packet. Teaching assistants should diff ledger keys, not prose writeups, when two submissions disagree about latency.

Where a free model path and a free server fit

Some teaching setups need a live generation path without asking every student for a paid key. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option, which can replace the generation stub later. The same free server option can host the inventory map if a local port is awkward in the classroom.

Treat those two availability claims as the only product facts this lab is willing to record. Do not pin model names, quotas, hardware notes, or promised duration onto the two-clock card itself. Those details are not part of the assignment contract, and inventing them would make traces incomparable. Keep the product name out of the pass rule so a vendor outage cannot redefine a correct shape.

Failure matrix

Classify planted failures with this matrix before anyone writes a narrative about how the demo felt. Each row names a clock owner, a forbidden response, and a replacement action students can actually run. Three planted failures are enough for an eighty-minute session; extra stories consume the remaining clock.

Symptom	Likely clock	Do not do	Do instead
Ledger has t_execute_ms: null and shape errors	generation / contract	retry until JSON looks nicer	fail the attempt; restore the schema
Both clocks large, status 200	mixed network	quote only the sum in a report	publish both clocks as separate fields
Fast generate, slow execute, 200	tool server	blame the model path	profile the frozen map and host
Slow generate, fast execute, 200	model path	restart the inventory server	keep the server; change only generation
Status 500 with a valid shape	tool server	loosen freeze_ok	keep freeze; fix the handler
Status 404 with UNK-0000	tool server (valid)	mark the lab broken	accept 404 as a passing execution

The matrix is the last graded artifact in the packet, sitting beside the JSONL ledger and shape card. A student who only screenshots a terminal without filling the matrix has not finished the workshop.

Limitations and who should skip this lab

This workshop is a classroom contract, not an observability platform and not a load-test harness. The recorder uses time.perf_counter on one machine, so it cannot separate server compute from local scheduling noise. The HTTP server is a frozen map, so it will not teach pooling, heavy tails, or authentication design. Clock values are teaching signals for attribution, not service objectives you should paste into a status page.

Skip this lab if you need production SLOs, multi-region probes, or a security review of tool calling. Skip it if your course cannot accept a local stub as the first graded path through generation. Skip it if the assignment requires named models, published quotas, or a vendor comparison table. Those claims sit outside this contract, and forcing them back in would reopen the mixed-clock problem.

The two-clock card also refuses to measure token spend, because spend is a different ledger entirely. Mixing token counts into latency traces recreates the original confusion this workshop exists to prevent. If a later course needs spend control, pin a separate card instead of extending this one.

Close

Pin the shape, split the clocks, and grade the ledger rather than the live demo window. Students can rerun the inventory example from the five commands above on any quiet evening. If you want a live generation path after the stub passes, try MonkeyCode's free model access next. Leave that access on the generation clock only, and keep the frozen inventory server in place.