The Witness Tape Workshop

On a Friday afternoon in a borrowed classroom, twelve laptops showed the same green check. An agent had declared a tiny Python kata finished. The facilitator asked one volunteer to reproduce the patch on a clean clone. The room went quiet. Chat history is not a build artifact. A screenshot of a passing test is not a receipt.

That gap is the workshop.

Public threads this month keep arguing whether models already outcode most developers. The stubborn problem in a teaching lab is smaller. Students can paste a passing run and still be unable to show how the tree changed. Vibe coding is not the failure mode. Calling an unreproducible transcript engineering is.

This session lasts eighty minutes. It gives students a tape recorder for agent work. The tape is a JSONL file, one event per line. A later exercise throws away the working tree and asks them to rebuild the same result from the tape plus a seed repository. If they cannot, the agent did not finish.

The seed that always breaks the same way

The facilitator starts from a repository so small it fits on one screen. The bug is boring on purpose. Boredom keeps the tape visible.

# src/mathutil.py
def mean(values):
    return sum(values) / len(values)

# tests/test_mathutil.py
from src.mathutil import mean

def test_mean_of_two():
    assert mean([2, 4]) == 3

def test_mean_empty_returns_none():
    assert mean([]) is None

A two-line pytest config keeps imports honest on a clean checkout.

# pytest.ini
[pytest]
pythonpath = .

Students run the obvious command from the repo root.

python -m pytest -q

The empty-list case fails with a division by zero. That failure is the whole plot. An agent that claims to fix the function without leaving a tape has only performed a demo.

Fifteen minutes: agree on the tape

The first block is not coding. It is naming. Students invent many schemas under pressure. The facilitator pins one, then refuses to extend it until replay works.

Each line is a JSON object with ts, event, and a small payload. Allowed events are plan, read, edit, test, and fail. Anything else is noise. A tape that cannot be replayed is a diary.

{"ts": "2026-09-17T14:02:01Z", "event": "plan", "goal": "mean([]) returns None"}
{"ts": "2026-09-17T14:02:08Z", "event": "read", "path": "src/mathutil.py", "sha256": "ab12"}
{"ts": "2026-09-17T14:02:21Z", "event": "edit", "path": "src/mathutil.py", "sha256_after": "cd34"}
{"ts": "2026-09-17T14:02:40Z", "event": "test", "cmd": "python -m pytest -q", "exit": 0}

The analogy is a flight recorder, not a novel. A recorder that writes feelings is useless in a crash review. A recorder that writes headings, altitudes, and switch positions can be replayed.

A tiny helper keeps the agent honest. Students paste it into tools/witness.py and call it from every tool wrapper. Empty __init__.py files in src/ and tools/ are enough to make the imports boring.

# tools/witness.py
import hashlib, json, time
from pathlib import Path

TAPE = Path("witness.jsonl")

def sha256_file(path: str) -> str:
    data = Path(path).read_bytes()
    return hashlib.sha256(data).hexdigest()

def record(event: str, **payload) -> None:
    line = {
        "ts": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
        "event": event,
        **payload,
    }
    with TAPE.open("a", encoding="utf-8") as fh:
        fh.write(json.dumps(line, sort_keys=True) + "\n")

No framework. No plugin. If the helper needs a package index, it is already too heavy for this room.

Twenty minutes: wrap a stub agent

The second block instruments a fake agent. Real model quality is not the point. The point is that every side effect goes through a function that writes a line first.

# tools/stub_agent.py
from pathlib import Path
from tools.witness import record, sha256_file

TARGET = Path("src/mathutil.py")

FIXED = """def mean(values):
    if not values:
        return None
    return sum(values) / len(values)
"""

def run() -> None:
    record("plan", goal="mean([]) returns None without raising")
    record("read", path=str(TARGET), sha256=sha256_file(TARGET))
    TARGET.write_text(FIXED, encoding="utf-8")
    record("edit", path=str(TARGET), sha256_after=sha256_file(TARGET))

if __name__ == "__main__":
    run()

Students run three commands in order. The order is the pedagogy.

export PYTHONPATH=.
python -m pytest -q || true
python -m tools.stub_agent
python -m pytest -q
cat witness.jsonl

A passing suite with an empty witness.jsonl is a failed exercise. The facilitator treats that case as a red build, even when pytest is green. That single rule does more than a lecture on professional ethics.

Rooms that already use a hosted coding agent can keep the same wrappers. MonkeyCode is one option when a class needs a model endpoint and a remote runner without standing up billing on day one. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Free model access and a free server option are enough to drive the stub above against a shared seed repo. They are not a substitute for the tape. A hosted run that cannot emit witness.jsonl is still a demo.

Twenty-five minutes: replay on a clean tree

The third block is the mean one. Students copy witness.jsonl aside, restore the broken seed, and run a replay script. The script does not trust the agent's story. It trusts hashes and exit codes.

# tools/replay.py
import hashlib, json, shlex, subprocess, sys
from pathlib import Path

def sha256_file(path: str) -> str:
    return hashlib.sha256(Path(path).read_bytes()).hexdigest()

def main(tape_path: str) -> int:
    expected = None
    for raw in Path(tape_path).read_text(encoding="utf-8").splitlines():
        ev = json.loads(raw)
        kind = ev["event"]
        if kind == "edit":
            expected = ev["sha256_after"]
        if kind == "test":
            proc = subprocess.run(shlex.split(ev["cmd"]))
            if proc.returncode != ev["exit"]:
                print("test exit mismatch", file=sys.stderr)
                return 2
    actual = sha256_file("src/mathutil.py")
    if expected and actual != expected:
        print(f"hash mismatch: {actual} != {expected}", file=sys.stderr)
        return 3
    print("replay ok")
    return 0

if __name__ == "__main__":
    raise SystemExit(main(sys.argv[1]))

The clean-tree dance is short. It should sting.

git checkout -- src/mathutil.py
python tools/replay.py witness.jsonl

Replay fails after checkout, because the file is broken again. That failure is the lesson. The tape recorded a hash of a file that no longer exists. Students must re-apply the edit, or store a patch beside the tape. The facilitator prefers a unified diff named witness.patch, committed next to witness.jsonl.

python -m tools.stub_agent
git diff -- src/mathutil.py > witness.patch
git checkout -- src/mathutil.py
git apply witness.patch
python tools/replay.py witness.jsonl
python -m pytest -q

Now the green check has a neighbor. The neighbor is a file another machine can read. Conversation logs can stay in the chat product. They can be interesting. They are not the work.

Twenty minutes: two runners, one tape

The last block splits the room. Half the students replay on the laptop. The other half replay on a remote shell, including a free server if the room has one. The tape, the patch, and the seed commit hash travel together. The model does not.

git rev-parse HEAD
sha256sum witness.jsonl witness.patch src/mathutil.py
PYTHONPATH=. python tools/replay.py witness.jsonl

If the hashes diverge, the students debug the tape, not the model. Clock skew, CRLF, and missing final newlines show up here. Those bugs are gifts. They teach why "it passed on my laptop" is a weak sentence.

A hosted agent is useful when the classroom network is the only place a model can run. It is harmful when it becomes the only copy of the work. The tape has to leave the vendor. Students who cannot download witness.jsonl and witness.patch have rented a demo, not built a change.

One comparison table is enough, drawn on a whiteboard rather than slides. Laptop replay must match remote replay on three values: seed commit, patch hash, and final src/mathutil.py hash. Model name is not a column. Latency is not a column. If those numbers start to matter, the room has left the teaching artifact and started a benchmark they cannot defend in eighty minutes.

What this lab refuses to claim

The stub agent is labeled as a stub. It does not measure model quality, latency, or cost. It does not prove that any product is better at coding than a junior developer. Those claims belong to papers with datasets, not to an eighty-minute lab.

The tape is not a supply-chain attestation. It is not signed. It does not handle secrets. Students must keep tokens out of JSONL, out of patches, and out of pytest output. Anyone doing incident response, medical data, or production billing should not treat this format as evidence. A teaching recorder is not a court recorder.

Teams that already have CI, signed commits, and golden tests may find the workshop slow. That is fine. The audience is the student who still thinks a chat window is a history file. The audience is also the facilitator who is tired of grading screenshots.

When the eighty minutes end, the facilitator asks for three artifacts and nothing else: witness.jsonl, witness.patch, and the seed commit. If a room wants a model endpoint and a server for the two-runner block, MonkeyCode's free access is enough to try the same seed repo, then delete the project after class. The tape still has to live in git.