The Witness Mark Workshop

A Friday handoff failed in a quiet way. The agent run had stopped, the ticket was still open, and the only note in the log was the word done. The next maintainer could not tell whether the job had finished, stalled on a constraint, or been cut short by a person. In a machine shop, a witness mark is the thin remaining line that shows where the cutter actually traveled, and a transcript without that line forces the next shift to guess.

This workshop spends 75 minutes on the habit. Six students share one fixture, a JSONL note, and a small Python checker. They do not tune a model, and they do not chase a score. They practice leaving a mark that a partner can rerun without a hallway conversation.

The lesson stays useful on a laptop with the standard library. A hosted bench is optional, and it is not the grader.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. The only product claims used here are operator-supplied: free model access, and a free server option, either of which can serve as that spare bench when a class wants a second place to draft a messy note. Quotas, model names, hardware, and duration are omitted because they were not verified for this draft.

The first ten minutes stay on paper. The instructor reads the night-shift note aloud and requires a card with five fields: intent, constraint, evidence, stop reason, and the exact command a stranger would type. Vague praise is rejected in the room before it can hide inside a file. Students hear the difference between a status word and a mark while the keyboard is still closed.

Minutes ten through thirty move to disk. Each student creates the bad twin, runs the checker, and writes the faults in ordinary words before editing anything. The repaired twin is the worked example, saved as a single JSONL line so a cold machine can rerun it.

mkdir -p samples
cat > samples/night_shift.jsonl << 'EOF'
{"intent":"close ticket 441","constraint":"stop on a repeated error","evidence":"looks fine","stop_reason":"done","rerun_command":"python3 witness_mark.py samples/night_shift.jsonl"}
EOF
cat > samples/repaired.jsonl << 'EOF'
{"intent":"close ticket 441 by confirming the retry budget","constraint":"stop if the same error id appears twice","evidence":"error_id=E17 count=2 at log line 88","stop_reason":"stopped after E17 repeated at line 88, before a third retry","rerun_command":"python3 witness_mark.py samples/repaired.jsonl"}
EOF
python3 witness_mark.py samples/night_shift.jsonl
python3 witness_mark.py samples/repaired.jsonl

A held file prints HOLD and leaves a non-zero exit. A cleared file prints CLEAR and then the line all marks clear. The contrast is the point of the half hour. One note can be replayed by a stranger, and the other only sounds finished.

The checker is a proposal for the room. It is not a production auditor, and it has not been scored against a live corpus. Failing output is a teaching signal, not a certificate. The fixture expects Python 3.9 or newer, because the annotations use built-in generic types.

#!/usr/bin/env python3
"""Proposal: classroom witness-mark checker. Not a production auditor."""

import json
import sys
from pathlib import Path

VAGUE = {"done", "ok", "fixed", "success", "completed", "finished", "lgtm"}
REQUIRED = ("intent", "constraint", "evidence", "stop_reason", "rerun_command")

def load_marks(path: Path) -> list[dict]:
    rows = []
    for lineno, line in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
        text = line.strip()
        if not text:
            continue
        try:
            row = json.loads(text)
        except json.JSONDecodeError as exc:
            raise SystemExit(f"{path}:{lineno}: invalid json: {exc}") from exc
        if not isinstance(row, dict):
            raise SystemExit(f"{path}:{lineno}: expected an object")
        row["_line"] = lineno
        rows.append(row)
    if not rows:
        raise SystemExit(f"{path}: no marks")
    return rows

def judge(row: dict) -> list[str]:
    faults = []
    for key in REQUIRED:
        value = row.get(key)
        if not isinstance(value, str) or not value.strip():
            faults.append(f"missing {key}")
    reason = str(row.get("stop_reason", "")).strip().lower().rstrip(".")
    if reason in VAGUE:
        faults.append("stop_reason is a slogan, not a mark")
    elif len(reason.split()) < 8:
        faults.append("stop_reason needs a verb, an object, and a boundary")
    evidence = str(row.get("evidence", ""))
    if evidence and not any(ch.isdigit() for ch in evidence):
        faults.append("evidence should name a line, id, or count")
    command = str(row.get("rerun_command", "")).strip()
    if command and not (
        command.startswith("python3 ")
        or command.startswith("pytest ")
        or command.startswith("./")
    ):
        faults.append("rerun_command must be a local python3, pytest, or ./ command")
    return faults

def main() -> None:
    if len(sys.argv) != 2:
        raise SystemExit("usage: python3 witness_mark.py <marks.jsonl>")
    path = Path(sys.argv[1])
    failed = 0
    for row in load_marks(path):
        faults = judge(row)
        label = f"{path}:{row['_line']}"
        if faults:
            failed += 1
            print(f"HOLD {label}")
            for fault in faults:
                print(f"  - {fault}")
        else:
            print(f"CLEAR {label}")
    if failed:
        raise SystemExit(f"{failed} mark(s) held")
    print("all marks clear")

if __name__ == "__main__":
    main()

A held note is not an insult. It means the next person would have to invent a story. The instructor walks the room once and refuses oral rescues, because a reason that cannot survive in the file cannot survive the shift change. Clever chat does not count as a witness mark, any more than a verbal claim counts on a bench where the layout line has been polished away.

The middle twenty minutes belong to the partner swap. Students exchange only the JSONL file, with no side thread and no shared screen. The partner reruns the command already written inside the note and decides whether the stop is understandable.

If the partner must ask what E17 means, the evidence field is still thin, even when the checker prints CLEAR. The script catches missing structure. It cannot catch a number that points at the wrong line. That gap is discussed on purpose, so nobody leaves believing a green line is the same thing as understanding.

The optional hosted pass takes the next ten minutes, and only if the spare bench is actually available. A student drafts one deliberately bad note there, copies it back, and runs the same local checker. The bench is not the grader. If access is missing, slow, or unfamiliar, the class skips the pass and stays with the two sample files.

The last fifteen minutes are the limitation review. The checker does not prove the agent was right. It only shows that the note has the fields a stranger needs in order to try the same command. Creative tasks and open-ended research will fight the required shape, and they should not be forced through it.

Teams that need a formal audit, a signed control, or a retention policy should not adopt this fixture as evidence. It stores whatever the student typed. Secrets, customer payloads, and private prompts do not belong in the JSONL, on a laptop or on a free server. A class without a human reviewer for this closing block will over-trust the word CLEAR.

The vague-word list needs the same honesty. The word done is banned because it hides the boundary, not because short writing is a moral failure. A precise five-word reason can be true and still fail the eight-word heuristic. Instructors should allow a paper override when the spoken review shows a real boundary that the script was too blunt to accept.

The heuristic is a teaching rail, not a law of clear writing. Empty notes were the problem in this room. Comments, as a habit, were not.

A solo developer who already writes replayable stop notes will learn little from the timing. A group that wants vendor benchmarks, model rankings, or a promised token budget will not find those measurements here, and should not invent them to fill the gap. The artifact is a note linter with a shop-floor rule: leave the line visible for the next person.

The closing move stays quiet. Each student keeps two files, the held note and the cleared note, beside the checker. Next week the same commands still run. That is the property the night shift needed.

Readers who want a spare bench for the optional pass can review MonkeyCode's current free model access and free server option, then confirm the live terms before a class depends on them.