A Bench Ledger for Shared Free Model Sessions

A shared free bench fails quietly. Two drivers open the same session, and neither one sees the other. The second run dies at minute nine with a timeout nobody can explain.

The fix is one append-only file. It records who opened the bench, what the run touched, and how it ended. A lock file prevents overlap, and a single query turns the log into a rework count on Friday.

This playbook fits teams of two to six people sharing one free lane. Above six, the lock becomes a queue, and the team needs real scheduling instead.

Three roles, one page

A bench is a shared resource, so access has to be owned. Someone drives it, someone reviews the output, and someone keeps the record. The roles can rotate weekly, but they cannot be empty.

The driver holds the lock and runs the session. The driver writes one note per hypothesis, then closes the entry. Nobody else touches the bench while that lock file exists.

The reviewer reads the note before anything merges. Review is one word in the log: pass or fail. A silent merge is exactly the failure this role exists to prevent.

The archivist is the lightest job on the bench. That person runs the weekly query and posts the numbers into the team channel. Ten minutes a week covers it.

Handoffs follow one rule. A driver who leaves early writes a note and closes with abandoned. The next driver opens a fresh entry, reads the last three notes, and starts clean.

Where the two lanes meet

MonkeyCode is an open-source option that fits this structure in two places. Its free model access covers the driver's session, and its free server option covers the shared box. The ledger stays neutral about lanes, because it only records open, note, and close.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Treat both lanes as availability supplied by the project, not a permanent guarantee. Free allowances and server terms change between releases. Check the project page before writing any number into your SOP.

The ledger itself

Install the tool per the project README, then drop this script next to the bench notes. It needs bash and jq, nothing else.

#!/usr/bin/env bash
# bench.sh - one append-only ledger for a shared free-model bench.
set -euo pipefail

BENCH_DIR="${BENCH_DIR:-$HOME/bench}"
LEDGER="$BENCH_DIR/ledger.jsonl"
LOCK="$BENCH_DIR/holder"
mkdir -p "$BENCH_DIR"

utc() { date -u +%Y-%m-%dT%H:%M:%SZ; }

case "${1:-help}" in
  open)
    handle="${2:?handle required}"; task="${3:?task required}"
    if [[ -f "$LOCK" ]]; then echo "held by $(cat "$LOCK")" >&2; exit 3; fi
    printf '%s %s\n' "$handle" "$(utc)" > "$LOCK"
    jq -cn --arg h "$handle" --arg t "$task" --arg ts "$(utc)" \
      '{event:"open",handle:$h,task:$t,utc:$ts}' >> "$LEDGER"
    ;;
  note)
    jq -cn --arg b "${2:?note required}" --arg ts "$(utc)" \
      '{event:"note",body:$b,utc:$ts}' >> "$LEDGER"
    ;;
  close)
    jq -cn --arg r "${2:?result required}" --arg ts "$(utc)" \
      '{event:"close",result:$r,utc:$ts}' >> "$LEDGER"
    rm -f "$LOCK"
    ;;
  who)
    [[ -f "$LOCK" ]] && cat "$LOCK" || echo free
    ;;
  stale)
    o=$(jq -sr '[.[]|select(.event=="open")]|length' "$LEDGER" 2>/dev/null || echo 0)
    c=$(jq -sr '[.[]|select(.event=="close")]|length' "$LEDGER" 2>/dev/null || echo 0)
    if (( o > c )); then echo "unclosed=$((o-c))"; exit 4; else echo clean; fi
    ;;
  *) sed -n '2p' "$0" ;;
 esac

Two behaviors matter more than the code. The lock makes overlap impossible rather than impolite. The stale check surfaces the session somebody forgot to close.

The one-page run

Paste this block into the team wiki and nothing else. The whole SOP fits on one screen.

00 PREFLIGHT  ./bench.sh who             -> free, or a holder name
01 OPEN       ./bench.sh open <handle> "<one-line task>"
02 WORK       one hypothesis per note, small diffs only
03 NOTE       ./bench.sh note "<what changed, what broke>"
04 REVIEW     a second person reruns the failing case before merge
05 CLOSE      ./bench.sh close pass|fail|abandoned
06 WEEKLY     jq summary pasted into the team channel

Step 03 carries the real cost. A note that says "still broken" teaches the next driver nothing. A note that names the input, the command, and the observed error survives the handoff.

Test it in ninety seconds

The script above is small enough to verify before the team depends on it. Run this in a scratch directory and check three outputs.

export BENCH_DIR="$(mktemp -d)"
./bench.sh open ana "retry flaky parser test"
./bench.sh open bo "second driver"; echo "exit=$?"   # expect held by ana, exit=3
./bench.sh note "reproduced with a 3-line input"
./bench.sh close pass
./bench.sh stale                                      # expect clean
wc -l < "$BENCH_DIR/ledger.jsonl"                     # expect 3

A refused second open is the test that matters. If bo gets in while ana holds the lock, the playbook is decoration.

Weekly review is two commands. The first counts outcomes, and the second replays the last notes for the handoff.

jq -sr '[.[]|select(.event=="close")]|group_by(.result)|map({r:.[0].result,n:length})' "$LEDGER"
jq -r 'select(.event=="note")|"\(.utc) \(.body)"' "$LEDGER" | tail -3

A rising abandoned count is the useful signal. It usually means sessions are too long for one sitting, not that the drivers are careless.

Choosing a lane

Situation	Lane	Reason
Shared box, long build, several drivers	free server option	one lock protects one machine, not five laptops
Single driver, short refactor	free model access	coordination cost exceeds the run
Work under compliance review	neither	this ledger is not an audit trail

The table is deliberately short. A longer matrix invites arguments that the log would settle in a minute.

Limits and misfits

This approach is wrong for solo developers. The lock and the notes cost more than they save when one person holds the bench all day.

It is also wrong for regulated code, because ledger.jsonl records intent, not approval. Teams that need per-run cost accounting should add a real timekeeping system, not a JSON line.

Finally, free lanes move. Quotas, model availability, and server lifetimes change without notice. Any hardcoded number in a wiki page goes stale within a release cycle.

Start with one week of entries. Ask a single teammate to drive the ledger and post the Friday numbers. The rework count will decide whether the bench deserves a second driver.