A Token Ledger for the 90-Minute Spike

At twenty past four on a Friday, an engineer closed three browser tabs and told the team the spike was dead. Ninety minutes had gone into asking whether a retry wrapper double-counted 5xx responses, and the honest answer was still maybe. The hypothesis had been reasonable and the probes had been real; the notes simply never recorded which minute, or which token, crossed the line.

The uncomfortable question arrived afterwards. Nobody could say what the probe had spent, because the wall clock had been watched and the token meter had not. A spike that ends without a spend record cannot be replayed, defended in a review, or declared dead with a straight face.

Two budgets, one stop condition

Almost every time-boxed spike carries a single constraint on the whiteboard: ninety minutes. That number survives standups because it is easy to defend, and it is also the constraint least likely to bind when the environment hands out model calls for free. The budget that actually ends these sessions is spend, meaning prompt plus completion tokens, and it moves on a different curve from the clock.

Whichever budget crosses first ends the spike, and the record has to name the one that fired. Without that name, the next engineer re-runs the same probe, pays the same tokens, and lands on the same maybe.

A ledger small enough to live inside the spike

The script below uses only the Python standard library, appends one JSON line per probe, and exposes a single gate subcommand whose exit code is the stop signal. There is no async, no provider SDK, and no database involved. Save it as spike/ledger.py and leave the rest of the harness in shell.

#!/usr/bin/env python3
"""Stdlib-only token ledger for a time-boxed spike.

    python ledger.py init --hypothesis 'retry wrapper double-counts 5xx'
    python ledger.py step --name probe-1 --prompt 1800 --completion 400 --evidence 0
    python ledger.py gate    # exit 1 means: stop and write the verdict
"""
import argparse, json, os, sys, time

LEDGER = os.environ.get('SPIKE_LEDGER', 'spike/ledger.jsonl')
META = LEDGER + '.meta'
DEADLINE_S = int(os.environ.get('SPIKE_DEADLINE_S', 5400))
BUDGET = int(os.environ.get('SPIKE_TOKEN_BUDGET', 250000))
QUIET_STEPS = int(os.environ.get('SPIKE_QUIET_STEPS', 6))


def read_meta():
    with open(META, encoding='utf-8') as fh:
        return json.load(fh)


def read_steps():
    if not os.path.exists(LEDGER):
        return []
    with open(LEDGER, encoding='utf-8') as fh:
        return [json.loads(line) for line in fh if line.strip()]


def cmd_init(args):
    os.makedirs(os.path.dirname(LEDGER) or '.', exist_ok=True)
    if os.path.exists(META) and not args.force:
        sys.exit('ledger already initialised; pass --force to restart the clock')
    with open(META, 'w', encoding='utf-8') as fh:
        json.dump({'t0': time.time(), 'hypothesis': args.hypothesis}, fh)
    open(LEDGER, 'w', encoding='utf-8').close()


def cmd_step(args):
    t0 = read_meta()['t0']
    rec = {
        'name': args.name,
        'prompt_tokens': args.prompt,
        'completion_tokens': args.completion,
        'tokens': args.prompt + args.completion,
        'evidence': args.evidence,
        'note': args.note,
        'elapsed_s': round(time.time() - t0, 1),
    }
    with open(LEDGER, 'a', encoding='utf-8') as fh:
        fh.write(json.dumps(rec) + '\n')
    print(json.dumps(rec))


def cmd_gate(_args):
    meta, steps = read_meta(), read_steps()
    spent = sum(s['tokens'] for s in steps)
    evidence = sum(s['evidence'] for s in steps)
    elapsed = round(time.time() - meta['t0'])
    reasons = []
    if elapsed > DEADLINE_S:
        reasons.append('deadline')
    if spent > BUDGET:
        reasons.append('budget')
    if len(steps) >= QUIET_STEPS and evidence == 0:
        reasons.append('no-evidence')
    report = {
        'hypothesis': meta['hypothesis'],
        'state': 'CONTINUE' if not reasons else 'STOP',
        'stop_reasons': reasons,
        'elapsed_s': elapsed,
        'tokens_spent': spent,
        'evidence_steps': evidence,
        'steps': len(steps),
        'tokens_per_evidence_step': spent // evidence if evidence else None,
    }
    print(json.dumps(report, indent=2))
    return 0 if not reasons else 1


def main(argv):
    p = argparse.ArgumentParser()
    sub = p.add_subparsers(dest='cmd', required=True)
    i = sub.add_parser('init')
    i.add_argument('--hypothesis', required=True)
    i.add_argument('--force', action='store_true')
    i.set_defaults(func=cmd_init)
    s = sub.add_parser('step')
    s.add_argument('--name', required=True)
    s.add_argument('--prompt', type=int, required=True)
    s.add_argument('--completion', type=int, required=True)
    s.add_argument('--evidence', type=int, choices=[0, 1], default=0)
    s.add_argument('--note', default='')
    s.set_defaults(func=cmd_step)
    g = sub.add_parser('gate')
    g.set_defaults(func=cmd_gate)
    args = p.parse_args(argv)
    return args.func(args)


if __name__ == '__main__':
    sys.exit(main(sys.argv[1:]))

Every step record stores the probe name, the counts taken from the provider's usage field, a zero-or-one flag for whether the step produced evidence about the hypothesis, and the seconds elapsed since init. The flag is a human judgement, and that is deliberate, because the ledger should capture what the engineer believed at the time rather than what the postmortem decided a week later.

Two small details keep this from decaying into another abandoned notebook. The init subcommand stamps the clock exactly once, so elapsed time never resets when a step crashes halfway through. The gate only reads the log and never mutates it, which means it can run from a shell loop, a CI job, or a colleague's terminal without side effects.

The harness that lets the gate decide

Because the gate returns a non-zero exit code when any budget is crossed, it can serve directly as the loop condition. The timeout stays in place only as a backstop for a step that hangs, and it should never be the thing that ends a healthy spike.

#!/usr/bin/env bash
set -euo pipefail
export SPIKE_LEDGER='spike/ledger.jsonl'
export SPIKE_DEADLINE_S=5400 SPIKE_TOKEN_BUDGET=250000 SPIKE_QUIET_STEPS=6

python ledger.py init --hypothesis 'retry wrapper double-counts 5xx'

# Gate is the primary stop condition; timeout only catches a hung step.
timeout 5400 bash -c 'while python ledger.py gate > /dev/null 2>&1; do ./one_step.sh; done' || true

python ledger.py gate | tee "spike/verdict-$(date +%s).json"

When the gate and the timeout disagree about why the loop ended, that disagreement is a ledger bug and deserves the same attention as the hypothesis itself. The step body has to read real usage numbers back from the response, because a placeholder that quietly reports zero spend is worse than a crash. The sketch below keeps the raw response next to the counts so an audit can compare the two later.

#!/usr/bin/env bash
# one_step.sh: one real probe. Counts are placeholders and MUST be wired.
set -euo pipefail
stamp="$(date +%H%M%S)"
curl -sS "$SPIKE_ENDPOINT" -o "spike/${stamp}-response.json" -w '%{http_code}' > "spike/${stamp}.code"
python ledger.py step --name "probe-${stamp}" --prompt 0 --completion 0 --evidence 0 --note 'wire real usage counts'

Reading two numbers instead of one

The gate prints tokens_per_evidence_step beside the raw spend, and that ratio is the figure worth arguing about at the sixty-minute mark. A spike that reaches one confirmed observation on thirty thousand tokens is a different artifact from one that reaches the same observation on two hundred thousand, even when both end with the same verdict.

By minute 60	at least one evidence step	no evidence step yet
under 40% of budget	continue, tighten the claim	change the instrument, not the prompt
over 80% of budget	freeze scope, write the verdict	kill the spike, record why

The bottom-right cell is where teams bargain hardest with themselves. A probe that burns four fifths of its budget without producing a single observation is evidence about the shape of the probe, not about the strength of the model, and a longer prompt rarely repairs it.

Where free access shifts the arithmetic

Environment choice is the third variable here, and it changes which budget binds first rather than whether budgets exist at all. A spike that runs on free model access and a free server option removes the invoice from the early part of the session, which leaves elapsed time and evidence quality as the constraints that actually fire. As of this writing the operator behind MonkeyCode, an open-source coding project, describes free model access as a promotional token allowance on the order of ten million tokens plus a free server option; those figures belong to a promotional period and should be re-checked against the project's current terms before a plan depends on them. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Under that arrangement the ledger keeps its value for a narrower reason. Spend stops being the thing that ends the session, so the honest metric to report becomes tokens per evidence step alongside wall-clock minutes, and both live in one JSON line. Free capacity also makes over-probing cheap and therefore likely, which is precisely when a quiet-steps rule earns its keep.

What the ledger cannot do

Provider-side usage counts and gateway-side counts can disagree, and only one of them is the invoice that finance sees. The evidence flag is subjective, so a team that wants to look productive can flag every step and defeat the gate in silence. A green gate is a stop condition rather than a passing test, and nobody should ever read it as coverage.

The approach suits single-hypothesis spikes that need a defensible stopping point, along with teams that keep getting asked what a probe cost. It does not suit work that needs parallel branches, because a linear log will misattribute spend across them, and it should never replace ordinary review of a patch or a security assessment. Anyone whose spike output is expected to become production code has already left the territory where a ninety-minute box means anything.

The verdict stays a short file: hypothesis, decision, evidence steps, tokens, minutes. The ledger is what makes those five fields truthful instead of remembered, and a kill rule that lives in the script travels to the next spike while a rule in someone's head does not. For a first run on free capacity, MonkeyCode's free model access is one place to point the harness at, and the exit code should be trusted the same way regardless of who pays the bill.