Letter to Yesterday-Me: The Three Mistakes That Ate a Full Day of Agent Runs

Yesterday began with one prompt and a clean checkout. I remember it differently than it happened.

By 18:40 I had one merged diff and a vague story. A ledger would have said: fourteen attempts, three accepted, one day gone.

This is the letter I would send to 09:00-me. Three mistakes, in the order they cost me time.

The setup I should have used

Two parts of the loop are free right now. MonkeyCode offers free model access for the writing step, and a free server option for the runner step. The operator of this account states a 10M-token allowance on the free tier.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Free tiers move. Confirm the current allowance, server specs, and terms on the project page. Do that before you build a week around them.

Mistake 1 — I logged outcomes, not attempts

I remembered the good runs and forgot the eleven bad ones. That is not data. That is a mood.

Without an attempt ledger, you cannot separate a weak model from a broken harness. You also cannot compare two models without guessing.

Step 1: append one JSON line per attempt

This script uses the standard library only. Write it before the first prompt, not after the fifth failure.

#!/usr/bin/env python3
# attempt_ledger.py - append-only record of every model attempt.
import argparse, json, sys
from datetime import datetime, timezone
from pathlib import Path

LEDGER = Path('runs/attempts.jsonl')

def append(rec):
    LEDGER.parent.mkdir(parents=True, exist_ok=True)
    rec['ts'] = datetime.now(timezone.utc).isoformat(timespec='seconds')
    with LEDGER.open('a', encoding='utf-8') as f:
        f.write(json.dumps(rec, sort_keys=True) + '\n')

def cmd_log(a):
    append({'attempt': a.attempt, 'model': a.model, 'verdict': a.verdict,
            'wall_s': a.wall_s, 'tokens': a.tokens, 'gate': a.gate})

def cmd_summary(_a):
    if not LEDGER.exists():
        sys.exit('no ledger yet')
    rows = [json.loads(l) for l in LEDGER.read_text().splitlines() if l.strip()]
    agg = {}
    for r in rows:
        m = agg.setdefault(r['model'], {'n': 0, 'acc': 0, 'wall': [], 'tok': 0})
        m['n'] += 1
        m['acc'] += 1 if r['verdict'] == 'accept' else 0
        m['wall'].append(r['wall_s'])
        m['tok'] += r['tokens']
    print('{:<24}{:>4}{:>7}{:>8}{:>10}'.format('model', 'n', 'acc%', 'med_s', 'tok/acc'))
    for model, m in sorted(agg.items()):
        wall = sorted(m['wall'])
        med = wall[len(wall) // 2]
        per = m['tok'] / m['acc'] if m['acc'] else float('inf')
        print('{:<24}{:>4}{:>6.0f}%{:>8.0f}{:>10.0f}'.format(
            model, m['n'], 100 * m['acc'] / m['n'], med, per))

p = argparse.ArgumentParser()
sub = p.add_subparsers(dest='cmd', required=True)
lg = sub.add_parser('log')
lg.add_argument('--attempt', type=int, required=True)
lg.add_argument('--model', required=True)
lg.add_argument('--verdict', choices=['accept', 'reject', 'error'], required=True)
lg.add_argument('--wall-s', type=float, required=True)
lg.add_argument('--tokens', type=int, default=0)
lg.add_argument('--gate', default='none')
lg.set_defaults(fn=cmd_log)
sm = sub.add_parser('summary')
sm.set_defaults(fn=cmd_summary)
args = p.parse_args()
args.fn(args)

Run it after every attempt. Do not batch the entries at end of day.

python3 attempt_ledger.py log --attempt 7 --model free-tier --verdict reject \
  --wall-s 212 --tokens 8400 --gate build
python3 attempt_ledger.py summary

The summary prints one row per model. Treat tok/acc as your cost column. The block below shows the output shape only. It is not a measured result:

model                       n   acc%   med_s   tok/acc
free-tier                   9     33%     210      9800

One rule matters here. A reject still needs a gate name. Use build, scope, or proof. No gate, no row.

Mistake 2 — I treated the runner box as a snowflake

The free server option is the tempting part. It is also where drift hides.

My sixth attempt died on a lockfile mismatch. Nothing was wrong with the prompt. The box had moved on since yesterday.

Step 2: freeze the environment before the agent runs

Pin the lockfiles. Verify the hash at the top of every run.

# once, on a green tree
sha256sum package-lock.json > env.lock

# preflight.sh - run before each attempt
#!/usr/bin/env bash
set -euo pipefail
test -f env.lock || { echo 'missing env.lock'; exit 1; }
sha256sum -c env.lock > /dev/null || { echo 'environment drifted'; exit 1; }
git diff --quiet || { echo 'dirty tree, commit first'; exit 1; }
echo 'preflight ok'

Three failures this catches: a stale lockfile, uncommitted edits, and a rerun on top of old state. Each of those cost me an hour or more.

Run preflight before the prompt, not after the diff. The order is the entire point.

Mistake 3 — I reviewed diffs without a rubric

'Looks right' is not a gate. It is a coin flip with extra typing.

Every vibe-accepted diff came back later as a review comment. That is the expensive way to learn.

Step 3: three gates, one verdict

Score every diff against the same three gates. Reject on the first failure.

Gate	Question	Fail action
build	Does the test suite pass?	log with --gate build
scope	Does the diff touch only allowed paths?	log with --gate scope
proof	Did a new test fail before the change and pass after?	log with --gate proof

The proof gate is the one people skip. Run the new test against the old code first. Watch it fail. Then accept the diff.

That single step converts 'the model seems good' into a claim you can defend in review.

When the free tier is the wrong tool

Free access is not a universal answer. It is a budget choice, and budgets have edges.

Situation	Free model + free server?	Reason
Throwaway refactor in a scratch repo	Yes	Low blast radius
Comparing two candidate approaches	Yes	The ledger gives a fair read
Code you cannot send to a third party	No	Data handling is your call, not the tier's
A release gate that must be deterministic	No	Free capacity is best-effort, not an SLA
Latency-sensitive interactive tooling	No	Queue behaviour shows up in your UX

Write the boundary down before you start. Deciding at hour six is how the day disappears.

The loop, end to end

1. Commit a clean tree.
2. Run preflight.sh.
3. Prompt with the failing test attached.
4. Apply the diff.
5. Run the three gates.
6. Log the verdict and the gate that failed.
7. Repeat. Run summary only at end of day.

Seven steps, about ten minutes of setup. The rest is attempts.

MonkeyCode sits at steps 3 and 5: free model access for the writing, the free server option for the runs. If you want to test this loop without a card, the free tier carries steps 1 through 3.

What this does not fix

A ledger will not make a weak model strong. It only tells you the truth sooner.

Free capacity is best-effort. Do not put a deadline you cannot move behind it.

And no rubric replaces reading the diff. It replaces remembering the diff.

Send this to 09:00-me and the day costs four hours. Not eight.