DEV Community

Quinn Li
Quinn Li

Posted on

Letter to Past Me: Three Mistakes That Cost a Day of Agent Runs

Letter to Past Me: Three Mistakes That Cost a Day of Agent Runs

Monday, 09:00. I asked an agent to split one module into three files.

Monday, 22:00. Same module. Same failing test. Three stale branches. Zero merged work.

The model was never the problem. My loop was. Past me, here is the letter I should have opened at 09:00.

One afternoon, three unpinned variables

I changed the working directory, the retry budget, and the runtime, all silently.

  • The tree changed between prompts.
  • Token counts looked fine, so I never measured wall-clock.
  • The loop ran on the laptop I was also typing on.

Each mistake is survivable alone. Together they turn one afternoon into a full day.

Mistake 1: every run inherited yesterday's state

I reused one directory for nine prompts. Prompt five read files that prompt three had half-rewritten. The agent then "fixed" code that only existed in that dirty tree.

The fix is a disposable worktree per prompt.

#!/usr/bin/env bash
# fresh.sh — one disposable worktree per prompt, built from a fixed commit
set -euo pipefail
RUN_ID="${1:?usage: fresh.sh <run-id>}"
git worktree add --detach "runs/$RUN_ID" HEAD
echo "ready: runs/$RUN_ID at $(git rev-parse --short HEAD)"
Enter fullscreen mode Exit fullscreen mode

Then:

  1. Point the agent at runs/$RUN_ID, never at your live checkout.
  2. Capture the diff before cleanup.
  3. Remove the tree, so the next prompt starts clean.
git -C "runs/$RUN_ID" diff > "runs/$RUN_ID.patch"
git worktree remove --force "runs/$RUN_ID"
git worktree prune
Enter fullscreen mode Exit fullscreen mode

Why this holds: HEAD cannot move during a run. Your editor stays on main, untouched. The agent gets a sandbox built from the same commit every time.

Mistake 2: I counted tokens and ignored the clock

Free model access changes the economics of retrying. A retry costs nothing in money, so you retry more. The bill moves to wall-clock and review attention, where nobody is watching.

Here is an illustrative ledger. These numbers are examples, not a benchmark. Run the script below to get your own.

Iteration Wall-clock Diff lines Tests Verdict
1 2m 10s 180 fail kept
6 3m 40s 240 fail kept
9 1m 05s 0 fail wasted
11 4m 20s 12 pass merged

Iteration 9 is the expensive one. It burned a minute and produced nothing. Without a ledger that minute is invisible; across eleven iterations it becomes the afternoon.

So past me: stop budgeting tokens. Budget seconds, diff lines, and accepted edits.

#!/usr/bin/env bash
# ledger.sh — append one row per agent iteration
set -euo pipefail

LEDGER="${LEDGER:-runs/ledger.csv}"
LABEL="${1:?usage: ledger.sh <label> <command...>}"
shift
mkdir -p "$(dirname "$LEDGER")" runs

if [ ! -s "$LEDGER" ]; then
  echo "utc,label,seconds,exit,diff_lines,verdict" > "$LEDGER"
fi

start=$(date +%s.%N)
set +e
"$@" > "runs/${LABEL}.out" 2>&1
code=$?
set -e
end=$(date +%s.%N)

secs=$(awk -v a="$start" -v b="$end" 'BEGIN{printf "%.1f", b-a}')
diff_lines=$(git diff --numstat | awk '{s+=$1+$2} END{print s+0}')
echo "$(date -u +%FT%TZ),$LABEL,$secs,$code,$diff_lines," >> "$LEDGER"
Enter fullscreen mode Exit fullscreen mode

Fill the verdict column by hand at review time: kept, wasted, or merged. That single column turns a log into an argument.

Mistake 3: I ran the loop where I also worked

The laptop was my editor, my test runner, and my agent host. One runaway loop made all three slow. I also lost the boundary between my changes and the agent's changes.

The fix is a runner you are willing to throw away. MonkeyCode is an open-source project that the operator describes as offering free model access and a free server option. That is their claim, passed to me; I have not benchmarked it and I do not present it as a measured result.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

As of 2026-09-15, the same operator describes the free allowance as 10M tokens. Treat any free allowance as a moving target. Verify the current number, the data policy, and the server terms on the project's own page before you plan a sprint around them.

What matters for my loop is the split, not the price:

  1. Model calls and long iterations run on the remote runner.
  2. Diff capture, the ledger, and the acceptance gate stay local.
  3. Nothing proprietary leaves my machine unless the policy check passes.

That split survives a price change. A landing page does not.

Decision table: where should this iteration run?

Situation Local Free remote server CI
Prompt touching secrets or customer data yes no only if approved
Iteration under five minutes, interactive yes optional overkill
Long retry loop you will not babysit no yes yes
Needs a GPU or persistent disk state no no provisioned only
Must reproduce on a clean machine verify yes yes
Needs an audit trail for compliance no no yes

Read each row left to right. Your first "yes" is the runner. If a row has no "yes", do not start the loop.

A 30-minute test plan you can copy

  1. Pick a refactor with a fast, non-flaky test suite.
  2. Drop ledger.sh and gate.sh into the repo.
  3. Run three prompts from the same commit, each in its own worktree.
  4. Stop the run when an iteration produces zero diff lines and failing tests.
  5. Summarize the ledger and compute median seconds per kept diff.
  6. Write the stopping rule into the loop before you prompt again.

The gate that saved me the most time:

#!/usr/bin/env bash
# gate.sh — stop the loop when an iteration produced nothing useful
set -euo pipefail
TREE="${1:?usage: gate.sh <worktree>}"
diff_lines=$(git -C "$TREE" diff --numstat | awk '{s+=$1+$2} END{print s+0}')
if [ "$diff_lines" -eq 0 ]; then
  echo "stop: empty iteration, do not resend the same prompt" >&2
  exit 3
fi
Enter fullscreen mode Exit fullscreen mode

And the summarizer:

import csv, statistics

rows = list(csv.DictReader(open("runs/ledger.csv")))
kept = [float(r["seconds"]) for r in rows if r["verdict"] in {"kept", "merged"}]
wasted = sum(float(r["seconds"]) for r in rows if r["verdict"] == "wasted")

print(f"accepted diffs: {len(kept)}")
print(f"median seconds per accepted diff: {statistics.median(kept):.1f}")
print(f"wasted minutes: {wasted / 60:.1f}")
Enter fullscreen mode Exit fullscreen mode

Run it once a day. If wasted minutes exceed accepted minutes, change the prompt, not the model.

Limitations, and who should skip this

  • A free server option is usually a shared resource. Expect queueing and no SLA.
  • Confirm the data policy before sending proprietary code. I only test this on my own repos.
  • Free allowances change without notice. Hardcoding a token number into a plan is a mistake.
  • The ledger needs an automatic pass/fail signal. If your gate is manual review, it measures noise.
  • I ran no comparative benchmark between providers. The table above is illustrative, not evidence.
  • Skip this workflow if compliance forbids third-party runners, or if your job needs a GPU or long-lived state.

What I would tell past me

Pin the tree. Log the seconds. Move the loop off the machine you work on.

Do those three things and Monday ends at 18:00, with one merged branch.

If you want the current free-tier terms, read them on the project's page, not in a blog post. Then run the ledger for one day and trust your own numbers.

Top comments (0)