Dear past me,
You will wire a review agent after dinner.
The demo will look fine in ten minutes.
Then three silent mistakes will steal a day.
The agent will not crash with a stack.
It will assume, retry, and keep talking anyway.
Your laptop fan will be the only alarm.
This letter is a postmortem, not a memoir.
Treat every snippet as a labeled example only.
Do not treat any of it as production telemetry.
The day you will lose
You will ask the agent to review a diff.
It will call read_file on the wrong path.
Then it will "fix" a file that never existed.
You will watch tokens move and feel progress.
Progress without a budget is still not progress.
It is only a loop with better copy.
The model will speak in complete sentences.
Complete sentences are not complete tool results.
That gap is where the day disappears.
Stop now and read these three mistakes first.
Mistake 1: No loop budget
You will let the model call tools freely.
Free models still cost both time and context.
Unbounded steps will hide several bad tool choices.
Set a hard budget before the first call.
A budget is a stop reason, not a vibe.
If you cannot name the stop, you cannot test it.
Numbered fix
- Cap
maxStepsat a small integer tonight. - Cap
maxToolCallsin a separate counter. - Abort after three repeated identical tool payloads.
- Log every step as one structured event.
- Canonicalize every payload before you hash them.
Whitespace changes should not reset the repeat counter.
{"path":"a.ts"} and { "path": "a.ts" } are one call.
Normalize JSON before you store the seen key.
Here is a labeled harness written in Node.js.
// labeled example: loop budget, not production telemetry
export function canonicalToolKey(name, payload) {
let normalized = String(payload).trim();
try {
normalized = JSON.stringify(JSON.parse(payload));
} catch {
// keep trimmed text when the payload is not JSON
}
return `${name}:${normalized}`;
}
export function createLoopBudget({ maxSteps = 8, maxToolCalls = 6 } = {}) {
const seen = new Map();
return {
steps: 0,
toolCalls: 0,
shouldStop(event) {
this.steps += 1;
if (event.type === "tool") {
this.toolCalls += 1;
const key = canonicalToolKey(event.name, event.payload);
seen.set(key, (seen.get(key) || 0) + 1);
if (seen.get(key) >= 3) {
return { stop: true, reason: "repeated_tool" };
}
}
if (this.steps > maxSteps) {
return { stop: true, reason: "max_steps" };
}
if (this.toolCalls > maxToolCalls) {
return { stop: true, reason: "max_tool_calls" };
}
return { stop: false, reason: null };
},
};
}
Run a dry check before you add networking.
Node 18 or newer already ships node:test.
node --test agent/loop-budget.test.js
Write the failing test before you add the budget.
Green tests with no fixtures are not evidence.
Why the same tool fires three times
The model does not see your disk.
It sees the last tool result you returned.
If that result is vague, it retries the same read.
"File not found" without the path invites a guess.
Return the path, the cwd, and a stop hint.
Then the next step can halt instead of riffing.
// labeled example: make missing files expensive to retry
export function missingFileResult(path, cwd) {
return {
ok: false,
error: "missing_file",
path,
cwd,
hint: "do_not_retry_same_path",
};
}
Feed that object back as the tool output.
Do not feed a prose apology from the model.
Prose apologies become new guesses on the next step.
Mistake 2: Partial JSON as a tool call
You will stream model tokens into a local buffer.
A brace will appear and you will parse early.
JSON.parse will throw, or worse, succeed with garbage.
A truncated path value is not a real path.
Your agent will invent the rest of the filename.
Then it will patch a neighbor file instead.
Streaming is for display. Parsing is for completion.
Do not mix those two jobs in one function.
The UI can render partial text without calling tools.
Numbered fix
- Buffer tokens until the stream signals
done. - Reject payloads that fail a schema check.
- Require
nameandargumentsas one pair. - Never repair JSON with a second model call.
- Guard file paths before any disk read.
// labeled example: complete-object gate
const ALLOWED = new Set(["read_file", "list_dir", "apply_patch"]);
export function parseToolCall(buffer, { done }) {
if (!done) {
return { ok: false, reason: "incomplete_stream" };
}
let parsed;
try {
parsed = JSON.parse(buffer);
} catch {
return { ok: false, reason: "invalid_json" };
}
if (parsed === null || typeof parsed !== "object" || Array.isArray(parsed)) {
return { ok: false, reason: "schema_mismatch" };
}
if (!ALLOWED.has(parsed.name)) {
return { ok: false, reason: "schema_mismatch" };
}
if (
parsed.arguments === null ||
typeof parsed.arguments !== "object" ||
Array.isArray(parsed.arguments)
) {
return { ok: false, reason: "schema_mismatch" };
}
return { ok: true, value: parsed };
}
export function guardReadFile(args, { exists }) {
if (!Object.hasOwn(args, "path")) {
return { ok: false, reason: "missing_path" };
}
if (typeof args.path !== "string" || args.path.length === 0) {
return { ok: false, reason: "empty_path" };
}
if (args.path.split("/").includes("..")) {
return { ok: false, reason: "unsafe_path" };
}
if (!exists(args.path)) {
return { ok: false, reason: "missing_file" };
}
return { ok: true, value: args.path };
}
Pick one contract and reject the rest.
Names like readFile and read_file are different contracts.
Fuzzy aliases feel helpful and then hide retries.
Assumed paths are how a half-day disappears.
The model will speak as if the file exists.
Your tools must not share that belief.
Mistake 3: No replay fixture
You will debug live against a moving model.
Each retry will change the story in small ways.
You will chase those ghosts for a full day.
A saved transcript is cheaper than another prompt.
Record one failing run as a JSONL fixture.
Replay that fixture without touching the live network.
Numbered fix
- Write each event into
transcripts/fail-01.jsonlnow. - Store prompts, tool calls, and stop reasons.
- Replay the run with the model stubbed out.
- Assert the budget still trips in the same place.
- Keep a second fixture for schema failures.
// labeled example: replay without a live model
import { test } from "node:test";
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import { createLoopBudget } from "./loop-budget.js";
import { parseToolCall } from "./parse-tool-call.js";
function loadEvents(path) {
const raw = readFileSync(path, "utf8").trim();
if (raw.length === 0) {
throw new Error("empty transcript");
}
return raw.split("\n").map((line) => JSON.parse(line));
}
test("repeated read_file stops the loop", () => {
const events = loadEvents("transcripts/fail-01.jsonl");
const budget = createLoopBudget({ maxSteps: 8, maxToolCalls: 6 });
let last = { stop: false, reason: null };
for (const event of events) {
last = budget.shouldStop(event);
if (last.stop) break;
}
assert.equal(last.reason, "repeated_tool");
});
test("partial json is not a tool call", () => {
const result = parseToolCall('{"name":"read_file","arguments":{"path":"src/",', {
done: false,
});
assert.equal(result.ok, false);
assert.equal(result.reason, "incomplete_stream");
});
Sample fixture. Keep it tiny and ugly.
{"type":"tool","name":"read_file","payload":"{\"path\":\"src/missing.ts\"}"}
{"type":"tool","name":"read_file","payload":"{\"path\":\"src/missing.ts\"}"}
{"type":"tool","name":"read_file","payload":"{ \"path\": \"src/missing.ts\" }"}
If replay cannot fail, you do not have a test.
You have a chat log with extra ceremony.
The third line proves canonicalization is doing work.
Two fixtures beat twelve live retries.
Live retries train you to prompt harder.
Fixtures train you to stop sooner.
Decision table for the first night
| Signal | Likely cause | Stop reason to assert |
|---|---|---|
| Stream ends mid-object | Parser ran before done
|
incomplete_stream |
| Same tool, same path, 3x | Missing budget or vague error | repeated_tool |
| Twelve reads, no patch | Step cap set too high | max_tool_calls |
| Live run differs from replay | Unrecorded side effect | Add the missing event |
| Patch on a missing file | Path guard never ran | missing_file |
Print this table and keep it beside your terminal.
Pick one row and prove it with a fixture.
Then, and only then, call a live model.
A workflow you can run tonight
- Copy the budget module into
agent/loop-budget.js. - Add
parseToolCallandguardReadFilebeside it. - Record one real failure as a JSONL file.
- Run
node --testuntil replay is red, then green. - Only after green, point the loop at a model.
mkdir -p agent/transcripts
# save loop-budget.js, parse-tool-call.js, and the tests
node --test agent/*.test.js
If tests pass with an empty transcripts folder, stop.
Your loader is too forgiving so fix that first.
An empty file must throw, not report green.
Record the stop reason in the same JSONL line.
Future you will not remember the terminal scroll.
The fixture will hold the reason still.
Where a free model actually helps
You still need a model for the first transcript.
A local stub cannot invent the first failure shape.
A free remote option is enough for that capture.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode documents free model access for this kind of loop.
It also documents a free server option for hosting the harness.
Use either as an input to the budget.
Do not treat them as a substitute for checks.
Do not skip the parser because the server is free.
Do not raise maxSteps because tokens feel cheap.
The day you will lose is still a calendar day.
If the harness helps, read the project docs.
Try the free server only on a throwaway transcript.
Limitations
This harness does not prove the patch is correct.
It only proves the loop stops for known reasons.
Schema checks do not equal a real security review.
JSONL replay ignores clock skew and rate limits.
It also ignores truncated HTTP bodies at the proxy.
Add those events if your failures live there.
The allow-list will lag your real tool list.
Update the allow-list in the same commit as the tool.
Otherwise replay will fail for the wrong reason.
The path guard is not a sandbox.
It only rejects missing or parent-escaping paths.
This path split ignores backslash separators on Windows.
Process isolation is a different article.
Who should not use this
Do not use this loop on production credentials.
Do not point these tools at customer data.
Do not run apply_patch outside a throwaway branch.
Skip this if you do not control tool execution.
A chat UI with no tools does not need a budget.
A single-shot summarizer does not need this replay.
Teams with an existing agent runtime should extend that.
Do not run a second loop beside a vendor orchestrator.
Two budgets will lie in two opposite directions.
Skip it for one-off curl experiments as well.
A budget around a single HTTP call adds noise.
Save the harness for loops that call tools.
Close the letter
Dear past me, ship the budget before the prompt.
Ship the parser before you accept the first stream.
Ship the replay fixture before the second retry.
The model will still assume, and that is normal.
Your job is to make each assumption expensive.
Cap eight steps, six tools, and one recorded failure.
Then go to sleep so the fan can rest too.
Top comments (0)