If the Remainder Doesn't Shrink, It's a Zero: A Bootcamp Lab on Agent Loop Progress

Your agent can thrash for forty turns and still be stuck. Busy is not progress. In this lab you fail unless every iteration names a remainder and that remainder strictly shrinks.

I do not grade the model's vibe. I grade a number that goes down.

Why this lab, and why now? Coding agents look productive because they emit files, tool calls, and confident summaries. That is theater. If the work left to do does not get smaller, you built a spinner with extra logging.

The failure I am tired of grading

A student ships an agent that "refactors the repo." It lists files. It rereads the same module. It rewrites a comment. Then it lists files again. The transcript is long. The remainder is identical.

Sound familiar? That loop is not searching. It is pacing.

Halt contracts tell you when to stop. Token budgets tell you what the run cost. Traces tell you what happened. None of those prove the loop moved. This lab is the missing piece: a progress contract.

What a remainder is

A remainder is a typed, comparable quantity of unfinished work. Not a paragraph. Not "almost done." A number plus a unit plus a proof rule.

Pick one remainder kind for the assignment. Stick with it for the whole run.

• unresolved_files: count of files still failing the assignment tests
• unresolved_assertions: count of failing asserts in a frozen test file I provide
• open_todos: count of TODO(lab) markers the agent is allowed to create, then must delete
• schema_violations: count of JSON records that fail a schema you freeze in schema.json

If you cannot compare remainder n with remainder n-1, you do not have a remainder. You have a diary.

Lab setup

You need Node 20+, git, and a throwaway directory. Do not run this against a real repo. The agent will write files. That is the point of a sandbox, not a personality trait.

mkdir agent-remainder-lab && cd agent-remainder-lab
git init
npm init -y
mkdir src tests fixtures

Freeze the human oracle first. The agent does not get to rewrite the tests that define "done." If the model authors the exit condition, the remainder can shrink by cheating. We already buried that lesson. Not this week.

cat > tests/oracle.test.mjs <<'EOF'
import test from 'node:test';
import assert from 'node:assert/strict';
import { readFileSync, existsSync } from 'node:fs';

test('inventory.json exists and parses', () => {
  assert.equal(existsSync('fixtures/inventory.json'), true);
  JSON.parse(readFileSync('fixtures/inventory.json', 'utf8'));
});
EOF

Seed a broken fixture so the first remainder is honest.

printf 'not-json\n' > fixtures/inventory.json
node --test tests/oracle.test.mjs; echo "oracle exit: $?"

That failing oracle is your starting remainder. Write it down before the agent starts. If you skip the baseline, later "progress" is fan fiction.

Artifact: a progress harness the grader can rerun

Every agent iteration must append one JSON line to remainder.ndjson. No pretty print. No extra keys the harness does not know. If the file is missing, the run is a zero.

// remainder-contract.mjs
export const ALLOWED_KINDS = new Set([
  'unresolved_files',
  'unresolved_assertions',
  'open_todos',
  'schema_violations',
]);

export function parseLine(line, lineno) {
  let row;
  try {
    row = JSON.parse(line);
  } catch {
    throw new Error(`line ${lineno}: not JSON`);
  }
  for (const key of ['iteration', 'kind', 'value', 'unit', 'note']) {
    if (!(key in row)) throw new Error(`line ${lineno}: missing ${key}`);
  }
  if (!Number.isInteger(row.iteration) || row.iteration < 1) {
    throw new Error(`line ${lineno}: iteration must be an integer >= 1`);
  }
  if (!ALLOWED_KINDS.has(row.kind)) {
    throw new Error(`line ${lineno}: unknown kind ${row.kind}`);
  }
  if (!Number.isInteger(row.value) || row.value < 0) {
    throw new Error(`line ${lineno}: value must be an integer >= 0`);
  }
  if (typeof row.unit !== 'string' || row.unit.length === 0) {
    throw new Error(`line ${lineno}: unit must be a non-empty string`);
  }
  return row;
}

export function assertProgress(rows) {
  if (rows.length === 0) throw new Error('no remainder rows');
  const kind = rows[0].kind;
  const unit = rows[0].unit;
  let prev = Number.POSITIVE_INFINITY;
  let stalled = 0;
  for (const row of rows) {
    if (row.kind !== kind) throw new Error('kind changed mid-run');
    if (row.unit !== unit) throw new Error('unit changed mid-run');
    if (row.value < prev) {
      stalled = 0;
      prev = row.value;
      continue;
    }
    stalled += 1;
    if (stalled > 1) {
      throw new Error(`no shrink for ${stalled} consecutive iterations at ${row.value} ${unit}`);
    }
  }
  return { kind, unit, start: rows[0].value, end: rows[rows.length - 1].value, iterations: rows.length };
}

One stall is allowed. Two consecutive non-shrinks are a fail. Why the slack? Tools flake. Networks hiccup. I am not grading luck. I am grading whether the loop can recover and still reduce the remainder.

// remainder-harness.mjs
import { readFileSync } from 'node:fs';
import { parseLine, assertProgress } from './remainder-contract.mjs';

const lines = readFileSync('remainder.ndjson', 'utf8')
  .split('\n')
  .map((l) => l.trim())
  .filter(Boolean);

const rows = lines.map(parseLine);
const summary = assertProgress(rows);
if (summary.end !== 0) {
  throw new Error(`remainder ended at ${summary.end} ${summary.unit}, expected 0`);
}
console.log(JSON.stringify(summary, null, 2));

Run it like a grader, not like a demo.

printf '%s\n' \
  '{"iteration":1,"kind":"unresolved_assertions","value":3,"unit":"asserts","note":"oracle red"}' \
  '{"iteration":2,"kind":"unresolved_assertions","value":3,"unit":"asserts","note":"reread same file"}' \
  '{"iteration":3,"kind":"unresolved_assertions","value":1,"unit":"asserts","note":"fixed parse"}' \
  '{"iteration":4,"kind":"unresolved_assertions","value":0,"unit":"asserts","note":"oracle green"}' \
  > remainder.ndjson

node remainder-harness.mjs

Swap the last non-zero pair so values go 3, 3, 3 and watch it throw. That is the whole lab. A spinner should hurt.

Checkpoints

Do these in order. Skipping ahead is how people invent a remainder after the fact.

1. Baseline. Run the frozen oracle. Write iteration 1 by hand. Same number the test runner reported. No rounding. No "about three."
2. Kind lock. Commit kind and unit in LAB.md. Changing them later is a rewrite of the assignment.
3. Loop wrapper. Your agent may call tools. After every tool batch, it must append one remainder line, then reread the oracle. Guessing the remainder from memory is a fail.
4. Stall rule. One flat iteration is legal. The second flat iteration must be a declared blocked stop with a reason, or the harness fails you.
5. Zero means zero. value: 0 is accepted only if node --test tests/oracle.test.mjs exits 0 on a clean checkout of your submission.

Checkpoint 3 is where most people cheat. They let the model narrate progress, then they type a shrinking sequence. I rerun the oracle. Your NDJSON is not evidence unless it matches the test runner.

A legal agent loop, not a vibe loop

Here is a skeleton. It is a lab wrapper. It is not a product.

// loop.mjs — proposed student wrapper, not a claimed production agent
import { appendFileSync } from 'node:fs';
import { spawnSync } from 'node:child_process';

function oracleFailures() {
  const r = spawnSync('node', ['--test', 'tests/oracle.test.mjs'], { encoding: 'utf8' });
  const fails = (r.stderr + r.stdout).match(/not ok /g);
  return fails ? fails.length : (r.status === 0 ? 0 : 1);
}

function record(iteration, value, note) {
  const row = {
    iteration,
    kind: 'unresolved_assertions',
    value,
    unit: 'asserts',
    note,
  };
  appendFileSync('remainder.ndjson', JSON.stringify(row) + '\n');
}

// You plug your tool-calling agent into step().
// step() may edit files. It may not edit tests/oracle.test.mjs.
async function run({ maxIters = 8, step }) {
  let prev = oracleFailures();
  record(1, prev, 'baseline');
  let stalled = 0;
  for (let i = 2; i <= maxIters; i++) {
    await step(i, prev);
    const now = oracleFailures();
    record(i, now, `after step ${i}`);
    if (now === 0) return { ok: true, iteration: i };
    if (now < prev) stalled = 0;
    else stalled += 1;
    if (stalled > 1) {
      throw new Error('remainder did not shrink; stop instead of spinning');
    }
    prev = now;
  }
  throw new Error('hit maxIters with remainder > 0');
}

export { run, oracleFailures };

Notice what is missing. No retry storm. No "one more pass." The loop is allowed to work only while the remainder falls. That is harsh. It is also how you keep a weekend lab from becoming an unbounded cloud bill.

Where a free coding agent actually belongs

You can implement step() with any model that can edit files. The grade is the harness, not the vendor.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I mention MonkeyCode here for one lab reason: students who do not want to paste personal API keys into a bootcamp wrapper can use its free model access and the free server option to run step() off their laptop. That is convenience. It is not extra credit. If the remainder file is fake, the free server will not save you.

Do not treat the remote box as a second brain. Treat it as a machine that still has to leave remainder.ndjson and a green oracle in the repo I clone.

Fair grading rubric

I grade artifacts I can rerun. I do not grade screenshots of a chat.

Score	What I clone	Remainder rule	Oracle
0	Missing remainder.ndjson, or tests edited	n/a	n/a
1	NDJSON parses, kind locked	Values wander or stall twice	Still red
2	NDJSON monotonic with one stall max	Ends above 0	Red or flaky
3	Harness exits 0	Ends at 0	Green on my machine
4	Score 3 plus a blocked path	Declares block instead of spinning	Green or honest block

Stretch does not replace a broken remainder. A beautiful agent that cannot shrink a number is still a 1.

Stretch goals

Only after a 3.

• Replay. Delete node_modules, recopy fixtures, rerun loop.mjs. Same remainder sequence shape, same final zero. Hidden local caches are a different lab. Do not smuggle them back in.
• Two remainders, one primary. You may log a secondary metric (diff lines, tool calls) but the harness still keys off one kind. Secondary numbers are comments. They are not the grade.
• Poisoned tool output. Feed step() a tool result that claims the oracle is green when it is not. The wrapper must remeasure with oracleFailures(), not trust the tool text.
• Budget sibling. Cap iterations, not vibes. If you already did a token-budget lab, do not double-count tokens here. This grade is progress, not spend.

That third stretch is the one I actually care about. Models will believe their own tools. Your wrapper should not.

Limitations, and who should skip this

This remainder model is crude. Some real work is non-monotone: you add a failing test on purpose, remainder jumps, then it falls. I ban that jump in the core lab because I cannot fairly grade "strategic getting worse" in a two-hour window. If your research loop needs exploratory regressions, this rubric will punish you. Use a different contract.

Skip this lab if you are driving production incident response. A shrinking unit-test count is not a blast radius. Skip it if your agent only answers questions and never touches files. There is no remainder to shrink. Skip it if you cannot freeze an oracle. Then you are grading prose again.

Also skip any remainder you cannot recompute from the repo. "Confidence: 0.81" is not a remainder. I cannot rerun your model's self-esteem.

What I want in your writeup

Keep it short. I read the NDJSON first.

• The kind and unit you locked, in one sentence
• The baseline command and the number it printed
• One iteration that stalled, if any, and why you allowed it
• The final harness output
• One thing the agent wanted to do that would have grown the remainder

If your writeup is longer than the remainder file, you are hiding. Cut it.

Close

An agent loop without a shrinking remainder is a while (true) with better copy. Name the leftover work. Measure it after every step. Stop when it hits zero, or when it refuses to fall.

That is the whole grade. Everything else is narration.