DEV Community

Cover image for The simplest loop that works: a write-test-fix agent, step by step
Ramón Chancay 👨🏻‍💻
Ramón Chancay 👨🏻‍💻

Posted on Originally published at ramonchancay.me

The simplest loop that works: a write-test-fix agent, step by step

In the previous post we saw what an agent loop is: the pattern in the abstract. Now we're going to build it, in the smallest agent that actually does something useful. It's the write-test-fix loop: you write a specification and some tests by hand, the model writes the code, the program runs the tests, and the error feeds back to the model until everything passes. By the end of the post you have the whole cycle in code that runs.

TL;DR

  • The write-test-fix loop is the minimal agent: spec and tests written by hand, the model generates the code, it runs, and the test failure feeds back as context for the next attempt.
  • The stopping condition comes from the tests: green and it stops, red and it retries, up to an iteration cap. The model doesn't need to "decide" it's done.
  • Feedback is everything. A test that only says "failed" fixes nothing; one that says "expected [9,5,25] and got [null,0,0]" gives the model exactly what it needs to fix it.

From theory to code: the simplest possible agent

The previous post left the loop in pseudocode: a while with three components—state, action, and stopping condition—around a model that decides the next action over and over. Useful for understanding the pattern, but abstract. Here we'll instantiate it in the smallest case that's still a real agent, and at the end you have code that runs.

The mapping is direct. Each piece of the loop has a concrete equivalent in this example:

Loop component In the write-test-fix loop
State The spec, the tests, and the history of attempts and errors
Action The model writes (or fixes) the code
Observation The test runner's output
Stopping condition Tests green, or the iteration cap is reached

If you understood that table, you understood the post. The rest is watching it work.

Of those four pieces, the one most worth keeping in your head is the state, because it's the one that moves. It doesn't reset on each turn: it starts with the spec and the tests, and grows with every attempt by the model and every error from the runner. Here's how that accumulation looks across iterations:

Iteration 0          Iteration 1               Iteration 2
                     (+ what 1 produced)       (+ what 2 produced)

spec                 spec                      spec
tests                tests                     tests
                     code (attempt 1)          code (attempt 1)
                     runner errors             runner errors
                                               code (attempt 2)
                                               runner errors
Enter fullscreen mode Exit fullscreen mode

That accumulation is exactly what lets the model fix instead of repeating the same error: on iteration 2 it sees what it wrote before and why it failed. In this example the state grows by appending every previous attempt, nothing more. Later in the series we'll see how it's managed when the context no longer fits and you have to summarize or drop.


Keep reading

Write-test-fix loop illustration: one node writes code, another tests it, and the failure feeds back as context until the tests pass

That is the first half. The full walkthrough — with the rest of the implementation, the trade-offs and the things that only show up in production — is on my blog:

Read the full post on ramonchancay.me →

Originally published at www.ramonchancay.me/blog/write-test-fix-agent-loop.

Top comments (0)