DEV Community: Jason Zhou

Crabbox: Cloud Sandboxes for Parallel Coding Agents

Jason Zhou — Thu, 16 Jul 2026 14:00:00 +0000

Crabbox: Isolated Cloud Sandboxes for Parallel Coding Agents

When you run ten or fifteen coding agents in parallel, writing code stops being the bottleneck. Merging it becomes the bottleneck. Every agent produces a pull request, and every PR carries the risk of breaking the system, so the real work shifts from "make the agent write code" to "let the agent prove its code works before a human ever looks at it."

That is the problem Crabbox - the new project from OpenClaw creator Peter Steinberger - is built to solve. Here is the full walkthrough, why parallel testing breaks on your laptop, and the open-source codebase harness that productizes the fix.

The merge bottleneck is the new bottleneck

Once your team sets up loops - agents that get triggered on their own, pick up work, and ship it - you end up with a lot of sessions running at once. At almost any given time I have at least ten agent sessions going; Peter has posted screenshots of fifteen-plus in parallel. The result is a massive volume of PRs that simply was not possible before.

That volume is the problem. Each PR has to be reviewed, and each one might break production. So the constraint moves up the stack:

   OLD bottleneck            NEW bottleneck
 ┌────────────────┐        ┌────────────────────┐
 │ writing code   │  ───►  │ getting code MERGED │
 │ (agents solved │        │ verifying + trusting│
 │  this)         │        │ 15 parallel PRs     │
 └────────────────┘        └────────────────────┘

The fix is not "review faster." It is to give each agent the tools to verify its own work and attach evidence - a passing test, a screenshot, a screen recording - to the PR, so a human is reviewing proof, not guessing. That is the first job of a codebase harness: make the repo agent-ready so agents can run, test, and verify before they hand work back.

Why parallel testing breaks on your laptop

Giving one agent a browser tool and a dev server works fine. The setup falls apart when many agents run at once, because they share your machine:

One database, one schema. If a local Supabase or Postgres instance is shared, one agent trying a new schema migration can break every other session at once.
Hardcoded ports. Most repos are not set up to run multiple instances side by side - a port is pinned for a good reason, and two dev servers collide.
One Docker daemon, one OS. Parallel sessions share the same daemon and resources, so they step on each other.
Resource limits. A modern production repo is heavy. Running many full instances locally is slow and often impossible.

Git work trees solve the code isolation problem - each agent gets its own checkout to modify - but they do not solve the runtime problem. Agents still need a live dev server and database to actually test against, and that is what collides.

The fix: one isolated sandbox per work tree

The right architecture is to stop running every session on your local machine and instead give each agent its own isolated cloud environment - its own box, its own database, its own dev server - that cannot affect any other session.

            LOCAL (work trees: code isolation only)
   agent A ─┐   agent B ─┐   agent C ─┐
            │            │            │
            v            v            v
   ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
   │ CLOUD BOX A │ │ CLOUD BOX B │ │ CLOUD BOX C │   runtime isolation
   │  own DB     │ │  own DB     │ │  own DB     │
   │  own server │ │  own server │ │  own server │
   └─────────────┘ └─────────────┘ └─────────────┘
       test           test            test
   (no shared schema, ports, or daemon - they can't break each other)

Building that pipeline by hand is real work: spin up a machine on spot, mount a disk, copy the code in, install dependencies, start the app, open a browser to test, then tear it all down afterward. My team built exactly this, and it unlocked a lot - but one rough edge remained: when an agent finds a bug mid-test, getting the local fix back into the cloud box is awkward. The normal commit-push-CI flow floods the repo with throwaway commits, and you do not want to rebuild the box every time.

What Crabbox does

Crabbox closes that last gap. It lets an agent warm up a cloud box, sync the uncommitted diff straight from the local work tree, and run tests against it in real time - so you can change a file locally and retest in seconds.

The workflow an agent runs:

Warm up a box - spin up an isolated cloud sandbox on demand.
Run any command in the cloud - the agent runs bash commands as if local, but they execute in the box. Each run first syncs the dirty diff from your local work tree (no commit needed - the folder just has to be git-initialized), then runs the command against the latest code.
Generate evidence - collect screenshots, record video of the run, and publish artifacts to storage (such as an S3 bucket) so they can be posted inline as PR comments.
Stop the box - tear it down and delete it when the task is done.

  agent finds a bug ─► fixes file LOCALLY ─► next `run` auto-syncs the diff
        ▲                                              │
        └──────────────── retest in the cloud ◄────────┘
   no extra commits, no box rebuild - always testing the latest version

The whole thing is configured with three files:

A Dockerfile that encapsulates everything your local machine has - Node, Docker, the Supabase CLI, any tool the app needs.
A crabbox.yml that defines the sandbox provider (the video uses Daytona for fast startup via prebuilt snapshots), the default work root, which folders to exclude from sync (heavy or unneeded ones - node_modules and build dirs are usually gitignored already), and environment variables to pass. Those vars are pushed to the box over an encrypted SSH connection, so the data plane stays relatively safe.
A setup.sh so the agent runs one script to install dependencies and bring the dev server up, instead of stepping through it each time.

Exact command names and flags live in the project and the video above - treat the steps here as the shape of the workflow, not copy-paste syntax.

The codebase harness that productizes this

You do not have to wire all of this together from scratch. We packaged the setup my team runs into an open-source Claude Code plugin: the AI Builder Club skills repo. It ships two flagship skill sets:

Codebase harness - make any repo agent-ready so agents can run, test, verify, and ship, including an isolated cloud box per agent so loops can ship code in parallel.
Loops - spin up compounding agent loops on a shared, file-based knowledge base.

Install it in Claude Code:

/plugin marketplace add AI-Builder-Club/skills
/plugin install skills@ai-builder-club

Then the two entry points:

/setup-codebase-harness - run it in the repo your agents work in, so they can run, test, and verify their own work (this is the harness Crabbox plugs into).
/new-loop - run it where your agent's memory should live; it bootstraps the shared knowledge base and scaffolds your first compounding loop.

If you want the full walkthrough of the concept - the four ingredients of a loop and how the harness fits - the loop engineering guide is the written companion, and AI Agent Reliability and Cost Control covers why agent-generated evidence is what makes parallel work safe to merge.

Go deeper:

AI Agent 101 Course - build agents that run, test, and verify their own work, then deploy them
Claude Code 101 Course - CLAUDE.md, hooks, subagents, and work trees, the foundation a codebase harness sits on

Want the step-by-step build alongside a team running this in production? Join AI Builder Club.

Start Here

Pick the repo your agents already work in and run /setup-codebase-harness from the AI Builder Club skills repo. Get one agent running its own tests and attaching evidence to a PR before you scale to ten. The merge bottleneck only gets worse with volume, so solve verification first.

For the full build alongside a team running this in production, join the AI Builder Club.

Join AI Builder Club

Originally published on AI Builder Club, where the full free agent-engineering curriculum lives.

Codebase Memory MCP: Give Your Coding Agent a Map (2026)

Jason Zhou — Thu, 16 Jul 2026 05:46:17 +0000

Codebase Memory MCP: Stop Making Your Coding Agent Read Your Repo as Flat Text

Ask a coding agent to change something in a big codebase and you know what happens. It greps, gets a wall of matches, opens 20 files one by one, and still misses half the places that might break.

Here is the thing though: your codebase is already a map. Every import is an edge. Every function call is an edge. Your agent throws that structure away and reads the whole thing as flat text.

Codebase Memory MCP fixes this by indexing your repo into a persistent code graph - functions, classes, call chains, routes, cross-file and cross-service links - that the agent queries instead of grepping. It is written in C, parses 158 languages via tree-sitter, indexes most repos in seconds, and answers structural queries in under a millisecond. When I ran the same architecture question with and without it on my own monorepo, token consumption dropped from ~38,000 to ~11,000. Over a two-question session: 64,000 vs 33,000. About half.

Prefer to watch? Here is the full walkthrough:

Why grep burns your context window

The default exploration loop every coding agent runs on a structural question looks like this:

Grep for the identifier. Get matches across dozens of files.
Read the files one by one. Each read dumps hundreds or thousands of lines into context.
Follow the imports. More reads.
Try to hold the reconstructed structure in the context window while doing the actual work.

Every step pays full price because grep returns text, not structure. The agent has to rebuild the relationships between symbols from scratch, on every task, by reading source. Claude Code has subagents to contain this (the search runs in a separate context), which helps with pollution but is slow, and the cost still gets paid somewhere.

This is context engineering's ugliest corner: in production agents, roughly 100 input tokens are consumed for every output token, and on large-repo tasks a huge share of those inputs is just re-discovering how the code fits together. The project's arXiv evaluation across 31 real-world repositories puts numbers on it: graph-based exploration answered structural questions with 10x fewer tokens and 2.1x fewer tool calls than file-by-file exploration, at 83% answer quality. The README's extreme benchmark: five structural queries cost ~3,400 tokens via the graph versus ~412,000 via grep-and-read.

What Codebase Memory MCP actually is

Codebase Memory MCP is an open-source (MIT) MCP server that extracts every function, class, method, route, and import from your code and builds a relationship graph out of them. Cross-file, cross-package, even cross-repo: if service A calls an HTTP route in service B, that edge is in the graph.

Two design decisions make it different from the pile of codebase-index tools you have already seen and forgotten:

1. No LLM in the indexing pipeline. Earlier projects ran a language-model pass to write a "knowledge map" of your repo. Those maps cost money to build, took minutes to hours, and went stale the moment the code changed. Codebase Memory MCP parses with tree-sitter grammars in pure C. Indexing is purely programmatic: an average repo indexes in seconds, and the Linux kernel - 28 million lines across 75,000 files - indexes in about 3 minutes. Rebuilding on change is cheap enough that the graph just stays current.

2. It meets the agent where it already works. More on the hook pattern below, because it is the most copyable idea in the project.

The agent gets a toolset that replaces the grep-read-repeat loop:

Tool	What the agent uses it for
`get_architecture`	Quick overview: languages, packages, routes, hotspots, clusters
`search_graph`	Locate the node for a symbol by name, label, or file pattern
`trace_path`	Follow the call chain: who calls this, what does it touch
`detect_changes`	Map a git diff to affected symbols, with risk classification
`query_graph`	Cypher-like queries, e.g. "callers of handle_order with no test coverage"
`get_code_snippet`	Pull just the source of one function by qualified name

That last pairing is the token win in miniature: instead of reading a 2,000-line file to see one function, the agent asks the graph where the function is and pulls only its body.

detect_changes is the sleeper feature for teams. Point it at a PR diff and it returns the blast radius - every symbol the change can reach - which turns agent code review from "read everything and hope" into a graph query.

The hook pattern: why this MCP works when others get ignored

Here is the failure mode that killed most code-search MCPs: the agent forgets to use them. You install a beautiful semantic search tool, and the agent... greps anyway. Claude Code and Codex are heavily optimized around their built-in search tools, and no system prompt reliably overrides that habit.

Codebase Memory MCP acknowledges this instead of fighting it. On Claude Code it registers a PreToolUse hook: when the agent calls its normal Grep or Glob tool, the hook intercepts the call, runs the graph lookup for matching symbols, and injects the structured context - call chains, relationships, definitions - alongside the ordinary grep results. The grep still runs. The agent just gets the map with the matches, whether or not it remembered the MCP exists.

If you are building MCP tools yourself, steal this. Do not rely on the model choosing your tool over its built-ins; use hooks to enrich the tools it already prefers. It is the difference between shipping a tool and shipping a behavior.

The real test: tracing a hidden lock through a monorepo

I indexed the codebase for Superdesign, our vibe-design platform. It has a backend function called createDesignDraftNode - the tool an agent calls to add a design to the infinite canvas. Because multiple agents can generate designs on the same canvas at once, everything flows through a queue guarded by a canvas lock.

The catch: the lock is invisible from where the work starts. createDesignDraftNode never calls it directly; the protection is delegated down through layers. Grep that file for "lock" and you get nothing. Any agent (or human) reading the entry point would conclude the protection does not exist.

So I asked two questions, in two sessions - one with the graph, one with MCP explicitly disabled:

Question	With Codebase Memory MCP	Without (default grep)
"Trace the createDesignDraftNode and canvas lock flow"	~11,000 tokens, full flow returned in seconds	~38,000 tokens
"What breaks if I change the lock?" (cumulative)	~33,000 tokens, all 13 call sites identified	~64,000 tokens

Same model, same repo, same questions. Half the tokens, and the graph version found every call site - which matters more than the cost, because the expensive failure is the new engineer (or agent) who decides the lock "slows things down," tweaks it, and ships a race condition. Ask "what will break if I change this?" and the graph answers with the actual dependency list instead of a best-effort sample.

Those are my numbers on one repo, measured from Claude Code's context inspector; your ratio will move with repo size and question type. The pattern held across every structural question I threw at it: the bigger and more cross-cutting the question, the wider the gap.

How to set it up (2 commands, ~10 minutes)

1. Install. One command on macOS/Linux:

curl -fsSL https://raw.githubusercontent.com/DeusData/codebase-memory-mcp/main/install.sh | bash

Want the graph visualization UI too:

curl -fsSL https://raw.githubusercontent.com/DeusData/codebase-memory-mcp/main/install.sh | bash -s -- --ui

The installer auto-detects your installed agents (Claude Code, Codex CLI, Gemini CLI, Zed, OpenCode, Antigravity, Aider, KiloCode, VS Code, OpenClaw, Kiro) and configures the MCP entries and hooks for each. Single static binary, zero runtime dependencies.

2. Index. Open your coding agent in the repo and tell it: "Help me use codebase memory MCP." It will run the indexing, and if your repo already has decent documentation it applies ignore filters on its own - mine correctly skipped the legacy folders. My semi-complex monorepo indexed in a few seconds.

3. Verify. Ask it to trace a function you know is load-bearing. You should see search_graph and trace_path calls come back with the full chain, no file-by-file reading.

4. (Optional) Look at your graph. With the UI install: codebase-memory-mcp --ui=true plus a port gives you a web view of the whole graph. Is it useful? Honestly not sure. It does look really cool.

Where this fits: the codebase harness

A code graph is one ingredient, not the whole meal. The repos where agents ship reliably have a full harness: a way to run the app locally in one command, end-to-end tests that gate PRs, isolated sandboxes for parallel work, and now a structural map of the code.

That is why we added Codebase Memory MCP to /setup-codebase-harness in the open-source AI Builder Club skills repo. Run the skill in your repo and it sets up the MCP and index alongside the rest: e2e test gates, the script toolkit that gets your local server running, and Crabbox for parallel agent testing in remote sandboxes. One command, and your codebase goes from flat text to something an agent can actually navigate, run, and verify against.

If you are optimizing agent costs more broadly, the graph attacks the input side; reducing Claude Code API costs covers the rest of the bill.

Start Here

Install Codebase Memory MCP in the repo where your agent does real work, index it, and ask one question you already know the answer to: "what breaks if I change X?" Compare what comes back against what you know. That single test tells you whether your agent has been navigating your codebase or guessing at it.

Then make it part of the harness: run /setup-codebase-harness from the AI Builder Club skills repo so every agent session starts with the map.

For the step-by-step build alongside a team running this in production, join the AI Builder Club.

Join AI Builder Club

Originally published on AI Builder Club, where the full free agent-engineering curriculum lives.

Loop Engineering Guide (2026)

Jason Zhou — Sun, 05 Jul 2026 11:41:52 +0000

Loop engineering is the discipline of designing the loop an agent runs inside - what it does between tool calls, when it checks its own work, and how it decides it's finished - instead of hand-writing each prompt. It's the 2026 successor to prompt engineering. The model writes the prompts now. The scarce skill is defining what "good" and "done" mean, and the part almost every explainer skips is this: in any loop, the verifier is the bottleneck, not the model.

A few mornings ago I typed one sentence into Claude Code and went to make coffee. By the time I got back, something like forty agents had spun up, written code, checked each other's work, thrown away the bad attempts, and left me three pull requests to review.

I didn't write forty prompts. I wrote one loop.

If you've felt the ground shift under "prompt engineering" lately, this is why. Here's the decode - not the hype version, the version you can use this afternoon.

AI Jason breaks down the same shift in his "Loop Engineer: Systemization and Artifacts" video - what the term actually means, the four core ingredients, and how his team designs loops that compound:

How Did We Get From Prompt Engineering to Loop Engineering?

It helps to see the line this sits on:

Year	Era	What you do	Your role
2024	Prompt engineering	Write a good prompt, get a good output	Operator
2025	Parallel agents	Stop babysitting one chat, run several at once	Manager
2026	Loop engineering	Build the loop that runs the agents for you	System designer

The people behind Claude Code have said the quiet part out loud: some mornings they aren't really writing the prompts anymore - another model is - and they're managing hundreds, even thousands, of agents at a time. That sounds like a flex until you see what it implies: if you're not in the loop pressing enter between every step, something else has to decide when the work is good enough. That something else is the whole game.

What Is a Loop, Exactly?

Strip the jargon and a loop is four moves on repeat:

discover → plan → execute → verify → (repeat until a condition is met)

You used to be the loop. You were the thing standing between the agent's steps, reading the output, catching the mistake, deciding what happens next, telling it to try again. Loop engineering is the discipline of stepping out of that inner cycle and up to designing the track the agent runs on.

Here's the simplest possible version. Give the agent a goal and a stopping condition, and let it run:

Goal: get the test suite passing.
Loop: run the tests, read the failures, fix the most likely cause, run again.
Stop when: all tests green, or you've tried 6 rounds (then summarize what's left).

That's a loop. And it genuinely works on a simple task. But point a loop at something open-ended - "improve this app," "make this page better," "research X and write it up" - and it either produces something great or it quietly turns into a very expensive slop machine. The difference between those two outcomes is the part nobody talks about.

Loops Nest: Andrew Ng's Three-Loop Model

The discover → plan → execute → verify cycle above is the inner loop. Zoom out and you'll find it wrapped inside slower loops. Andrew Ng laid this out cleanly in a June 2026 Batch letter - and the fact that he wrote it is itself the signal: "'Loop engineering' is a hot buzzphrase after mentions of it by Boris Cherny (Claude Code's creator) and Peter Steinberger (OpenClaw's creator) went viral." When Andrew Ng devotes a whole letter to a term, it has left buzzword territory.

His model is three loops running at different speeds:

Loop	Cadence	Who runs it	What it decides
Agentic coding loop	Seconds to minutes	The agent	Write code, test it, iterate until it's bug-free and meets the spec (and evals, if you have them)
Developer feedback loop	Tens of minutes to hours	You	Review the product, steer the agent, update the spec
External feedback loop	Hours to weeks	The world	Alpha testers, A/B tests, production data - feeds your vision back into the spec

  ┌──────────────────────────────────────────────────────┐
  │  EXTERNAL FEEDBACK LOOP            (hours → weeks)     │
  │  alpha testers · A/B tests · production data          │
  │   ┌────────────────────────────────────────────────┐ │
  │   │  DEVELOPER FEEDBACK LOOP     (minutes → hours)  │ │
  │   │  you review · steer · update the spec          │ │
  │   │   ┌──────────────────────────────────────────┐ │ │
  │   │   │  AGENTIC CODING LOOP    (seconds → mins) │ │ │
  │   │   │  generate → test → verify → repeat       │ │ │
  │   │   └──────────────────────────────────────────┘ │ │
  │   └────────────────────────────────────────────────┘ │
  │              vision ──► spec ──► agent                 │
  └──────────────────────────────────────────────────────┘

Watch what changes as you move outward: the loop gets slower, and the verifier gets more human. The inner loop can verify itself with tests. The outer loops can't - they need you. Ng makes the sharpest point right here, and it's a friendly amendment to the "taste is the reward function" idea coming up next. People call the human contribution "taste," he writes, but he prefers "context advantage": "So long as the human knows something the AI does not, human-in-the-loop is needed to inject that knowledge." Taste sounds innate and unteachable. "Context advantage" tells you exactly what to encode into the loop: the things you know about your users and your problem that the model doesn't. That's your verifier, written down.

And here's his tell for when to graduate from eyeballing output to a real verifier: "If you find that the system repeatedly runs into certain problems, building a set of evals for the agent becomes useful." Same lesson as the landing-page example below - the moment you're checking the same thing twice, write it into the loop.

Why Is the Verifier the Bottleneck, Not the Generator?

Every loop has two halves. The generator produces work - that's the model, and models are now extremely good. The verifier judges whether that work is good. Put it plainly: a loop is just a generator wired to a verifier, and the generator was never the bottleneck. The verifier is.

For two years we obsessed over the generator. We tuned prompts, swapped models, argued about temperature. But in a loop, the generator runs over and over for nearly free. The thing that decides whether all that motion produces value is the verifier.

And the freer you let the loop run, the more everything rides on the verifier. A loop with a weak "good enough?" check doesn't fail loudly. It succeeds at producing garbage, confidently, hundreds of times.

This is the same shift Addy Osmani keeps pointing at: the bottleneck moved from writing code to proving it works. Review, judgment, taste, knowing what "correct" looks like - that's now the most leveraged skill an engineer has. It's the same lesson agent evaluation teaches at the production layer: a model grading its own homework always gives itself an A. In a loop-engineering world, your taste isn't a soft skill anymore. It's the reward function.

Open Loop vs Closed Loop: Which Should You Build?

Once you accept that the verifier is the point, the engineering decision gets clear. Every loop sits somewhere between two poles.

	Open loop	Closed loop
What it is	Give a goal and loose conditions, let it explore a wide space	Pin success criteria in advance, evaluate every step, define an explicit stop
Upside	Genuinely novel output. Surprising solutions live here	Runs on a normal budget. Predictable. Safe to leave alone
Downside	Burns tokens. Degrades into slop fast with loose criteria	Won't surprise you. Does what you specified, not more
Lives or dies by	The verifier (even more so)	The verifier

The actual engineering is two decisions:

Choose open or closed for this specific task. Roughly: how much do I need novelty, and how much budget am I willing to risk?
Write the verifier that matches. A closed loop needs hard, checkable passes. An open loop needs an even better verifier, because it's the only thing standing between exploration and slop.

What Does a Loop With a Weak Verifier Actually Produce?

The most useful thing isn't a loop that works. It's the same task run two ways.

Here's Take 2 written out:

Goal: improve landing-page conversion clarity.
Done when ALL pass:
  - Lighthouse accessibility score >= 95
  - Exactly one primary CTA above the fold
  - Hero headline states the value prop in <12 words
  - No layout shift (CLS < 0.1)
Loop: propose a change -> run the checks -> keep it only if every check still passes
      -> stop when all green or after 5 rounds.

Now the loop converges, because every iteration clears a bar you defined. Then comes the trick that rescues open loops too: keep those hard checks as the floor, and add one open instruction ("surprise me with the headline"). Now you get exploration that can't degrade below your standard.

The lesson isn't "closed loops are better." It's: the verifier is what makes either kind ship.

What Tools Can Run the Loop For You?

You don't have to build the loop machinery from scratch. There's a clear lineage, from rigid to autonomous:

Tool / pattern	How it decides "done"	Best for
"Ralph" loop	A bare shell `while` loop re-fires the same prompt until you stop it - no smart "done," you are the stop button	Mechanical, repetitive tasks. Crude but bulletproof
Claude Code `/goal`	A small fast model judges the stop condition after every turn	Fuzzy "done" that still needs evaluating
Goal-tracking setups	Agent tracks its own progress in files, defines "done" up front	Long runs that need to stay oriented
Self-hosted agents (Hermes-style)	Runs continuously on its own infrastructure, keeps state across sessions	Always-on agents you set up once and leave running

/goal is the cleanest entry point: you type a completion condition once, and after each turn a Haiku-class evaluator reads the transcript and decides yes or no, looping until it holds (or you hit your turn cap). One catch worth knowing - the evaluator only sees what Claude already printed, so your condition has to be provable from the agent's own output.

Pick the least autonomous tool that does the job. Autonomy is not the goal. A shipped result is the goal. When the work fans out wide and repetitive, push the loop into a script with dynamic workflows instead of babysitting it.

What Does a Real Loop System Look Like?

A single loop is useful. A system of loops that share a brain is where the leverage compounds. This is what AI Jason runs inside his own company.

Start with the simplest one: a support loop. Every 30 minutes a cron wakes an agent. It pulls every support ticket, replies to the ones it can answer, and logs the frictions and product ideas it spots into a shared folder called signals. That loop alone earns its keep.

Now make it compound. The same loop doesn't just log a bug - it spawns a coding agent to fix it, monitors whether the fix held, and tells the customer it shipped. If people still hit the issue, that means it wasn't fixed at the root, so the loop tries again.

The real trick is that every loop reads and writes the same shared folders. Jason runs several at once:

Loop	Trigger	What it does	What it writes
Support	Every 30 min	Answer tickets, spot friction	`signals`, engineer tasks
SEO	Daily, 9am	Pull data, research topics, publish pages	pages, conversion-gap `signals`
Product growth	Daily	Prioritize experiments from analytics + signals	tasks
Reddit	Scheduled	Draft on-brand comments	comment artifacts

Because they share one file system, the SEO loop's "this keyword converts but we have no organic content" signal feeds the content loop. The support loop's repeated-bug signal gets picked up by the product loop. Each loop runs every hour or every day, reading what the others learned. The shared brain is what makes it compound - and Jason's quoted output from this setup is 20 to 40 high-quality pages a day driving traffic without him looking at it.

The 4 Ingredients of a Loop That Compounds

Jason's framework boils down to four parts. Most people nail the first three and skip the fourth - which is the one that actually decides whether autonomous work is possible.

Triggers. What wakes the agent. A cron job, a webhook, another agent, a server incident. The point is the agent runs without you pressing enter.
File structure. The most important design decision. Where artifacts, contracts, and logs live (covered below).
Tools and connectors. The skills and scripts that let the agent do real work - Intercom to fetch tickets, Stripe to check subscriptions, Supabase to debug, Playwright to test.
An agent-ready codebase. The setup that lets many agents work in parallel and verify their own output. This is the one everyone misses.

How Do You Make a Codebase Agent-Ready?

Before any loop works, the environment has to let an agent operate solo. Three properties:

Legible - the agent can find where to change what. Keep AGENTS.md / CLAUDE.md as a ~100-line index that points to deeper docs (OpenAI keeps theirs around 100 lines). Then bake rules into custom lints so the agent gets a warning automatically instead of you hoping it reads the right doc. Example: lint-fail any import from a legacy folder you don't want touched.

Executable - the agent starts work with the dev server already up, at near-zero token cost. Write a dev local script so it doesn't burn 3-5 minutes booting the app every run. Make the repo worktree-friendly so five parallel agents each spin up their own server without colliding. Add scripts that jump to a specific state to test scenarios fast.

Verifiable - give the agent tools to test and prove it worked. The Playwright CLI is the standout: it drives the browser and records a video clip you can attach to the GitHub PR, so review takes seconds. Back it with end-to-end tests on the flows you never want broken - sign-up, upgrade, core action.

One hard rule from the video, and it lines up exactly with the verifier point above: don't let an agent self-verify. It doesn't work well. Jason's PR skill always spawns a separate read-only verifier agent with the detailed spec. Generator and verifier stay different agents.

The File System: Artifacts, Contracts, Logs

This is ingredient #2, and it's the heart of the system. Three file types, three jobs:

Artifacts - the shared knowledge layer. The output of each loop's work. Types include docs, signals, tasks, tickets, even campaigns for an ads loop. Each artifact type gets its own folder with a README defining what goes in, what doesn't, the process for adding an item, and the schema. Each artifact file carries front-matter metadata, a body, and a timeline of every change.

A signal is the unit that makes loops compound. It captures a product idea, a friction, or a missed opportunity, links to its raw sources (a support ticket, a customer quote), and any loop can read or write it.

Contracts - one per loop, usually a README in the loop's folder. It states the goal, the workflow, the boundaries, the outstanding backlog, and a timeline. Every time the loop fires, it reads its contract first - goal, workflow, what happened last time - then takes the next best action.

Logs - a single global work-log file. Different from the timelines because your day mixes reviewing loop output with hands-on copilot work. Before an agent starts a big task it reads the last 5-10 entries; when it finishes it appends what it did. That's how cross-domain context survives between sessions.

The workflow to stand one up: run the loop manually once as a test, calibrate the workflow with the agent, then ask it to write the contract and register the trigger. Test run first, loop second.

Why Writing a Verifier Is Like Defining a Reward Function

If you've touched reinforcement learning, this clicks instantly. In RL you don't tell the agent every move - you define the reward, and the agent iterates toward it on positive and negative signal.

That's exactly what you're doing here. You are not training the model. You are defining the reward: the end goal, and what counts as good. Your domain knowledge - knowing what correct looks like in your problem - is the moat. The model is a commodity. The reward function is yours.

This is also why the field keeps drifting toward harness engineering: the leverage isn't in the phrasing anymore, it's in the system around the model - context, tools, state, and the evaluation loop that decides when to stop. Loop engineering is that same move, named from the loop's point of view.

And watch where the vocabulary goes next. Once you accept that the verifier is the point, the natural follow-up question is how do you write a good one - which is exactly the ground "evals" cover. Ng's own advice ("build a set of evals when the system keeps hitting the same problems") points straight at it, and the term already circulating for this next layer is eval engineering: formalizing the verifier into a versioned dataset you can measure against. If loop engineering is designing the loop, eval engineering is designing the bar the loop clears. Same lineage, one level deeper.

Which is why I keep saying it: writing the verifier is the new prompt engineering.

Is Loop Engineering Just a New Buzzword?

I'd be doing you a disservice not to flag the counter-take, because in the three weeks after the term went viral, the skeptic case grew real teeth. The strongest version has four distinct arguments, not one:

The vocabulary mockery. One widely-shared post maps the whole lexicon back to CS primitives - "a while loop becomes 'loop engineering'... unit tests become 'evals'" - and it's now a genre: a 1,800-comment Hacker News thread arguing agents are "just a while loop with an LLM call," and an entire site (extra-steps.dev) dedicated to the bit, both rounded up in PostHog's "WTF is loop engineering". Anyone who's written a CI pipeline or a Kubernetes reconciliation loop has been "designing loops" for years.
The autonomy check. The Register's June 24 take: the latest AI buzzword "still needs humans in the loop." The demos hide how much steering real work still takes.
The economic critique. Ed Zitron's version: the trend amounts to "celebrating and evangelizing autonomous token consumption" - spending the model vendors would very much like to stimulate. And he has a live exhibit: Uber reportedly capped engineers at $1,500/month for agent tooling after burning through its annual AI budget in four months. An unattended loop with a weak verifier doesn't fail loudly; it fails at token prices, all night.
The sampling-bias point. The subtlest one, from developer @_kboy_ on X: Claude Code solving "writing software" is real, but that doesn't "give anyone the evidence to start defining how everyone should develop software" - the vendors' data comes from people already using their product.

Both things are still true at once, and that's the honest read. The word is riding a wave and the shift under it is real. The tell is who's using it seriously: Boris Cherny (Claude Code's creator), Peter Steinberger (OpenClaw's creator), and Andrew Ng - who wrote an entire letter mapping out his three loops - are not chasing a hashtag. When the people building the tools and the person who taught half the industry machine learning independently converge on the same frame, the frame is load-bearing. (And the wave is still rising: three weeks in, "Loop Engineer" is already a module title in GenAI bootcamp curricula, and Jensen Huang was echoing "prompt engineering is dead" on stage.)

So here's your filter. Ignore the word. Look at whether the underlying shift is real:

Are you increasingly designing systems that run agents, instead of prompting agents directly? Yes.
Is the scarce, valuable part now defining "done" and "good," rather than phrasing the request? Yes.
Does a loop with a weak verifier reliably produce expensive garbage? Also yes - see Uber's invoice.

Here's the thing to notice: every serious criticism above attacks weak loops, not the discipline. Loops that still need un-budgeted human steering. Loops that burn tokens with no halt condition or spend cap. Loops graded on the vendor's own telemetry. The skeptics aren't refuting loop engineering - they're describing what happens when you skip the verifier, which is the entire argument of this guide. The label is optional. The shift is not. Whether you call it loop engineering or just "building agents that don't waste my money," the move is the same: stop perfecting prompts, start writing verifiers.

Your Loop Engineering Starting Checklist

If you build one loop this week, run it through this:

Define "done" in measurable terms before you write a single instruction.
Pin the passes - the tests, the rubric, the eval - up front, not after.
Choose open vs closed by need-for-novelty x budget-you'll-risk.
Always attach the run data to whatever you hand back to a human.
Use the least autonomous harness that gets the result.
Set the trigger - cron, webhook, or another agent - so it runs without you.
Give it shared folders - artifacts, a loop contract, a global log - so the next run, and every other loop, can build on this one.

Do that and you've crossed the line from someone who prompts AI to someone who engineers the system that does the work. That's the whole skill. The rest is reps.

Start Here

Pick one repetitive task this week. Before you write a single instruction, write down what "done" means in measurable terms. Then pin the checks, choose open or closed, and let the loop run against your bar instead of the model's.

For closed-loop templates, verifier checklists, and teardowns of loops that shipped (and loops that burned money), join the AI Builder Club - come ship something real.

Join AI Builder Club

Originally published on AI Builder Club, where the full free agent-engineering curriculum lives.

DEV Community: Jason Zhou

Crabbox: Cloud Sandboxes for Parallel Coding Agents

Crabbox: Isolated Cloud Sandboxes for Parallel Coding Agents

The merge bottleneck is the new bottleneck

Why parallel testing breaks on your laptop

The fix: one isolated sandbox per work tree

What Crabbox does

The codebase harness that productizes this

Related Content

Start Here

Codebase Memory MCP: Give Your Coding Agent a Map (2026)

Codebase Memory MCP: Stop Making Your Coding Agent Read Your Repo as Flat Text

Why grep burns your context window

What Codebase Memory MCP actually is

The hook pattern: why this MCP works when others get ignored

The real test: tracing a hidden lock through a monorepo

How to set it up (2 commands, ~10 minutes)

Where this fits: the codebase harness

Related Content

Start Here

Loop Engineering Guide (2026)

How Did We Get From Prompt Engineering to Loop Engineering?

What Is a Loop, Exactly?

Loops Nest: Andrew Ng's Three-Loop Model

Why Is the Verifier the Bottleneck, Not the Generator?

Open Loop vs Closed Loop: Which Should You Build?

What Does a Loop With a Weak Verifier Actually Produce?

What Tools Can Run the Loop For You?

What Does a Real Loop System Look Like?

The 4 Ingredients of a Loop That Compounds

How Do You Make a Codebase Agent-Ready?

The File System: Artifacts, Contracts, Logs

Why Writing a Verifier Is Like Defining a Reward Function

Is Loop Engineering Just a New Buzzword?

Your Loop Engineering Starting Checklist

Related Content

Start Here