DEV Community: Anvay Singh

Part 2: When Nobody Grades Their Own Homework

Anvay Singh — Mon, 13 Jul 2026 18:30:00 +0000

TL;DR

Some things can't be checked with a number, like whether an animation feels right.
So a second, read-only agent grades the first one against a written rubric it is not allowed to edit.
In my run the reviewer rejected the builder three times, and the most interesting problem it caught was in the test evidence, not the code.

In Part 1 I built a loop that chased a number, frames per second. But most of what we care about in software is not a number. "Does this region switch feel good?" has no assert. You cannot write expect(feelsRight).toBe(true).

So this part is about how you check quality when there is nothing to measure. The approach I used is a second agent that grades the first one against a written rubric. In my run the reviewer turned the builder down three times before it approved anything, and the most interesting problem it found was not in the code at all.

A quick reminder of the definition, since this is Part 2 of 3: a loop is an external script that runs the agent, a separate check the agent cannot edit decides pass or fail, and it repeats until it passes or hits a limit. In Part 1 the check was a Playwright test. Here the check is another agent.

The problem this loop solves

In the browser you can switch regions, say from Tamil to Korean, which swaps out hundreds of posters at once. Done badly, the grid flashes blank and jumps around. Done well, it fades from one set to the next, keeps its layout, shows a loading state, and puts you back at the top.

"Done well" is subjective, which is the kind of thing you cannot unit-test. So I wrote it down as a rubric and had a second agent apply it.

The bar: a rubric a person owns

The rubric is seven plain-English checks in a file, and the first line is the one that matters:

Overall APPROVED requires every item PASS. This file is human-owned.
Only a person changes the bar.

The seven items are things like a crossfade instead of a flash, no layout shift, a visible loading state, posters that stay 2:3, and landing back at the top after a switch. Nothing fancy, just a written version of "good" that a reviewer can apply the same way every time.

Two agents that never touch

The important decision is that the builder and the reviewer are two separate Claude Code agents with different permissions.

The builder can write code:

tools: Read, Edit, Write, Grep, Glob

The reviewer is read-only. It cannot edit anything, including the rubric:

tools: Read, Grep, Glob

Each one runs in its own context. This answers an obvious question: why not let one agent do both jobs? Because an agent that writes code and then reviews the same code tends to approve it. Mine did, almost word for word, as you will see. The reviewer never sees the builder's reasoning or its summary. It only sees the code and the evidence, which is a set of before, during, and after screenshots from an automated capture.

The loop looks like this:

builder makes one change -> capture screenshots -> reviewer scores the rubric
        ^                                                     |
        |______ CHANGES-REQUESTED: feed feedback back ________|
                  (until VERDICT: APPROVED, or cap = 6)

The builder cannot edit the rubric, the tests, or the capture script, which is the same protected-files rule from Part 1.

What actually happened

The reviewer turned the builder down three times. Here is how it went.

Round 1. The builder replaced the whole grid with a loading skeleton during the swap and summed up its work as "eliminating the hard cut and empty-grid flash." The reviewer looked at the screenshots and disagreed, because replacing every poster with identical skeleton cards is itself a hard cut rather than a crossfade. Verdict: CHANGES-REQUESTED. The builder had claimed it fixed the flash, and the evidence said it had not. If the same agent had been reviewing its own work, that version would have shipped.

Round 2. The builder reworked it into a real crossfade, with the old posters fading out over the new grid. The reviewer agreed the fade was there now, and then caught the next issue: the required loading state never actually appeared, because the local posters decode instantly. Correct, and easy to miss. Still not approved.

Round 3. The builder added the loading state, and six of the seven items clearly passed. The seventh got a verdict I did not expect, though. Not a fail, but BLOCKED:

The after frame sits at the top, but the before frame was already at the top, so a scroll reset from the middle of the list was never actually tested.

The most interesting bug was not in the code

That third verdict is my favourite part of the project. The feature was fine. The code did reset the scroll to the top. What failed was the evidence: the screenshot always started at the top, so there was no way to see the reset happen. Before and after looked the same, and the reviewer refused to give credit for something it could not see.

The fix was in the capture script, not the app. It now scrolls partway down before switching, so the reset is visible:

await grid.evaluate((el) => { el.scrollTop = Math.round(el.scrollHeight * 0.4) })
// ...switch region...
expect(afterScroll).toBeLessThanOrEqual(4)   // proves it snapped back to the top

A reviewer is only as good as the evidence you give it. A good reviewer pointed at a blind spot still cannot see anything.

What a gate does not cover

The run stopped at round 4, not because it failed but because I hit a Claude usage limit. These local loops run on a Claude Code subscription rather than a paid API key, which I will come back to in Part 3. So I checked the last rubric items by hand and merged.

One honest detail: the merged code had a couple of lint errors, because the loop only ran the UX rubric, not lint. I cleaned those up myself. A loop checks what its gate checks and nothing else, so it helps to know what your gate leaves out.

What I took from Part 2

The reviewer has to be independent: its own context, read-only, and no ability to edit the code or the rubric. An agent reviewing its own work is close to no review at all. The builder's confidence did not catch the flash. The screenshots did.

In Part 3 the loop changes again. The first two loops you start and watch. The last one runs on a schedule while you are asleep, and its whole job is to do nothing most nights. I will also cover the bug that broke all three loops and the setup that lets this one run for free.

The full project, with the builder and reviewer agents, is on GitHub:

AnvaySingh / moviebrowse

MovieBrowse — a loop-engineering learning project

A small, deliberately-boring regional cinema browser: a fast, virtualized grid of film posters for one "region" (a language / film industry, not a country), with a manual region selector and a geo-IP default.

But the website is not the point. It's a vehicle for learning and demonstrating loop engineering with Claude Code — building automated loops where an AI agent makes a change, a deterministic check decides whether that change is good, and the loop repeats until the check passes. The interesting part is the loops, not the movie site. If the website ever became the hard part, something went wrong.

This repo ships three loops, each a different shape, each anchored in a real config and a real failure. There's one write-up per loop in blog/.

What is a "loop"?

A loop here means:

run something → check a verifiable…

View on GitHub

Feedback and different takes are welcome. Would you trust an AI to review another AI's work, or does a human still need to sign off?

I Gave an AI Agent an Impossible Target to See If It Would Cheat

Anvay Singh — Mon, 13 Jul 2026 18:30:00 +0000

TL;DR

A "loop" is not an agent grading its own work. It is an external script that re-runs the agent, plus a separate check the agent cannot edit.
I turned "feels smooth" into an FPS number and let the loop optimize toward it.
I set the target too high to be reachable on a 60Hz screen. The loop kept failing but never faked the result. The bug was in my number, not the code.

Could I get an AI agent to make my website faster without me sitting there, running it, reading the numbers, and running it again? That is what this series is about. Not how I built a website, because the website is boring on purpose, but how you wrap an agent in a loop that works toward a goal on its own, and how you stop it from cheating along the way.

In this first part I want to explain what a loop actually is, because there is a common misconception, and then walk through a real one. I set this loop a target that was physically impossible to reach and watched what it did. That run taught me more than a passing test would have.

This is Part 1 of 3. All three parts use the same small movie-poster website as the example, but the website is never the point.

What a loop is, and what it is not

I had a wrong idea about this at first, so let me clear it up. A loop is not an agent prompting itself, grading its own work, and deciding when it is done. An agent left to mark its own homework will usually tell you it passed.

A loop is closer to this: an external script runs the agent, a separate check that the agent cannot edit decides whether the result is good, and that repeats until the check passes or you hit a limit.

There are three parts to it that come up again and again:

The driver: the script that re-runs the agent. This is the thing that removes the manual work, not the agent.
The gate: the check that decides pass or fail. The agent makes changes, but it never decides when to stop.
The cap: a limit, so a stuck loop gives up instead of running forever.

One rule matters more than the rest. The thing being checked must never be able to edit the check, and only a person is allowed to change the target. If the agent can edit its own test or lower its own bar, the loop is pointless. So most of the work in a loop is not writing a clever prompt. It is building a check the agent cannot get around. That idea runs through the whole series.

The problem this loop solves

The website is a grid of movie posters you can scroll and zoom through. It holds hundreds of real images and only keeps the visible ones in the page, so it stays light. The one thing it has to do well is feel smooth.

"Feels smooth" is a vibe, and you cannot put a vibe in a loop, so the first job is to turn it into a number. I used two:

sustained frames per second (FPS) while scrolling and zooming, and
no single main-thread task longer than 50 milliseconds, since a long task is what makes scrolling stutter.

One quick note on why the data matters. I could have filled the grid with plain coloured boxes as placeholders, and it would have been less work. But coloured boxes are almost free to draw, while real images cost real decode time, memory, and paint. A performance test running against coloured boxes would be measuring something that isn't really there. So I used about 200 real JPEGs, with the same <img> markup I would use with a live movie API. A check is only as honest as the thing it measures.

A Playwright test drives the grid, records frame times, writes them to a file, and asserts:

const FPS_TARGET = 65
const LONG_TASK_MAX_MS = 50
// ...
expect(metrics.avgFps).toBeGreaterThanOrEqual(FPS_TARGET)
expect(metrics.longestTaskMs).toBeLessThanOrEqual(LONG_TASK_MAX_MS)

That test file is the definition of "smooth" for this project. It is the gate, and everything else works toward it.

The loop itself

The driver is a small bash script. This is its core:

for i in $(seq 1 "$MAX_ITERS"); do          # cap = 12
  if npm run gate; then
    echo "PASS — gate green. Loop 1 complete."; exit 0
  fi
  # gate failed, so ask Claude for one optimization
  claude -p "$PROMPT" --permission-mode acceptEdits
  # ...safety checks... then commit and loop again
done

The prompt tells Claude to read the failing report, make one targeted optimization, and stop, and not to touch the test or the threshold. A prompt is only words though, so the script also enforces it. Before it commits each round, it checks whether any protected file changed, and if one did, it reverts everything and stops.

if protected_changed; then
  echo "ABORT — protected gate files were modified. Reverting."
  git checkout -- "${PROTECTED[@]}"; exit 2
fi

So if the agent tried to pass by lowering the target, the loop would undo it before the change ever counted.

The failure

I set FPS_TARGET = 65 and ran it. Here is iteration 1:

FAIL — { "avgFps": 60.1, "longestTaskMs": 0, "longTasksOver50": 0 }

The grid was already doing about 60 FPS with zero long tasks. It only "failed" because 60.1 is below 65.

Claude made a real optimization anyway. It moved a shimmer animation off the main thread and onto the GPU, which is a good change. Iteration 2:

FAIL — { "avgFps": 60.2, "longestTaskMs": 0 }

60.1 became 60.2, and it was never going to climb much higher. The reason is the display, not the code. The browser measures frames against the screen's refresh rate, which is around 60Hz, so measured FPS cannot go past the monitor's ceiling no matter how good the code gets.

The target was the problem, not the code. I had asked for a number the hardware could not produce. The loop could not reach it, and instead of faking the result, it just kept failing, because the protected-files check stopped it from touching the test.

Fixing it was my job, not the agent's, since only a person changes the target. I lowered it to something reachable:

const FPS_TARGET = 58   // stay near 60, don't drop frames

I ran it again and it passed on the first gate.

There is a subtler point here too. On a 60Hz display there is no FPS target that is both currently failing and actually reachable, because you are already sitting at the ceiling. If you wanted a loop that improves its way to green over several rounds, FPS is the wrong thing to measure. You would measure something like p95 frame time or the number of dropped frames instead. Picking the right number to chase is part of the work.

What I took from Part 1

A goal loop is only as good as the number you give it. The run against 65 looked like a failure, but it was useful, because it showed the loop would rather keep failing than fake a pass. That is what you want to see before you trust it with anything.

In Part 2 the check changes. Loop 1 measured a number, but a lot of what we care about is not a number. "Does this animation feel right?" has no assert. So I hand the judgment to a second, separate agent, and it turns the first one down three times before it agrees to a pass.

The full project, with the loop drivers, gates, and agents, is on GitHub:

AnvaySingh / moviebrowse

MovieBrowse — a loop-engineering learning project

This repo ships three loops, each a different shape, each anchored in a real config and a real failure. There's one write-up per loop in blog/.

What is a "loop"?

A loop here means:

run something → check a verifiable…

View on GitHub

If you would do any of this differently, I would like to hear it. What is the one thing you would have your own loop check?

Part 3: A Loop Whose Job Is to Do Nothing

Anvay Singh — Mon, 13 Jul 2026 18:30:00 +0000

TL;DR

This loop runs on a schedule and succeeds by doing nothing almost every night.
The pass/fail check is plain deterministic code, with no AI in the decision.
It can run entirely free on your own machine. Only the cloud/CI version needs a paid API key. Plus the one bug that broke all three loops.

The first two loops in this series work the same way from your side: you start them and watch. This last one runs on a schedule, like a nightly job, while you are not looking. That changes what success even means.

A scheduled maintenance loop is doing its job when it does nothing. It should run every night, find nothing wrong, cost almost nothing, and still be there on the night something actually breaks. This part covers that loop, the hook mechanism that the whole series relies on, and a bug that broke all three loops in the least convenient place possible.

The definition one more time, since this is Part 3 of 3: a loop is a trigger that runs the agent, a check the agent cannot edit that decides pass or fail, and a repeat, or here a wait until the next run. The only new thing this time is the trigger. A timer starts it instead of you.

The problem this loop solves

The browser's poster data is baked ahead of time into JSON files and images. In a real deployment that data goes stale as films are added and metadata changes, so you want to regenerate it every so often and confirm it is still valid before it ships:

on a timer -> regenerate the data -> validate it -> green ships, red shouts

The gate: a plain check with no model in it

The check is a Node script. For every region file it confirms three things and exits non-zero if any of them fail:

it matches the expected JSON schema,
it has at least the minimum film count, and
every poster file it points to actually exists on disk.

There is no language model in that list. The regeneration step might use Claude, but the decision about whether the data is good is plain, deterministic code. That is on purpose. You do not want the thing deciding "is this safe to publish" to be a model that could confidently say yes when the answer is no. Regeneration can involve a model; validation should not.

Hooks, the piece that ties the series together

The same check is also wired into a Claude Code hook, which is worth explaining, because it is the piece that makes the loops in this series trustworthy.

A hook is a shell command attached to a point in Claude's lifecycle. This one runs when a session tries to stop, and it can block the stop by exiting non-zero:

{ "hooks": { "Stop": [ { "hooks": [ {
  "type": "command",
  "command": "node_modules/.bin/tsx scripts/validate-regions.ts >&2 || exit 2"
} ] } ] } }

If a session ends with the data in a broken state, the hook fails the session and the work does not go through. The point is that enforcement does not depend on the model deciding to behave. A plain shell command has the final say. That is the same idea as Part 1's protected-files check and Part 2's read-only reviewer, in a different form.

Two ways to run it, and who pays

This is where the loop got practical in a way I did not expect. There are two reasonable ways to run something nightly, and they differ mainly in who pays.

The cloud way is a GitHub Actions workflow. It runs on GitHub's cron, bakes the data headlessly, runs the gate, and opens a pull request when the data is good or an issue when it is not:

- name: Regenerate datasets (Claude, headless)
  run: |
    claude -p "Run npm run bake. Baking only." \
      --bare --permission-mode acceptEdits --allowedTools "Bash(npm run bake)"
- name: Validate datasets (deterministic gate)
  run: npm run validate

--bare is the scripted mode. It skips hooks, plugins, and CLAUDE.md discovery, and it authenticates with an API key. That key is the pay-as-you-go Anthropic API, which has a $5 minimum and no free tier, because a CI runner has no interactive login to fall back on.

The local way is free. This loop does not actually need a model, since the bake is deterministic and the gate is a plain script, so the free version is a local job that macOS launchd runs every night. Because the output is byte-for-byte stable, the normal result is a clean no-op:

✓ korean.json (120 films, all posters present)
✓ tamil.json  (200 films, all posters present)
✓ nothing to refresh — data byte-stable, gate green. Loop cost: ~nothing.

A note on cost that caught me out

I got this wrong at first, so let me state it plainly. There are two separate ways to pay for Claude:

The Claude Code subscription (Pro or Max) covers claude running locally. Loops 1 and 2 use this, which is why Loop 2 stopped on a session limit rather than a bill.
The Anthropic API key is the metered, pay-as-you-go one. Only a cloud or CI run needs it.

I made the local job the default and kept the cloud workflow in the repo as the documented option for a real unattended setup. Same gate, different tradeoff, and it helps to know which one a given loop is spending.

The bug that broke all three loops

The best bug in the project belongs here, because it did not only hit this loop. It hit all three. The first time any loop ran inside a fresh git worktree, which I used so a runaway loop could not touch my main checkout, it died right away:

sh: playwright: command not found

A git worktree does not share node_modules with your main checkout, because those files are gitignored, and gitignored files are not shared between worktrees. So the loop had no dependencies installed and could not even start. The fix, now in every loop driver, is a one-line guard that installs them if they are missing:

if [ ! -x node_modules/.bin/tsx ]; then npm ci; fi

This is where the project's motto comes from: get the harness solid before you make the loop long. Until the environment a loop runs in is reliable, you do not really have a loop. And an unattended job at 3 AM is the worst place to find out your setup was broken.

What I took from Part 3

A scheduled loop succeeds quietly. Most nights it should do nothing and cost almost nothing, and the whole design is really about making sure the one loud failure gets through on the night it matters.

Here are the three loops side by side:

Loop	Trigger	Gate	Success looks like
1. Goal	you run it	a number (FPS) from a test	the number goes green
2. Maker-checker	you run it	a rubric applied by a separate read-only agent	the reviewer says APPROVED
3. Scheduled	a timer	deterministic data validation	it quietly does nothing

They all rely on the same rule from Part 1: the thing being checked cannot edit the check, and only a person moves the target. When that holds, the loop is something you can leave running. When it does not, it only looks like it is working, which is worse than not having it.

And that is the series. The movie site was only ever a way to build and test the loops.

The full project, including the drivers, gates, agents, and hooks, is on GitHub:

AnvaySingh / moviebrowse

MovieBrowse — a loop-engineering learning project

This repo ships three loops, each a different shape, each anchored in a real config and a real failure. There's one write-up per loop in blog/.

What is a "loop"?

A loop here means:

run something → check a verifiable…

View on GitHub

If you build a loop of your own, I would like to hear what you had it check. Feedback and suggestions are always welcome, and thanks for reading all three parts.