DEV Community: Sudarshan Shenvi

Your Agents Don't Need a Better Loop. They Need an Org Chart

Sudarshan Shenvi — Wed, 12 Aug 2026 05:10:46 +0000

This is a follow-on to Set It and Ship It: How I Let AI Agents Build My Java Services While I Sleep -start there if you haven't.

Last time, the whole thing came down to a setup I still run every day: a builder agent and a separate critic, working one service at a time against requirements written tightly enough that a machine could grade them. It works, and I wasn't planning on a sequel.

Then I tried to scale it, and something broke. It wasn't the loop - the loop held up fine. The problem was everything happening between the loops, which, it turned out, I'd never really thought about.

Parallel isn't the same as connected

That first setup once ran three agents at once - a reconciliation service, a Kafka consumer, a cache - and by morning I had three clean reports. What I skipped over, mostly because I hadn't clocked it myself, is why that worked: none of those three tasks touched the others. Three strangers in three separate rooms, basically.

Most real work isn't like that. Sooner or later one agent needs a contract that doesn't exist yet because the agent writing it is still going. Or two of them open the same file and the second one quietly paves over the first. Every loop passes its own gate and the feature is still broken, because each gate only ever looks inward, at its own little job.

A refund flow, and where it fell apart

Let me make this concrete with an engineer I'll call Loknath - really a stand-in for a few good people I've watched hit this same wall.

Loknath has the one-loop workflow down cold, so he reaches for something bigger: a refund flow. It touches a lot of the stack - a schema change to store refunds, an API endpoint to kick one off, auth so not just anyone can, and the actual write against the ledger. Same move as before. Four requirement lists, four agents, into the always-on box overnight.

He comes back to a mess. The API agent had built its endpoint against a refunds table the data agent hadn't shipped yet, so half of it was wired to a schema that moved underneath it. Two agents had both been editing the ledger module. And one of them, doing its honest best to satisfy "trigger a refund," had gone and issued a real refund against a test account, just to prove the path worked end to end. Four green reports. One broken feature, and one transaction nobody could take back.

Here's what stuck with me: none of that was a bad loop. Every agent did solid work inside its own room. What went wrong was that nobody had decided who owned what, who was allowed to talk to whom, or which step needed a person to look at it first.

Loknath hadn't been running workers. He'd been running an organization, and no one had drawn the org chart.

The rung above the loop

There's a ladder people talk about that helps here. A prompt controls a single response. Context controls what the model sees. A loop controls one agent's cycle of doing and checking its own work. My first post lived entirely on that loop rung.

The rung above it is the topology - which agents exist, what each one owns, who's allowed to talk to whom, how the work moves through them. Not one loop running well, but a bunch of them wired together on purpose. And if you don't wire it on purpose, it still gets wired; it just takes the shape of whatever order you happened to launch things in.

Give every agent a mandate

The thing that helped Loknath most was to stop launching anonymous agents at "the repo" and start handing each one a mandate - one narrow thing it owns, which quietly defines the much larger set of things it has no business touching.

security-agent   owns  auth, permissions, audit logs
data-agent       owns  schema, migrations
api-agent        owns  endpoints, request/response contracts

An agent with no mandate is basically an intern with commit access to prod and a lot of enthusiasm. It'll help everywhere, including the places you didn't want helped. Once the lines are drawn, the API agent that needs the refunds table can't just conjure one up - that's the data agent's turf, so the request has to cross an edge and wait its turn. The "who talks to whom" question I couldn't answer earlier stops being a question, because ownership went and answered it. It's the same fix testable requirements gave me inside a single loop, just one floor up.

The one edge where you put a human

In the first post I was pretty adamant about turning the permission prompts off, and I still am - asking a human to approve every step is how you turn an overnight run back into a full-time babysitting job.

Scaling up taught me the more careful version of that rule. Not zero gates, but zero gates in the boring places and one real gate in the place that counts. The edges in a graph aren't equal. Most of them carry cheap, reversible stuff. A few carry the things you can't undo - the payment, the migration, the deploy - and those tend to sit right where one agent hands off to the next.

data-agent ──► api-agent ──► [ refund write ]  ◄── human approves here
   (auto)        (auto)          (gated)

That's the exact spot Loknath's phantom refund would have died. One approval on the irreversible edge, hands off everywhere else. You've got one interruption to spend - spend it there.

"The graph did it" is not an audit answer

Getting it to work is half the job. Being able to see what happened afterward is the other half, and I underrated it badly until the first time something broke and I couldn't tell which agent had done it.

Give each agent its own identity. When five of them act as the same account, "who did this" has no answer, and you find that out at the worst possible moment.
Decide which agent can reach which tool, on purpose. Saying the frontend agent can't touch payments is dull work, and it's most of your safety.
Tag every request with the graph, the run, and the node. That's how you hold what actually happened up against what you meant to happen, and the space between those two is where the real bugs live.
Budget each agent, not just the project. Graphs fan out, and the token cost I flagged last time doesn't quietly add up here so much as multiply.

The ways this goes wrong

Nearly every failure I've seen comes back to a handful of things. The topology was never designed, so it's just whatever fell out of the launch order. Everything shares one identity, so there's nothing to grab when it breaks. Fan-out and retries run the bill up without anyone noticing. A bad instruction slips into one agent and rides the edges straight into the next. Or there's simply no gate on the one step you'd have most wanted to catch.

What the bottleneck becomes

The first post moved my bottleneck from writing code to writing contracts. This one moved it again - from writing a contract to designing an organization: who exists, what they own, where the boundaries sit, and which single edge out of fifty is worth a person's attention. If you've ever watched a team go from three people to thirty, none of this will feel new. It's an old problem in different clothes, and about the only thing that's really changed is that the org I'm designing now is made of agents.

The usual disclaimer, and it matters more here than it did last time. Everything above is what's worked for me, and none of it is a promise - the more agents you wire together, the more ways there are for the arrangement itself to be the thing that's wrong. Watch the cost especially: a graph multiplies token usage in ways a single loop never did. Give each agent a scoped identity and scoped tools, meter every one of them, cap the whole run, and keep a human on the edges you can't undo. Start with two connected agents before you try ten. The structure lowers the risk; it doesn't erase it.

Set It and Ship It: How I Let AI Agents Build My Java Services While I Sleep

Sudarshan Shenvi — Sat, 01 Aug 2026 08:52:58 +0000

I've been doing this job long enough to be suspicious of anything that promises to make it easy. So I want to be upfront: what follows still feels a little unreal to me, even after weeks of watching it work.

Here's the short version. The current crop of agents can run for five hours straight and build almost anything you ask for, at a quality level that genuinely holds up. There's a catch, and it's the whole point of this post: they only do that if you write your requirements carefully and force the work through hard gates before it comes back to you. Skip that part and you get five hours of confident, plausible garbage. Do it well and the ceiling is set by how clearly you can think, not by the model.

Why this is the thing to get right

Give a capable agent a loose prompt and a few hours, and it will produce a mountain of output. The problem is that the mountain drifts. Little assumptions pile up, the thing wanders off from what you actually wanted, and by hour three it's polishing something you never asked for.

Gates fix that. Three of them, specifically, and they work together:

First, requirements that are actually testable - no vibes, no "make it good." Second, a separate critique agent whose only job is to check the work against those requirements and refuse to pass it until every single one is met. Third, somewhere for all this to run uninterrupted, with the permission pop-ups turned off so nobody has to babysit it.

None of these is clever on its own. Together they're the difference between a demo and a workflow you'd actually trust.

The rules I don't break

I've boiled it down to five things I won't skip, because every time I've skipped one I've regretted it:

Write the requirements before the agent starts. No exceptions, even when the task feels obvious.
Treat every requirement as a gate, not a suggestion.
Let a different agent judge the work. The one that built it doesn't get a vote.
Demand a pass on all of them. Not most. Not "basically done." All.
Run it always-on with the permission prompts off.

Writing requirements that hold up

The requirements list is the contract, and honestly it's where the real work moved to. If the list is fuzzy, the output is fuzzy and the agent will happily fill every gap with a guess.

What I aim for is requirements a machine can grade without me in the room. Each one names something specific: a behaviour, a file, an interface, a limit. Each one can be marked pass or fail with no argument. Between them they cover the boring stuff too - edge cases, performance, security, docs because that's exactly what gets quietly dropped otherwise.

A quick before-and-after. Here's the version that'll burn you:

"The API should be fast and secure."

And here's the version that actually protects you:

"All endpoints respond in under 200ms at p95 under 100 concurrent requests. Every input is validated server-side. No secrets in logs. Auth required on every route except /health."

Same intent, wildly different outcome. Time spent sharpening this list pays for itself many times over across a long run, so I don't rush it anymore.

The critique gate

A requirements list nobody enforces is just a wish. The enforcement is a separate critique agent, and keeping it separate is the whole trick and you don't let the builder grade its own homework.

The loop is simple. The builder produces the work. The critique agent gets that work plus the full requirements list, then goes down it item by item and marks each one pass or fail. Anything that fails goes back with the specific gap called out, and round it goes again. The work only lands on my desk once every requirement has a clean pass.

A few things I insist on from the critic: it checks everything, not a sample. It gives me a real verdict per item, not a warm "looks good." And I write its prompt to be adversarial on purpose since I want it hunting for the gap, not looking for a reason to approve.

Where it runs matters more than you'd think

Five hours is a long time for a laptop lid to stay open. So this runs in the cloud or on a machine that never sleeps, never times out, never loses state halfway through.

The part people flinch at is turning off the permission prompts. I get it. But every "can I edit this file?" pop-up turns an autonomous five-hour build back into a job you have to sit and watch, which defeats the entire point. The gates and the critique agent are what keep the run honest and not you clicking Allow forty times.

To be clear, "no permission gates" is about the agent's inner workflow, not about throwing safety out the window. Scoped credentials, a sandbox, an isolated branch, a spend limit; all of that stays. An unattended run should be contained. It just shouldn't need a chaperone.

A Tuesday with Mithun

Let me make this concrete with someone I made up who behaves exactly like the good engineers I know. Call him Mithun. He's on a Spring Boot team, and here's how one Tuesday goes now that the agents do the building.

8:45 AM - coffee and the contract

A ticket lands: add a PaymentReconciliation service that matches settled transactions against ledger entries and flags the mismatches. A couple of years ago that's two or three days of work. Mithun doesn't open his IDE. He opens requirements.md and writes the contract instead:

# Requirements: PaymentReconciliation Service

## Functional
- REST endpoint POST /reconciliation/run accepts a date range (from, to).
- Matches Transaction records against LedgerEntry by transactionId + amount.
- A mismatch = amount delta > 0.00 OR missing counterpart on either side.
- Persists a ReconciliationReport (JPA entity) with counts and line-item results.

## Non-Functional
- Java 21, Spring Boot 3.x, Maven build.
- p95 latency < 500ms for a 10,000-record range.
- 100% of new code covered by JUnit 5 tests; mismatch edge cases tested explicitly.
- No secrets in logs; all inputs validated with Bean Validation.
- Follows existing package layout under com.blute.reconciliation.

## Definition of Done
- mvn verify passes clean (compile + test + checkstyle + spotbugs).
- OpenAPI spec regenerated and committed.
- README section added describing the endpoint and mismatch rules.

Fifteen lines. That's the job, really and the rest is typing he no longer has to do.

9:10 AM - launch and walk away

He points the builder agent at the repo and the requirements file, running in the team's always-on environment. The last line of his prompt is the one that matters:

"All work must be verified by separate critique agents against every requirement in requirements.md. Do not return until a critique agent confirms a pass on 100% of them. mvn verify must be green."

Then he closes the laptop and goes to standup. No pop-ups chase him.

9:15 AM to 11:30 AM - it runs without him

While he's in meetings, the whole thing plays out on its own. The builder scaffolds the service, the JPA entity, the controller, the matching logic. It writes the JUnit tests, including the fiddly ones - zero-delta matches, missing counterparts, rounding right at the cent boundary.

Then the critique agent picks up the diff and starts marking. It fails two items: the latency requirement isn't actually proven because there's no benchmark, and checkstyle is unhappy about the package layout. Both go back. The builder adds a timing test over a 10k-record fixture and moves two classes where they belong. The critic runs again, and this time it's clean - fifteen of fifteen, build green, OpenAPI regenerated, README updated. Nobody touched any of it.

11:35 AM - he reviews the output, not the process

Mithun gets back to a finished branch and a report waiting for him:

POST /reconciliation/run implemented ......... PASS
Mismatch rule (delta > 0.00 | missing) ....... PASS
ReconciliationReport persisted ............... PASS
Java 21 / Spring Boot 3.x / Maven ............ PASS
p95 < 500ms @ 10k records (measured 410ms) ... PASS
100% new-code test coverage .................. PASS
Bean Validation on all inputs ................ PASS
No secrets in logs ........................... PASS
Package layout com.acme.reconciliation ....... PASS
mvn verify clean (checkstyle + spotbugs) ..... PASS
OpenAPI regenerated & committed .............. PASS
README section added ......................... PASS
... 15/15 PASS

He still reads the matching logic and the edge-case tests. Not to catch bugs - the gate already did that - but to check the approach lines up with how the team thinks about reconciliation. It does. He leaves one note about a naming choice he'd have made differently, then adds that preference to the team's requirements template so next time it's a gate instead of a comment.

2:00 PM - two more in parallel

Here's the part that reorganised his day. Because he's not hand-writing code, his bottleneck is now just thinking clearly about what he wants. So he spends the afternoon writing two more requirement lists - a Kafka consumer for settlement events, a caching layer for the ledger reads - and kicks both off to run into the evening.

6:30 PM - set it and ship it

He logs off. The critique loops grind away overnight. By morning there'll be two more branches, each with its own clean report waiting for a human read. His job quietly stopped being about writing lines and became about writing contracts, and the throughput that unlocks is honestly still hard for me to wrap my head around.

How I actually run this, step by step

If you want to copy the workflow, this is the shape of it:

Scope the task and write the requirements list, every item testable.
Put the hard rule in the prompt: separate critique agents check all work, 100% passes before it comes back.
Provision an always-on environment with the permission gates off and the guardrails on.
Launch the builder with the task and the full list.
Let the critique loop run - pass/fail per item, failures bounce back to the builder.
Accept the work only on a full pass.
Review the output, not the process, and feed anything you learned back into the template.

The ways this goes wrong

Almost every failure I've seen traces back to one of these:

Vague requirements, where "make it good" gives the agent nothing to be checked against. Self-review, where the builder blesses its own work and the whole gate collapses. Partial passes, where "most of it's done" quietly lets the drift back in. Babysitting, where leftover permission prompts throw away the exact capability you were reaching for. And an under-powered environment, where a five-hour job dies on a machine that went to sleep at hour two.

Get the requirements right, keep the critic honest and separate, and give it somewhere to actually run - that's most of the battle. The rest is learning to trust the gates enough to close the laptop.

A quick disclaimer. Your mileage will vary. Everything above reflects what's worked for me on specific tasks with specific codebases, and none of it is a guarantee - model behavior, task complexity, and your existing test and review infrastructure all change the outcome. Treat this as a starting point, not a recipe. Run agents against isolated branches and sandboxed environments, keep credentials scoped and spend capped, and have a human read the output before anything ships to production. One thing to watch closely: token usage. Multi-hour runs with builder-plus-critique loops burn tokens fast, and every failed-and-retried requirement adds another pass over the work - costs can climb quickly and unpredictably. Set hard budget limits, monitor consumption during long runs, and price out a small task before you turn agents loose on a big one. The gates reduce risk; they don't remove it. Use judgement, start small on low-stakes work, and scale up only as you build confidence for your own context.