AI Latency Is a Feature, Not a Bug

Marc Limotte — Wed, 29 Jul 2026 15:37:40 +0000

The itch

A spinner, and a climbing token count. I've just handed my coding agent a spec that took two mornings to write — tight enough that I already know roughly what the diff should look like. For the next ten minutes, maybe thirty, the work is out of my hands. And here comes the itch: do something, open another terminal.

There are two well-worn ways to scratch it.

Camp one says scale up. Agents managing agents: Steve Yegge's Gas Town orchestrator is built to run 20–30 Claude Code instances at once, and he describes the destination as agent fleets — a hundred-plus agents, supervised by yet more agents. Camp two says the itch is the injury. A BCG Henderson Institute study of 1,488 workers put a name on the damage — "brain fry" — and located the cause: "the most mentally taxing form of AI engagement was oversight." Not using AI. Watching it. Sufferers reported 39% more major errors and 33% more decision fatigue. Camp two's prescription is guardrails: cap your concurrent streams, batch your check-ins, protect recovery time. Sane advice that stops one step short — it says what to stop doing, not what the reclaimed attention is for.

About the title of this post. The latency is real — minutes at least, sometimes most of an hour — and the only workaround we've found for it is multi-tasking. So yes, calling the latency a feature is a little inflammatory, since what it hands you first is the burden of juggling. But the juggling is now a fact of the job the way meetings are a fact of the job, and the useful question is no longer whether to multi-task but how to do it best — I'd argue doing this well is quietly becoming one of the defining skills of the era.

So here's a third answer, somewhere between the camps. Camp one buys throughput by overdriving the engineer; you and the product both pay eventually. Camp two protects the engineer but treats the reclaimed time as something to guard rather than use — restful, and a little unsatisfying. My version: keep camp two's limit, then spend what it frees on the kind of thinking the job never used to leave room for. Those ten quiet minutes while the agent works are the most undervalued asset in an AI Engineer's day.

Spend the latency, don't fill it

The old loop:

Think
Code
Think
Code

The new one:

Think
Delegate
Think bigger
Review
Delegate
Architect
Review
Refactor

What changed isn't the speed of the loop; it's the shape.

My setup, offered as one practitioner's habits and not a prescription. Inside any one project: one coding task at a time, so agents never collide in the same working copy. Across everything: one to three foreground agents, period.

So if you're capped at three and one of them just took your task, what do you do with yourself? You switch — and there are two kinds of switch.

Switch projects: pick up a different repo entirely. Right now I'm rotating between three:

repackaging and documenting quandary, my wrapper around OR-Tools' CP-SAT solver, for open-source release
a new feature that computes the 2D geometry for non-orthogonal corners in a physical kitchen layout
a refactor to replace a JavaScript-heavy, spreadsheet-like interface with a Datastar architecture

Constraint solvers, computational geometry, front-end architecture. The flavors are so different that moving between them barely feels like task-switching — nothing overlaps, nothing couples, and each return comes with fresh eyes.

Or switch functions: stay with the same project but move up a level. While the agent is implementing, testing, or reviewing this feature, you think hard about the next one — or about the fix you've been deferring, or the refactor that would make the next three features cheap. (The Datastar work above is this kind: there's no code yet. I'm still working out the spec.) Don't multitask across agents; multitask across levels of abstraction.

I've spent the last five years as CTO of my own startup — nominally in control of the schedule — and the push was still always from one feature directly into the next; stepping back never got a line item. Agent latency installs the pause whether you asked for it or not — ten unhurried minutes with nothing to type, which is a good size for questioning things: is this abstraction still the right analogy? Why does this module keep needing exceptions?

There's a mechanism under this, and many engineers already know it through folklore. Rich Hickey's "Hammock Driven Development" talk makes the claim precisely: your background mind does the heavy lifting on hard problems, and you can't direct it — you can only feed it, by making a problem an agenda item during your waking hours. Every engineer has the corroborating data: the bug that solves itself in the shower, the design that unknots first thing in the morning. Cognitive scientists call it incubation, with a detail worth noticing: answers tend to surface during mildly engaging activity — the walk, the shower — after the problem has been deliberately loaded. The latency window is for the loading. Vigilance-monitoring ten agents, by contrast, is the dead zone: too demanding to let the mind wander, too shallow to feed it anything. It happened to me mid-draft on this very article: I spent one latency window chewing on the brute-force argument you'll meet in the next section, and the realization that it was all assertion and no support arrived in the middle of the night, unprompted.

Feeding isn't the window's only job. Plenty of windows go to plain dispatch — writing prompts for tasks that don't depend on anything currently running. But a prompt is direction in miniature, a spec that fits in a paragraph, and more than once a prompt I meant to dash off kept growing until it had become the next spec. It's the same work at different depths: authoring direction for the agents, loading problems into your own head. The best windows do both.

The loading has its own variety. Some windows go to reading a problematic interface, or a stretch of code you suspect needs a refactor — holding it in view long enough for the background mind to pick it up. Some go to re-walking the whole workflow, asking whether its shape is anything more than an accumulation of stale decisions. And some go to the next spec.

That last form has an industrialized version: Spec Driven Development. Work the spec, the research notes, the plan into discrete artifacts before any agent writes code — tools like GitHub's Spec Kit and Amazon's Kiro are built around exactly this loop. Hickey again: the first step is writing the problem down. The artifacts pay twice. Producing them is the thinking — writing the spec is a feeding session on paper. And handing them off is clean: when the direction is a document rather than a notion in your head, delegation becomes a closure event. You can put the task down completely and come back without having left half your attention behind.

What this costs

The strongest objection to my pitch first. Task-switching isn't free. Sophie Leroy's research named the cost — "attention residue," the fraction of your mind that stays lodged in the previous task — and found it heaviest when that task was left unfinished. An article recommending multi-tasking has to answer for this. Part of the answer is the clean handoff from the last section: delegation that ends in a written artifact is a finish, not an interruption, and the residue research is about interruptions. The other part is symmetry: when an agent finishes while you're mid-thought, let it wait. Complete the thought, then attend. Camp two calls this batching your check-ins, and camp two is right.

Not everything deserves the full ceremony, either. An engineer at Kilo describes the practice as gardening rather than building: "take care of that, remove that weed." But weeds don't need a separate fleet — they're the short-cadence residents of the same 1–3: the rename, the doc fix, small enough that prompting one fits inside a bigger task's latency window. Some windows go exactly that way: scan the kanban board, find something that fits the gap, write its prompt. A working moment for me usually looks like one spec in progress, one long implementation running, and a weed on its way out.

Now the steelman. Orchestration — agents managing agents — surely wins for some problem shapes: bulk migrations, wide mechanical sweeps, tuning your product's prompts against an eval suite, ongoing maintenance of a domain the fleet already knows. It may earn its keep most clearly outside coding altogether — continuous data QA, say, or ingestion pipelines that watch for new data around the clock. If that's your work, run the fleet. For feature work, though, two problems survive even when you personally touch only a few foreground agents at a time.

The first is review debt. The fleet generates code faster than any human can honestly read it, and the gap compounds: unreviewed decisions become the foundation the next round of generation builds on.

The second is that it's brute force. We've known since The Mythical Man-Month that adding developers to a late project makes it later — coordination costs grow faster than output. Agents are headcount. Cheaper and tireless, but headcount, and a fleet of them attached to a thin spec is the Mythical Agent-Month. Statistically, the fleet is running best-of-N sampling against a distribution centered on the quality of your direction; a better spec moves the whole distribution, while more agents only search its tails. And the N isn't free: Microsoft's Experiences and Devices group canceled most of its Claude Code licenses this spring after per-engineer token bills reached $500–$2,000 a month. So the trade on offer is: spend real money to replace your own deep engagement with rapid switching and surface-level review, at exactly the moment your understanding of the code is lightest. I'd rather spend the same attention improving the direction.

From the outside, I'll admit, my desk looks like theirs — a person talking to a few agents. The difference is what's behind it: no unreviewed fleet, every line of direction authored by me, and the latency going toward thinking instead of fleet management. If your answer is that reviewer agents can carry the review debt, notice what that is — more sampling to compensate for thin direction. Agents can check each other's correctness. Whether the work matches intent can only be checked by whoever holds the intent.

Now the confession. "Greater throughput and higher quality" is a claim I can't prove; nobody has run the study comparing a focused practice against an orchestrated one. I also haven't lived the other side: I tried Gas Town, and Conductor, and stopped at "overkill" — so this is a critique of the mechanism and the economics, not a survivor's tale. What I can offer: review debt observably compounds, direction quality — judgment, really — multiplies across every token generated downstream, and the adjacent evidence says perceived speed lies — in the one randomized trial we have, experienced developers estimated AI made them 20% faster when it had actually made them 19% slower. The tooling in that study is dated now; the perception gap is the part I'd bet still holds.

One more objection: models keep getting faster, so won't the latency — and this article — evaporate? I doubt it. Capability grows the scope of what we delegate; the tasks just become bigger.

Monday morning

The practice, compressed:

Cap foreground agents at what you can honestly guide and review. For me that's one to three. The BCG data agrees: productivity gains flattened at three concurrent AI streams and fell after that.
When you feel the itch to spawn another agent, write the prompt for the next spec instead. The itch is real energy; point it at direction rather than supervision.
Finish your thought before attending a finished agent. Clean handoffs run in both directions.
Keep a hammock list — the few questions you're currently feeding your background mind. Glance at it whenever a window opens.
Let the small stuff be gardening. Weeds share the same one-to-three slots; scan the kanban (or what have you) during a window, find one that fits the gap, prompt it.

I don't have to prove this one. Teams are running every version of this workflow right now, and time will tell which approaches compound. What I want to plant is simpler: the fleet is not the only path. There's an alternative that keeps you thinking instead of monitoring, and each engineer can judge it for themselves. One score you can trust from the inside: whether you end the week fried or fed.

This piece wasn't written in one window; it accumulated across a week of them. There is a spinner going one pane over right now, though. So let me end on the state of the board:

This article — an agent and I are still passing the draft back and forth.
The Datastar migration — mid-discussion. Claude reached for Signals everywhere it could; I think it overused them, and I'm applying a friend's advice to mange the client state as server-side session state.
The kitchen geometry — wrapped. The non-orthogonal corners went out on the strength of a spec I only partly wrote. Most of it came from the architect — the kind who designs buildings, not software.

DEV Community: Marc Limotte

AI Latency Is a Feature, Not a Bug

The itch

Spend the latency, don't fill it

What this costs

Monday morning