Put a fast model inside a bad workflow and watch what happens.
It scans the repository, receives a wall of search output, asks permission for a harmless command, loses the important constraint in the noise, rereads the same files, and finally runs the test that could have run automatically ten minutes earlier. The model may be quick. The task is still slow.
Coding-agent performance is an end-to-end property. Inference speed matters, but only for the slice of the run spent waiting on inference. The rest is context collection, tool execution, approvals, validation, retries, and recovery.
Before paying for faster reasoning, remove the turns that never needed reasoning.
Count handoffs, not chat messages
A round trip is any handoff that must finish before useful work can continue. It might be another model turn, a tool invocation, a human decision, or a recovery cycle after the run loses state.
A normal coding-agent loop looks roughly like this:
context -> reason -> act -> observe -> approve -> validate -> recover or finish
None of those phases is automatically waste. The waste appears when boundaries are vague.
A tool dumps an entire log when the next step needs one error. An approval covers one command instead of a defined risk scope. A failed check returns prose instead of a machine-readable result. A resumed session rebuilds context from the transcript because nobody saved a durable checkpoint.
Map those handoffs before changing models. Here is an illustrative version, not a proposed standard:
context_acquisition:
actor: retrieval tool
input_boundary: task contract and allowed repository scope
output_artifact: context manifest
repeat_trigger: a named evidence gap
model_reasoning:
actor: coding agent
input_boundary: context manifest and current goal
output_artifact: decision or bounded plan
stop_condition: next action and success condition are explicit
tool_execution:
actor: deterministic tool
input_boundary: fixed command, files, and limits
output_artifact: bounded result plus full-log reference
retry_rule: retry only a classified transient failure
approval:
actor: human or policy service
scope: named effects and resources
output_artifact: decision record
invalidation: scope, target, or reviewed artifact changes
validation:
actor: test, linter, type checker, or other named check
input_boundary: candidate artifact
output_artifact: structured pass or fail evidence
failure_route: repair, escalate, or stop
recovery:
actor: runtime
input_boundary: durable checkpoint and classified failure
output_artifact: restored state and allowed next action
stop_condition: state cannot be restored safely
The indentation mistake in many real systems is conceptual: everything sits under model_reasoning. Retrieval, command selection, policy, validation, and recovery all become prompts. The agent keeps talking because the workflow gave it no other way to move.
Remove the model from deterministic leaf work
Models are useful when the next action depends on incomplete evidence, tradeoffs, or interpretation. They are an expensive control structure for a transform with fixed inputs and a checkable output.
Formatting is the obvious example. File conversion, schema validation, asset resizing, linting, and many code-generation substeps can also be deterministic. Give them typed inputs, run the tool directly, and return a small result.
If a generated release image needs platform-ready dimensions, a browser-local utility such as Resize Image For can handle that fixed resize-and-export step without uploading source pixels for processing. Asking an agent to rediscover dimensions, choose an image library, write a temporary script, and inspect the output adds uncertainty to a settled operation.
The same principle applies inside a repository. If the project already has a formatter command, run it. If a JSON schema decides whether a file is valid, call the validator. If a codemod expresses a known migration, let the agent choose whether the codemod fits, then let the codemod perform the repetitive edits.
Direct tools do not make the workflow safe by themselves. A deterministic command can still target the wrong files or receive bad input. The benefit is a narrower contract: the agent decides what should happen, while the tool performs a repeatable operation whose output is easier to inspect.
Bound context without starving the task
"Use less context" is bad advice when applied blindly. Missing a package convention or an acceptance requirement can create more retries than a smaller prompt saves.
The useful goal is bounded context. Start each run with a stable task contract:
- the repository state and task goal
- allowed files, commands, and effects
- relevant project instructions
- the expected artifact
- named acceptance checks
- stop and escalation conditions
Retrieve repository material against that contract. Return the matching symbols, nearby tests, and local instructions instead of dumping a broad search transcript. When a tool produces a large log, keep the full output as an artifact and send the agent the failing check, exit status, and a reference to the detail.
This also changes recovery. A durable task contract, context manifest, current diff, and latest check result are much cheaper to resume from than a long conversation full of superseded plans.
Alera's project description makes these surrounding surfaces visible: real PTYs for CLI agents, isolated Git worktrees, lifecycle activity, resource tracking, and persisted sessions. That does not establish a speed advantage. It does show that process state, workspace state, and session state belong in the performance model.
A fast response cannot compensate for rebuilding all three after every interruption.
Put approvals at risk boundaries
Approval prompts are often treated as a choice between safety and speed. That framing produces two bad designs: interrupt the operator for every small action, or grant broad authority so the chart looks faster.
An approval should describe a meaningful boundary. Scope it to the effects that carry risk: changing deployment files, using a credential, writing outside the worktree, opening a network connection, merging a pull request, or publishing an artifact. Let routine operations inside an already approved scope proceed without renegotiation.
The approval also needs an invalidation rule. Permission to deploy one reviewed artifact should not silently transfer to a different artifact after the agent edits it. A decision tied to a target, scope, and artifact state is useful evidence. A floating "yes" in a transcript is not.
Proliferate's README presents reusable workflows, isolated task worktrees, parallel agents, and human approval gates as explicit orchestration primitives. Again, that is a project-authored description rather than measured proof. The useful design signal is the separation: workflow steps and approval boundaries exist outside the model's conversational improvisation.
Some reviews must remain serial. A consequential change may deserve a person reading the final diff even if that adds latency. Optimize the placement and scope of that decision; do not delete it to win a benchmark.
Validation should return evidence, not reassurance
An agent saying "the tests should pass" is not a validation phase. Neither is a final message that says the task is complete.
Where acceptance can be mechanical, let the named checks decide it. The runtime can run tests, lint, type checks, schema checks, build commands, or task-specific assertions without asking the model to narrate each command. Return structured results to the loop:
check: pnpm-typecheck
status: failed
exit_code: 2
summary: 2 errors in src/auth/session.ts
artifact: logs/pnpm-typecheck-017.txt
next_route: bounded-repair
The model still has a job when a check fails: diagnose the evidence and propose a repair. It should not need to persuade the system that a failed check is close enough.
This boundary matters after tool upgrades too. A current coding-tool roundup reports changes involving permissions, secret redaction, MCP configuration and trust, and experimental worktree behavior. Those changes sit around inference. An upgraded model can answer faster while the run loses time to a stale integration, a changed entry point, or a permission failure.
Re-run the workflow checks after an upgrade. Old latency assumptions do not survive a changed execution path merely because the model name stayed familiar.
Parallel agents change the shape of waiting
Parallelism can reduce elapsed time when tasks are genuinely independent. It can also create more handoffs: worktree setup, task partitioning, shared-state coordination, competing patches, and a larger review queue.
Count accepted outcomes, not active terminals. Two agents investigating the same failure may produce twice the activity and no additional progress. Four patches waiting on one reviewer are still waiting.
This is why resource and lifecycle visibility matter. For each worker, preserve the task boundary, current state, produced artifact, and owner of the next decision. Concurrency without those records turns recovery into archaeology.
Parallel work earns its complexity when overlap removes waiting that cannot be removed another way. It is not a substitute for making one loop legible.
Benchmark the loop, not the launch demo
The launch discussion for Bullet is useful because its maker attributes speed to several workflow choices: model routing, targeted search, bounded tool output, context hygiene, and fewer round trips. Commenters questioned the comparison methodology. Both sides point to the same requirement: "fast" needs a task envelope.
Compare representative runs from the same starting state and task class. Record:
- latency by phase
- round-trip count by type
- retries and their causes
- the artifact produced
- the named checks that passed or failed
- whether the outcome was accepted
Do not blend a typo fix with a cross-package migration and call the average meaningful. Do not count a run as fast because it stopped before validation. Do not compare one agent with precise repository instructions against another starting from a vague sentence.
Fewer round trips are a hypothesis, not a guarantee. One extra retrieval can prevent several repair cycles. One deliberate approval can prevent a costly effect. A model turn that catches a bad assumption may be the quickest part of the run.
The target is avoidable latency. Preserve the handoffs that carry judgment, authority, or proof.
That distinction also protects engineering understanding. Removing repetitive work is useful. Removing the points where someone has to understand the architecture, assess a tradeoff, or review a risky diff is how output outruns comprehension.
Use a round-trip budget before a model leaderboard
For one week, instrument the workflow you already have. You do not need a grand observability platform. A task record with timestamps, handoff types, retries, artifacts, and check results is enough to expose obvious waste.
Look for repeated patterns:
- repository searches that fetch the same material
- raw tool output repeatedly injected into context
- low-risk actions approved one at a time
- deterministic transforms performed through model turns
- validation delayed until a large patch accumulates
- sessions that cannot resume from a durable artifact
Fix one boundary at a time. Replace repeated discovery with a context manifest. Add a summary contract to a noisy tool. Move a fixed transform behind a direct command. Tie approval to a real risk scope. Run a focused check as soon as the relevant artifact exists.
Then compare models under the improved workflow. The faster model may still win, and now the comparison measures more than its ability to survive your orchestration overhead.
Use this order: remove unnecessary turns, bound the remaining handoffs, then compare models.
Before paying for a faster model, remove the turns that never needed a model.
Source notes
Top comments (0)