DEV Community: hefty

Your Coding Agent Needs an `unverified` State

hefty — Fri, 18 Sep 2026 06:39:08 +0000

Imagine a coding agent patches a checkout bug. The focused unit test passes. The process exits cleanly. Its final message says the task is done.

One problem: the request was about a browser checkout flow, and nobody ran that flow.

The patch may be correct. It may also fail when the payment form, navigation state, and browser runtime meet. The agent has evidence for one claim and no evidence for another. Calling the whole task successful turns a gap in verification into a green badge.

Coding-agent systems need an explicit unverified outcome. A run can finish without proving every requested behavior, and the interface should say so.

"Done" is carrying too many meanings

Most agent workflows collapse several events into one status:

The process stopped.
The tool changed something.
The requested behavior was accepted.

Those events are related, but they are not interchangeable.

A zero exit code tells you that a process completed according to its own exit convention. Reading the file back tells you that bytes were written. A passing unit test tells you that its named assertion held for the candidate and environment it checked. None of those facts automatically proves the whole task.

The gap gets easier to miss as agents gain more tools. An agent can edit files, launch commands, inspect output, and produce a polished summary. More activity creates more observations. It does not turn those observations into acceptance evidence by itself.

The workflow needs two axes:

execution_status: Did the attempt finish, fail, or remain blocked?
claim-level verification: Which requested behaviors were verified, contradicted, or left unverified?

An attempt can be finished while one of its claims remains unverified. That is not a system error. It is an honest description of the available evidence.

Track claims, not confidence

A single task badge hides too much. Break the request into claims that can be checked independently.

For the hypothetical checkout patch, the claims might be:

the price calculation returns the expected total for a named case
the payment button becomes enabled after valid input
the browser completes the requested checkout path
the patch does not change an unrelated account flow

These claims may not all need the same evidence. A focused unit test could cover the calculation. A browser check could cover the interaction. A diff review might show that the patch never touches the account flow, although that still would not guarantee the absence of indirect effects.

Each claim gets one of three outcomes:

verified: a named check produced evidence for the claim
contradicted: a named check produced evidence against the claim
unverified: the workflow does not have enough relevant evidence either way

Unverified is not a softer word for failure. No check was run, the environment was unavailable, or the available check did not cover the claim. Contradicted means relevant evidence exists and points the other way. Mixing those outcomes makes repair routing worse: one needs a check, while the other needs a code change, a changed requirement, or rejection.

Claim-level outcomes prevent the opposite mistake too. One failed check should not stain every claim in the task. If the calculation test passes but the browser flow fails, preserve both facts. The system now knows what not to redo and where the repair belongs.

Use a small verification record

The record does not need to become a new observability platform. It needs enough structure to stop prose from laundering uncertainty.

Here is an illustrative format, not an existing protocol or a claim about any project's API:

execution_status: finished
candidate_ref: "<exact patch or revision under review>"
claims:
  - id: checkout-total
    expected: "<named calculation behavior>"
    outcome: verified
    evidence: "<focused test artifact>"
  - id: browser-checkout
    expected: "<requested browser behavior>"
    outcome: unverified
    evidence: null
    gap: "<browser environment was not available>"
next_action: "<run the named browser scenario against this candidate>"

The angle brackets matter. They keep the example from looking like a real execution log. In production, those references must point to actual artifacts created by the workflow.

A useful record answers four practical questions:

What exact candidate was evaluated?
What did each check claim to establish?
Where is the evidence?
What bounded action would resolve the remaining gap?

It should not say all tests passed when only one test command ran. It should name the check and the requirement it covers. It should not use a file hash as proof of behavior. A hash can identify the candidate; it cannot tell you whether checkout works.

Evidence expires when the relevant candidate changes

Verification belongs to a candidate, not to a conversation.

Suppose the browser flow passes, then the agent edits the checkout component again. The old browser evidence may no longer apply. Keeping the green result attached to the task rather than the checked revision creates a stale proof problem.

That does not mean every edit invalidates every check. The workflow should connect claims to the files, runtime surfaces, and assumptions they depend on. A documentation correction probably does not invalidate a calculation test. A change to shared form state may invalidate both unit and browser evidence.

The useful rule is narrower: when the candidate changes, reassess the evidence for affected claims. Preserve unaffected evidence and rerun or downgrade the rest.

Aiden's published design separates attempts, effects, evidence, verification, and verdicts. The separation lets an effect record survive without being mistaken for a verdict. A parent process can inspect what a worker changed, then decide whether the supplied evidence covers the assigned claim.

Parent agents must verify the handoff

Multi-agent systems make false completion easier.

A helper returns a confident paragraph: "Implemented the fix and verified the result." The parent sees clean prose, not the boundary of the check. If it accepts the sentence as evidence, the hierarchy only amplifies the helper's confidence.

The parent should evaluate the handoff against the assignment:

Is there a candidate artifact to inspect?
Does the evidence belong to that candidate?
Does the check address the assigned claim?
Are unresolved gaps still visible?

This is not a demand for the parent to rerun every command. It is a demand that the parent judge evidence, not tone. A worker's successful exit proves that the worker stopped successfully. Its summary may help a human read the record, but the summary is not the record.

The same rule applies to human review. Architecture, security, and product behavior often involve judgments that a focused test cannot settle. The system should route those claims to review instead of silently treating machine-checkable evidence as universal approval.

Route the gap instead of hiding it

An unverified state can look like unfinished work. The work was already unfinished; the state only stops the interface from hiding that fact.

A good unresolved handoff should include:

the last candidate that was checked
the claims that were verified
the exact claim that remains open
the evidence collected so far
the reason verification stopped
one safe next action

The reason matters. Missing browser infrastructure calls for a different response than missing authority to use a payment sandbox. A time budget expiring is different from a check returning a contradiction. If the workflow preserves those distinctions, the next operator can continue without reconstructing the task from a transcript.

An unresolved handoff also needs a stop condition. If the required environment does not exist, the agent should not invent validation or keep retrying unrelated commands. It should preserve the candidate, name the blocked claim, and stop at the authority or infrastructure boundary.

Preserving the gap gives the next step a typed input instead of a vague success message.

Bring the missing check back into view

Return to the checkout patch. The unit test passed, so the calculation claim can be verified for the checked candidate. The browser flow was never run, so that claim is unverified. The task is not globally green, and it is not globally failed.

The next action is small: run the named browser scenario against the same candidate. If it passes, attach the evidence and update that claim. If it fails, mark the claim contradicted and route the patch back for repair. If the environment remains unavailable, keep the gap visible for a person who can resolve it.

A coding agent should be allowed to finish its run without pretending it proved the result. An honest unverified state gives the next developer something better than confidence: it gives them the exact boundary of what is known.

Source notes

The Fastest Coding-Agent Step Is the One You Remove From the Loop

hefty — Mon, 14 Sep 2026 10:40:09 +0000

Put a fast model inside a bad workflow and watch what happens.

It scans the repository, receives a wall of search output, asks permission for a harmless command, loses the important constraint in the noise, rereads the same files, and finally runs the test that could have run automatically ten minutes earlier. The model may be quick. The task is still slow.

Coding-agent performance is an end-to-end property. Inference speed matters, but only for the slice of the run spent waiting on inference. The rest is context collection, tool execution, approvals, validation, retries, and recovery.

Before paying for faster reasoning, remove the turns that never needed reasoning.

Count handoffs, not chat messages

A round trip is any handoff that must finish before useful work can continue. It might be another model turn, a tool invocation, a human decision, or a recovery cycle after the run loses state.

A normal coding-agent loop looks roughly like this:

context -> reason -> act -> observe -> approve -> validate -> recover or finish

None of those phases is automatically waste. The waste appears when boundaries are vague.

A tool dumps an entire log when the next step needs one error. An approval covers one command instead of a defined risk scope. A failed check returns prose instead of a machine-readable result. A resumed session rebuilds context from the transcript because nobody saved a durable checkpoint.

Map those handoffs before changing models. Here is an illustrative version, not a proposed standard:

context_acquisition:
  actor: retrieval tool
  input_boundary: task contract and allowed repository scope
  output_artifact: context manifest
  repeat_trigger: a named evidence gap

model_reasoning:
  actor: coding agent
  input_boundary: context manifest and current goal
  output_artifact: decision or bounded plan
  stop_condition: next action and success condition are explicit

tool_execution:
  actor: deterministic tool
  input_boundary: fixed command, files, and limits
  output_artifact: bounded result plus full-log reference
  retry_rule: retry only a classified transient failure

approval:
  actor: human or policy service
  scope: named effects and resources
  output_artifact: decision record
  invalidation: scope, target, or reviewed artifact changes

validation:
  actor: test, linter, type checker, or other named check
  input_boundary: candidate artifact
  output_artifact: structured pass or fail evidence
  failure_route: repair, escalate, or stop

recovery:
  actor: runtime
  input_boundary: durable checkpoint and classified failure
  output_artifact: restored state and allowed next action
  stop_condition: state cannot be restored safely

The indentation mistake in many real systems is conceptual: everything sits under model_reasoning. Retrieval, command selection, policy, validation, and recovery all become prompts. The agent keeps talking because the workflow gave it no other way to move.

Remove the model from deterministic leaf work

Models are useful when the next action depends on incomplete evidence, tradeoffs, or interpretation. They are an expensive control structure for a transform with fixed inputs and a checkable output.

Formatting is the obvious example. File conversion, schema validation, asset resizing, linting, and many code-generation substeps can also be deterministic. Give them typed inputs, run the tool directly, and return a small result.

If a generated release image needs platform-ready dimensions, a browser-local utility such as Resize Image For can handle that fixed resize-and-export step without uploading source pixels for processing. Asking an agent to rediscover dimensions, choose an image library, write a temporary script, and inspect the output adds uncertainty to a settled operation.

The same principle applies inside a repository. If the project already has a formatter command, run it. If a JSON schema decides whether a file is valid, call the validator. If a codemod expresses a known migration, let the agent choose whether the codemod fits, then let the codemod perform the repetitive edits.

Direct tools do not make the workflow safe by themselves. A deterministic command can still target the wrong files or receive bad input. The benefit is a narrower contract: the agent decides what should happen, while the tool performs a repeatable operation whose output is easier to inspect.

Bound context without starving the task

"Use less context" is bad advice when applied blindly. Missing a package convention or an acceptance requirement can create more retries than a smaller prompt saves.

The useful goal is bounded context. Start each run with a stable task contract:

the repository state and task goal
allowed files, commands, and effects
relevant project instructions
the expected artifact
named acceptance checks
stop and escalation conditions

Retrieve repository material against that contract. Return the matching symbols, nearby tests, and local instructions instead of dumping a broad search transcript. When a tool produces a large log, keep the full output as an artifact and send the agent the failing check, exit status, and a reference to the detail.

This also changes recovery. A durable task contract, context manifest, current diff, and latest check result are much cheaper to resume from than a long conversation full of superseded plans.

Alera's project description makes these surrounding surfaces visible: real PTYs for CLI agents, isolated Git worktrees, lifecycle activity, resource tracking, and persisted sessions. That does not establish a speed advantage. It does show that process state, workspace state, and session state belong in the performance model.

A fast response cannot compensate for rebuilding all three after every interruption.

Put approvals at risk boundaries

Approval prompts are often treated as a choice between safety and speed. That framing produces two bad designs: interrupt the operator for every small action, or grant broad authority so the chart looks faster.

An approval should describe a meaningful boundary. Scope it to the effects that carry risk: changing deployment files, using a credential, writing outside the worktree, opening a network connection, merging a pull request, or publishing an artifact. Let routine operations inside an already approved scope proceed without renegotiation.

The approval also needs an invalidation rule. Permission to deploy one reviewed artifact should not silently transfer to a different artifact after the agent edits it. A decision tied to a target, scope, and artifact state is useful evidence. A floating "yes" in a transcript is not.

Proliferate's README presents reusable workflows, isolated task worktrees, parallel agents, and human approval gates as explicit orchestration primitives. Again, that is a project-authored description rather than measured proof. The useful design signal is the separation: workflow steps and approval boundaries exist outside the model's conversational improvisation.

Some reviews must remain serial. A consequential change may deserve a person reading the final diff even if that adds latency. Optimize the placement and scope of that decision; do not delete it to win a benchmark.

Validation should return evidence, not reassurance

An agent saying "the tests should pass" is not a validation phase. Neither is a final message that says the task is complete.

Where acceptance can be mechanical, let the named checks decide it. The runtime can run tests, lint, type checks, schema checks, build commands, or task-specific assertions without asking the model to narrate each command. Return structured results to the loop:

check: pnpm-typecheck
status: failed
exit_code: 2
summary: 2 errors in src/auth/session.ts
artifact: logs/pnpm-typecheck-017.txt
next_route: bounded-repair

The model still has a job when a check fails: diagnose the evidence and propose a repair. It should not need to persuade the system that a failed check is close enough.

This boundary matters after tool upgrades too. A current coding-tool roundup reports changes involving permissions, secret redaction, MCP configuration and trust, and experimental worktree behavior. Those changes sit around inference. An upgraded model can answer faster while the run loses time to a stale integration, a changed entry point, or a permission failure.

Re-run the workflow checks after an upgrade. Old latency assumptions do not survive a changed execution path merely because the model name stayed familiar.

Parallel agents change the shape of waiting

Parallelism can reduce elapsed time when tasks are genuinely independent. It can also create more handoffs: worktree setup, task partitioning, shared-state coordination, competing patches, and a larger review queue.

Count accepted outcomes, not active terminals. Two agents investigating the same failure may produce twice the activity and no additional progress. Four patches waiting on one reviewer are still waiting.

This is why resource and lifecycle visibility matter. For each worker, preserve the task boundary, current state, produced artifact, and owner of the next decision. Concurrency without those records turns recovery into archaeology.

Parallel work earns its complexity when overlap removes waiting that cannot be removed another way. It is not a substitute for making one loop legible.

Benchmark the loop, not the launch demo

The launch discussion for Bullet is useful because its maker attributes speed to several workflow choices: model routing, targeted search, bounded tool output, context hygiene, and fewer round trips. Commenters questioned the comparison methodology. Both sides point to the same requirement: "fast" needs a task envelope.

Compare representative runs from the same starting state and task class. Record:

latency by phase
round-trip count by type
retries and their causes
the artifact produced
the named checks that passed or failed
whether the outcome was accepted

Do not blend a typo fix with a cross-package migration and call the average meaningful. Do not count a run as fast because it stopped before validation. Do not compare one agent with precise repository instructions against another starting from a vague sentence.

Fewer round trips are a hypothesis, not a guarantee. One extra retrieval can prevent several repair cycles. One deliberate approval can prevent a costly effect. A model turn that catches a bad assumption may be the quickest part of the run.

The target is avoidable latency. Preserve the handoffs that carry judgment, authority, or proof.

That distinction also protects engineering understanding. Removing repetitive work is useful. Removing the points where someone has to understand the architecture, assess a tradeoff, or review a risky diff is how output outruns comprehension.

Use a round-trip budget before a model leaderboard

For one week, instrument the workflow you already have. You do not need a grand observability platform. A task record with timestamps, handoff types, retries, artifacts, and check results is enough to expose obvious waste.

Look for repeated patterns:

repository searches that fetch the same material
raw tool output repeatedly injected into context
low-risk actions approved one at a time
deterministic transforms performed through model turns
validation delayed until a large patch accumulates
sessions that cannot resume from a durable artifact

Fix one boundary at a time. Replace repeated discovery with a context manifest. Add a summary contract to a noisy tool. Move a fixed transform behind a direct command. Tie approval to a real risk scope. Run a focused check as soon as the relevant artifact exists.

Then compare models under the improved workflow. The faster model may still win, and now the comparison measures more than its ability to survive your orchestration overhead.

Use this order: remove unnecessary turns, bound the remaining handoffs, then compare models.

Before paying for a faster model, remove the turns that never needed a model.

Source notes

Your Coding Agent UI Needs an Event Model, Not a Chat Transcript

hefty — Sat, 12 Sep 2026 08:25:09 +0000

Imagine a coding agent has been running for 25 minutes. Its transcript is busy. It searched the repository, announced a plan, edited three files, ran a command, apologized for an error, and said it was making one final adjustment.

The cursor is still moving. What you cannot see is more important:

Which phase is active?
Is the agent blocked or still making progress?
What was the last durable artifact?
Did the required checks pass?
What action is safe now?

A chat transcript can contain every sentence the agent produced and still fail to answer all five.

Chat is useful for instructions and explanations. It is a bad state model. Once an agent runs tools, waits for approval, creates artifacts, and validates work over several minutes, its operator UI needs a structured event stream underneath the conversation.

Chat is an explanation layer

Chat became the default agent interface for an obvious reason: models produce text, and people know how to type into a box. That works well when the exchange is short and the output is the product.

The shape changes when the output is a patch, pull request, deployment plan, or test result. Now the text describes work happening elsewhere. A sentence such as "I'm checking the failing tests" may describe intent, an active process, a retry, or stale narration left behind after a crash.

The frontend should not have to parse that sentence to decide whether to show a spinner, an approval button, or a recovery action. Prose changes between models and versions. It can arrive late, contradict an earlier message, or sound finished before the acceptance checks run.

Keep the conversation. Let it carry goals, questions, tradeoffs, and rationale. Put operational truth on a separate channel with stable semantics.

Current tools are already exposing pieces of this distinction. Microsoft's Foundry Toolkit release notes describe richer model profiling and clearer Agent Inspector handling for events and latency. A recent DEV.to comparison evaluates coding agents through practical concerns such as repository permissions, tests, and knowing when to stop. These are project and author descriptions, not a shared standard, but they point at the same product problem: useful agent interfaces must represent the workflow around generation.

Give the run a small event contract

You do not need to invent a universal agent protocol. Start with the information your interface must render without guessing.

Here is an illustrative TypeScript model:

type Phase = "planning" | "editing" | "validation";

type EventBase = {
  runId: string;
  eventId: string;
  sequence: number;
  timestamp: string;
  phase?: Phase;
  correlationId?: string;
  summary: string;
};

type AgentEvent = EventBase & (
  | { type: "run_queued" }
  | { type: "phase_started"; phase: Phase }
  | { type: "tool_started"; operation: string }
  | {
      type: "tool_finished";
      operation: string;
      outcome: "succeeded" | "failed";
      durationMs?: number;
    }
  | { type: "artifact_created"; artifactRef: string }
  | {
      type: "approval_required";
      requiresAction: true;
      approvalId: string;
      scope: string[];
    }
  | {
      type: "approval_resolved";
      approvalId: string;
      decision: "approved" | "denied";
    }
  | {
      type: "check_finished";
      checkRef: string;
      outcome: "passed" | "failed";
      durationMs?: number;
    }
  | {
      type: "run_failed";
      errorCode: string;
      recoverable: boolean;
    }
  | { type: "run_completed"; checkRefs: string[] }
);

Treat the names as placeholders; the boundaries are what matter.

runId, eventId, and sequence give the frontend identity and ordering. A correlation ID can join a tool start to its finish or connect a retry to the operation it replaced. The summary stays short enough for a timeline. Large outputs live behind artifactRef or checkRef instead of flooding the event payload.

Durations belong only on operations the runtime can measure. If the system cannot separate model time from network delay, it should not manufacture that distinction for a polished chart.

An append-only stream is often a good fit because it preserves transitions. The UI can rebuild a projection after reconnecting instead of trusting one giant mutable session object whose history has vanished. That is an implementation option, not a requirement. The contract matters more than whether you call the storage pattern event sourcing.

Model the states people care about

"Streaming," "running," and "done" are too coarse for an agent that can pause, ask permission, validate a patch, and fail in ways that may or may not be recoverable.

A practical UI might project events into these states:

queued
running
waiting_for_approval
validating
failed_recoverable
failed_terminal
completed

This is a proposed state set, not an industry standard. Your workflow may need fewer states or more precise ones. What matters is that each displayed state follows from an event rather than a mood inferred from the latest message.

waiting_for_approval deserves special treatment. The run is not progressing, but it has not failed. The interface should preserve what action was requested, the scope of that request, and the decision that resumed or stopped the run. A modal that disappears after one click leaves the timeline unable to explain why the agent gained permission to continue.

Failure needs similar honesty. A failed test may permit a bounded retry. A revoked credential may require operator action. A corrupted workspace may end the run. Showing the same red "failed" badge for all three forces the user back into logs to discover the available next step.

Completion should be strict. If the task contract names tests, review, or another acceptance check, run_completed should reference that evidence. The agent's final "done" message is commentary. It is not an acceptance event.

Activity is not progress

A transcript looks productive whenever text keeps arriving. An event timeline can make the same mistake if it rewards volume.

Ten tool calls might represent ten useful steps. They might also be the same failed search repeated with slightly different wording. Counting events does not tell you which one happened.

Progress indicators should follow durable milestones: a phase transition, a new diff, an approval decision, or a named check result. If the workflow has no defensible percentage, skip the percentage. "Validation: 2 of 4 checks finished" tells the truth. "87% complete" usually does not.

The same rule applies to status copy. Prefer "waiting for approval to modify deployment files" over "the agent is thinking." One describes an observable condition and gives the operator a decision. The other decorates uncertainty.

Latency needs semantics

One timer cannot explain a long-running agent.

A run can spend time in a queue, waiting on a model, executing a tool, waiting for a person, or validating an artifact. Those delays call for different responses. Faster model inference will not fix a five-hour approval wait. A quicker shell command will not help a run that keeps choosing the wrong tool.

Recent launch discussions make this measurement gap visible. Makers describe fewer round trips, bounded tool output, and lower cost, while community replies ask for stronger evidence. Rather than adding another "fast" badge, give operators a latency view that connects measurable phases to outcomes.

For each correlated operation, record the start and finish when the runtime owns both boundaries. Then show where elapsed time accumulated and whether the operation produced an artifact, a check result, a retry, or a failure. Keep unobserved time labeled as unobserved rather than assigning it to the model by subtraction.

That makes performance work concrete. It also prevents approval delay and validation time from being misreported as generation latency.

Project one stream into several views

The event stream is infrastructure, not the interface itself. Different users need different projections of the same run.

A compact status panel can show the current phase, blocker, last durable artifact, acceptance state, and allowed next action. A timeline can preserve transitions for diagnosis. An approval inbox can collect unresolved requests across runs. Artifact and check views can connect a run to its diff, worktree, pull request, logs, and validation results. A latency view can group measured duration by phase and operation.

This shape matches what is appearing around current multi-agent products. Project discussions connect agent runs with documents and human approval gates. Product descriptions connect them with worktrees, pull requests, artifacts, remote access, and recovery. Those descriptions do not prove reliability or productivity. They do show why a transcript-only UI runs out of room: the work has relationships that paragraphs cannot represent cleanly.

Keep the event payload bounded. A test log belongs in controlled artifact storage, with the event pointing to it. The same goes for a diff or pull request. References let each view fetch the evidence it needs without turning the timeline into a second database of duplicated blobs.

Recovery starts with an honest last state

An event model alone cannot make recovery safe, but it gives the interface enough information to stop pretending.

After a disconnect or crash, the UI should be able to show the last accepted sequence, the last durable artifact, whether the failure was marked recoverable, and which action the runtime currently permits. It should also expose uncertainty. A stale connection should look stale, not "running."

The projection code must expect duplicates, delayed delivery, and reconnects. Event IDs support idempotent handling. Per-run sequence numbers expose gaps and out-of-order delivery. If event 42 arrives before event 41, the UI can wait, refresh, or mark the view incomplete instead of quietly building a fictional timeline.

This does not replace checkpointing or idempotent tools. It keeps the frontend focused on the information needed to present recovery without guessing at backend state.

Do not turn telemetry into a data leak

Structured events are easier to query and display. They are also easier to retain forever by accident.

Tool arguments, file paths, prompts, and logs can expose source code or secrets. Capture the smallest summaries and references required to operate the run. Redact deliberately, restrict access, and set retention rules for event and artifact stores. A beautiful inspector is not worth copying sensitive payloads into five projections.

Small synchronous assistants may not need any of this. If a tool answers in seconds, produces no external artifact, and never waits for permission, a conversation can be enough. The event model earns its complexity when the workflow becomes long-running, tool-using, approval-bearing, or recoverable.

The transcript is not the control surface

A transcript tells the story of a run. An event model lets the product operate it.

With explicit events, the interface can answer the questions that matter: what phase is active, what is blocked, which artifact exists, what passed, and what action is available. Chat can then do what it is good at: helping a person steer the work and understand why the agent made a choice.

If your frontend has to parse reassuring prose to decide whether the work is complete, it does not have a run model yet.

Source notes

Token Price Is a Terrible Metric for Coding-Agent Cost

hefty — Thu, 10 Sep 2026 22:50:10 +0000

Run A uses the cheaper model. It edits the right files, fails a test, retries with a wider change, burns through more tool calls, and stops with a confident completion message. An engineer spends the next hour figuring out which half of the patch can be kept.

Run B uses a more expensive model. It works from the same repository state, stays inside scope, runs the named checks, and returns a patch the reviewer accepts with one small correction.

The model line item can make Run A look cheaper. The engineering outcome says otherwise.

Token price is an input price. It does not tell you what a verified patch cost. If coding agents are doing repository-level work, the useful internal metric is cost per accepted task, with every retry, failed check, recovery step, execution environment, and human intervention left in the ledger.

Your invoice measures consumption, not completion

Tokens matter. Tool calls matter. Model time and sandbox time matter. You should track all of them.

They still stop short of the result.

A provider invoice cannot tell you whether a migration was safe, a visual regression was caught, or the reviewer rejected the patch. It usually cannot associate an abandoned worktree with the accepted patch that replaced it. If the agent switched models halfway through, the bill may split one engineering task across several unrelated line items.

Current model releases are starting to frame efficiency around the whole agent loop. Meta's Muse Spark 1.3 announcement, for example, claims fewer tokens and tool calls alongside better long-horizon coding behavior. Those are vendor claims, not independently reproduced results here. More importantly, fewer calls are only useful if the run still reaches the acceptance boundary.

A loop that stops early because it gave up is not efficient. Neither is a loop that says "done" before the test suite disagrees.

The accounting needs three layers:

consumption: model usage, tool calls, execution time, and isolated compute
failure load: retries, failed checks, abandoned branches, and recovery work
outcome: accepted, rejected, or incomplete under named checks

The third layer keeps the first two honest.

Define the task before measuring the agent

Cost per accepted task becomes nonsense if every run gets a different task.

A typo fix with a deterministic test is not comparable to a dependency migration across six packages. A generated frontend with visual and interaction requirements is not comparable to renaming a private function. Averaging them into one number creates a tidy dashboard and a bad purchasing decision.

Define a task envelope before the agent starts:

repository revision and working state
task class and goal
allowed file and command scope
authority and time limits
expected artifact
acceptance checks
review decision required

Then compare runs inside the same task class. Use the same starting revision when practical. Keep the validation contract stable. If one model gets a precise brief and another gets "fix the app," you are measuring prompt quality as much as model performance.

The acceptance boundary also has to be explicit. A zero exit code may be one check, but it is not a general definition of success. The boundary could include unit tests, type checks, a migration dry run, browser screenshots, an accessibility scan, or human review of a risky diff. Different tasks need different evidence.

This metric is local by design. Your repository, harness, and review policy shape the result. Cost per accepted task is useful for comparing choices inside one engineering system, not for declaring a universal winner.

Put the receipt in the harness

Most teams already have enough raw data to build a useful receipt. The problem is that it lives in separate places: provider usage, terminal logs, worktrees, CI, review comments, and somebody's memory of the failed first attempt.

The harness is where those records can meet.

Alera's README describes separate terminals, Git worktrees, lifecycle hooks, quota tracking, and resource attribution in one agentic development environment. Mivia documents isolated worktrees, durable step records, lifecycle hooks, provider choice, and local workflow files. These are project-authored descriptions, not proof that either tool measures every cost correctly. They do show where task-level accounting can attach to real execution.

A compact receipt could look like this:

task:
  id: auth-callback-fix-017
  repository_revision: 84c19e2
  class: bounded-bug-fix
  goal: fix duplicate callback processing

agent:
  provider: example-provider
  model: example-model

limits:
  scope:
    - src/auth/callback.ts
    - tests/auth/callback.test.ts
  time_minutes: 45
  allowed_effects:
    - edit_worktree
    - run_local_tests
  forbidden_effects:
    - deploy

acceptance:
  checks:
    - pnpm test --filter auth-callback
    - pnpm typecheck
  reviewer_decision_required: true

consumption:
  input_tokens: recorded_by_provider
  output_tokens: recorded_by_provider
  tool_calls: recorded_by_harness
  execution_duration: recorded_by_harness
  isolated_compute_duration: recorded_by_runtime

failure_load:
  retries: 1
  failed_checks:
    - first auth-callback test run
  abandoned_worktrees: 0
  recovery_steps:
    - reverted out-of-scope config edit

human_load:
  approvals: 0
  intervention: reviewer requested one scope correction

result:
  artifact: worktrees/auth-callback-fix-017.diff
  status: accepted

The example does not force every field into dollars. Human review may stay qualitative unless the team records time consistently. The point is traceability: one task, its spend, its failures, its evidence, and its final disposition.

Store rejected and incomplete receipts too. If the ledger keeps only successful runs, the apparent cost will improve every time the agent fails.

Failed loops are part of the price

Coding-agent cost discussions often drift toward routing: use a cheaper model for easy work, fall back to a stronger model when needed. That can work. It can also hide a pile of waste behind the word "fallback."

Suppose the first model makes three attempts, expands the diff, and leaves the worktree in an uncertain state. The second model then spends time reconstructing the task and cleaning up before producing the accepted patch. The accepted result belongs to the full chain, not just the final model call.

The same rule applies to parallel agents. More workers can finish independent tasks faster, but they can also duplicate investigation, collide on shared state, or produce several patches that compete for one review slot. Per-session resource attribution matters because concurrency can raise throughput and waste in the same afternoon.

This is where durable steps and lifecycle hooks earn their keep. A step boundary gives the harness somewhere to record spend and check outcomes. A validation hook can stop a broken loop before it starts another expensive revision. An abandoned worktree remains associated with the task instead of disappearing from the success story.

Human intervention belongs in the receipt as well, but do not fake precision. "Reviewer resolved an out-of-scope config edit" is better data than a guessed dollar amount. Over time, repeated interventions reveal where the task contract or harness needs work.

Segment by task class before comparing models

A useful evaluation set should look like the work your team wants agents to do.

For small deterministic edits, acceptance may be cheap: one focused test plus diff review. A migration needs stronger state checks and rollback evidence. Debugging needs a reproduced failure before the patch and a clear signal afterward. Generated frontend work needs browser behavior and visual evidence, not merely code that compiles.

Keep those classes separate. Within each class, select representative tasks, fix the envelope, and compare accepted outcomes. For a narrow frontend study, a collection of Generative UI research papers can help identify domain-specific systems and evaluation ideas worth turning into a task set. The directory is a research surface, not a coding-agent benchmark and not evidence that any model will save money.

Track the distribution, not only the average. One model may be cheap and predictable on bounded edits while occasionally exploding on migrations. Another may cost more per call but fail less often on long tasks. A single blended number hides the difference that should drive routing.

Vendor efficiency claims can feed this process as hypotheses. If a release claims fewer tokens or tool calls, run representative local tasks under the same envelope. Check whether accepted-task cost falls, whether the failure rate changes, and whether review gets easier or harder. The local receipt decides whether the claim matters to your workflow.

Optimize the harness before the token

The cheapest improvement may have nothing to do with model price.

Add stop rules so an agent cannot repeat the same failed approach indefinitely. Bound concurrency so every session has an owner and a budget. Persist step records so a fallback does not need to reconstruct the run from chat history. Trigger validation at meaningful boundaries, not only after a large patch has accumulated.

Execution belongs in the accounting too. Cloudflare's documented Cursor integration separates the agent loop from customer-controlled isolated execution. That architecture does not prove lower cost or better security. It does make the operating boundary visible: commands, repositories, tools, and compute can live outside the model provider, and their cost does not vanish because the token invoice ignores them.

Once the harness can associate consumption, failures, checks, and acceptance with one task, model pricing becomes useful again. You can see where a cheaper model is genuinely economical, where a stronger model reduces recovery, and where the task contract is causing both to waste time.

Until then, a low token bill can be an expensive way to produce work nobody should merge.

Source notes

A Bigger Context Window Won't Make Your Coding Agent Resumable

hefty — Fri, 04 Sep 2026 12:24:11 +0000

Your coding agent has been running for eight hours. It edited twelve files, started a dev server, attempted a migration, hit two failing tests, and then lost its session.

The transcript is still there. It might even fit inside the model's context window. But a fresh process still cannot answer the questions that matter:

Which changes were successfully applied?
Is the migration safe to run again?
Is that dev server still alive?
Which failing test blocked the last step?
Should the next process resume, roll back, or stop for review?

The transcript survived. The job did not.

Long-running agent work needs a resume protocol. More context can help a model continue a conversation, but it cannot turn chat history into durable job state.

Context is not a checkpoint

Conversation context preserves messages, tool calls, and explanations. That is useful. It can remind a model what the user asked and how the session reasoned about the task.

A checkpoint has a different job. It records what is true about the work now.

That distinction gets expensive when a run is interrupted. A transcript might say, "I'll update the schema and rerun the tests." It does not necessarily prove that the schema changed, identify which command ran, or preserve its exit code. If the session disappeared between the tool call and the final response, the prose may describe an intention rather than the repository's actual state.

A replacement process needs a compact operational record:

the run's contract
the last completed step
changes and external side effects already made
exact validation results
the next safe decision

Without that record, recovery starts with archaeology.

Give the run a bounded contract

"Keep working until it is done" sounds productive, but it is a terrible recovery contract. Done is undefined, authority is open-ended, and a replacement process cannot tell whether the run stopped normally or simply ran out of road.

A useful contract names the boundary before execution begins:

a run ID and owner
repository and worktree
the goal and stop condition
time, tool, and authority limits
the expected artifact
the checks that decide whether the artifact is acceptable

Nightshift's author describes this shift explicitly: long agent runs become disk-backed, time-bounded jobs with contracts, recovery paths, receipts, and archives. That is a project-authored design, not independent evidence that every run will recover correctly. Still, the operating model is the interesting part. A shift has an end, and its state exists somewhere other than the active chat.

The stop condition deserves special attention. "Implement authentication" leaves too much room for interpretation. "Prepare a reviewable patch for the login callback, stop before deployment, and record the named test results" gives the next operator something they can inspect.

Persist facts at step boundaries

Do not wait until the final response to save state. By then, the process may already be gone.

Checkpoint after meaningful steps: dependency installation, migration generation, code modification, test execution, browser validation, or any external mutation. The record should stay small. Its purpose is fast re-entry, not a polished diary of everything the agent thought about.

Mivia's documented workflow offers a useful implementation signal here. Its README describes isolated workflows, per-step run records, lifecycle hooks, and local files. Those features create places where a harness can persist facts before moving forward.

The word "facts" matters. Save the command and its real result, not "tests look good." Save the files changed, not "updated the frontend." If a check failed, keep the failure visible instead of smoothing it into a progress summary.

A good checkpoint might say:

last_completed_step: generated migration 0042_add_job_receipts
changed_files:
  - db/migrations/0042_add_job_receipts.sql
  - src/jobs/receipt.ts
checks:
  - command: pnpm test --filter job-receipts
    exit_code: 1
    outcome: 2 tests failed in receipt-retry.test.ts
pending_action: inspect whether retries duplicate an existing receipt

That is far more useful than another page of confident narration.

Track side effects separately

File changes are only part of an agent run. The process may have started services, modified a database, opened a pull request, published an artifact, sent a message, or changed remote configuration.

Those effects need their own ledger because replaying the original prompt can repeat them.

A blind retry might apply the same migration twice. It might create a second deployment while the first one is still running. It might overwrite uncertainty with a new sequence of actions that happens to produce cleaner logs.

A side-effect ledger should answer three questions:

What changed outside the transcript?
Can the operation be checked safely?
Is it repeatable, reversible, or neither?

This does not make every action idempotent. It makes the uncertainty visible before another process acts on it.

Recovery must live outside the failed session

A stalled process cannot be the sole owner of its recovery plan. The plan needs to be readable by a fresh process or a human operator with no access to the original model state.

Recovery should begin with inspection, followed by an explicit decision:

Resume when the recorded state matches the environment and the pending action is still valid.
Roll back when a known reversible change should be removed before work continues.
Escalate when the evidence is incomplete, the environment drifted, or another action could compound an unknown side effect.

This is why recovery is not the same as retry. Retry repeats an operation. Recovery first establishes what already happened.

PI-Desktop's project documentation points at the operator-facing side of this problem through project sessions, message-scoped diffs, approvals, and guarded rollback. Those controls do not guarantee safe recovery, and local-first software is not automatically isolated or secure. They do make the unit of change more inspectable than one opaque stream of agent activity.

A resume protocol can be simple. Here is an operating pattern, not a standard or a security guarantee:

run:
  id: auth-callback-2026-09-04-01
  owner: overnight-agent
  repository: web-app
  worktree: ../worktrees/auth-callback

contract:
  goal: prepare a reviewable login-callback patch
  stop_when: named tests pass or a migration needs approval
  time_budget_minutes: 90
  allowed_effects:
    - edit_worktree
    - run_local_tests
  forbidden_effects:
    - deploy
    - modify_production_data
  expected_artifact: git_diff

checkpoint:
  last_completed_step: add receipt persistence
  pending_action: inspect two retry test failures
  changed_files:
    - src/jobs/receipt.ts
    - db/migrations/0042_add_job_receipts.sql

side_effects:
  services_started:
    - name: web-dev-server
      port: 3102
  external_mutations: []

evidence:
  checks:
    - command: pnpm test --filter job-receipts
      exit_code: 1
      outcome: 2 retry tests failed

recovery:
  inspect_first:
    - git diff --stat
    - pnpm test --filter job-receipts
  resume_when: migration is unchanged and failures reproduce
  rollback: discard the isolated worktree
  escalate_when: database state differs from the checkpoint

receipt:
  status: blocked
  reviewer_action: decide whether receipt creation must be idempotent

The schema is deliberately boring. Boring is good when a process wakes up at 3 a.m. and needs to decide whether a migration already ran.

Finish with a receipt, not "done"

A completion message is not evidence. The final receipt should contain the artifacts produced, the exact checks and outcomes, known failures, assumptions, and the next reviewer action.

This also gives review tools a clean handoff. R3's launch describes a local interface that turns scattered comments into explicit feedback for a coding agent. It is a small, low-engagement project signal rather than adoption evidence, but the shape of the loop is useful: review state becomes an artifact that can follow the job into another session.

A receipt does not prove correctness. It tells the next reviewer what was checked and what was not. That is enough to replace a vague claim of completion with a reviewable boundary.

Let old runs improve new contracts

An archive earns its keep when it changes the next run.

The ZeroShot launch describes local capture of agent sessions, clustering repeated friction, and drafting reusable workflow improvements. Those are author claims, not measured productivity results. The useful idea is narrower: repeated interruptions can reveal missing fields in the contract.

If three runs stall because nobody recorded which browser profile owns port 3102, add that ownership to the template. If retries keep duplicating external actions, require idempotency keys or an escalation rule. If reviewers repeatedly ask which checks actually ran, make command outcomes mandatory in the receipt.

The loop is practical: archive the run, find recurring ambiguity, then remove that ambiguity from the next contract.

Apply the replacement-process test

Before letting an agent run unattended for hours, imagine the current session vanishes halfway through.

Can a different process inspect durable artifacts and determine the contract, current state, side effects, evidence, and next safe action? Can it choose resume, rollback, or escalation without treating the old transcript as ground truth?

If the answer is no, a larger context window will only preserve more material to reread. Make the job resumable before making it longer.

Source notes

Four Coding Agents Need Four Workspaces, Not Four Chat Windows

hefty — Mon, 31 Aug 2026 06:35:44 +0000

Opening four coding-agent sessions feels like scaling. On a shared machine, it is closer to giving four fast contributors the same repository, shell, credentials, ports, caches, and merge queue without deciding who owns any of them.

The first failure probably will not come from model quality. One task will restart a dev server while another is testing it. Two workers will touch the same lockfile. A branch will pass its own checks and still conflict with a migration waiting in the merge queue.

Four chat windows create concurrency. Four owned workspaces plus one deliberate merge queue create a system.

Parallelism multiplies shared state

Tasks that sound independent in a prompt can overlap in the environment.

A frontend change and an API change may both edit generated types. Two test runs may expect the same database or browser profile. Separate worktrees can still launch services on the same port, read the same environment variables, and write to shared caches. The agents do not collide in the prompt. They collide in everything the prompt lets them touch.

This is why adding a second agent changes the job. With one worker, the operator can keep a surprising amount of state in their head. With four, every unstated assumption becomes a race condition or a review problem.

The fix is to make ownership visible before execution starts.

A worktree is the start, not the boundary

Git worktrees are a sensible first step. Each task gets its own branch and working files, so one agent is less likely to overwrite another agent's edits by accident.

That is useful isolation, but it is narrow isolation.

A worktree does not reserve a port. It does not separate process trees, temporary directories, credentials, network access, browser state, or external services. Treating it as a sandbox gives the workflow more confidence than the boundary deserves.

Proliferate is an instructive project example because its documented design pairs isolated task worktrees with visible review state. The important idea is the pairing. Filesystem separation tells you where a patch was produced; task state tells you what should happen to that patch next. The project description is not independent proof that every boundary is solved, and it does not need to be. The pattern is useful on its own.

For each concurrent task, define at least these ownership boundaries:

Workspace ownership

Assign one repository scope, branch, and worktree. State which generated files or shared configuration the task may change. If two tasks own the same file, they are coupled work and should be queued accordingly.

Process ownership

Reserve ports, dev servers, test databases, caches, temporary paths, and browser profiles. A process should have a named task owner and a cleanup rule instead of becoming shared background state.

Artifact ownership

Name the output before the run begins: a patch, a report, a screenshot set, a migration plan, or some other reviewable object. "Improve the app" is not an artifact contract.

Treat authority as a task budget

The easiest setup is to launch every worker from the operator's normal shell. It is also the setup with the least useful separation.

A task that edits a local component rarely needs every token, deployment command, network destination, and destructive tool available to the human running it. Grant the repository, commands, credentials, and network surface required for that task. Leave the rest out.

Current agent harnesses do not all enforce these boundaries in the same way. If a tool cannot restrict a credential or network path, write that limitation into the run record and reduce the task's authority. Do not silently treat the operator's full shell as a reasonable default.

This also makes failures easier to understand. When a task has a small authority budget, the failure surface is smaller: you can see which files, processes, and external systems were in play without reconstructing the whole machine after the fact.

Keep integration serial on purpose

Implementation can run in parallel. Integration should have one owner and an explicit order.

Two patches can both be locally correct and still disagree about an API shape, migration sequence, dependency version, generated file, or user-visible behavior. Git may merge the text cleanly while the combined system is wrong. That is an integration conflict even when there are no conflict markers.

Give one person, or one tightly bounded integration role, responsibility for:

choosing the review order
resolving overlapping decisions
rerunning checks against the combined state
deciding what lands and what returns for another pass

The merge queue is where parallel work becomes one product. Making it serial is coordination, not wasted parallelism.

Completion needs evidence, not a status message

An agent saying "done" proves that it generated a completion message. It does not prove the patch works.

Every task should return an evidence bundle with:

the changed files and the intent of the diff
the exact checks run and their real results
screenshots or browser evidence when behavior is visual
known failures, skipped checks, and unresolved assumptions
merge-order or conflict notes for the integration owner

Repository-native agent systems are starting to expose this control layer directly. GitHub Agentic Workflows defines bounded jobs and safe-output handling. Its August 24 update describes safe-output validation, run steering, inspectable model and catalog state, and clearer startup diagnostics. GitHub Copilot's August releases describe task management, queued prompts, plan/autopilot behavior, and rewind. These are product descriptions, not reliability benchmarks, but the direction is useful: long-running work needs intervention, recovery, and inspectable output.

The evidence rule also applies to non-code artifacts. If a frontend task produces launch or social graphics, a narrow browser-local step such as Resize Image For can handle resizing, fitting, cropping or padding, previews, and export while keeping source-image processing in the browser. That is easier to review than handing the source files to another remote worker with a broad instruction to "prepare the assets."

Write the contract before launching the second agent

A small task record is enough. The exact schema matters less than forcing the decisions into the open.

task: fix-mobile-navigation
owner: agent-2
stop_when: mobile navigation passes the named browser checks

workspace:
  repository_scope: apps/web
  worktree: ../worktrees/fix-mobile-navigation
  shared_files: [pnpm-lock.yaml]

process:
  ports: [3102]
  services: [web-dev-server]
  browser_profile: agent-2-mobile

authority:
  credentials: []
  network: [localhost]
  destructive_tools: false

expected_artifacts:
  - code-diff
  - test-results
  - mobile-screenshots
  - known-issues

integration:
  merge_owner: human-reviewer
  review_after: shared-design-token-change
  rollback: discard-worktree-and-branch

This is an operating record, not a universal security configuration. The point is to make shared state, authority, evidence, and merge order reviewable before they become an incident report.

If the task cannot fill in these fields, it is probably too vague or too coupled to run concurrently.

Know when serial work wins

Parallelize tasks with separate state and separate validation paths. A documentation change in one package and an isolated test addition in another may fit well. Shared-schema changes, coupled migrations, global dependency upgrades, and work that relies on one fragile validation environment usually deserve a queue.

There is no prize for keeping every agent busy. If coordination and verification cost more than the execution time you saved, reduce the agent count.

Before opening another session, provision its workspace, authority, integration path, and evidence contract. If that is too expensive for the task, keep the work serial.

Source notes

Agent Plugins 1.0 Solves Packaging, Not Trust

hefty — Sun, 30 Aug 2026 12:57:18 +0000

A portable agent plugin can be identical on disk and still become a different operational system in two clients.

One client may need its own manifest namespace or installation path. Another may expose a different capability set. Repository access, process execution, network policy, secret handling, failure behavior, and run evidence can all depend on the host.

If a team reviews only the portable files, it has reviewed the package. It has not reviewed every runtime that can give that package power.

Agent Plugins 1.0 addresses a useful problem: packaging Agent Skills and MCP servers in a vendor-neutral format. That should cut duplicated integration work. It does not make distribution, permissions, UX, or client-specific behavior identical, and the official overview leaves those concerns to each client.

The practical rule is to standardize the package and test the host separately.

Packaging is real progress

Agent integrations have accumulated plenty of one-off setup: tool definitions, skill directories, MCP configuration, marketplace metadata, and client-specific instructions. A portable core gives publishers and client authors a common shape to work with.

That common shape is an interoperability floor. It can tell a client how a component is packaged and reduce the need to rebuild the same integration from scratch. It also gives reviewers a more stable object to inspect, version, and compare.

The boundary matters, though. The Agent Plugins overview keeps distribution, permissions, UX, and client-specific capabilities under client control. Package compatibility and runtime equivalence are therefore separate claims.

A client can support the package format without matching another client's install flow, defaults, effective authority, or operator experience. Both clients may be conforming to the same packaging idea while making different runtime decisions.

This is not an argument for forcing every host into one security model. A local coding tool, a managed enterprise agent, and a restricted CI worker may need different policies. Teams just need to stop treating "portable" as shorthand for "already trusted everywhere."

The host turns files into authority

Reviewing a package can establish useful facts: where it came from, which revision is present, and which skills or MCP components it contains. Those facts do not establish what a particular invocation can do.

The host still decides how the component is discovered and started. It maps tools into the agent's working context, applies repository and filesystem scope, controls process and network access, and determines whether credentials are available. It also decides what happens after a tool error and what evidence survives the run.

Consider a hypothetical plugin installed in two coding clients. Client A requires an explicit per-project setup and grants only the tools enabled for that workspace. Client B loads the component through a different adapter and applies a broader existing workspace policy. The package did not change. The effective authority did.

That is where the trust decision lives: in the combination of package, client, environment, and invocation. A compatibility badge that names only the package leaves most of that decision unstated.

Client adapters are production code

Apache Magpie's marketplace documentation shows the compatibility work in concrete form. Its portable plugin.json, skills, and MCP core coexist with client-specific manifests, namespaces, and installation paths.

This is one project's implementation perspective, not the normative definition of Agent Plugins. It still demonstrates why adapters deserve a normal code review. They can affect how a client discovers the package, which setup path an operator follows, and which capabilities become available.

Treat the adapter as part of the release:

Pin its revision with the portable core.
Review client namespaces and installation instructions.
Record the tools and behavior the client is expected to expose.
Test updates, removal, and rollback instead of checking only the first install.

Calling these files "setup glue" makes them easy to skip during review. That is a mistake. They sit on the path between portable metadata and a running component.

Use four contracts instead of one compatibility label

A single compatible/incompatible result cannot carry enough information for an operational trust decision. A more useful review separates four contracts. These are working terms for teams, not terminology from the Agent Plugins specification.

1. Package contract

The package contract identifies the artifact being evaluated. Record its publisher or source, immutable revision, integrity information, and expected contents. A version label is useful only if it resolves to the same material during install and review.

This contract answers: "Which package did we inspect?"

2. Installation contract

The installation contract covers how that artifact enters a client. It includes the marketplace or source, client adapter, review policy, update behavior, revocation path, and rollback procedure.

Installing a reviewed version today should not silently authorize an unknown replacement tomorrow. Client-specific setup also belongs here because it can change without modifying the portable core.

This contract answers: "How did this exact package become available in this client?"

3. Runtime contract

The runtime contract defines authority for one environment or invocation. It covers repository and filesystem scope, processes, network access, secrets, enabled tools, lifetime, and any approval boundary.

Installation should not imply unlimited runtime authority. A component can be allowed in the environment while individual runs receive narrower capabilities. The host must enforce that distinction because the package format cannot know every workspace policy.

This contract answers: "What can this installed component do here and now?"

4. Evidence contract

The evidence contract defines what the run must leave behind: actual tool calls, outputs, failures, changed resources, validation results, and policy exceptions. The receipt needs enough detail for a reviewer to compare expected behavior with what happened.

A clean exit code is thin evidence. It says little about which tools ran, which resources changed, or whether the client recovered from an error by taking an unexpected path.

This contract answers: "What proves the component behaved within the accepted boundary?"

The four contracts can change independently. Updating a package affects the first. Moving to a new marketplace or adapter affects the second. Opening network access affects the third. Dropping tool-call logs affects the fourth. Each change deserves the review that matches it.

Small tool surfaces are easier to inspect

qwen-dap-mcp provides a useful adjacent example of capability design. Its project documentation describes a debugger-focused MCP surface for stacks, registers, locals, disassembly, dumps, and bounded fix-and-verify loops. Ordinary source editing and build or test authority remain with the coding agent.

The project is not evidence of an Agent Plugins 1.0 requirement, nor should it be assumed to be an implementation of that standard. The useful lesson is narrower: a component can expose evidence without absorbing every nearby capability.

A reviewer can reason about a small debugger surface more easily than an all-purpose integration with file editing, shell access, debugging, deployment, and unrestricted network calls bundled together. Packaging decides how a component travels. Capability design decides how much power the integration exposes.

Review those decisions separately.

Test the same package in every supported client

A package should earn acceptance per client and environment. Run the same pinned revision through a compact matrix and keep the results with the release record.

Test area	What to record
Install identity	Source, immutable package revision, client manifest or namespace, and adapter revision
Effective capabilities	Capabilities requested, capabilities granted, and any client defaults added during install or invocation
Resource scope	Repository and filesystem boundaries, process execution, network access, and secret exposure
Tool behavior	Tool schemas, validation rules, timeouts, cancellation, and visible error behavior
Context cost	Startup instructions and tool context consumed before useful task context is added
Verification	Commands the host runs, stopping conditions, and behavior when validation fails
Lifecycle	Update policy, revocation, removal, and rollback to a known revision
Run evidence	Actual tool calls, changed resources, results, exceptions, and reviewer-visible receipts

This matrix is an operational checklist, not a claim about fields mandated by Agent Plugins. Its purpose is to expose host differences before a real task finds them for you.

Do not test only the happy path. Give a tool invalid input. Deny a requested capability. Interrupt a process. Remove network access. Check whether the client reports the failure clearly, stops at the declared boundary, and preserves enough evidence to diagnose the run.

Two clients that both install the package may still produce different acceptance records. That is useful information, not a reason to hide the difference behind a shared compatibility label.

The harness still owns the boring controls

Repository instructions, forbidden paths, validation commands, process isolation, review flows, and context budgets do not disappear when components become portable. They remain part of the harness around the model and its plugins.

Current builder discussions reflect that split. A DEV.to workflow describes repository rules, memory, tools, and validation commands as startup context. Hacker News project posts include hook guards, persistent agent filesystems, guided review, and process isolation. Reddit users discuss MCP compatibility and context overhead as practical constraints on local coding setups.

Those reports are workflow examples and community sentiment, not proof that one harness design works everywhere. They do point to work the package format does not perform. A portable component cannot choose the right forbidden paths for your repository, set your risk tolerance, or decide how much context its tools may consume before the task begins.

Teams still need to design that environment. Portability simply gives them a cleaner component to place inside it.

Standardize the package, then verify the power

Agent Plugins 1.0 can make skills and MCP servers easier to distribute across clients. That is worth having. Repeated bespoke packaging wastes publisher effort and makes integrations harder to inspect.

The standard stops at a sensible boundary. The client still controls installation behavior, permissions, UX, and client-specific capabilities. The surrounding harness still controls repository rules, validation, context, and review. Runtime evidence still has to show what occurred.

When a package passes the acceptance matrix in one client, keep that result attached to that client and environment. Do not copy the trust decision to another host just because the same files install there.

The operating rule is simple: standardize the package, then verify every host that gives it power.

Source notes

A Confirm Button Is Not a Coding-Agent Security Boundary

hefty — Wed, 26 Aug 2026 07:13:33 +0000

A coding agent asks for permission to modify packages/web/generated/client.ts. The path looks harmless, the proposed diff sounds reasonable, and the user clicks Approve.

That approval means very little if the system has not resolved where the write will land. A symlink or another layer of indirection can make the effective destination different from the path in the dialog. The human reviewed one operation while the machine executed another.

A confirm button becomes a security boundary only when approval and execution refer to the same action. The action executed must be the action reviewed.

This is narrower than the usual "keep a human in the loop" advice. A human can be present, attentive, and still approve the wrong thing because the interface showed a story about the action instead of the computed action itself.

Approval needs an invariant

There are three separate objects in an agent workflow:

the user's intent
the model's description of what it plans to do
the operation the runtime has resolved and will execute

Teams often collapse them into one approval prompt. That works until a path resolves somewhere unexpected, a skill changes between review and execution, or a broad session permission gets reused for a materially different tool call.

The model's narration is useful context. It is not the authorization object. Models summarize, omit details, and can be wrong. The runtime already knows more precise facts: the effective resource, operation, credentials, network access, loaded tool code, and environment that will receive the action.

Approval should bind to those facts. If any material fact changes, the old approval is invalid. The system should prepare a new proposal and ask again.

That rule also cuts through a lot of fuzzy permission design. "Allow file edits for this session" is easy to implement and difficult to reason about. "Allow this write to this resolved path, using this tool version, before this expiry" is much less convenient. It is also something a reviewer can understand.

Prepare the operation before asking

A practical approval envelope can stay small. It needs enough information to identify the proposed effect without pretending to be a universal protocol.

proposal_id: apv_7f31
requested_resource: packages/web/generated/client.ts
resolved_resource: /workspace/task-184/packages/web/generated/client.ts
operation: write
capabilities:
  - filesystem:write:/workspace/task-184/packages/web/generated/client.ts
skill:
  name: api-client-generator
  version: 2.4.1
expires_at: 2026-08-26T06:15:00+08:00
expected_evidence:
  - changed-paths
  - diff
  - test:generated-client

The exact fields will vary. A database migration needs a target database and transaction policy. A deployment needs an environment, artifact identity, and rollback path. A browser action may need an account, origin, and allowed side effect.

The sequence matters more than the exact field names:

Prepare: resolve resources, operation, capabilities, and tool identity.
Display: show computed facts alongside the agent's explanation.
Bind: attach consent to one proposal and invalidate it after material changes.
Execute: run with the declared workspace, credentials, network policy, and lifetime.
Verify: compare the actual effects with the approved proposal.
Record: preserve the decision, result, exceptions, and review evidence.

A proposal hash can help detect mutation, but a hash does not prove that the underlying operation is safe. It only proves that the bytes did not change. The user still needs a readable view of the facts being authorized, and the runtime still needs to enforce them.

A loaded skill is part of the action

Natural-language intent does not fully identify an agent operation. The same request can behave differently depending on the tool, plugin, MCP server, or reusable skill loaded into the run.

That matters because agent skills have become a software supply-chain surface. Security reporting has described typosquatted skills that instructed agents to fetch credential-stealing code. The broad lesson is not that every shared skill is malicious. It is that "generate the API client" does not describe the full action when an external instruction package decides which commands to run.

Bind the relevant identity into the proposal:

package or skill name
source and owner
version or immutable revision
review or trust status
capabilities requested for this run

Installation trust and run-time authority are different decisions. A team may allow a reviewed skill to exist in its environment without allowing every invocation to access credentials or the network. The reverse is also true: approving a task does not make a newly substituted dependency trustworthy.

If the skill revision changes after approval, the proposal changed. Ask again. Quietly preserving consent across a tool update defeats the point of naming the tool in the first place.

Isolation contains mistakes; it does not validate consent

Parallel coding-agent products increasingly give each task an isolated worktree, terminal, conversation, and review state. That is a sensible operating model. Vendor tooling is also moving toward persistent isolated environments and resumable agent state.

Isolation limits the blast radius. It does not prove that the action inside the boundary matches what the human approved.

An agent can execute the wrong command in the correct sandbox. It can write to an unexpected resolved path that still exists inside the worktree. It can use an untrusted skill with tightly scoped permissions. Containment may turn a severe failure into a recoverable one, which is worth doing, but approval integrity is a separate property.

Treat the two controls as complementary:

the approval envelope defines the authorized effect
the execution boundary limits what can happen if the action, dependency, or runtime goes wrong

A worktree is useful for filesystem isolation and review. A sandbox can restrict process, network, and credential access. Neither should be presented as proof that the reviewer saw the exact operation that ran.

The approval UI should show facts, not confidence

Most approval interfaces spend too much space on the model's prose. "I will safely update the generated client and run the relevant tests" sounds reassuring, but it is not specific enough to authorize anything.

Give the computed fields the visual priority:

Requested path: packages/web/generated/client.ts
Resolved path:  /workspace/task-184/packages/web/generated/client.ts
Operation:      write
Capabilities:   one-file write, test command
Skill:          api-client-generator@2.4.1
Expires:        06:15 CST
Evidence:       changed paths, diff, generated-client test

Show a warning when requested and resolved resources differ. Make capability expansion obvious. Put the tool or skill revision near the approval control instead of burying it in a log panel. If the system cannot compute a field, label it unknown rather than filling the gap with the model's confidence.

This interface problem is getting more relevant as agents render their own controls and task-specific surfaces. Builders comparing component catalogs, MCP UI tools, renderer patterns, and trusted-rendering approaches can use Generative UI resources as one reference set. Whatever stack you choose, keep the authorization facts under host control; the agent should not get to invent the fields that authorize its own action.

Low-risk operations can still use broader policy approval. Formatting Markdown in a disposable branch does not need the same ceremony as rotating credentials or modifying a release workflow. The broader policy must still name its resource and capability bounds. "Do harmless things" is a preference, not an enforceable policy.

Execution needs a receipt

Approval records intent. A receipt records what happened.

At minimum, preserve the proposal identity, actual changed resources, tool calls, exit state, policy exceptions, and the evidence shown to the reviewer. For repository work, that often includes a diff and test output. It may also require checking effects that ordinary git status does not reveal.

This is where path-resolution failures become visible. Suppose the approved proposal names one resolved destination, but the receipt shows another. The system should not call the run successful and tuck the mismatch into a warning. It should mark the transaction failed or require a new approval before continuing.

The same rule applies when execution discovers that it needs more authority. If a one-file edit now requires a package install and outbound network access, stop. Prepare another proposal. Do not stretch the first approval until it covers whatever the agent wants next.

Receipts also make review state durable. Parallel tasks can resume without relying on a conversational summary of what supposedly happened. A reviewer can inspect the approved operation, the actual effect, and any divergence between them. Current developer skepticism around coding agents often lands here: humans still have to reconstruct and review the accepted result. Better evidence does not eliminate that work, but it makes the work bounded.

Consent should fail closed when the proposal moves

The original dialog did not need a longer explanation. It needed a resolved destination, a bounded capability, a named skill revision, an expiry, and a promise about the evidence that would follow.

Once those facts were bound to one proposal, any material change should have invalidated the click. After execution, the receipt should have proved whether the runtime honored the proposal.

This model will not stop every prompt injection, dependency attack, credential leak, or runtime exploit. It does enforce one useful property that a plain confirm button cannot: the human approves an action the system can identify, constrain, and compare with the result.

Do not ask people to approve a sentence the agent wrote about itself. Ask them to approve the exact operation the runtime is prepared to enforce.

Source notes

Your AI Coding Stack Needs Change Control, Not More Tuning

hefty — Sun, 23 Aug 2026 06:38:59 +0000

Your AI Coding Stack Needs Change Control, Not More Tuning

A coding agent has one bad run. The team swaps the model, rewrites the system prompt, adds an MCP server, upgrades the harness, and changes the context rules. The rerun passes.

Everyone is relieved, but the result taught the team almost nothing. Nobody knows which change helped, whether the new setup costs more to operate, or whether it will survive a different task. Rolling back is awkward because five moving parts now depend on one another.

An AI coding stack needs change control. Each adjustment should be a bounded experiment with a baseline, one named variable, visible human and machine costs, and a decision at the end.

An upgrade is still a configuration experiment

The model is only one layer of the stack. Provider settings, harness versions, system instructions, MCP tools, repository memory, context selection, approval rules, and sandbox policy can all change the result.

When several layers move together, a successful run cannot tell you which layer caused the improvement. The same pile of changes also makes a later regression harder to diagnose. You have traded one uncertain configuration for another.

Start with a small task set drawn from work the team performs. A useful baseline task has a known starting state and an observable outcome. That might mean a named test turns green, a browser flow behaves correctly at specified viewports, or a patch satisfies an existing API contract without touching unrelated packages.

Forget the universal benchmark. Pick a task that exposes the failure this configuration change is supposed to reduce.

Change one layer at a time when that is practical. If a migration forces the model, provider adapter, and prompt format to move together, record them as one bundle. The bundle can still be evaluated, but it cannot support claims about which internal change deserves credit.

Preserve the decision, not only the latest config

Most configuration files describe the stack as it exists now. They say little about why it exists, what it replaced, or which evidence justified the last change. Overwriting the file also overwrites part of the team's operational memory.

A small, append-only stack-change record fixes that. Preserve the previous state, add corrections as later entries, and record both when an observation was made and when the configuration was active. The format can stay boring:

hypothesis: What specific failure should this change reduce?
baseline_task: Which fixed task and starting state will be compared?
variable: Which layer or explicitly named bundle is changing?
acceptance_checks: What observable result counts as success?
outcome: What happened against those same checks?
machine_cost: Wall time, requests, billed usage, and cache behavior when available
human_cost: Steering time, review time, and reconstruction work
regressions_and_uncertainty: What broke, what was not measured, and what weakened the comparison?
decision: keep | revert | inconclusive

Treat this as a working template. It makes the next decision explainable and reversible; it does not pretend to be an industry standard.

The record should not decide for the team. Evidence can support a choice without becoming an automatic enforcement rule. A test result may prove that one behavior works under one setup; it does not prove that the configuration is better for every repository task.

Do not let the changed stack grade itself

Write the hypothesis and acceptance checks before running the new configuration. Otherwise it is easy to inspect whatever the agent happened to produce and invent a success criterion afterward.

Keep the pre-run plan, execution artifacts, review, and final interpretation distinct. The changed stack can collect command output, diffs, traces, and browser observations. It should not turn its own confident summary into the pass condition.

This separation also makes failures useful. A run that misses the acceptance check may still reveal a broken tool boundary or a context problem. Preserve that evidence, then let the reviewer decide whether it explains the failure. Collapsing execution and judgment into one chat transcript makes the persuasive final paragraph compete with the underlying facts.

The checkpoints can stay small. Before execution, the record needs a hypothesis, a clean baseline, and acceptance checks. Before a keep decision, it needs the resulting patch or behavior, the evidence behind it, and a review performed against the original checks. Missing evidence should produce an inconclusive result rather than a story about why the run probably worked.

Count the costs the dashboard misses

A configuration can generate more code and still make the workflow worse.

Machine cost includes more than a token total. Record elapsed time, request count, billed usage, retries, and cache behavior when the provider exposes it. A recent issue for one Codex-on-Bedrock configuration reported unexpectedly high cache-write spend when the desired cache controls were unavailable. That report belongs to one provider path, not every coding-agent setup, but it shows why model names and aggregate token counts are too coarse for configuration work.

Human cost is easier to hide. Count the time spent steering the run, checking its claims, and reconstructing code the reviewer no longer understands. One practitioner account describes AI-assisted output increasing while review became more cognitively expensive and code understanding weakened. That is personal experience rather than a controlled productivity result, but the cost category is worth tracking in a team's own work.

Reviewer attention varies with task difficulty and familiarity, so fake precision will not help. A short note such as "the reviewer had to trace three unrelated packages to validate the patch" can be more useful than a made-up score. The comparison only needs enough detail to show whether the new setup moved work out of the agent and back onto the human.

Run the comparison on a real repository task

Suppose a team wants to change its context-selection policy for a monorepo. The hypothesis is that a generated dependency map will reduce incorrect edits outside the target package.

The team chooses a known pagination regression from a clean commit. The acceptance checks are already available: the contract tests must pass, an existing query-count limit must hold, and the patch must not modify packages outside the API path. The model, harness, tools, and approval settings stay fixed. Only the context policy changes.

The record now has something concrete to compare:

Did both configurations satisfy the same contract tests?
Did either run edit unrelated packages or require manual repair?
How many retries and billed requests did each run use?
How long did a reviewer need to understand and validate each patch?
Did missing telemetry or environmental differences make the comparison weak?

Those questions do not collapse into a universal score. They support a local decision. If the new policy passes the checks and lowers review reconstruction without introducing another cost, the team has a reason to keep it. If it fails a behavior the baseline handled, revert it. If the environment changed halfway through or the provider omitted the needed cost data, mark the result inconclusive and run a cleaner test only if the expected value justifies the time.

Use the same discipline for larger migrations. Name the bundle, freeze what can be frozen, preserve the old configuration, and narrow the claim. A successful provider migration can justify adopting that bundle without proving that every model, prompt, and cache setting inside it is optimal.

End every change with a decision

Configuration experiments tend to linger. A new MCP tool remains enabled because it might help later. A longer prompt survives because one run looked good. A second harness stays installed while the team debates which one to use. Each unresolved experiment adds another branch to the operating environment.

Close the record with one of three decisions:

Keep the change because it beat the baseline on the declared outcome without an unacceptable regression.
Revert it because it failed the checks or moved too much cost onto the reviewer or infrastructure.
Mark it inconclusive because the comparison was not strong enough to support either choice.

Do not preserve a change merely because it "feels better." Put that observation in the notes and design a task that can expose the suspected improvement. If the expected benefit is too small to justify another test, the old stable configuration wins by default.

Timebox the tuning work as well. Current developer discussions describe model, harness, MCP, and prompt optimization becoming a job of its own. That is anecdotal sentiment, but the failure mode is easy to recognize: the stack consumes the time it was meant to return.

The team in the opening example should have saved its starting configuration, named the failure, changed one layer, and compared the rerun with the same checks and costs. Without that record, a passing run is just a pleasant result. It is not evidence that the stack improved.

If a configuration cannot beat a stable baseline on real work, revert it. Otherwise the next failure will be harder to explain than the last one.

Source notes

Your First Coding Agent Should Be a Reviewer, Not an Author

hefty — Wed, 19 Aug 2026 11:30:00 +0000

Most teams introduce a coding agent backwards. They hand it a ticket, let it edit the repository, and only then ask a human to figure out whether the patch is correct.

The model gets the fun job. The reviewer pays the reverse-engineering tax.

Start with review instead. If an agent can find a concrete problem, attach evidence, expose its uncertainty, and give a human a cheap way to confirm the result, it has already created value. It can prove that value before the team makes code authorship its default role.

Review can pay before the agent writes code

A recent TechRadar report about Linux kernel work draws a useful line. It describes AI review and analysis agents surfacing actionable bug reports, while humans write the fixes and maintainers retain judgment over what lands.

That is one reported workflow, not proof that AI review is universally accurate or cheaper. The role split is still worth examining: the agent can reduce uncertainty without owning the patch.

Authorship bundles several questions together. Did the agent understand the bug? Did it choose the right fix? Did it preserve unrelated behavior? Did its tests prove what it says they proved? A generated patch forces the reviewer to answer all of them at once.

A review finding is narrower. The agent can point to one behavior and the evidence behind it. A human can confirm or reject that finding without first untangling a proposed implementation. False positives still cost time, but a compact, falsifiable report limits that cost.

The first test is whether the agent can make a decision easier before it starts changing code.

"Reviewer" is a workflow, not a prompt

Telling a model to "review this repository" is barely more specific than telling it to "write good code." Useful review needs a bounded surface, an environment that can produce evidence, and an output that another person can inspect.

Proliferate is a concrete product signal here. Its project documentation lists isolated worktrees or sandboxes, diff review, and reviewer agents as parts of the operating surface for multiple coding harnesses. The README does not prove that those reviewers are accurate or production-safe. It does show that review and isolation are becoming explicit product features rather than chores left outside the agent loop.

The surrounding controls matter as much as the reviewer model:

Give it one commit, diff, subsystem, or execution path to inspect.
Let it run named checks in an isolated environment when the finding depends on behavior.
Preserve test output, traces, browser observations, and repository history with the report.
Require it to stop when missing context prevents a defensible finding.

The same operational concerns appear in practitioner guidance about production coding agents: tests, Git history, prompts before destructive actions, explicit verification, and knowing when to stop. Those are not benchmark results, but they are better selection criteria than raw generation quality.

A reviewer with unlimited scope and no stop condition will produce commentary. A reviewer with a defined surface and inspectable evidence has a chance to produce a decision.

Require a review receipt

A confident paragraph is not a review artifact. "There may be a serious race condition" gives the next person a claim to investigate and no cheap way to investigate it.

Require a small receipt for every finding:

Finding: one concrete issue rather than a broad quality opinion.
Location: a file and line, execution path, or observable behavior.
Evidence: a test, trace, command output, diff, or browser observation another person can inspect.
Impact: what can fail and under which conditions, without inflating severity.
Uncertainty: assumptions, missing context, and plausible alternative explanations.
Next check: the smallest action that confirms or rejects the finding.

This is a proposed working format, not an industry standard. It makes a weak finding cheap to reject.

Consider a hypothetical service with a role-change endpoint and cached authorization. A useful agent report could look like this:

Finding: A removed project member can retain the previous permission until the cache entry expires.
Location: The role-update path returns before the existing authorization cache is invalidated.
Evidence: After a successful role removal, the named integration scenario still receives a 200 response for the protected project request. The trace and test output are attached.
Impact: Access may remain stale for the configured cache lifetime after this update path.
Uncertainty: The report has not established whether a background invalidation event is expected to close the gap.
Next check: Run the same scenario with the invalidation worker enabled and inspect whether it receives the role-change event.

The agent has not proved the architecture is insecure, and it has not earned permission to rewrite the cache layer. It has produced a specific claim with a short rejection path. A maintainer can inspect the trace, check the event flow, and decide whether a patch is needed.

A review that cannot fill the receipt should not silently graduate into an agent-authored fix. It should return the missing evidence or context.

Domain context belongs inside the evidence loop

Frontend review makes the weakness of vague judgment obvious. "This page looks wrong" is not a finding. Neither is "make it feel more polished."

The reviewer needs a selected reference for the intended interaction, a named design-system constraint, a browser observation under stated conditions, and a next check. For generated interfaces, a curated set of Generative UI cases can help a team choose relevant examples and translate them into its own observable criteria.

The examples are reference material, not proof and not an automated acceptance oracle. The team still has to state what matters in its product. That might be keyboard order, component usage, overflow at a named viewport, or the behavior of a streamed state transition. The agent's job is to compare the implementation with those declared constraints and show the mismatch.

This is why domain context should be attached to the review task instead of hidden in a prompt like "use good UX." References become useful only after the team turns them into checks that can fail.

Let the agent climb the role ladder

Authorship does not need to be the starting position. Increase the agent's ability to mutate the repository only after its output is useful at the previous step:

Explain a bounded area of the code and cite the relevant paths.
Reproduce one reported behavior without changing the repository.
Review a selected diff or execution path and produce a receipt.
Suggest a patch inside an isolated workspace, with the finding and evidence kept beside it.
Author a bounded change when the team can evaluate both the diagnosis and the result.

Advancement should stay local to the workflow. An agent that reviews dependency updates well has not earned a global trust score for authentication changes. A good result on a backend test does not prove it can judge a generated interface.

Ask practical questions at each step. How many findings did maintainers confirm? How quickly could they reject the wrong ones? Did the report remove uncertainty, or did it create another document to decode? These are evaluation questions, not universal metrics. Teams can answer them with their own repository history instead of importing a multiplier claim.

Authorship then becomes a downstream privilege. The agent already knows how to locate evidence, state uncertainty, and stop when the task exceeds its context. The patch is connected to a diagnosis that survived review rather than appearing as the first artifact in the process.

More output will not repair expensive review

Current developer discussion is skeptical of generation volume without validation. A Hacker News thread built around the "2x, not 10x" framing pushes toward experiments, constraints, and review. A Reddit post describes spending more time tuning models, harnesses, MCP tools, and prompts than completing the work the stack was meant to help with. Both are community sentiment, not measured productivity evidence.

That frustration suggests a narrower move. Adding another authoring agent can increase the amount of code waiting for judgment. A focused reviewer can work on the expensive part directly by making one claim easier to inspect.

The team that started by granting write access needed a finding a human could confirm, not a patch the human had to decipher. Once the agent can repeatedly make review cheaper for a named workflow, letting it propose the fix becomes a smaller step.

If it cannot make review cheaper, letting it generate more code only increases the bill.

Source notes

Give your coding agent a reproducible failure, not a vague prompt

hefty — Sun, 16 Aug 2026 06:06:48 +0000

"Fix checkout" is enough to start a conversation. It is a terrible unit of work for a coding agent.

The agent can inspect the repository, find something suspicious, write a plausible patch, and report success. The reviewer still has no shared answer to two basic questions: what exactly was broken, and what observable behavior should now be different?

That gap matters more as implementation gets cheaper. A coding task should begin with a known before-state and a declared after-state. The patch is only an attempt to move the system between them.

A task is a state transition, not a sentence

A vague prompt usually contains intent:

Fix the checkout timeout when a session expires.

It does not contain a task contract. Which revision has the bug? Which fixture exposes it? Does the request hang, return the wrong status, or discard the cart? What behavior must remain unchanged? Which command evaluates the result?

Without those details, the agent has to invent part of the acceptance test while implementing the fix. That is a bad delegation boundary. The same system is choosing what "done" means and announcing that it got there.

A reviewable task has two observable states:

The starting state can be recreated before any edit.
The target state can be evaluated after the edit without trusting the agent's summary.

The agent can choose the implementation between those states, while the task author owns the surrounding contract.

This framing also keeps the human out of low-value supervision. You do not need to narrate every code change if you have already defined the behavior, constraints, and evidence that will make the change acceptable.

Start from failure on purpose

CLI-Gym is a useful example of failure-first task construction. Its project documentation describes a pipeline that begins with healthy repository environments, introduces controlled failures, and turns those states into reproducible command-line tasks. The README reports 1,655 tasks from 29 repositories.

Those numbers are project-authored, and they do not prove benchmark quality or production reliability. The task design is more useful here than the headline count.

A controlled failure gives the agent and reviewer the same starting point. It can be reset. It can be observed before the patch. It can be checked again afterward. If the failure disappears for reasons unrelated to the patch, that becomes visible too.

Compare that with dropping an agent into a moving branch and saying, "The build is weird, please fix it." The agent may find a real problem. It may also repair an incidental symptom, update a flaky snapshot, or change enough code that nobody can tell which behavior mattered.

Failure-first does not mean every assignment needs an existing failing test. Feature work often starts without one. In that case, define an observable before-state and a concrete acceptance example. A missing endpoint, an unsupported interaction, or a fixture that currently produces no result can still anchor the transition.

Put the contract in an artifact

Chat is convenient for exploration. It is weak as the only record of a coding task.

A durable setup keeps intent, scope, checks, and evidence in artifacts that survive the session. A task file can be reviewed before execution. A plan can be compared with the allowed scope. Test output and the final diff can stay attached to the same work unit.

This changes the review from "Does the completion message sound convincing?" to a comparison:

Did the run begin from the declared base state?
Does the original reproduction now reach the target behavior?
Did the diff stay inside the stated boundaries?
Which named checks ran, and which did not?
Did the agent stop when the task became ambiguous?

Treat a confident completion paragraph as context, then verify it against the artifacts.

The contract should also remain implementation-neutral where possible. "Add a retry around refreshSession()" may be a reasonable plan, but it is already prescribing a patch. "An expired session returns the typed authentication error within the configured timeout and preserves the cart" gives the agent room to inspect the code before choosing a change.

A six-field task card

You do not need a large specification system to get this benefit. A compact task card is enough for many repository changes.

The following example is hypothetical. The commands and fixture names stand in for whatever your repository actually uses.

Base

Revision: <commit SHA>
Fixture: expired-session.json
Environment assumption: the local test services start with the documented setup command
Initial state: the cart contains one item and the session token is expired

Reproduce

Run pnpm test checkout -- --grep "expired session"
Current result: the checkout request waits until the test timeout
Save the pre-change failure output with the task

Target

Checkout returns the repository's existing typed authentication error before the configured timeout
The cart remains intact so the user can authenticate and try again

Boundaries

Do not change the session lifetime
Do not add a dependency
Preserve checkout behavior for valid sessions
Keep unrelated formatting and refactors out of the diff

Checks

Re-run the exact reproduction command
Run the existing checkout test group
Run the repository's type check
Inspect the final diff against the boundaries above

Stop

Hand the task back unresolved if:

the failure does not reproduce from the declared base
the fixture or local setup is broken
the expected result conflicts with the current API contract
a required service or credential is unavailable
the fix needs a wider product or architecture decision

Omitting the Stop section leaves the agent with no defined way to return an unresolved task. An agent that has only success criteria will keep searching for a patch. Explicit stop conditions let it return a useful failure report before it widens the change or guesses at product intent.

Passing checks is evidence with a boundary

Named checks make the transition reviewable. They do not certify the entire system.

A green targeted test says something about the behavior encoded in that test. It does not prove that the architecture is healthy, that the change is secure, or that every product expectation has been captured. A reproducible failure narrows uncertainty around the named behavior while leaving architectural and product questions for review.

The final diff still needs review. Compare the code with the declared target and non-goals, then decide whether broader tests or specialist review are needed. A two-line fix in an authentication path may deserve more scrutiny than a larger change in an isolated internal tool.

The task card helps make that judgment legible. It shows what the checks were supposed to prove and where their authority ends.

Let an unresolved run count as a valid result

The Hacker News and Reddit threads in the source notes are anecdotal, but some participants favor small, auditable loops: inspect, change, run a real check, review the diff, and preserve enough state for the next step. Tool preferences vary, while every unattended run still needs a clear handoff.

A run that reports "cannot reproduce from the named revision" has produced useful information. So has one that finds contradictory acceptance checks or a missing service. Those results protect the repository from a guessed patch and give the task author a specific problem to resolve.

Treating every run as patch-or-failure creates pressure to hide uncertainty. Treating a justified stop as part of the contract makes the workflow more honest. The agent can state what it inspected, where the transition broke down, and what decision or dependency is missing.

That is a better result than green output from a check nobody agreed would define success.

Define the transition before asking for the patch

Return to "fix checkout." With a base revision, fixture, reproduction command, behavioral target, boundaries, checks, and stop conditions, the assignment becomes reviewable before the agent writes a line of code.

The agent may discover that the expected fix is tiny. It may find that the task is underspecified. Both outcomes are easier to handle when the before-state and after-state are explicit.

A stronger model can propose a better implementation. It cannot recover a success condition the team never wrote down.

Source notes

A coding agent needs an authority budget, not just a better model

hefty — Fri, 14 Aug 2026 06:43:32 +0000

A coding agent can produce a patch before your team has agreed on what it was allowed to read.

That is a bad trade. A fast patch is not useful if nobody can explain which files shaped it, which tools it called, what data it retained, or what evidence makes the result trustworthy. The model may be capable. The workflow can still be unusable.

I think about this as an authority budget. The budget is the set of permissions and obligations around an agent: what it can read, write, call, retain, ask approval for, and prove afterward. The model matters, but it is only one part of the operating design.

Model quality is only one axis

Coding assistants now occupy different workflow lanes. Some live in an IDE. Some work in a terminal, browser, or pull-request flow. Privacy, team controls, cost, and the amount of context available also vary by tool.

That variety makes a universal ranking less useful than it looks. Choosing an agent is also choosing a permission surface.

An IDE assistant may see the file currently open and a slice of the repository. A terminal agent may be able to run commands and modify a wider set of paths. A browser-oriented workflow may have a different kind of access again. Each one creates a different review problem.

The question is not only, "Which model writes the best code?" It is also:

What context does this workflow expose?
What can the agent change without asking?
Which external systems can it call?
How does a reviewer know what happened?

A model comparison can help with the first question. It does not answer the rest.

Reading, writing, and reviewing are different powers

The Pair documents a simple split between a Mentor, which acts as a read-only reviewer, and an Executor, which performs the writing work. Whether that project catches more mistakes is a separate question. The design still makes an important permission distinction visible.

Inspection and mutation do not have to be the same authority.

That distinction is easy to lose when one agent can inspect a repository, edit files, run commands, and summarize its own work. The same system that made the change can also decide that the change looks fine. A successful response then becomes a weak substitute for independent review.

A stricter workflow can separate the powers:

One step gathers context and proposes a change.
Another step applies the change inside a bounded workspace.
A reviewer, human or automated, inspects the diff and the required checks.

This does not make the review correct by magic. It does make the review boundary explicit. Read access is not write access, and write access is not approval.

The harness around the model is part of the product

Google's developer highlights describe persistent isolated environments and Managed Agents as part of its developer tooling. The release signal matters because it treats execution space, persistence, and agent lifecycle as product concerns rather than details hidden behind a chat box.

The Product Hunt page for prjct/ takes a similar direction from a different angle. Its product positioning combines intent briefs, bounded repository context, persistent memory, guardrails, and evaluations around coding agents.

Those pages describe vendor or product-authored capabilities. They are not independent validation, and isolation or persistence does not automatically make a workflow safe. They do show where the engineering surface is moving: the wrapper around the model increasingly determines the context, tools, memory, and checks that shape a run.

That wrapper deserves the same design attention as the prompt.

If an agent has persistent memory, decide what enters it and how long it stays there. If it runs in an isolated environment, decide which credentials, network routes, and repositories can still reach the environment. If it has guardrails, make the restricted actions visible and test the failure path.

A label such as "sandboxed" is not a permission model.

Why teams say no

A recent Reddit discussion about a workplace that did not allow AI agents is anecdotal, not a measurement of how teams behave. It is still useful because the objections are concrete. People discuss code exfiltration, data retention, residency, and how a reviewer can examine agent-produced changes.

Those are authority questions.

A developer may want help with a repository. The organization may need to know whether source code leaves its boundary, whether prompts or logs persist, where artifacts are stored, and whether a human can reconstruct the change. Productivity does not remove those obligations.

This is why "the model is good enough" rarely settles an adoption decision. A team can accept the coding capability and reject the surrounding authority. It can also approve a narrow local workflow while refusing broad repository or production access.

Write the six lines before widening access

The authority budget does not need to be a grand governance program. Start with six explicit lines in the task definition.

Read. Name the repositories, directories, files, secrets, and external context the agent may see. Keep sensitive material out of scope by default.
Write. Name the paths and branches the agent may change. Make generated artifacts and protected files clear.
Call. List the commands, tools, network destinations, and services available during the run. Treat a new network permission as a design change.
Retain. Decide where prompts, source snippets, memory, logs, diffs, and output artifacts persist, and for how long.
Approve. Mark the actions that require a person or a policy gate, such as changing dependencies, accessing production, or widening the workspace.
Prove. Require the diff, named check results, relevant event or log evidence, and an explicit failure state before the run counts as complete.

The last line is where many workflows become honest. A process exit of zero is not the same as an accepted change. The agent's statement that it finished is not the same as a passing test suite. A useful result should let the next reviewer see what changed, what ran, what failed, and what the agent left untouched.

For an automated coding task, I would start with a read-only repository slice or a narrow writable path. Then I would add one permission for one named workflow need. If the agent needs network access, record why. If it needs persistent memory, define its contents and lifetime. If it needs to edit a new directory, make that expansion reviewable.

This keeps the failure radius small while the team learns how the workflow behaves.

Spend authority only when the run can show its receipt

Better models will make agents more useful. They will not decide what a team should expose to an agent or what evidence should unlock the next step.

That decision belongs in the workflow contract.

The better coding agent is often not the one with the most impressive demo. It is the one whose authority a team can explain, whose changes fit inside that authority, and whose result leaves a receipt that another person or system can review.