DEV Community: hefty

Four Coding Agents Need Four Workspaces, Not Four Chat Windows

hefty — Mon, 31 Aug 2026 06:35:44 +0000

Opening four coding-agent sessions feels like scaling. On a shared machine, it is closer to giving four fast contributors the same repository, shell, credentials, ports, caches, and merge queue without deciding who owns any of them.

The first failure probably will not come from model quality. One task will restart a dev server while another is testing it. Two workers will touch the same lockfile. A branch will pass its own checks and still conflict with a migration waiting in the merge queue.

Four chat windows create concurrency. Four owned workspaces plus one deliberate merge queue create a system.

Parallelism multiplies shared state

Tasks that sound independent in a prompt can overlap in the environment.

A frontend change and an API change may both edit generated types. Two test runs may expect the same database or browser profile. Separate worktrees can still launch services on the same port, read the same environment variables, and write to shared caches. The agents do not collide in the prompt. They collide in everything the prompt lets them touch.

This is why adding a second agent changes the job. With one worker, the operator can keep a surprising amount of state in their head. With four, every unstated assumption becomes a race condition or a review problem.

The fix is to make ownership visible before execution starts.

A worktree is the start, not the boundary

Git worktrees are a sensible first step. Each task gets its own branch and working files, so one agent is less likely to overwrite another agent's edits by accident.

That is useful isolation, but it is narrow isolation.

A worktree does not reserve a port. It does not separate process trees, temporary directories, credentials, network access, browser state, or external services. Treating it as a sandbox gives the workflow more confidence than the boundary deserves.

Proliferate is an instructive project example because its documented design pairs isolated task worktrees with visible review state. The important idea is the pairing. Filesystem separation tells you where a patch was produced; task state tells you what should happen to that patch next. The project description is not independent proof that every boundary is solved, and it does not need to be. The pattern is useful on its own.

For each concurrent task, define at least these ownership boundaries:

Workspace ownership

Assign one repository scope, branch, and worktree. State which generated files or shared configuration the task may change. If two tasks own the same file, they are coupled work and should be queued accordingly.

Process ownership

Reserve ports, dev servers, test databases, caches, temporary paths, and browser profiles. A process should have a named task owner and a cleanup rule instead of becoming shared background state.

Artifact ownership

Name the output before the run begins: a patch, a report, a screenshot set, a migration plan, or some other reviewable object. "Improve the app" is not an artifact contract.

Treat authority as a task budget

The easiest setup is to launch every worker from the operator's normal shell. It is also the setup with the least useful separation.

A task that edits a local component rarely needs every token, deployment command, network destination, and destructive tool available to the human running it. Grant the repository, commands, credentials, and network surface required for that task. Leave the rest out.

Current agent harnesses do not all enforce these boundaries in the same way. If a tool cannot restrict a credential or network path, write that limitation into the run record and reduce the task's authority. Do not silently treat the operator's full shell as a reasonable default.

This also makes failures easier to understand. When a task has a small authority budget, the failure surface is smaller: you can see which files, processes, and external systems were in play without reconstructing the whole machine after the fact.

Keep integration serial on purpose

Implementation can run in parallel. Integration should have one owner and an explicit order.

Two patches can both be locally correct and still disagree about an API shape, migration sequence, dependency version, generated file, or user-visible behavior. Git may merge the text cleanly while the combined system is wrong. That is an integration conflict even when there are no conflict markers.

Give one person, or one tightly bounded integration role, responsibility for:

choosing the review order
resolving overlapping decisions
rerunning checks against the combined state
deciding what lands and what returns for another pass

The merge queue is where parallel work becomes one product. Making it serial is coordination, not wasted parallelism.

Completion needs evidence, not a status message

An agent saying "done" proves that it generated a completion message. It does not prove the patch works.

Every task should return an evidence bundle with:

the changed files and the intent of the diff
the exact checks run and their real results
screenshots or browser evidence when behavior is visual
known failures, skipped checks, and unresolved assumptions
merge-order or conflict notes for the integration owner

Repository-native agent systems are starting to expose this control layer directly. GitHub Agentic Workflows defines bounded jobs and safe-output handling. Its August 24 update describes safe-output validation, run steering, inspectable model and catalog state, and clearer startup diagnostics. GitHub Copilot's August releases describe task management, queued prompts, plan/autopilot behavior, and rewind. These are product descriptions, not reliability benchmarks, but the direction is useful: long-running work needs intervention, recovery, and inspectable output.

The evidence rule also applies to non-code artifacts. If a frontend task produces launch or social graphics, a narrow browser-local step such as Resize Image For can handle resizing, fitting, cropping or padding, previews, and export while keeping source-image processing in the browser. That is easier to review than handing the source files to another remote worker with a broad instruction to "prepare the assets."

Write the contract before launching the second agent

A small task record is enough. The exact schema matters less than forcing the decisions into the open.

task: fix-mobile-navigation
owner: agent-2
stop_when: mobile navigation passes the named browser checks

workspace:
  repository_scope: apps/web
  worktree: ../worktrees/fix-mobile-navigation
  shared_files: [pnpm-lock.yaml]

process:
  ports: [3102]
  services: [web-dev-server]
  browser_profile: agent-2-mobile

authority:
  credentials: []
  network: [localhost]
  destructive_tools: false

expected_artifacts:
  - code-diff
  - test-results
  - mobile-screenshots
  - known-issues

integration:
  merge_owner: human-reviewer
  review_after: shared-design-token-change
  rollback: discard-worktree-and-branch

This is an operating record, not a universal security configuration. The point is to make shared state, authority, evidence, and merge order reviewable before they become an incident report.

If the task cannot fill in these fields, it is probably too vague or too coupled to run concurrently.

Know when serial work wins

Parallelize tasks with separate state and separate validation paths. A documentation change in one package and an isolated test addition in another may fit well. Shared-schema changes, coupled migrations, global dependency upgrades, and work that relies on one fragile validation environment usually deserve a queue.

There is no prize for keeping every agent busy. If coordination and verification cost more than the execution time you saved, reduce the agent count.

Before opening another session, provision its workspace, authority, integration path, and evidence contract. If that is too expensive for the task, keep the work serial.

Source notes

Agent Plugins 1.0 Solves Packaging, Not Trust

hefty — Sun, 30 Aug 2026 12:57:18 +0000

A portable agent plugin can be identical on disk and still become a different operational system in two clients.

One client may need its own manifest namespace or installation path. Another may expose a different capability set. Repository access, process execution, network policy, secret handling, failure behavior, and run evidence can all depend on the host.

If a team reviews only the portable files, it has reviewed the package. It has not reviewed every runtime that can give that package power.

Agent Plugins 1.0 addresses a useful problem: packaging Agent Skills and MCP servers in a vendor-neutral format. That should cut duplicated integration work. It does not make distribution, permissions, UX, or client-specific behavior identical, and the official overview leaves those concerns to each client.

The practical rule is to standardize the package and test the host separately.

Packaging is real progress

Agent integrations have accumulated plenty of one-off setup: tool definitions, skill directories, MCP configuration, marketplace metadata, and client-specific instructions. A portable core gives publishers and client authors a common shape to work with.

That common shape is an interoperability floor. It can tell a client how a component is packaged and reduce the need to rebuild the same integration from scratch. It also gives reviewers a more stable object to inspect, version, and compare.

The boundary matters, though. The Agent Plugins overview keeps distribution, permissions, UX, and client-specific capabilities under client control. Package compatibility and runtime equivalence are therefore separate claims.

A client can support the package format without matching another client's install flow, defaults, effective authority, or operator experience. Both clients may be conforming to the same packaging idea while making different runtime decisions.

This is not an argument for forcing every host into one security model. A local coding tool, a managed enterprise agent, and a restricted CI worker may need different policies. Teams just need to stop treating "portable" as shorthand for "already trusted everywhere."

The host turns files into authority

Reviewing a package can establish useful facts: where it came from, which revision is present, and which skills or MCP components it contains. Those facts do not establish what a particular invocation can do.

The host still decides how the component is discovered and started. It maps tools into the agent's working context, applies repository and filesystem scope, controls process and network access, and determines whether credentials are available. It also decides what happens after a tool error and what evidence survives the run.

Consider a hypothetical plugin installed in two coding clients. Client A requires an explicit per-project setup and grants only the tools enabled for that workspace. Client B loads the component through a different adapter and applies a broader existing workspace policy. The package did not change. The effective authority did.

That is where the trust decision lives: in the combination of package, client, environment, and invocation. A compatibility badge that names only the package leaves most of that decision unstated.

Client adapters are production code

Apache Magpie's marketplace documentation shows the compatibility work in concrete form. Its portable plugin.json, skills, and MCP core coexist with client-specific manifests, namespaces, and installation paths.

This is one project's implementation perspective, not the normative definition of Agent Plugins. It still demonstrates why adapters deserve a normal code review. They can affect how a client discovers the package, which setup path an operator follows, and which capabilities become available.

Treat the adapter as part of the release:

Pin its revision with the portable core.
Review client namespaces and installation instructions.
Record the tools and behavior the client is expected to expose.
Test updates, removal, and rollback instead of checking only the first install.

Calling these files "setup glue" makes them easy to skip during review. That is a mistake. They sit on the path between portable metadata and a running component.

Use four contracts instead of one compatibility label

A single compatible/incompatible result cannot carry enough information for an operational trust decision. A more useful review separates four contracts. These are working terms for teams, not terminology from the Agent Plugins specification.

1. Package contract

The package contract identifies the artifact being evaluated. Record its publisher or source, immutable revision, integrity information, and expected contents. A version label is useful only if it resolves to the same material during install and review.

This contract answers: "Which package did we inspect?"

2. Installation contract

The installation contract covers how that artifact enters a client. It includes the marketplace or source, client adapter, review policy, update behavior, revocation path, and rollback procedure.

Installing a reviewed version today should not silently authorize an unknown replacement tomorrow. Client-specific setup also belongs here because it can change without modifying the portable core.

This contract answers: "How did this exact package become available in this client?"

3. Runtime contract

The runtime contract defines authority for one environment or invocation. It covers repository and filesystem scope, processes, network access, secrets, enabled tools, lifetime, and any approval boundary.

Installation should not imply unlimited runtime authority. A component can be allowed in the environment while individual runs receive narrower capabilities. The host must enforce that distinction because the package format cannot know every workspace policy.

This contract answers: "What can this installed component do here and now?"

4. Evidence contract

The evidence contract defines what the run must leave behind: actual tool calls, outputs, failures, changed resources, validation results, and policy exceptions. The receipt needs enough detail for a reviewer to compare expected behavior with what happened.

A clean exit code is thin evidence. It says little about which tools ran, which resources changed, or whether the client recovered from an error by taking an unexpected path.

This contract answers: "What proves the component behaved within the accepted boundary?"

The four contracts can change independently. Updating a package affects the first. Moving to a new marketplace or adapter affects the second. Opening network access affects the third. Dropping tool-call logs affects the fourth. Each change deserves the review that matches it.

Small tool surfaces are easier to inspect

qwen-dap-mcp provides a useful adjacent example of capability design. Its project documentation describes a debugger-focused MCP surface for stacks, registers, locals, disassembly, dumps, and bounded fix-and-verify loops. Ordinary source editing and build or test authority remain with the coding agent.

The project is not evidence of an Agent Plugins 1.0 requirement, nor should it be assumed to be an implementation of that standard. The useful lesson is narrower: a component can expose evidence without absorbing every nearby capability.

A reviewer can reason about a small debugger surface more easily than an all-purpose integration with file editing, shell access, debugging, deployment, and unrestricted network calls bundled together. Packaging decides how a component travels. Capability design decides how much power the integration exposes.

Review those decisions separately.

Test the same package in every supported client

A package should earn acceptance per client and environment. Run the same pinned revision through a compact matrix and keep the results with the release record.

Test area	What to record
Install identity	Source, immutable package revision, client manifest or namespace, and adapter revision
Effective capabilities	Capabilities requested, capabilities granted, and any client defaults added during install or invocation
Resource scope	Repository and filesystem boundaries, process execution, network access, and secret exposure
Tool behavior	Tool schemas, validation rules, timeouts, cancellation, and visible error behavior
Context cost	Startup instructions and tool context consumed before useful task context is added
Verification	Commands the host runs, stopping conditions, and behavior when validation fails
Lifecycle	Update policy, revocation, removal, and rollback to a known revision
Run evidence	Actual tool calls, changed resources, results, exceptions, and reviewer-visible receipts

This matrix is an operational checklist, not a claim about fields mandated by Agent Plugins. Its purpose is to expose host differences before a real task finds them for you.

Do not test only the happy path. Give a tool invalid input. Deny a requested capability. Interrupt a process. Remove network access. Check whether the client reports the failure clearly, stops at the declared boundary, and preserves enough evidence to diagnose the run.

Two clients that both install the package may still produce different acceptance records. That is useful information, not a reason to hide the difference behind a shared compatibility label.

The harness still owns the boring controls

Repository instructions, forbidden paths, validation commands, process isolation, review flows, and context budgets do not disappear when components become portable. They remain part of the harness around the model and its plugins.

Current builder discussions reflect that split. A DEV.to workflow describes repository rules, memory, tools, and validation commands as startup context. Hacker News project posts include hook guards, persistent agent filesystems, guided review, and process isolation. Reddit users discuss MCP compatibility and context overhead as practical constraints on local coding setups.

Those reports are workflow examples and community sentiment, not proof that one harness design works everywhere. They do point to work the package format does not perform. A portable component cannot choose the right forbidden paths for your repository, set your risk tolerance, or decide how much context its tools may consume before the task begins.

Teams still need to design that environment. Portability simply gives them a cleaner component to place inside it.

Standardize the package, then verify the power

Agent Plugins 1.0 can make skills and MCP servers easier to distribute across clients. That is worth having. Repeated bespoke packaging wastes publisher effort and makes integrations harder to inspect.

The standard stops at a sensible boundary. The client still controls installation behavior, permissions, UX, and client-specific capabilities. The surrounding harness still controls repository rules, validation, context, and review. Runtime evidence still has to show what occurred.

When a package passes the acceptance matrix in one client, keep that result attached to that client and environment. Do not copy the trust decision to another host just because the same files install there.

The operating rule is simple: standardize the package, then verify every host that gives it power.

Source notes

A Confirm Button Is Not a Coding-Agent Security Boundary

hefty — Wed, 26 Aug 2026 07:13:33 +0000

A coding agent asks for permission to modify packages/web/generated/client.ts. The path looks harmless, the proposed diff sounds reasonable, and the user clicks Approve.

That approval means very little if the system has not resolved where the write will land. A symlink or another layer of indirection can make the effective destination different from the path in the dialog. The human reviewed one operation while the machine executed another.

A confirm button becomes a security boundary only when approval and execution refer to the same action. The action executed must be the action reviewed.

This is narrower than the usual "keep a human in the loop" advice. A human can be present, attentive, and still approve the wrong thing because the interface showed a story about the action instead of the computed action itself.

Approval needs an invariant

There are three separate objects in an agent workflow:

the user's intent
the model's description of what it plans to do
the operation the runtime has resolved and will execute

Teams often collapse them into one approval prompt. That works until a path resolves somewhere unexpected, a skill changes between review and execution, or a broad session permission gets reused for a materially different tool call.

The model's narration is useful context. It is not the authorization object. Models summarize, omit details, and can be wrong. The runtime already knows more precise facts: the effective resource, operation, credentials, network access, loaded tool code, and environment that will receive the action.

Approval should bind to those facts. If any material fact changes, the old approval is invalid. The system should prepare a new proposal and ask again.

That rule also cuts through a lot of fuzzy permission design. "Allow file edits for this session" is easy to implement and difficult to reason about. "Allow this write to this resolved path, using this tool version, before this expiry" is much less convenient. It is also something a reviewer can understand.

Prepare the operation before asking

A practical approval envelope can stay small. It needs enough information to identify the proposed effect without pretending to be a universal protocol.

proposal_id: apv_7f31
requested_resource: packages/web/generated/client.ts
resolved_resource: /workspace/task-184/packages/web/generated/client.ts
operation: write
capabilities:
  - filesystem:write:/workspace/task-184/packages/web/generated/client.ts
skill:
  name: api-client-generator
  version: 2.4.1
expires_at: 2026-08-26T06:15:00+08:00
expected_evidence:
  - changed-paths
  - diff
  - test:generated-client

The exact fields will vary. A database migration needs a target database and transaction policy. A deployment needs an environment, artifact identity, and rollback path. A browser action may need an account, origin, and allowed side effect.

The sequence matters more than the exact field names:

Prepare: resolve resources, operation, capabilities, and tool identity.
Display: show computed facts alongside the agent's explanation.
Bind: attach consent to one proposal and invalidate it after material changes.
Execute: run with the declared workspace, credentials, network policy, and lifetime.
Verify: compare the actual effects with the approved proposal.
Record: preserve the decision, result, exceptions, and review evidence.

A proposal hash can help detect mutation, but a hash does not prove that the underlying operation is safe. It only proves that the bytes did not change. The user still needs a readable view of the facts being authorized, and the runtime still needs to enforce them.

A loaded skill is part of the action

Natural-language intent does not fully identify an agent operation. The same request can behave differently depending on the tool, plugin, MCP server, or reusable skill loaded into the run.

That matters because agent skills have become a software supply-chain surface. Security reporting has described typosquatted skills that instructed agents to fetch credential-stealing code. The broad lesson is not that every shared skill is malicious. It is that "generate the API client" does not describe the full action when an external instruction package decides which commands to run.

Bind the relevant identity into the proposal:

package or skill name
source and owner
version or immutable revision
review or trust status
capabilities requested for this run

Installation trust and run-time authority are different decisions. A team may allow a reviewed skill to exist in its environment without allowing every invocation to access credentials or the network. The reverse is also true: approving a task does not make a newly substituted dependency trustworthy.

If the skill revision changes after approval, the proposal changed. Ask again. Quietly preserving consent across a tool update defeats the point of naming the tool in the first place.

Isolation contains mistakes; it does not validate consent

Parallel coding-agent products increasingly give each task an isolated worktree, terminal, conversation, and review state. That is a sensible operating model. Vendor tooling is also moving toward persistent isolated environments and resumable agent state.

Isolation limits the blast radius. It does not prove that the action inside the boundary matches what the human approved.

An agent can execute the wrong command in the correct sandbox. It can write to an unexpected resolved path that still exists inside the worktree. It can use an untrusted skill with tightly scoped permissions. Containment may turn a severe failure into a recoverable one, which is worth doing, but approval integrity is a separate property.

Treat the two controls as complementary:

the approval envelope defines the authorized effect
the execution boundary limits what can happen if the action, dependency, or runtime goes wrong

A worktree is useful for filesystem isolation and review. A sandbox can restrict process, network, and credential access. Neither should be presented as proof that the reviewer saw the exact operation that ran.

The approval UI should show facts, not confidence

Most approval interfaces spend too much space on the model's prose. "I will safely update the generated client and run the relevant tests" sounds reassuring, but it is not specific enough to authorize anything.

Give the computed fields the visual priority:

Requested path: packages/web/generated/client.ts
Resolved path:  /workspace/task-184/packages/web/generated/client.ts
Operation:      write
Capabilities:   one-file write, test command
Skill:          api-client-generator@2.4.1
Expires:        06:15 CST
Evidence:       changed paths, diff, generated-client test

Show a warning when requested and resolved resources differ. Make capability expansion obvious. Put the tool or skill revision near the approval control instead of burying it in a log panel. If the system cannot compute a field, label it unknown rather than filling the gap with the model's confidence.

This interface problem is getting more relevant as agents render their own controls and task-specific surfaces. Builders comparing component catalogs, MCP UI tools, renderer patterns, and trusted-rendering approaches can use Generative UI resources as one reference set. Whatever stack you choose, keep the authorization facts under host control; the agent should not get to invent the fields that authorize its own action.

Low-risk operations can still use broader policy approval. Formatting Markdown in a disposable branch does not need the same ceremony as rotating credentials or modifying a release workflow. The broader policy must still name its resource and capability bounds. "Do harmless things" is a preference, not an enforceable policy.

Execution needs a receipt

Approval records intent. A receipt records what happened.

At minimum, preserve the proposal identity, actual changed resources, tool calls, exit state, policy exceptions, and the evidence shown to the reviewer. For repository work, that often includes a diff and test output. It may also require checking effects that ordinary git status does not reveal.

This is where path-resolution failures become visible. Suppose the approved proposal names one resolved destination, but the receipt shows another. The system should not call the run successful and tuck the mismatch into a warning. It should mark the transaction failed or require a new approval before continuing.

The same rule applies when execution discovers that it needs more authority. If a one-file edit now requires a package install and outbound network access, stop. Prepare another proposal. Do not stretch the first approval until it covers whatever the agent wants next.

Receipts also make review state durable. Parallel tasks can resume without relying on a conversational summary of what supposedly happened. A reviewer can inspect the approved operation, the actual effect, and any divergence between them. Current developer skepticism around coding agents often lands here: humans still have to reconstruct and review the accepted result. Better evidence does not eliminate that work, but it makes the work bounded.

Consent should fail closed when the proposal moves

The original dialog did not need a longer explanation. It needed a resolved destination, a bounded capability, a named skill revision, an expiry, and a promise about the evidence that would follow.

Once those facts were bound to one proposal, any material change should have invalidated the click. After execution, the receipt should have proved whether the runtime honored the proposal.

This model will not stop every prompt injection, dependency attack, credential leak, or runtime exploit. It does enforce one useful property that a plain confirm button cannot: the human approves an action the system can identify, constrain, and compare with the result.

Do not ask people to approve a sentence the agent wrote about itself. Ask them to approve the exact operation the runtime is prepared to enforce.

Source notes

Your AI Coding Stack Needs Change Control, Not More Tuning

hefty — Sun, 23 Aug 2026 06:38:59 +0000

Your AI Coding Stack Needs Change Control, Not More Tuning

A coding agent has one bad run. The team swaps the model, rewrites the system prompt, adds an MCP server, upgrades the harness, and changes the context rules. The rerun passes.

Everyone is relieved, but the result taught the team almost nothing. Nobody knows which change helped, whether the new setup costs more to operate, or whether it will survive a different task. Rolling back is awkward because five moving parts now depend on one another.

An AI coding stack needs change control. Each adjustment should be a bounded experiment with a baseline, one named variable, visible human and machine costs, and a decision at the end.

An upgrade is still a configuration experiment

The model is only one layer of the stack. Provider settings, harness versions, system instructions, MCP tools, repository memory, context selection, approval rules, and sandbox policy can all change the result.

When several layers move together, a successful run cannot tell you which layer caused the improvement. The same pile of changes also makes a later regression harder to diagnose. You have traded one uncertain configuration for another.

Start with a small task set drawn from work the team performs. A useful baseline task has a known starting state and an observable outcome. That might mean a named test turns green, a browser flow behaves correctly at specified viewports, or a patch satisfies an existing API contract without touching unrelated packages.

Forget the universal benchmark. Pick a task that exposes the failure this configuration change is supposed to reduce.

Change one layer at a time when that is practical. If a migration forces the model, provider adapter, and prompt format to move together, record them as one bundle. The bundle can still be evaluated, but it cannot support claims about which internal change deserves credit.

Preserve the decision, not only the latest config

Most configuration files describe the stack as it exists now. They say little about why it exists, what it replaced, or which evidence justified the last change. Overwriting the file also overwrites part of the team's operational memory.

A small, append-only stack-change record fixes that. Preserve the previous state, add corrections as later entries, and record both when an observation was made and when the configuration was active. The format can stay boring:

hypothesis: What specific failure should this change reduce?
baseline_task: Which fixed task and starting state will be compared?
variable: Which layer or explicitly named bundle is changing?
acceptance_checks: What observable result counts as success?
outcome: What happened against those same checks?
machine_cost: Wall time, requests, billed usage, and cache behavior when available
human_cost: Steering time, review time, and reconstruction work
regressions_and_uncertainty: What broke, what was not measured, and what weakened the comparison?
decision: keep | revert | inconclusive

Treat this as a working template. It makes the next decision explainable and reversible; it does not pretend to be an industry standard.

The record should not decide for the team. Evidence can support a choice without becoming an automatic enforcement rule. A test result may prove that one behavior works under one setup; it does not prove that the configuration is better for every repository task.

Do not let the changed stack grade itself

Write the hypothesis and acceptance checks before running the new configuration. Otherwise it is easy to inspect whatever the agent happened to produce and invent a success criterion afterward.

Keep the pre-run plan, execution artifacts, review, and final interpretation distinct. The changed stack can collect command output, diffs, traces, and browser observations. It should not turn its own confident summary into the pass condition.

This separation also makes failures useful. A run that misses the acceptance check may still reveal a broken tool boundary or a context problem. Preserve that evidence, then let the reviewer decide whether it explains the failure. Collapsing execution and judgment into one chat transcript makes the persuasive final paragraph compete with the underlying facts.

The checkpoints can stay small. Before execution, the record needs a hypothesis, a clean baseline, and acceptance checks. Before a keep decision, it needs the resulting patch or behavior, the evidence behind it, and a review performed against the original checks. Missing evidence should produce an inconclusive result rather than a story about why the run probably worked.

Count the costs the dashboard misses

A configuration can generate more code and still make the workflow worse.

Machine cost includes more than a token total. Record elapsed time, request count, billed usage, retries, and cache behavior when the provider exposes it. A recent issue for one Codex-on-Bedrock configuration reported unexpectedly high cache-write spend when the desired cache controls were unavailable. That report belongs to one provider path, not every coding-agent setup, but it shows why model names and aggregate token counts are too coarse for configuration work.

Human cost is easier to hide. Count the time spent steering the run, checking its claims, and reconstructing code the reviewer no longer understands. One practitioner account describes AI-assisted output increasing while review became more cognitively expensive and code understanding weakened. That is personal experience rather than a controlled productivity result, but the cost category is worth tracking in a team's own work.

Reviewer attention varies with task difficulty and familiarity, so fake precision will not help. A short note such as "the reviewer had to trace three unrelated packages to validate the patch" can be more useful than a made-up score. The comparison only needs enough detail to show whether the new setup moved work out of the agent and back onto the human.

Run the comparison on a real repository task

Suppose a team wants to change its context-selection policy for a monorepo. The hypothesis is that a generated dependency map will reduce incorrect edits outside the target package.

The team chooses a known pagination regression from a clean commit. The acceptance checks are already available: the contract tests must pass, an existing query-count limit must hold, and the patch must not modify packages outside the API path. The model, harness, tools, and approval settings stay fixed. Only the context policy changes.

The record now has something concrete to compare:

Did both configurations satisfy the same contract tests?
Did either run edit unrelated packages or require manual repair?
How many retries and billed requests did each run use?
How long did a reviewer need to understand and validate each patch?
Did missing telemetry or environmental differences make the comparison weak?

Those questions do not collapse into a universal score. They support a local decision. If the new policy passes the checks and lowers review reconstruction without introducing another cost, the team has a reason to keep it. If it fails a behavior the baseline handled, revert it. If the environment changed halfway through or the provider omitted the needed cost data, mark the result inconclusive and run a cleaner test only if the expected value justifies the time.

Use the same discipline for larger migrations. Name the bundle, freeze what can be frozen, preserve the old configuration, and narrow the claim. A successful provider migration can justify adopting that bundle without proving that every model, prompt, and cache setting inside it is optimal.

End every change with a decision

Configuration experiments tend to linger. A new MCP tool remains enabled because it might help later. A longer prompt survives because one run looked good. A second harness stays installed while the team debates which one to use. Each unresolved experiment adds another branch to the operating environment.

Close the record with one of three decisions:

Keep the change because it beat the baseline on the declared outcome without an unacceptable regression.
Revert it because it failed the checks or moved too much cost onto the reviewer or infrastructure.
Mark it inconclusive because the comparison was not strong enough to support either choice.

Do not preserve a change merely because it "feels better." Put that observation in the notes and design a task that can expose the suspected improvement. If the expected benefit is too small to justify another test, the old stable configuration wins by default.

Timebox the tuning work as well. Current developer discussions describe model, harness, MCP, and prompt optimization becoming a job of its own. That is anecdotal sentiment, but the failure mode is easy to recognize: the stack consumes the time it was meant to return.

The team in the opening example should have saved its starting configuration, named the failure, changed one layer, and compared the rerun with the same checks and costs. Without that record, a passing run is just a pleasant result. It is not evidence that the stack improved.

If a configuration cannot beat a stable baseline on real work, revert it. Otherwise the next failure will be harder to explain than the last one.

Source notes

Your First Coding Agent Should Be a Reviewer, Not an Author

hefty — Wed, 19 Aug 2026 11:30:00 +0000

Most teams introduce a coding agent backwards. They hand it a ticket, let it edit the repository, and only then ask a human to figure out whether the patch is correct.

The model gets the fun job. The reviewer pays the reverse-engineering tax.

Start with review instead. If an agent can find a concrete problem, attach evidence, expose its uncertainty, and give a human a cheap way to confirm the result, it has already created value. It can prove that value before the team makes code authorship its default role.

Review can pay before the agent writes code

A recent TechRadar report about Linux kernel work draws a useful line. It describes AI review and analysis agents surfacing actionable bug reports, while humans write the fixes and maintainers retain judgment over what lands.

That is one reported workflow, not proof that AI review is universally accurate or cheaper. The role split is still worth examining: the agent can reduce uncertainty without owning the patch.

Authorship bundles several questions together. Did the agent understand the bug? Did it choose the right fix? Did it preserve unrelated behavior? Did its tests prove what it says they proved? A generated patch forces the reviewer to answer all of them at once.

A review finding is narrower. The agent can point to one behavior and the evidence behind it. A human can confirm or reject that finding without first untangling a proposed implementation. False positives still cost time, but a compact, falsifiable report limits that cost.

The first test is whether the agent can make a decision easier before it starts changing code.

"Reviewer" is a workflow, not a prompt

Telling a model to "review this repository" is barely more specific than telling it to "write good code." Useful review needs a bounded surface, an environment that can produce evidence, and an output that another person can inspect.

Proliferate is a concrete product signal here. Its project documentation lists isolated worktrees or sandboxes, diff review, and reviewer agents as parts of the operating surface for multiple coding harnesses. The README does not prove that those reviewers are accurate or production-safe. It does show that review and isolation are becoming explicit product features rather than chores left outside the agent loop.

The surrounding controls matter as much as the reviewer model:

Give it one commit, diff, subsystem, or execution path to inspect.
Let it run named checks in an isolated environment when the finding depends on behavior.
Preserve test output, traces, browser observations, and repository history with the report.
Require it to stop when missing context prevents a defensible finding.

The same operational concerns appear in practitioner guidance about production coding agents: tests, Git history, prompts before destructive actions, explicit verification, and knowing when to stop. Those are not benchmark results, but they are better selection criteria than raw generation quality.

A reviewer with unlimited scope and no stop condition will produce commentary. A reviewer with a defined surface and inspectable evidence has a chance to produce a decision.

Require a review receipt

A confident paragraph is not a review artifact. "There may be a serious race condition" gives the next person a claim to investigate and no cheap way to investigate it.

Require a small receipt for every finding:

Finding: one concrete issue rather than a broad quality opinion.
Location: a file and line, execution path, or observable behavior.
Evidence: a test, trace, command output, diff, or browser observation another person can inspect.
Impact: what can fail and under which conditions, without inflating severity.
Uncertainty: assumptions, missing context, and plausible alternative explanations.
Next check: the smallest action that confirms or rejects the finding.

This is a proposed working format, not an industry standard. It makes a weak finding cheap to reject.

Consider a hypothetical service with a role-change endpoint and cached authorization. A useful agent report could look like this:

Finding: A removed project member can retain the previous permission until the cache entry expires.
Location: The role-update path returns before the existing authorization cache is invalidated.
Evidence: After a successful role removal, the named integration scenario still receives a 200 response for the protected project request. The trace and test output are attached.
Impact: Access may remain stale for the configured cache lifetime after this update path.
Uncertainty: The report has not established whether a background invalidation event is expected to close the gap.
Next check: Run the same scenario with the invalidation worker enabled and inspect whether it receives the role-change event.

The agent has not proved the architecture is insecure, and it has not earned permission to rewrite the cache layer. It has produced a specific claim with a short rejection path. A maintainer can inspect the trace, check the event flow, and decide whether a patch is needed.

A review that cannot fill the receipt should not silently graduate into an agent-authored fix. It should return the missing evidence or context.

Domain context belongs inside the evidence loop

Frontend review makes the weakness of vague judgment obvious. "This page looks wrong" is not a finding. Neither is "make it feel more polished."

The reviewer needs a selected reference for the intended interaction, a named design-system constraint, a browser observation under stated conditions, and a next check. For generated interfaces, a curated set of Generative UI cases can help a team choose relevant examples and translate them into its own observable criteria.

The examples are reference material, not proof and not an automated acceptance oracle. The team still has to state what matters in its product. That might be keyboard order, component usage, overflow at a named viewport, or the behavior of a streamed state transition. The agent's job is to compare the implementation with those declared constraints and show the mismatch.

This is why domain context should be attached to the review task instead of hidden in a prompt like "use good UX." References become useful only after the team turns them into checks that can fail.

Let the agent climb the role ladder

Authorship does not need to be the starting position. Increase the agent's ability to mutate the repository only after its output is useful at the previous step:

Explain a bounded area of the code and cite the relevant paths.
Reproduce one reported behavior without changing the repository.
Review a selected diff or execution path and produce a receipt.
Suggest a patch inside an isolated workspace, with the finding and evidence kept beside it.
Author a bounded change when the team can evaluate both the diagnosis and the result.

Advancement should stay local to the workflow. An agent that reviews dependency updates well has not earned a global trust score for authentication changes. A good result on a backend test does not prove it can judge a generated interface.

Ask practical questions at each step. How many findings did maintainers confirm? How quickly could they reject the wrong ones? Did the report remove uncertainty, or did it create another document to decode? These are evaluation questions, not universal metrics. Teams can answer them with their own repository history instead of importing a multiplier claim.

Authorship then becomes a downstream privilege. The agent already knows how to locate evidence, state uncertainty, and stop when the task exceeds its context. The patch is connected to a diagnosis that survived review rather than appearing as the first artifact in the process.

More output will not repair expensive review

Current developer discussion is skeptical of generation volume without validation. A Hacker News thread built around the "2x, not 10x" framing pushes toward experiments, constraints, and review. A Reddit post describes spending more time tuning models, harnesses, MCP tools, and prompts than completing the work the stack was meant to help with. Both are community sentiment, not measured productivity evidence.

That frustration suggests a narrower move. Adding another authoring agent can increase the amount of code waiting for judgment. A focused reviewer can work on the expensive part directly by making one claim easier to inspect.

The team that started by granting write access needed a finding a human could confirm, not a patch the human had to decipher. Once the agent can repeatedly make review cheaper for a named workflow, letting it propose the fix becomes a smaller step.

If it cannot make review cheaper, letting it generate more code only increases the bill.

Source notes

Give your coding agent a reproducible failure, not a vague prompt

hefty — Sun, 16 Aug 2026 06:06:48 +0000

"Fix checkout" is enough to start a conversation. It is a terrible unit of work for a coding agent.

The agent can inspect the repository, find something suspicious, write a plausible patch, and report success. The reviewer still has no shared answer to two basic questions: what exactly was broken, and what observable behavior should now be different?

That gap matters more as implementation gets cheaper. A coding task should begin with a known before-state and a declared after-state. The patch is only an attempt to move the system between them.

A task is a state transition, not a sentence

A vague prompt usually contains intent:

Fix the checkout timeout when a session expires.

It does not contain a task contract. Which revision has the bug? Which fixture exposes it? Does the request hang, return the wrong status, or discard the cart? What behavior must remain unchanged? Which command evaluates the result?

Without those details, the agent has to invent part of the acceptance test while implementing the fix. That is a bad delegation boundary. The same system is choosing what "done" means and announcing that it got there.

A reviewable task has two observable states:

The starting state can be recreated before any edit.
The target state can be evaluated after the edit without trusting the agent's summary.

The agent can choose the implementation between those states, while the task author owns the surrounding contract.

This framing also keeps the human out of low-value supervision. You do not need to narrate every code change if you have already defined the behavior, constraints, and evidence that will make the change acceptable.

Start from failure on purpose

CLI-Gym is a useful example of failure-first task construction. Its project documentation describes a pipeline that begins with healthy repository environments, introduces controlled failures, and turns those states into reproducible command-line tasks. The README reports 1,655 tasks from 29 repositories.

Those numbers are project-authored, and they do not prove benchmark quality or production reliability. The task design is more useful here than the headline count.

A controlled failure gives the agent and reviewer the same starting point. It can be reset. It can be observed before the patch. It can be checked again afterward. If the failure disappears for reasons unrelated to the patch, that becomes visible too.

Compare that with dropping an agent into a moving branch and saying, "The build is weird, please fix it." The agent may find a real problem. It may also repair an incidental symptom, update a flaky snapshot, or change enough code that nobody can tell which behavior mattered.

Failure-first does not mean every assignment needs an existing failing test. Feature work often starts without one. In that case, define an observable before-state and a concrete acceptance example. A missing endpoint, an unsupported interaction, or a fixture that currently produces no result can still anchor the transition.

Put the contract in an artifact

Chat is convenient for exploration. It is weak as the only record of a coding task.

A durable setup keeps intent, scope, checks, and evidence in artifacts that survive the session. A task file can be reviewed before execution. A plan can be compared with the allowed scope. Test output and the final diff can stay attached to the same work unit.

This changes the review from "Does the completion message sound convincing?" to a comparison:

Did the run begin from the declared base state?
Does the original reproduction now reach the target behavior?
Did the diff stay inside the stated boundaries?
Which named checks ran, and which did not?
Did the agent stop when the task became ambiguous?

Treat a confident completion paragraph as context, then verify it against the artifacts.

The contract should also remain implementation-neutral where possible. "Add a retry around refreshSession()" may be a reasonable plan, but it is already prescribing a patch. "An expired session returns the typed authentication error within the configured timeout and preserves the cart" gives the agent room to inspect the code before choosing a change.

A six-field task card

You do not need a large specification system to get this benefit. A compact task card is enough for many repository changes.

The following example is hypothetical. The commands and fixture names stand in for whatever your repository actually uses.

Base

Revision: <commit SHA>
Fixture: expired-session.json
Environment assumption: the local test services start with the documented setup command
Initial state: the cart contains one item and the session token is expired

Reproduce

Run pnpm test checkout -- --grep "expired session"
Current result: the checkout request waits until the test timeout
Save the pre-change failure output with the task

Target

Checkout returns the repository's existing typed authentication error before the configured timeout
The cart remains intact so the user can authenticate and try again

Boundaries

Do not change the session lifetime
Do not add a dependency
Preserve checkout behavior for valid sessions
Keep unrelated formatting and refactors out of the diff

Checks

Re-run the exact reproduction command
Run the existing checkout test group
Run the repository's type check
Inspect the final diff against the boundaries above

Stop

Hand the task back unresolved if:

the failure does not reproduce from the declared base
the fixture or local setup is broken
the expected result conflicts with the current API contract
a required service or credential is unavailable
the fix needs a wider product or architecture decision

Omitting the Stop section leaves the agent with no defined way to return an unresolved task. An agent that has only success criteria will keep searching for a patch. Explicit stop conditions let it return a useful failure report before it widens the change or guesses at product intent.

Passing checks is evidence with a boundary

Named checks make the transition reviewable. They do not certify the entire system.

A green targeted test says something about the behavior encoded in that test. It does not prove that the architecture is healthy, that the change is secure, or that every product expectation has been captured. A reproducible failure narrows uncertainty around the named behavior while leaving architectural and product questions for review.

The final diff still needs review. Compare the code with the declared target and non-goals, then decide whether broader tests or specialist review are needed. A two-line fix in an authentication path may deserve more scrutiny than a larger change in an isolated internal tool.

The task card helps make that judgment legible. It shows what the checks were supposed to prove and where their authority ends.

Let an unresolved run count as a valid result

The Hacker News and Reddit threads in the source notes are anecdotal, but some participants favor small, auditable loops: inspect, change, run a real check, review the diff, and preserve enough state for the next step. Tool preferences vary, while every unattended run still needs a clear handoff.

A run that reports "cannot reproduce from the named revision" has produced useful information. So has one that finds contradictory acceptance checks or a missing service. Those results protect the repository from a guessed patch and give the task author a specific problem to resolve.

Treating every run as patch-or-failure creates pressure to hide uncertainty. Treating a justified stop as part of the contract makes the workflow more honest. The agent can state what it inspected, where the transition broke down, and what decision or dependency is missing.

That is a better result than green output from a check nobody agreed would define success.

Define the transition before asking for the patch

Return to "fix checkout." With a base revision, fixture, reproduction command, behavioral target, boundaries, checks, and stop conditions, the assignment becomes reviewable before the agent writes a line of code.

The agent may discover that the expected fix is tiny. It may find that the task is underspecified. Both outcomes are easier to handle when the before-state and after-state are explicit.

A stronger model can propose a better implementation. It cannot recover a success condition the team never wrote down.

Source notes

A coding agent needs an authority budget, not just a better model

hefty — Fri, 14 Aug 2026 06:43:32 +0000

A coding agent can produce a patch before your team has agreed on what it was allowed to read.

That is a bad trade. A fast patch is not useful if nobody can explain which files shaped it, which tools it called, what data it retained, or what evidence makes the result trustworthy. The model may be capable. The workflow can still be unusable.

I think about this as an authority budget. The budget is the set of permissions and obligations around an agent: what it can read, write, call, retain, ask approval for, and prove afterward. The model matters, but it is only one part of the operating design.

Model quality is only one axis

Coding assistants now occupy different workflow lanes. Some live in an IDE. Some work in a terminal, browser, or pull-request flow. Privacy, team controls, cost, and the amount of context available also vary by tool.

That variety makes a universal ranking less useful than it looks. Choosing an agent is also choosing a permission surface.

An IDE assistant may see the file currently open and a slice of the repository. A terminal agent may be able to run commands and modify a wider set of paths. A browser-oriented workflow may have a different kind of access again. Each one creates a different review problem.

The question is not only, "Which model writes the best code?" It is also:

What context does this workflow expose?
What can the agent change without asking?
Which external systems can it call?
How does a reviewer know what happened?

A model comparison can help with the first question. It does not answer the rest.

Reading, writing, and reviewing are different powers

The Pair documents a simple split between a Mentor, which acts as a read-only reviewer, and an Executor, which performs the writing work. Whether that project catches more mistakes is a separate question. The design still makes an important permission distinction visible.

Inspection and mutation do not have to be the same authority.

That distinction is easy to lose when one agent can inspect a repository, edit files, run commands, and summarize its own work. The same system that made the change can also decide that the change looks fine. A successful response then becomes a weak substitute for independent review.

A stricter workflow can separate the powers:

One step gathers context and proposes a change.
Another step applies the change inside a bounded workspace.
A reviewer, human or automated, inspects the diff and the required checks.

This does not make the review correct by magic. It does make the review boundary explicit. Read access is not write access, and write access is not approval.

The harness around the model is part of the product

Google's developer highlights describe persistent isolated environments and Managed Agents as part of its developer tooling. The release signal matters because it treats execution space, persistence, and agent lifecycle as product concerns rather than details hidden behind a chat box.

The Product Hunt page for prjct/ takes a similar direction from a different angle. Its product positioning combines intent briefs, bounded repository context, persistent memory, guardrails, and evaluations around coding agents.

Those pages describe vendor or product-authored capabilities. They are not independent validation, and isolation or persistence does not automatically make a workflow safe. They do show where the engineering surface is moving: the wrapper around the model increasingly determines the context, tools, memory, and checks that shape a run.

That wrapper deserves the same design attention as the prompt.

If an agent has persistent memory, decide what enters it and how long it stays there. If it runs in an isolated environment, decide which credentials, network routes, and repositories can still reach the environment. If it has guardrails, make the restricted actions visible and test the failure path.

A label such as "sandboxed" is not a permission model.

Why teams say no

A recent Reddit discussion about a workplace that did not allow AI agents is anecdotal, not a measurement of how teams behave. It is still useful because the objections are concrete. People discuss code exfiltration, data retention, residency, and how a reviewer can examine agent-produced changes.

Those are authority questions.

A developer may want help with a repository. The organization may need to know whether source code leaves its boundary, whether prompts or logs persist, where artifacts are stored, and whether a human can reconstruct the change. Productivity does not remove those obligations.

This is why "the model is good enough" rarely settles an adoption decision. A team can accept the coding capability and reject the surrounding authority. It can also approve a narrow local workflow while refusing broad repository or production access.

Write the six lines before widening access

The authority budget does not need to be a grand governance program. Start with six explicit lines in the task definition.

Read. Name the repositories, directories, files, secrets, and external context the agent may see. Keep sensitive material out of scope by default.
Write. Name the paths and branches the agent may change. Make generated artifacts and protected files clear.
Call. List the commands, tools, network destinations, and services available during the run. Treat a new network permission as a design change.
Retain. Decide where prompts, source snippets, memory, logs, diffs, and output artifacts persist, and for how long.
Approve. Mark the actions that require a person or a policy gate, such as changing dependencies, accessing production, or widening the workspace.
Prove. Require the diff, named check results, relevant event or log evidence, and an explicit failure state before the run counts as complete.

The last line is where many workflows become honest. A process exit of zero is not the same as an accepted change. The agent's statement that it finished is not the same as a passing test suite. A useful result should let the next reviewer see what changed, what ran, what failed, and what the agent left untouched.

For an automated coding task, I would start with a read-only repository slice or a narrow writable path. Then I would add one permission for one named workflow need. If the agent needs network access, record why. If it needs persistent memory, define its contents and lifetime. If it needs to edit a new directory, make that expansion reviewable.

This keeps the failure radius small while the team learns how the workflow behaves.

Spend authority only when the run can show its receipt

Better models will make agents more useful. They will not decide what a team should expose to an agent or what evidence should unlock the next step.

That decision belongs in the workflow contract.

The better coding agent is often not the one with the most impressive demo. It is the one whose authority a team can explain, whose changes fit inside that authority, and whose result leaves a receipt that another person or system can review.

Source notes

Your Coding Agent's CLI Should Not Be Your Automation API

hefty — Sat, 08 Aug 2026 02:42:53 +0000

Picture a CI job waiting on an interactive coding agent. The process streams reassuring prose, asks a question nobody can answer, edits a few files, and eventually prints that the task is complete. It exits with code 0.

The pull request still cannot safely move forward. The machine does not know whether the agent changed the intended files, ran the required checks, skipped a step, or quietly gave up after producing a plausible explanation.

A terminal can be an excellent surface for a developer and a terrible contract for automation. Humans can interpret a conversation, clarify a request, and notice when the agent has wandered. CI needs explicit state and artifacts.

Keep the CLI for operators. Put a small task, session, and result boundary between that CLI and the rest of the delivery system.

One agent has two very different consumers

Coding tools now show up as editor assistants, terminal agents, browser builders, pull-request bots, and API-driven services. That variety is useful because the workflows are different. An engineer exploring a codebase needs a different interaction from a CI job reviewing every new pull request.

The mistake is making every downstream system learn the conversational habits of whichever agent happens to be installed today.

An operator can...	Automation needs...
read streaming prose	structured status
answer a follow-up question	an explicit approval state
decide that a partial result is useful	named acceptance checks
interrupt when the task drifts	cancellation and timeout semantics
inspect the working tree afterward	a declared result artifact

Terminal formatting changes. Prompts change. Agent versions add new questions and new modes. If a pipeline depends on matching those details, the pipeline is coupled to an interface designed for a person.

That coupling is usually invisible until the first unattended run hangs on a prompt or reports success without producing the artifact the next step expected.

Put a small session boundary above the process

AgentAPI is one concrete version of this architectural move: it places an HTTP control layer in front of several coding-agent CLIs and documents use cases such as pull-request review and MCP integration. The project makes the integration seam concrete without proving that every agent behaves identically.

An automation boundary can stay small. It needs enough structure to answer questions such as:

What task did the caller submit?
Which session owns the work?
Is the run queued, active, waiting for approval, complete, failed, timed out, or canceled?
What may the caller retrieve when the run ends?

HTTP may fit a service. A local supervisor could expose the same contract through files, a socket, or a job queue. Either implementation can give machines structured state instead of terminal prose.

Here is an illustrative task request, not a proposed standard:

repository: payments-service
goal: add validation for expired checkout sessions
constraints:
  writable_paths:
    - src/checkout/**
    - tests/checkout/**
  network: disabled
acceptance:
  - npm test -- checkout
  - npm run lint
outputs:
  - patch
  - check-results
  - review-summary

The adapter translates that request into whatever the selected agent understands. It also translates process behavior back into session state. The workflow above the adapter should not need to know whether an agent prints a spinner, opens a full-screen terminal interface, or phrases a permission request differently after an update.

A structured boundary cannot make the output correct. It can make the run observable enough for another system to decide what happens next.

Version the work definition beside the code

Private prompt recipes are a bad foundation for team automation. Nobody can review when the recipe changed, which repository assumptions it contains, or why one engineer gets a different result from another.

Auggie combines an interactive terminal agent with codebase-aware custom commands and GitHub Actions workflows. That arrangement keeps repeatable work definitions in team-owned, versioned project context.

A repository task might define:

the files an agent may inspect or modify
required checks and their timeout
commands that always need approval
the artifacts a review job must return
the conditions that turn a partial run into a failure

Reviewing those rules does not guarantee good agent output. It does something more basic: it makes changes to the automation visible. A pull request can show that a team widened a writable path, removed a check, or changed the expected artifact before that decision reaches every future run.

The agent still gets room to reason inside the task. The delivery system keeps ownership of the boundary around it.

A successful process exit is not acceptance

Agent automation often collapses three different events into one:

the agent process stopped
the agent claimed it finished
the requested work passed its acceptance checks

Only the third event should unlock the next delivery step.

Intent is positioned around a lifecycle that starts with a feature description and continues through isolated work, verification, and delivery. That is a product-authored description, not independent proof of the workflow. It still points at the right integration question: what evidence crosses the boundary when the work is done?

For a coding task, a result contract may require:

a patch or commit containing changes within the allowed scope
the exit status and output of named checks
a short summary of changed and intentionally untouched areas
a machine-readable failure reason when an expected artifact is missing

Logs help with diagnosis, but dumping a transcript is not a result contract. The downstream reviewer should not have to replay a conversation to discover whether tests ran. Give it the diff, the checks, and the bounded summary directly.

Partial work needs the same precision. If the agent changed the code but a required test timed out, preserve the patch and return a failed acceptance state. The reviewer keeps the useful artifact without receiving a false success signal.

Normalize less than you want to

Once a team has two agents behind one boundary, the temptation is to design a universal agent protocol. That abstraction gets expensive quickly.

Agents differ in context handling, review modes, tool controls, permission models, and repository awareness. Flatten all of that into a lowest common denominator and the wrapper can erase the feature that justified choosing a particular agent.

Normalize the boring parts:

task input
session identity and lifecycle
execution scope
cancellation and failure
required result artifacts

Keep native capabilities behind deliberate, named extensions. A workflow that opts into an agent-specific review mode should say so. Portability is useful only when the contract tells the truth about what is portable.

Direct CLI use also remains the right choice for interactive local work, experiments, and one-off tasks led by an operator. There is no need to turn every terminal session into a service. The boundary becomes necessary when another machine depends on the outcome.

The five-part contract

Before wiring an agent into CI or an internal tool, define five things.

1. Task contract

Name the repository, goal, constraints, and acceptance checks. Avoid hiding durable requirements in prompt prose that only one caller knows.

2. Session contract

Represent start, progress, approval, cancellation, timeout, and failure as explicit states. Do not infer them from terminal output.

3. Execution contract

Declare the workspace, writable files, available tools, network access, and approval boundaries. An API wrapper cannot compensate for an undefined authority model.

4. Result contract

Require the diff, test result, log, summary, or other artifact that lets the next system evaluate the run. Process completion and task acceptance are separate facts.

5. Capability escape hatch

Allow a workflow to request a native feature on purpose. Mark that dependency instead of pretending the workflow will behave the same on every agent.

These five parts are a practical review checklist, not an industry standard. A small team may implement them with a local supervisor and a result directory. A platform team may use an API, job queue, and artifact store. Both implementations can enforce the same narrow contract.

Make the run reviewable first

Before adding a coding agent to an automated delivery path, the team should be able to answer five questions:

What exactly starts the task?
How does the caller observe, cancel, or time out the session?
Which resources and permissions bound the run?
Which artifacts and checks prove acceptance?
Which parts depend on a native agent capability?

If the answers live only in a prompt and a terminal transcript, the system is automating a conversation rather than relying on an API.

An honest boundary may also make the agent easier to replace. More immediately, it lets reviewers judge the work without having to trust the agent's final sentence.

Source notes

Local AI Is an Architecture Decision, Not a Hardware Flex

hefty — Tue, 04 Aug 2026 03:47:48 +0000

The local AI demo loop is seductive. Get a model to fit on a laptop, show it running, and declare the hard part solved.

The product review is less fun. How much memory can it consume? Where do intermediate files go? Can a user edit the result? What may the agent touch? How does anyone know the job finished correctly?

"It runs locally" answers none of those questions. It tells you where one computation happened.

Local-first has to survive the entire workflow. The model call, interface, files, tools, approvals, exports, and completion evidence all need boundaries. Miss one, and the architecture can quietly contradict the label on the box.

Locality has six boundaries

Before choosing a model or framework, write down this contract:

Resource boundary: What memory, compute, storage, and wait time may the feature consume?
Residency boundary: Which inputs and intermediate artifacts stay on the device? Which network dependencies remain?
Handoff boundary: What editable artifact does generation return?
Interface boundary: Which state transitions and behaviors are explicit and testable?
Authority boundary: What may the agent read, change, call, or publish without approval?
Evidence boundary: What report, diff, export, trace, or test result proves the step completed?

This is not a vendor standard. It is an architecture-review shortcut. If a team cannot answer one of these questions, it has found an unresolved product decision rather than a documentation problem.

Start with the resource envelope

Local-model discussions tend to collapse into a single number: how large a model can fit into how little VRAM.

Projects such as AirLLM make that constraint visible by documenting very large model execution under constrained VRAM. That is useful engineering. It still does not tell a product team whether a feature is usable.

"Can run" and "can support this workflow" are different claims. The latter needs decisions about:

acceptable wait time for an interactive action
storage available for model weights and generated artifacts
what else must run on the device at the same time
how the feature behaves when the resource ceiling is reached
whether the application queues work, degrades the capability, or stops visibly

Those are routing and UX decisions. A model that technically loads may still be wrong for a hot interaction path. A slower local path may be perfectly reasonable for a queued task with a visible progress state. The hardware limit does not settle the product design; it forces the product design to become explicit.

The residency boundary needs the same precision. A model can execute on-device while telemetry, retrieval, authentication, asset storage, or a later processing step still uses the network. That may be an acceptable system. It just should not hide behind a broad "local" claim.

Map every input, intermediate artifact, and outbound request. Then describe the boundary you actually built.

Generation is not a handoff

A raw model response is rarely the end of a real workflow.

MiniMax H3's ComfyUI integration moves the conversation one step past model availability. The announcement pairs an open-weight multimodal model, native stereo audio, and 2K video with a local, optimized ComfyUI path. The model matters, but so does the operator workflow around it. Generation becomes more useful when the result can move through inspection, revision, export, and the next production step.

A good handoff is editable and boring. It might be a video plus audio tracks, a still image, a project graph, a structured specification, or a set of exported files. The right artifact depends on the job. A final preview with no recoverable state is usually a dead end wearing a nice UI.

Consider a generated thumbnail or video still. The generation step may be local, but publishing still needs deterministic dimensions and an export the creator can inspect. For an Instagram asset, Resize Image for Instagram provides browser-local fit, fill, preview, and export across square, portrait, landscape, Story, and Reel presets. That is a small step, but it preserves the point of the workflow: the image remains an artifact the user can see and control before publishing.

Here is the harsher handoff test: remove the model after generation. If the user cannot continue with what it returned, the feature produced a dependency on another model call, not a durable artifact.

Local-first includes the frontend

Developers sometimes apply rigor to model placement and then treat the interface as a screenshot around a prompt box.

This gets the priorities backwards. A local system has more visible resource limits, longer-running operations, partial results, and more failure states to explain. Its frontend needs a stronger state model, not a prettier loading animation.

Bonsai takes the opposite route. It models reactive web applications with functional state machines, incremental recomputation, and executable DOM behavior tests. That does not mean every local AI product should use Bonsai or OCaml. It shows what "explicit" looks like: state has an owner, transitions have rules, and behavior can be exercised rather than admired in a screenshot.

The same distinction matters in Generative UI. A catalog of Generative UI resources can help a team compare schema-driven renderers, trusted component catalogs, protocols, and implementation patterns. Those examples are design input. They do not prove that a generated interface preserves state, handles interruption, respects permissions, or returns the right action to the application.

Treat these as separate jobs:

inspiration decides what kinds of interface might be useful
generation selects or composes an allowed surface
the application owns state and permissions
behavioral tests prove the important transitions

For a long local task, the interface should be able to distinguish queued, loading, generating, awaiting approval, exporting, completed, interrupted, and failed states when those states exist in the workflow. Do not flatten them into one spinner and a hopeful success toast.

A plausible screen is not proof. The user needs to know what happened, what can still change, and what the system expects next.

Local execution does not grant safe authority

Where a model runs and what an agent may do are separate decisions.

Nightcrawler's documented design keeps those questions apart. It puts a scope proxy, small-step loop, dashboard, and structured report around a local mobile agent. Together, those pieces define the allowed surface, expose progress, and leave something an operator can inspect afterward.

The project documentation is not an independent security audit, and the pattern does not guarantee safety. What is worth borrowing is the shape of the design: locality is not used as a substitute for authorization.

A local agent can still delete the wrong file, call an unintended tool, publish too early, or operate with credentials broader than its task. Keeping inference on-device does not reduce the need for scoped capabilities and approval gates.

Write the authority boundary as operations, not adjectives:

may_read:
  - project/media/inbox/**
may_write:
  - project/media/exports/**
may_call:
  - local-video-generator
  - local-image-resizer
requires_approval:
  - publish
  - overwrite-source
forbidden:
  - read-credentials
  - upload-source-media

The exact syntax is unimportant. The test is whether the runtime can enforce the policy and whether the interface can explain an approval request without making the user reconstruct the agent's history.

Receipts are part of the product

A completed animation is not evidence that a multi-step job completed.

For a media workflow, the receipt may name the source artifact, generation settings, exported files, failed outputs, and the directory where results were written. For a coding agent, it may be a diff plus test results. For an interface-generating agent, it may be the accepted component specification and behavioral checks.

Keep the receipt narrow. Recording every prompt, image, and intermediate state by default can create a second data-handling problem. Capture what a reviewer needs to verify the operation, redact sensitive values, and give the evidence a retention policy.

This is also where local-first claims become testable. A receipt can record which steps stayed on-device, which network calls occurred, which approvals were granted, and which artifact crossed the final boundary. Configuration describes intent. The receipt describes the run.

Apply the contract before picking the stack

Take a hypothetical local creator tool that generates a short video and prepares a still for social publishing. This example is not a claim about any of the projects above.

The team should be able to answer six questions before debating models:

Resources: What device envelope is supported, and what happens when generation exceeds it?
Residency: Do prompts, source media, previews, and exports stay local? Which checks or services still need a network connection?
Handoff: Does the user receive editable media and a predictable export, or only a rendered preview?
Interface: Can the UI represent partial output, cancellation, approval, retry, and failure without losing state?
Authority: Can the agent prepare files but never publish or overwrite a source without approval?
Evidence: Does the run end with an export manifest or report that names what happened?

A benchmark score answers none of those questions.

Benchmarks may help choose an implementation after the workflow is defined. They cannot define the workflow for you. Neither can an impressive hardware demo.

Local is the whole chain

A coherent local-first product makes a series of small, verifiable promises. It names the resource ceiling and shows where data moves. It returns an artifact the user can edit. The frontend exposes state, the runtime limits what the agent may do, and the workflow leaves evidence when the work ends.

One locally running model does not rescue a workflow that breaks those promises elsewhere. At that point, local execution is a placement detail rather than an architecture.

Ask the team to explain all six boundaries without naming a model benchmark. If the answer still begins with VRAM, the architecture review is not done.

Source notes

Your AI Fallback Chain May Be Quietly Changing the Job

hefty — Sun, 02 Aug 2026 10:24:38 +0000

A discovery stage needs source tools, a defined reasoning profile, and a way to preserve the evidence it finds. Its terminal artifact might be a source packet with explicit claim boundaries.

Now imagine that the primary provider times out. The orchestrator moves to the next model in a generic fallback list, but that route cannot use the required source tool. The setting is ignored, or the workflow quietly switches to text-only generation. A polished packet appears and the run turns green.

The workflow stayed available by changing the job. Fluent prose makes the failure harder to spot because every later stage receives an artifact that looks valid but was produced outside the discovery contract. If the dashboard records only "completed," that mismatch can survive all the way to the final output.

The model does not have to be bad for this to happen. The routing policy only has to treat unlike jobs as interchangeable. A reliable router defines each stage's contract, selects a route that can satisfy it, rejects incompatible substitutes, and records what actually ran. Availability fallback fits inside that policy; it does not get to override it.

One workflow contains several different jobs

"Use the best model" sounds decisive until a pipeline has several stages. Best at what? Discovery may need browsing or retrieval tools, enough context for the evidence set, and evidence capture. Planning works on a bounded packet and turns it into an argument. Drafting has platform, voice, length, and file constraints, while evaluation needs explicit checks tied back to the brief and sources.

Reasoning effort belongs in the route definition as well. It is a budget and behavior setting for a particular task, rather than a global quality switch that should automatically be pushed to maximum.

Current SDKs expose the seams needed to make those choices explicit. The OpenAI Agents SDK documents model and provider selection per agent, configurable reasoning effort, and strict validation for unsupported features. Vercel's AI SDK documents step preparation using runtime context, including model selection, along with reasoning controls and telemetry. Vercel's routing guide separates task-oriented routing from cost, latency, fallback, load balancing, and budget policies.

Those controls do not prove that every pipeline needs multiple models. Often one route is enough for the whole workflow. The useful change is that reuse becomes a checked decision instead of an inherited default: if the same route satisfies all four contracts, assign it four times and record the choice. The design is about preserving each job, not collecting model names.

Start with a four-stage contract

Build the route table from the jobs before adding model names. Here is a practical four-stage design for a source-bounded publishing workflow. It is an implementation example, not a benchmark claim about which kind of model performs best.

Stage	Required capabilities	Terminal artifact	Route rejection condition
Discovery	approved source tools, enough context for the evidence set, source evaluation and evidence capture	source packet with claim boundaries	required tool, context, or capture behavior is unsupported
Planning	bounded source-packet input, synthesis, argument and outline structure	source-linked argument and outline	route or stage setup cannot preserve the evidence boundary
Drafting	source-packet input, platform format, voice and length constraints, file output	non-empty draft at the expected path	route cannot preserve input, formatting, or artifact requirements
Evaluation	brief and source access, explicit checks, structured pass/fail result	evaluation record linked to the draft	route cannot execute the required checks or return the agreed structure

A useful boundary exists between the first two rows. Discovery gathers evidence with approved source tools, checks whether the material is relevant and readable, and records what each source can support. Planning receives that bounded evidence packet and decides the article's argument, order, and outline. It should not reopen broad discovery whenever it wants more material. If the packet has a real gap, the planner can return that gap for a deliberate discovery pass instead of silently widening its own job.

A compact contract can live in configuration rather than prompt prose:

version: 3
stage: discovery
task_class: source_bounded_research
requires:
  tools: [approved_source_reader]
  context_profile: research_packet
  evidence_capture: true
reasoning_profile: deliberate_research
expected_artifact: research/source-packet.md
approved_routes:
  - research-primary
  - research-secondary

The profile aliases resolve to concrete provider, model, reasoning, and tool settings elsewhere. That keeps infrastructure policy in orchestration state, where the runtime can inspect it, while the prompt describes the work. Real contracts may also need budget or latency boundaries. A field earns its place only when the router can validate it or the evaluator can check it; anything else is decoration.

The stage-routing loop

The mechanism is small enough to implement without building an internal routing product.

Declare the contract before selecting a model. Record the task class, required tools and features, context needs, reasoning profile, budget boundary, expected artifact, and approved substitutes.
Resolve a route from the contract and current runtime context. Put the choice in code or versioned configuration so an application-wide default cannot make it by accident.
Validate compatibility before execution. Check tools, schema behavior, context, settings, and output support. OpenAI's strict_feature_validation=True is one concrete example: an unsupported setting can become a visible error instead of being ignored. Other stacks will expose different checks, so the general rule is to test the requirements you depend on.
Classify a failure before choosing recovery. A timeout, a missing capability, and repeated provider instability call for different responses.
Emit a receipt for the route that ran. Configuration shows intent. The receipt shows execution.

The order matters. When fallback selection happens before compatibility validation, a generic list can quietly weaken the contract.

Fail closed on capability, fail over on availability

"Fail closed" can sound rigid in a system designed to keep working. The useful question is what failed.

Runtime event	Response
transient timeout or provider error	retry within a bounded policy, then try an approved compatible route
provider rejects the context size	use an approved larger-context route only if every other contract field still passes
required tool, schema, or setting is unsupported	reject that route; test another against the original contract or stop
repeated provider instability	open or observe a circuit according to policy
no approved compatible route	fail the stage visibly

Retries handle transient attempts, fallbacks choose alternate execution routes, and circuit breakers stop sending traffic into repeated failure. Capability rejection happens earlier: it says that a route cannot perform this job at all. Flattening those mechanisms into one chain produces deceptive uptime, with more green runs and less certainty about what "completed" means.

The practical consequence is simple. A generic text model cannot cover tool-backed discovery in an emergency, and an evaluation route with different schema behavior needs validation before it can substitute. Even a larger context window is irrelevant if the route loses a required tool. Every replacement has to pass the current contract rather than merely produce an answer.

This fail-closed rule is the article's operational recommendation, derived from the routing and validation controls in the sources. Neither vendor mandates that exact policy wording.

Verify the runtime, not the configuration

Even a good route table cannot tell you what happened after a retry, runtime override, or provider failure. Evidence from the run closes that gap.

The OpenAI Agents SDK documents nested tracing across runs, agents, model generations, tools, guardrails, and handoffs. Vercel describes telemetry spanning the root generation, model calls, steps, tools, usage, errors, and selected context. Those surfaces make a compact routing receipt practical.

A useful receipt might contain:

stage: discovery
contract_version: 3
selected_route: research-primary
actual_provider: <provider-id>
actual_model: <model-id>
reasoning_setting: <resolved-setting>
compatibility_check: passed
recovery:
  action: fallback
  reason: provider_timeout
terminal_status: completed
artifact: research/source-packet.md
trace_id: <trace-id>

This schema is a recommendation, not a standard supplied by either SDK. Its job is narrow: show the selected and actual routes, explain any change, record whether the substitute passed the contract, and point to the terminal artifact.

A receipt cannot prove that the draft is accurate or good. It helps an operator separate routing failures, tool failures, and output-quality problems. Evaluation still has its own work to do.

Tracing also creates a data-handling problem because generation and function spans may include sensitive inputs or outputs. Capture only what diagnosis requires, redact deliberately, restrict access, and set retention rules. Recording everything by default is a liability, not an observability strategy.

The cost and complexity objection is valid

A router adds code, configuration, tests, and another place to fail. That cost is real, so start with the two stages that have the clearest capability difference. Define their contracts, keep a tiny approved route table, and add one preflight check for the requirement most likely to disappear during fallback. Record a redacted receipt, then force one provider failure and confirm that the substitute preserves the contract or stops.

Route only where capability, risk, or artifact semantics materially change. A second model adds no value when the first route already passes all four stage contracts; in that case, the simplest correct table contains one model with four explicit assignments.

Cost control belongs here too, without a promise that routing automatically saves money. Reasoning effort, model tier, retries, and budget ceilings are policy dimensions you can assign per stage and measure. Extra reasoning is not universally better, and a lower-priced route is a poor bargain if it corrupts the artifact consumed by every later stage.

The complexity has to buy a visible property: the workflow performs the declared job or reports that it could not. Without that property, the router is only a more elaborate fallback chain.

Reliability means preserving the job

It is easy to mistake an AI stage's output for evidence that the stage worked, especially when the output reads well. The safer sequence is to declare the contract, resolve and validate an approved route, recover according to the failure class, and record the runtime that produced the artifact.

Then ask the test that generic fallback chains avoid: if the primary provider failed today, could you prove that the fallback performed the same job, or would you only know that it produced plausible text?

Source notes

Your coding agent needs a proof environment, not just a sandbox

hefty — Thu, 30 Jul 2026 09:52:15 +0000

A sandbox can contain a wrong answer.

That sounds obvious, but a lot of agent infrastructure still treats containment as the finish line. Put the agent in a disposable container. Limit its credentials. Tear the environment down afterward. Now the run is "safe."

Safer, yes. Correct, no.

Isolation can tell you that the agent did not overwrite your laptop or leak one task's filesystem into another. It cannot tell you that the checkout was reproducible, the tests measured the intended behavior, the page rendered correctly, or the final diff stayed inside the requested scope.

For production work, a coding agent needs a proof environment: an isolated runtime that starts from known state, checks the behavior you care about, and returns evidence another person can inspect.

Without that last part, you have a contained claim of success.

What a sandbox actually solves

Sandboxes are useful because agent runs are messy.

A realistic task may install dependencies, launch services, mutate a database, open a browser, create files, occupy ports, and leave background processes behind. Two agents sharing that state can interfere with each other in ways that are hard to reproduce. A failed experiment can poison the next attempt.

Per-task or per-user environments reduce that mess. Projects such as Agent-Sandbox expose isolated code, shell, browser, and computer sessions with explicit lifecycle management. Discussions around forkable coding environments push the same idea further: duplicate the full state before trying destructive tests or competing implementations.

That is a real improvement over handing every agent the same long-lived development machine.

But isolation answers a narrow question:

Where did this work run, and what was it allowed to affect?

Correctness is a different question:

What behavior changed, and what evidence supports the claim that the change is right?

A container boundary cannot answer that for you. Neither can an exit code.

Start with reproducible state

Before an agent can prove anything, the environment needs an identity.

"It passed in the sandbox" is weak if nobody knows what the sandbox contained. Record enough state to recreate the run:

repository commit and dirty-tree status
dependency lockfile and base image
installed toolchain versions
fixtures, seed data, and service versions
relevant environment inputs, with secrets redacted
viewport, device profile, and browser state
the exact task prompt or acceptance criteria

Browser and application state deserve special attention. A git worktree can reproduce source files, but it does not capture a logged-in session, a populated database, an in-memory queue, or a half-finished multi-step flow. That is why environment-level forks are interesting: two approaches can begin from the same application state instead of merely the same commit.

If the starting points differ, a comparison between agent runs is mostly theater. You do not know whether the patch, the fixture, or yesterday's leftover process caused the result.

The environment identity should travel with the output. A reviewer should not have to reverse-engineer it from CI logs.

Green commands are not a behavioral oracle

Agents are very good at satisfying visible checks. That becomes a problem when the visible checks are incomplete.

Suppose the task is "keep the form open and show an inline error when payment fails." The agent changes the component, updates a mock, rewrites a test, and reports:

typecheck: passed
unit tests: passed
build: passed

The page may still redirect on failure. The test may now assert the new, wrong behavior. The mock may never exercise the failure branch. The agent did not lie; it optimized against a weak contract.

A proof environment needs a behavioral oracle: a task-specific definition of what should happen.

For an API change, that might be golden inputs and outputs plus checks around side effects. For a migration, it might be the before-and-after schema, representative data, and a rollback run. For a frontend task, it might include explicit interaction steps, visible text, accessibility state, responsive behavior, console errors, and relevant network requests.

The oracle does not need to be fancy. It does need to be independent of the implementation the agent just wrote.

This is the uncomfortable part: an agent should not be allowed to redefine success while implementing the task. If it changes the acceptance test, that change needs separate scrutiny. Otherwise "all tests pass" can mean "the agent moved the goalposts cleanly."

Golden tests have their own risk. They can preserve behavior that was already wrong. Treat them as a lock on known behavior, not a substitute for deciding what the task should accomplish.

A screenshot is evidence, not proof

Frontend work exposes the gap quickly.

A build can pass while the page is clipped on mobile. A component test can pass while focus disappears after a dialog opens. A screenshot can look fine while the console fills with errors or an API request returns the wrong payload.

Browser tooling now makes it practical to collect much richer artifacts. Agent-browser, for example, documents accessibility snapshots, screenshots, snapshot and pixel diffs, traces, console messages, page errors, network inspection, device emulation, and saved session state.

Those primitives are useful because they let the environment return structured evidence instead of one confident paragraph from the agent.

Still, collecting artifacts is not the same as judging them.

A screenshot proves that some pixels appeared. It does not prove that they match the expected design. A trace proves that events occurred. It does not prove that the right user journey completed. A clean console says little about a silent backend data error.

Evidence becomes meaningful when it is paired with an expected outcome:

Given: a signed-in user with an expired card
When: the user submits the checkout form
Then:
  - the page remains on /checkout
  - the inline error is visible
  - focus moves to the error summary
  - no order is created
  - the payment failure is logged once

Now the screenshot, accessibility snapshot, network log, and backend record each answer a specific question. The proof is the relationship between the contract and the artifacts, not the artifact pile itself.

Return an evidence bundle, not a victory message

An agent's final message is a summary. It should never be the only record of what happened.

A useful evidence bundle includes:

the environment identity and starting revision
exact commands, exit codes, and failed checks
the final source diff
test results tied to named acceptance criteria
screenshots or visual diffs for relevant UI states
browser trace, console output, and page errors
network or backend logs when the behavior crosses that boundary
any changed tests, mocks, fixtures, or baselines
the cleanup or retention decision for the environment

Keep failures in the bundle. A flaky end-to-end check that passed on retry is part of the result. So is a mobile viewport that was skipped because the service failed to start. Deleting inconvenient evidence turns the bundle back into marketing.

The bundle also needs to be reviewable without asking the same agent to interpret it. Store the diff, logs, traces, and images as artifacts. Link checks to the acceptance criteria. Make the environment revision visible.

This does not remove human review. It makes human review less dependent on trust.

The four-layer contract

The whole setup can be reduced to four layers.

1. Reproducible state

Identify the source, dependencies, data, services, browser session, and task contract. A rerun should begin close enough to the original state that its result means something.

2. Isolation

Give the task its own processes, filesystem, ports, and browser state. Scope credentials and external access separately; a sandbox does not automatically make outbound side effects safe.

3. Behavioral oracle

Define expected outcomes outside the implementation. Use golden inputs and outputs, explicit UI assertions, invariants, or domain checks that match the task.

4. Reviewable evidence

Return the commands, diff, checks, failures, screenshots, traces, and logs needed to evaluate the claim. Preserve enough context for somebody else to disagree.

Skip any layer and the result gets weaker.

Reproducible but unisolated runs contaminate each other. Isolated runs without an oracle execute safely but prove little. Checks without artifacts force reviewers to trust a summary. Artifacts without expected outcomes become a folder of screenshots nobody can interpret.

A checklist for your agent harness

Before accepting an agent-written change, ask:

Can we identify and recreate the environment's starting state?
Did the task run in an isolated filesystem, process, service, and browser context?
Were credentials and external side effects scoped explicitly?
Did the checks measure the requested behavior rather than generic repository health?
Did the agent change any test, mock, fixture, snapshot, or baseline that defines success?
Does the evidence include failed checks and skipped coverage?
Can a reviewer inspect the diff and runtime artifacts without the agent's narration?
Does the environment have an explicit cleanup or retention outcome?

You do not need a giant platform to start. Add an environment manifest to the run. Keep acceptance criteria outside the generated patch. Save browser traces for UI changes. Treat changed tests as first-class review items. Attach the evidence bundle to the pull request.

Then ask a better question.

Do not ask whether the agent ran safely.

Ask whether another person can verify what it proved.

Source notes

Your coding agent's memory is a dependency. Treat it like one

hefty — Tue, 28 Jul 2026 03:24:19 +0000

An agent that forgets your repository every morning is expensive.

An agent that confidently remembers last month's architecture is worse.

Persistent memory looks like a context feature when you install it. But once it survives a session, it can shape future behavior and spread to teammates. It has quietly become infrastructure. At that point, "the agent remembers our codebase" is not a sufficient design.

You need to know what was stored, where it lives, when it expires, what gets injected into a session, and how to remove it.

That is dependency management, even if the dependency happens to be a graph, a Markdown file, or a learned instruction.

What persistent context actually fixes

The appeal is easy to understand. Coding agents burn time rediscovering the same repository:

Where is this symbol defined?
Which modules call it?
What tests cover this path?
Which package owns the boundary?
Did this convention already get documented somewhere?

A persistent code graph can answer those questions without another wide grep-and-read loop. A small set of durable project instructions can stop the agent from relearning the same constraints in every task. Session summaries can carry a decision forward without dragging an entire transcript behind them.

This is better than stuffing the whole repository into a prompt. Targeted structural queries leave more room for the task itself.

Projects such as codebase-memory-mcp document this model directly: index the repository into a local graph, query its structure, watch for changes, and optionally share a compressed graph artifact with the team. Everything Claude Code takes a broader approach, with session memory, learned instructions, hooks, retention controls, injection limits, and separate data roots for different harnesses.

I have not tested either project, and their performance claims are their own. The interesting part is the operational surface they expose. Persistent context is no longer one prompt. It has installation behavior, storage, refresh logic, configuration, and removal.

Once those things exist, memory has a lifecycle.

Installation needs a receipt

Package managers show a plan before they modify your dependency graph. Database migrations can be reviewed before they run. Infrastructure tools can produce a dry run.

Agent-memory installers should meet the same bar.

The codebase-memory-mcp project documents installation across several agent clients. Depending on the target, that can involve MCP registrations, instruction files, skills, hooks, backups, and local graph data. The convenience is obvious. So is the trust problem.

One issue on the project proposed an install-plan receipt: a machine-readable preview of every mutation before the installer writes anything. It was a proposal, not proof that the feature exists. But the primitive is exactly right.

Before installing a memory layer, I want a list like this:

add: project/.agent/instructions.md
edit: user MCP configuration
add: post-checkout refresh hook
create: local graph index
backup: existing client configuration

No mystery writes. No "we configured everything for you" followed by an afternoon of diffing dotfiles.

The receipt also gives uninstall a contract. If the installer claims ownership of five integrations, removal should name those same five integrations, preserve user-owned edits, and ask before deleting stored indexes.

That is boring machinery. Good. Infrastructure earns trust by being boring.

Scope memory before memory scopes you

"Persistent" answers how long context lives. It says nothing about who should see it.

A memory can be:

project-local or user-global
private to one agent harness or shared across several
stored outside the repository or committed as a generated artifact
written by a human, derived from code, or learned from previous agent output

Those are different authority levels. Mixing them produces surprising behavior.

Suppose an agent works on Resize Image For. One project invariant is worth remembering: source-image processing stays in the browser, and adding server uploads requires an explicit product decision. That constraint belongs with the project. A personal preference about terminal aliases does not.

Now consider a curated catalog such as Generative UI resources. Its category model and canonical URL rules are durable project context. The current number of entries is not. One can guide future edits; the other will become stale as soon as the catalog changes.

This is the test I use: would a new teammate need this fact to make a safe change, and can the repository remain the authority for it?

If yes, keep it close to the project and review it like project configuration. If it is a personal workflow preference, keep it in a user-scoped store. If it is a generated snapshot, label it as generated and define who refreshes it.

Do not let two harnesses silently share a memory directory because both happened to choose the same default path. Do not commit personal learned instructions because sharing sounds convenient. Scope should be a decision, not an accident.

Freshness is part of correctness

Stale memory is not harmless background noise. It changes what the agent looks at and what it ignores.

A repository graph can point to renamed modules. A session summary can preserve an abandoned approach. A learned instruction can keep enforcing a convention the team removed. A security rule can refer to an old boundary. Each item still looks like context. None announces that it has expired.

Memory systems need invalidation triggers, not vague promises to "stay up to date."

Refresh or invalidate stored context when:

the repository changes beyond a defined threshold
an architecture decision is replaced
indexed paths move or disappear
the memory tool or schema is upgraded
a generated artifact no longer matches its source revision
a learned instruction falls below its confidence or retention limit

File watchers and git-change detection help with repository graphs. Versioned artifacts help teams see when a snapshot changed. Retention windows help with learned context. None is universal, and none removes the need to inspect the live code for a high-risk task.

Persistent context should speed up orientation. It should not outrank the repository, tests, or current runtime evidence.

Stored does not mean prompt-worthy

Once teams can retain context, they tend to retain too much of it.

That recreates the original problem in a new place. Instead of dumping the repository into every prompt, the system dumps a large memory store into every prompt. Old decisions compete with current code. Weak inferences sit next to explicit rules. The agent spends tokens sorting history before it can do the task.

Storage and injection need separate policies.

Everything Claude Code's documented controls include retention windows, context-size limits, confidence thresholds, and separate data roots. Whether or not you use that project, those are useful knobs to steal.

For each memory class, decide:

How long may it remain stored?
What event invalidates it?
What confidence or provenance is required?
How much may enter one session?
Can the agent retrieve it on demand instead of receiving it by default?

A human-written project rule can have a long lifetime and high priority. A generated task summary should probably decay. A structural graph may stay stored but get queried only when the task touches that area. A speculative "preference" learned from one accepted patch should not silently become policy.

Memory should reduce search, not pre-decide the answer.

Rollback is a feature, not cleanup

Practitioner discussions about AI-assisted coding are full of inconsistent experiences. Some developers get excellent results after investing in repository guides and constraints. Others spend their time correcting style drift, security misses, and code they do not trust. These are anecdotes, not controlled measurements, but they point at the same operational fact: context quality changes outcomes.

That makes rollback part of the product.

You should be able to disable memory injection without deleting stored data. You should be able to rebuild a graph from a known revision. You should be able to inspect which learned instruction influenced a session. You should be able to remove an integration without guessing which hooks or config entries it touched.

And when the agent behaves strangely, "clear everything" should not be the only debugging tool.

Version the parts that affect team behavior. Record provenance for generated or learned memory. Keep backups of edited configuration. Make the enabled state visible. Give operators one narrow switch at a time.

If memory cannot be audited or disabled, it is not helping the agent understand your system. It is creating a second system that your team does not understand.

A dependency-grade adoption checklist

Before giving a coding agent durable memory, answer these questions:

What files, registrations, instructions, skills, hooks, backups, or indexes will installation change?
Is each memory project-local, user-global, harness-specific, or intentionally shared?
Which repository, architecture, tool, or confidence changes invalidate it?
What may enter a session automatically, and what must be retrieved on demand?
Was this written by a human, derived from code, or learned from previous agent output?
Can a reviewer see what changed and why the agent received it?
Can injection stop without destroying the stored artifact?
Can every owned integration and artifact be rolled back cleanly?

If a tool cannot answer those questions yet, that does not make it useless. It tells you where to limit the rollout.

Start with one repository. Keep the memory local. Inspect the installation. Make refresh explicit. Watch what enters the prompt. Then widen the boundary only after the team can explain the behavior.

An agent that forgets everything wastes time.

An agent that remembers without boundaries spends trust.

Source notes