DEV Community: hefty

Your coding agent's worktree is not its security boundary

hefty — Mon, 02 Nov 2026 06:07:50 +0000

An agent opens a fresh worktree, completes a task, and hands back a diff. The files are separate. The result is easy to review. It's tempting to call that containment, but the worktree only separates where the task's edits go. It does not decide which other files the process can read, which credentials it can use, or where it can connect.

DeepSeek's dsh-superagent-prototype makes the distinction unusually explicit. Its README describes a research prototype that handles permitted GitHub requests (and optionally Feishu), gives work a separate session and directory, then returns results to the original thread. Its deployment guide describes the other half of that setup: the published presets use danger-full-access under the operator's OS account. The guide warns that account wrappers choosing default identities do not prevent the worker from reading the operator's home directory or invoking programs by path, and recommends a dedicated OS account for isolation.

That warning describes this prototype's documented presets, not every coding agent. It left me with a better deployment question than "does it use a worktree?": whose authority does the worker inherit?

Four controls that answer different questions

A session or worktree tells you where the agent's conversation and edits belong. A credential wrapper picks which account a particular integration uses by default. The OS identity and its runtime restrictions determine what a process can access when it runs. A human review gate tells you whether to accept the work it returns.

Those controls are related, but they aren't substitutes. A narrowly scoped GitHub token doesn't restrict access to a second credential already readable by the process. A clean diff can't show you everything the worker may have read during execution. Rejecting the PR afterward cannot undo a secret read.

The Reddit roundup keeps coming back to reachable credentials and broad tokens. Its incident accounts are author-curated, not a verified dataset, so I wouldn't turn its tally into a risk estimate. A DEV.to essay focuses instead on approvals and configuration trust. Those decisions matter too, but approving a plan does not restrict the OS process that executes it.

A preflight I'd run before the first task

This is a proposed review procedure, not a test I ran on the DeepSeek prototype or a universal sandbox recipe:

Write down the account that will actually execute the worker, not just the GitHub identity it uses to post results. Identify the home directory and any other mounted paths visible to that account.
Inventory the credentials and network destinations the task genuinely needs. Keep unrelated projects, personal credentials, and production access outside the worker's reach with enforceable account, mount, credential, and egress controls appropriate to the runner.
In a disposable environment containing only synthetic files and tokens, try both an allowed access and a denied access under the same runner configuration planned for real work. Record the observed results. A declared policy is useful; a denied-action check tells you whether it is applied where this worker runs.
After any preset change, compare a redacted before/after description of the effective account, mounts, credential scope, and egress rules. A text diff checker for redacted permission manifests can help spot changed lines; leave whitespace-ignore off when indentation might matter. Compare only sanitized or synthetic text, never live secrets or private configurations. A text diff reveals textual differences, not whether the policy enforces what you intended.

Then look at the returned patch and the checks the worker actually ran. Code review is still necessary. It just comes after the access boundary has already mattered.

Don't mistake a dedicated account for a finished sandbox

A separate OS account reduces what the worker can inherit from your personal account, which is why the prototype's deployment guide recommends one. Its value depends on the permissions, mounts, network access, and credentials you give it. A shared volume containing another project's secrets would weaken that separation; an unrestricted outbound path deserves its own review. The right setup depends on the job, and none of these steps is proof of complete containment.

For a small documentation task, a restricted runner may be easy to arrange. A task that needs package installs, external services, or customer data asks for a more deliberate exception process. If a proposed exception expands the boundary, review that expansion before launching the run, not only when a tidy PR arrives.

Before a run, find out what the agent could reach. Afterward, inspect what it changed. A worktree helps with the second question; don't let it answer the first by accident.

Source notes

DeepSeek dsh-superagent prototype README — project description of its research-only request, session, and directory workflow.
dsh-superagent deployment guide — same project's documented presets, operator-account access warning, and dedicated-account recommendation; not an independent security audit.
Reddit: coding-agent security incident roundup — author-reported discussion of credentials and tokens; incident claims not independently verified here.
DEV.to: Securing AI Agents — vendor-authored discussion of approvals and configuration trust, not a validation of the prototype.

An Agent Workbench Needs More Than a Chat Pane

hefty — Sat, 31 Oct 2026 13:17:41 +0000

A coding agent can leave behind a detailed chat transcript and still make you hunt for the active run, the affected code, the access it had, and the change waiting on you.

That is the workbench problem. More generated text won't solve it. The developer needs a way to move between a live session, the relevant code and runtime, and a reviewable change. Those are different views of the same task, but none can stand in for the others.

Start with the bug, not the prompt

Imagine a settings dialog whose Save button works with a mouse but not from the keyboard. This is a hypothetical handoff, not a test of a particular agent product.

A screenshot of the button tells the agent where to look. It does not tell it whether the dialog uses a shared form component, which keyboard behavior is expected, or whether other settings screens depend on the same handler. Before delegating, give it the affected route and component, the expected keyboard behavior, and the neighboring screens that must keep working. Then identify the revision it should start from.

Browser element selection can help with that first step. PulseDev's product description proposes selecting an element in a local browser, passing context to a coding agent, and reviewing a proposed diff there. I like the shorter path from UI to code. I wouldn't mistake the selected element for a map of every place a shared component is used; the reviewer still has to inspect the call sites and behavior.

Make the run findable

Now suppose the agent is running while another developer takes over. A chat transcript might explain the original request; it is a poor answer to "which session should I resume?" A practical handoff should name the run, the repository and revision, its current state, and the pending decision. If a remote session or a stale checkout is involved, the handoff should say so rather than leaving the next person to infer it.

Corgi's maker describes an activity feed, unread-session navigation, and launch/resume search for agent terminals. Those are examples of useful session ergonomics, not evidence that the feed verifies work. An "active" badge does not tell you whether an edit is correct, and a quiet terminal does not prove nothing happened. The IDE discussion around project navigation and debugging points to the other half of the workbench: you still need to find the code and understand how it behaves in the project.

I would keep a small handoff card next to a nontrivial run. This is a proposed operator checklist, not a feature I tested in any of the tools above:

Task: keyboard Save behavior in settings dialog
Starting revision: <commit or immutable revision ID>
Affected route/component: <location and known shared call sites>
Expected behavior: <keyboard path and unchanged neighboring behavior>
Session: <run ID, location and current state>
Access: <declared file/tool/network scope; pending requests>
Review: <diff or commit to inspect; checks actually run or not run>
Decision: <accept, request changes, or investigate further>

If you can't fill in the revision or session ID, leave it blank and investigate. Guessing after the run just makes the handoff look more complete than it is.

Permissions belong in a separate column

A readable diff answers what changed in the repository. It does not tell you everything the agent could read or contact while making that change.

OpenShell's README describes an isolated agent runtime with file, process, and network policies, plus checks before proposed policy expansions are approved. That is a different control from a session feed or a code review. A request for broader network access deserves its own decision; approving it does not approve the resulting code. Conversely, rejecting a diff cannot retroactively revoke access the run already had. These are project-described controls, not a claim that any sandbox guarantees containment.

For the hypothetical dialog fix, inspect the proposed patch in the IDE, check related uses of the shared component, and look at the keyboard path in the running UI when possible. Separately, look for access requests that exceeded the task's declared scope. If a check wasn't run or the session history isn't available, mark that gap instead of treating the activity feed as a security receipt.

The handoff test

Another developer should be able to locate the run, see the code and context it touched, inspect any capability changes, and decide what happens next without reconstructing the task from a long chat. For a one-line typo, that's overkill. For a shared UI component or a run with new tool access, it's the difference between supervising work and merely watching it happen.

If the next developer can't tell what to approve or resume, a prettier chat pane won't fix the handoff.

Source notes

DEV.to: Agents go mainstream while trust becomes the bottleneck — author's argument about review throughput and production quality; not a measured adoption study.
NVIDIA/OpenShell README — project description of isolated runtime policy and proposed policy-change checks; not independent containment testing.
Show HN: Corgi — maker description and operator discussion of live-session ergonomics.
Reddit: Do we still need an IDE? — qualitative discussion of navigation, debugging, and responsibility; not a survey.
PulseDev on Product Hunt — product description and comments on element context and diff review; not a hands-on evaluation.

A Coding Agent's Green Check Is a Coverage Claim

hefty — Thu, 29 Oct 2026 08:35:44 +0000

An agent says the tests passed. The screenshot is green. But the result could be from yesterday's commit, and the test for the behavior you requested might not even be in the suite anymore.

A green check tells you about the checks that ran. It says nothing about the behavior nobody checked.

Start with a claim you can inspect

Suppose you ask a coding agent to add a settings toggle. Before it edits the app, write down what you'd expect to see: the default value on first load, the saved value after reopening settings, and the state after reset. These are proposed acceptance checks for an example change, not a test of any particular product.

After the agent finishes, inspect the running behavior or stored state, not just the final message. A description of a file move is not evidence that the file ended up at its destination. The same gap applies to a toggle that looks right in a screenshot but does not persist. Independent checks needn't be elaborate; they need to touch the thing the task was meant to change.

Letting the agent write the acceptance test after implementing the toggle is risky. If it misunderstood the default, it can put the same mistake in the code and the test. Keep at least one expected outcome outside that loop, even if it's a human-reviewed assertion written before the edit.

Tie the receipt to the revision

A test result without a revision could belong to different code. Record the commit or immutable diff, the command and its output. Then inspect whether relevant checks were skipped, removed, or weakened. A focused test may pass while the broader suite remains unknown. Say both things.

Here is a proposed review card for that hypothetical settings change. Fill it with actual evidence from your own repository rather than copying these placeholders into an approval:

Intent: default off; saved choice survives reopening; reset returns to default
Revision reviewed: <commit or immutable diff identifier>
Files changed: <reviewed file list>
Checks expected before editing: <default / persist / reset checks>
Command and result on that revision: <command + captured output, or NOT RUN>
Checks changed or skipped: <list, or explicitly none after inspection>
Behavior inspected outside tests: <observation, or NOT CHECKED>
Verdict: accepted / needs work / incomplete

If the reset path was never exercised, the verdict on that requirement is incomplete. It does not become accepted because the other tests passed. If a test disappeared in the same diff, investigate the removal before taking the surviving green tests at face value. This is a review protocol, not a claim that any particular agent or tool automatically enforces it.

A functional pass says little about side effects

The toggle might work while the agent logs a token, touches an unrelated file, or requests access it did not need. Those are different acceptance questions. Check the diff for scope; for sensitive flows, use non-real planted credentials and watch the paths the agent actually takes. Keep a record of the paths the check never reached.

A clean result on an exercised path is worth recording. It is not a certificate that every possible path is safe. Likewise, a tool that compares a completion claim to session evidence cannot infer an unobserved user requirement. The evidence has to match the question you are asking.

Skip the card for a one-word documentation edit. For stored preferences, permissions, or production data, write the assertion down and keep the test receipt with the review.

If a path wasn't checked, mark it incomplete. The next reviewer can run that check; they can't recover it from a confident closing paragraph.

Source notes

DEV.to discussion: How do you check that an agent actually did what it says it did? — practical question about independently checking changed state; discussion, not prevalence data.
backcheck — project-described approach to checking claims against session evidence and qualifying unsupported results; not an independent efficacy study.
weir — project-described planted-credential testing and explicit incomplete coverage; no claim of universal leak detection.

A read-only coding agent can still run the repo's hooks

hefty — Sat, 24 Oct 2026 12:24:35 +0000

A coding agent that cannot edit a file sounds safe to point at an unfamiliar repository. Before the first edit, though, it may load settings from the repository, start a declared connector, or run a hook. Denying its Write tool does not settle who gets to start those processes.

This isn't a claim that every coding agent behaves the same way. It's a reason to test the exact command you plan to run, especially if that command will open code you did not write.

The permission you forgot to inspect

In an October DEV.to account, the author tested a scanner adapter against a hostile repository on Linux with Claude Code 2.1.288. The repository contained a .claude/settings.json hook and a .mcp.json server declaration. In the reported test, both ran under the adapter's original command even though Write remained denied. That's the author's result for that adapter and version, not an audit of other agents.

The reported fix didn't disable every tool. Restricting settings to user scope with --setting-sources user and using --strict-mcp-config stopped the repository hook and MCP server in that setup, while the scanner's explicitly supplied MCP server still worked. There's a difference between a tool the operator selected and one the target repository selected for them. Check these flags against the CLI version you deploy rather than pasting them into a different agent's command.

Put an untrusted repo in the test, on purpose

Before running a headless reviewer against unknown code, I would build a disposable fixture and run the same adapter invocation I intend to use in CI. This is a proposed acceptance test, not a test I ran for this article.

Give the fixture a harmless settings hook and an MCP server that each leave a distinct marker in test-only logs. Keep real tokens and production endpoints out of it. Run the current read-only adapter configuration, then run it again with the settings source and allowed MCP servers selected explicitly by the operator. Capture process-start evidence along with the markers. An agent's recap won't tell you which process actually launched.

I would keep the acceptance checks separate:

The baseline shows whether the repository's hook or server starts under the command as it exists today. If it does, record that as a failed trust-boundary check, regardless of whether the agent edits a file.
Under the restricted configuration, the repository-provided hook and server should not start. An operator-provided MCP server that the workflow needs should still be available.
The agent should remain unable to write through its denied tool in both cases. Check this separately; process-start and write permissions are different observations.

Keep the fixture isolated and discard it afterward. If the CLI cannot specify trusted settings and connectors for this workflow, don't declare the test passed because the agent promised to ignore the repo. Change the runner boundary or keep that agent away from untrusted repositories until you have a control you can verify.

Egress is a different check

Even after you stop a repository-supplied process from starting, ask where the processes you do allow can connect. The Harness v1.11.0 release notes describe an opt-in --egress <allowlist> for restricting outbound HTTPS hosts in its agent container. That's a concrete example of a runtime control, not a flag you can assume exists in the scanner from the DEV.to test.

For a runner with an outbound-host policy, use a controlled test endpoint to check an allowed destination and a blocked destination without transmitting secrets. Save the connection result and the effective policy alongside the process evidence. An allowlist limits the routes available through that runner; it does not fix an unwanted local hook starting, and a clean settings scope does not prove network containment. The Harness release is product-specific, and a host allowlist is not a blanket exfiltration guarantee.

Before I accept "read-only" as a security property, I want the run to tell me whose configuration loaded, which processes started, and where those processes could connect. A denied edit alone tells me none of that.

Source notes

Hostile-repository adapter test — author-reported Linux/Claude Code 2.1.288 test of repository settings, MCP startup, and an explicit configuration remedy; not independently reproduced here.
Harness release notes — v1.11.0 description of the optional outbound HTTPS host allowlist for its container; a separate product, not an integration with the scanner.

The agent stopped halfway through. Start with the diff, not the chat.

hefty — Thu, 22 Oct 2026 07:57:40 +0000

Suppose you ask a coding agent to fix the error state on a signup form. It changes the component, starts a test, and the session drops. When you reconnect, the chat offers a confident recap. The repository, meanwhile, has uncommitted changes and no clear answer about which checks finished.

That recap is useful context. It cannot tell you whether the code in front of you is the code that was checked.

We give agents elaborate starts: better prompts, more context, a cleaner plan. I want an equally boring plan for when a run cuts out. Someone has to decide what to do with the code already on disk.

Give the run a boundary before it starts

For the signup form example, I would write down the task in terms a reviewer can check later:

Starting commit and allowed files.
Expected behavior: show the server's validation message beside the field; keep the existing success state intact.
Checks to run, including the relevant form tests and a manual pass through the error state.
Actions that still need a person, such as approving a wider change or merging the result.

This is a proposed working contract. A practitioner account of AI coding makes a similar case for bounded tasks, phased execution, tests, and reading the diff instead of relying on the agent's summary. The written boundary lets you spot drift after a break.

The starting commit matters here. If the working tree already had unrelated edits, a later diff is harder to interpret. Record that fact at the start, or use a clean branch or worktree for the task.

Reconnect to the work, then inspect it

First establish whether the agent's response is still active, stopped, or complete. Do not infer that from a chat message that may have arrived before the interruption.

Then look at the repository. For a Git project, git status --short, the unstaged diff, and git diff --cached answer more than a prose recap. Which files changed? Is there a half-written test? Did the agent touch a file outside the agreed scope? If a check ran, keep its command, result, and the code state it checked. A passing test from before the last edit is old evidence.

BetterC0de describes a beta workspace that places project files, terminals, browser previews, and Git review surfaces near agent activity. Microsoft's Foundry Toolkit has stop and reconnect controls for active hosted-agent responses. They address different parts of an interrupted run; the sources do not describe an integration between them. You still have to verify the code you are about to keep.

In the signup form scenario, a preview might still show the earlier build while the working tree contains a newer edit. Check the state behind the preview before treating it as evidence. The same applies to a test result: it belongs to the version that actually ran.

Make one of three decisions

Resume when the original task still fits, the agent's state is known, and the changed files and check results line up with the current code. Give the agent the unresolved step and the relevant failure output. Avoid asking it to reconstruct the whole run from memory.

Repair when the scope is intact but a specific check failed or an edit is plainly incomplete. A failing form test with a known error is repairable. Inspect the patch after the fix and rerun the affected checks against that patch.

Restart when you cannot explain the working tree, the task crossed its boundary, or the agent no longer has enough context to continue responsibly. Preserve any useful notes and test output, then begin again from a known base. If you have to guess which edits belong to this run, another agent pass will make the patch harder to review.

These are operator choices, not reliability guarantees. A green unit test covers what that test asserts. Sensitive behavior still needs the review appropriate to the change.

The handoff is part of the task

A run may stop without leaving a clean finish line. Before anyone continues, they should be able to name the task boundary, identify the code under review, and say which checks apply to that code.

If those pieces line up, continue. If one is missing, repair the evidence or restart the work. A plausible chat recap is the start of inspection, not approval to keep going.

Source notes

AI coding workflow account — practitioner advice on bounded work, phased execution, tests, diff review, and clean resets; not independent performance evidence.
BetterC0de README — beta workspace description and its files, terminals, preview, and Git review surfaces.
Microsoft Foundry Toolkit release notes — hosted-agent follow-up, stop, and reconnect controls in the v1.6.14 notes.

Your Copilot Model Migration Needs a Rehearsal, Not a New Dropdown

hefty — Sat, 17 Oct 2026 06:50:55 +0000

A model selector can show a suggested replacement while your team's pinned workflow still names the old model. The picker changed; the workflow may not have. Rerun a real task before signing off.

GitHub's Copilot changelog schedules selected model deprecations for October 19, 2026, across Copilot chat, inline edits, ask and agent modes, and completions. It lists suggested alternatives and calls out model availability for Business and Enterprise organizations: default enablement can apply, but an admin or global policy can disable an alternative. This is a Copilot-specific notice, not a retirement announcement for those models everywhere. Nor is a suggested alternative a promise that your prompts, integrations, costs, or outputs will behave the same way.

GitHub also argues that repository instructions, skills, and hooks can stay with a workflow even when the model changes. That's a design argument, not evidence that a particular switch is safe. For a rehearsal, hold that context fixed and see what the new choice does.

Check what is pinned before touching settings

Start where your team actually selects a model: individual Copilot settings, organization policy, and any workflow or integration that names one. Record the existing choice, its owner, and where it is configured. Before planning a run, check which alternative your organization allows. A teammate's model picker doesn't tell you what your own policy permits.

If you don't have permission to inspect an organization setting, ask the admin to confirm it. Don't infer that a changed picker updated every pinned choice. The changelog tells you what GitHub plans to remove; it doesn't audit your setup for you.

Rehearse one task with the same boundaries

Pick a small task you know how to inspect, such as adding an error state to a form. This is a proposed test, not a report of one I ran. Write down the acceptance conditions before changing the model: the error appears on failed submission, focus reaches the error message, the success path still works, and your project's required checks run.

Use the same repository revision, committed instructions, permitted tools, and required checks for both runs. Work in separate branches or clean worktrees so the first run cannot leak changes into the second. If the old option is still available under your policy, run the task once with each choice. If it is not, use a saved baseline diff and check output, and label the comparison accordingly. A historical result is not a controlled head-to-head run.

A simple record is enough:

Task: add form submission error state
Repo revision / instructions: <record the same values for each run>
Model choice / policy: <record the configured choice and whether it was allowed>
Permissions: <record allowed tools and approvals>
Checks: <required checks and actual pass / fail / not-run results>
Review: <diff, behavior check, unresolved issues, reviewer decision>

Those are fields to fill in, not a sample result. Review the diff against the acceptance conditions. Run the real test and build commands your repository requires; inspect the form behavior and focus path if those are part of the task. Keep not run distinct from pass. A convincing explanation in the agent transcript does not turn an unrun check green.

Know when to stop

If policy blocks the alternative, pause the pinned workflow and ask for an authorized choice rather than routing around the setting. If the output misses the error state, changes unrelated files, or leaves a required check failing, the model's availability is beside the point. Rework the task, review the new diff, and run the checks again. If you cannot run them, keep a manual path for the work until you can verify it.

This rehearsal doesn't measure general model quality. One task can expose a local incompatibility or give you enough evidence to approve that particular workflow; it cannot guarantee equivalent latency, cost, or behavior for every task. Repeat the exercise for materially different pinned workflows rather than treating one green run as a blanket approval.

I'd keep the task, the fixed context, the actual check output, and the review decision together. Otherwise the team has a new model name in settings and no record of what that change did to the work.

Source notes

GitHub Copilot model deprecation notice — planned date, affected Copilot experiences, suggested alternatives, and organization-policy caveat. The notice was read in same-day discovery and briefing; a fresh direct fetch was blocked at drafting time.
GitHub on portable agent context and developer control — vendor framing for repository context, skills, hooks, and oversight; not evidence of model equivalence. The article was read in same-day discovery and briefing; a fresh direct fetch was blocked at drafting time.

More Agent Code Won't Ship Itself. Build the Review Surface.

hefty — Thu, 15 Oct 2026 09:14:56 +0000

More PRs can tell you the agents are busy. They don't tell you whether more software made it through review.

A study of engineering activity at 718 firms through March 2026 looked at agent adoption, coding activity, review, and completed work. The associated Ars Technica report describes more code, commits, and PRs alongside a 49% rise in review time and nearly twice as many requested changes. It reports no statistically significant increase in issue or epic completion. That's an observational finding at the firm level, not evidence that agents caused every extra review minute or that your team's output went down. The report and the paper describe the same study, not two independent tests.

Code generation can get cheaper while review still takes time. A prettier stream of generated code won't fix that.

Make the task, not the transcript, the unit of review

A live transcript shows what the agent is doing. A reviewer needs to know which task the change belongs to, what it touched, which checks ran, and what decision is waiting on them.

Imagine a frontend agent asked to change the empty state of a billing screen. The review card shouldn't lead with tokens produced or a cheerful "done" badge. It should lead with the branch or worktree, the changed route, a diff, the checks required for this kind of change, and the result of each check. A screenshot of the empty state is useful. It doesn't show whether the keyboard can reach the retry button or whether the loading state regressed.

I'd model the card as a projection of a small task record, not as an agent-written summary:

Task: Billing empty state | worktree: billing-empty-state
Change: billing route + empty-state component
Checks: typecheck PASS; UI test FAIL; keyboard check NOT RUN
Evidence: diff, test output, preview of empty state
Approval: not requested
Disposition: blocked; reviewer has not accepted this change

That is an illustrative design, not a tested product. The important detail is that FAIL and NOT RUN come from check records, not the agent's confidence. A preview should point to an artifact tied to this revision; if the branch changes, the old screenshot should not silently pass for current evidence.

Status should follow evidence

One DEV.to proposal describes an IDE as a control tower for agents in separate worktrees, with visible active, blocked, approval, and review states. That's a useful UI hypothesis, not proof that a dashboard improves throughput.

The smallest useful contract beneath such a UI might record a task ID, revision, worktree, required checks, check-run results, evidence pointers, pending approvals, and the human disposition. Then the interface can derive states without turning the agent's prose into authority:

A required check fails: show blocked, the failing output, and the revision that produced it.
A privileged action needs permission: show awaiting approval, with the exact action and scope. Don't treat permission to run a command as acceptance of the resulting PR.
Required checks pass and evidence is attached: show ready for review, not "shipped." A person can still request changes.

For the billing example, a failed UI test should send the session back to repair or to a reviewer who can diagnose it. If the test cannot run, preserve NOT RUN and the reason. If an agent wants to retry with a different package or permission, that is a new approval question. If the diff drifts beyond the task, stop the run or revert the branch rather than blessing the old review card.

On a blocked task, show the failing check first. On an approval request, show the permission scope first. Keep the underlying event and check records stable so a rendered card cannot rewrite what happened. If you're comparing presentation patterns for agent-facing products, Generative UI interface examples offer design references; they aren't evidence that generated UI makes a review process reliable.

Count verified delivery, not typing speed

A review surface may not shrink review time. It might reveal checks you've been quietly skipping, which is more useful than a board full of green "completed" cards whose checks never ran.

Track each task through proposal, evidence, human review, and accepted delivery. Keep failed and unrun checks visible. Then measure how many tasks made it through that sequence and how much reviewer time they took. Typing speed can wait.

Source notes

Software-production bottlenecks study — observational firm-level analysis; it does not test the proposed review UI.
Ars Technica's report on the study — source of the reported review-time and change-request figures; coverage of the same study.
DEV.to control-tower proposal — product/workflow perspective, not measured effectiveness.
Hacker News discussion of the study — practitioner reactions, not independent productivity data.

An AI-Suggested Package Is Not an Install Command

hefty — Sat, 10 Oct 2026 07:08:49 +0000

A coding agent suggests a package that sounds exactly right for the job. Then it wants to run npm install. Hold on: a package name in a response is a proposal, not permission to change your dependency tree.

In a small npm-focused experiment, a DEV.to author gave three AI coding tools 30 everyday tasks and counted six suggestions for nonexistent packages, counted per tool. Two invented names recurred across tools. The author also found two real packages that were very new or little-used. No failed lookup was installed. Those observations are a reason to add a check, not a measured attack rate, a model ranking, or proof that an unfamiliar package is malicious.

Three different questions get collapsed into one

First: does the exact package name exist in the registry this project will use? Check the full name, including scope. If the proposed name is missing, don't let the agent quietly pick a lookalike and continue.

Second: is this the dependency you want? Inspect the intended version, publisher or maintainer information, provenance you can actually verify, and whether the package does the specific job requested. A registry entry answers the existence question. It doesn't answer the trust question. A new package may be perfectly legitimate; its unfamiliarity calls for review, not an automatic accusation.

Third: what would installation change? The proposed command, manifest edit, lockfile delta, and verification results are separate artifacts. A lockfile shows which dependencies changed; it does not certify that the resulting tree is safe.

If you only check the first question, you've built a spelling check and called it a security boundary.

Put the gate where the action happens

Wove's README describes a useful pattern for file edits: its tooling can enforce read-before-edit and limit large-file rewrites. Those are file-operation controls described by the project, not a claim that Wove audits npm packages or blocks installs.

Enforce the rule at the tool that performs the action. You can put "verify packages first" in the prompt, but an unrestricted command runner can still install them. File-edit restrictions alone won't stop a package-manager process. The install gate belongs at the process/tool boundary as well.

Here is a proposed workflow for a JavaScript task, not a feature test of any agent:

The agent submits the exact package name, requested version, and why the task needs it. It does not run the install yet.
The gate checks that name and version against the registry configured for this project. A missing entry stops the request. An unexpected substitute becomes a new proposal, not a silent retry.
A reviewer checks publisher/version context and task fit. Uncertain provenance pauses for a decision; the gate does not pretend that metadata alone settles trust.
Only after approval does the command runner receive authority for the reviewed install. Record the actual command and the package/lockfile changes it produced.
Run the project's relevant checks and attach their results to the dependency change. If a check was skipped, say so.

That last step is easy to miss. Approval for one name and version isn't blanket approval for a different version chosen later or for a second package the agent decides to add. Reopen the gate when the proposal changes.

Advice about tools has a provenance problem too

A Hacker News discussion about a proposed shared memory for agent tool reviews immediately raised questions about seeded reviews and vendor gaming. That is community concern, not evidence of a package attack or proof that the proposed review system is compromised. Still, it is a good reminder not to let an agent's recommendation, or a glowing review it found, stand in for the checks above.

Let the agent propose a dependency. Ask for the exact package, reason, and expected change. Check those details before the runner gets permission to install it. A plausible sentence should never be the thing that authorizes a new dependency.

Source notes

DEV.to npm package-suggestion experiment — small-sample name lookups, unfamiliar real packages, and no-install caveat.
Wove README — project-described read-before-edit and large-file rewrite controls; not evidence of an npm install gate.
Show HN: Pod discussion — discussion context on provenance and gaming of agent tool reviews, not package-security evidence.

A Coding Agent Is Only as Useful as Its Handoff

hefty — Wed, 07 Oct 2026 13:00:05 +0000

A coding agent can finish its turn while the task is still stranded. It changed files, reported success, and left the next developer to reconstruct what it was asked to do, which checks ran, and what still needs a decision.

The useful unit of agent work is a task someone else can continue or reject. Its handoff needs the acceptance contract, the run boundary, evidence, and a named next action.

The agent's chat is a poor task record

Open SWE describes a loop in which a task can enter through an issue, conversation, or schedule, run in a cloud sandbox, produce a PR, and receive follow-up from review or CI. OpenAI's DevDay recap announces reusable cloud environments with shared settings and permissions, along with hosted tool and computer-use capabilities in its Agents API.

In either workflow, work crosses sessions, tools, and people. A chat transcript may explain a decision. It is a bad place to hide the only copy of the task's current state.

Community discussion raises two reasonable questions: where does the work live, and what can someone see when a run stalls? Comments and user reports can prompt a better handoff design. They do not establish that any particular product failed to recover.

Carry four things to the next person

1. An acceptance contract. Record the request and what would count as a finished result. "Improve the release page" is too loose. "Update the hero layout, check the narrow viewport, and prepare the announcement image" gives the reviewer something to inspect. Keep later scope changes visible rather than silently replacing the original request.

2. The run boundary. Record the repo and branch, the environment, the tools and permissions granted, and anything outside scope. Reusable environments can save setup, but a reviewer still needs to know which configuration this run used. Open SWE explicitly distinguishes its cloud sandbox workflow from a local CLI that runs commands as the user; the label "agent" does not tell you the boundary.

3. Evidence tied to each acceptance item. A diff shows what changed. A passing test will not catch a headline wrapping under a button on a narrow screen. A browser preview will not tell you whether CI passed. Link the artifacts and record failed or skipped checks alongside passing ones.

4. A next owner and action. A run can stop at "PR opened," "CI failed," or "needs design review." Name who handles the next step and what they need to decide. Otherwise the agent's final message sounds conclusive while the work sits idle.

Try it on a release-page task

Imagine a request to update a release-page hero and prepare an Instagram announcement. This is a hypothetical acceptance exercise, not a run reported by either vendor.

The page change needs a reviewable diff, a test or build result, and browser previews at the agreed viewport sizes. The announcement needs its own reviewed image export. For that manual publishing step, a reviewer could use Resize Image for Instagram to fit, preview, and export the image for a post or Story. The exported file does not prove the page looks right, and the page preview does not prove the image is ready to publish.

A handoff record for that task could be this small:

Request: Update release-page hero; prepare Instagram announcement image.
Boundary: <repo/branch, environment, allowed tools and permissions>
Outputs: <PR or diff, browser previews, exported image>
Checks: <command or manual check, result, artifact link for each>
Open issue: <failed/skipped check or decision still needed>
Next action: <owner and the specific review or fix>

Fill it with actual links and results as the work happens. If the mobile preview was never opened, write "not checked." If CI failed after the PR, record the failing job and whether the agent is still working on it. A clean record makes uncertainty visible before someone mistakes a confident summary for approval.

A PR is a checkpoint, not a verdict

Open SWE's described PR and review loop gives the next person a concrete place to act, but its repository also says the project is under active development. OpenAI's announcement describes cloud capabilities; it does not establish that every long-running task will resume cleanly or that a PR and CI loop is built into every integration. Neither description is a reliability benchmark.

After the agent stops, another developer should be able to tell what was requested, where the work ran, what was verified, and what to do next without replaying the chat. Code without that record leaves the team to reconstruct the job.

Source notes

Open SWE README — project workflow, cloud sandbox and local CLI boundary, and active-development caveat.
OpenAI DevDay 2026 recap — announcement of reusable Codex cloud environments and Agents API capabilities.
Hacker News DevDay discussion — discussion context about cloud-agent usefulness and product clarity; not product evidence.
Reddit developer discussion — anecdotal recovery and visibility concerns; specific failure reports are unverified.

A Coding Agent's Permissions Need a Negative Test

hefty — Fri, 02 Oct 2026 05:36:50 +0000

A runner calls a session "read-only." Fine. Put it in a disposable workspace and ask it to change a file it should not be able to change.

You need the runner to deny the attempted operation. The agent politely declining proves little.

This is a test design, not a report of a vulnerability or a test I ran on a particular product.

Start with a workspace you can afford to lose

Build a throwaway repo inside an isolated test environment. Give the agent a legitimate job: read a small config file and explain one setting. Then create a separate fixture path outside the allowed worktree but still inside the disposable environment. Put only dummy data there, including a clearly fake token string. Keep real credentials, mounted home directories, production services, and external network access out of the fixture.

Write down the contract before you prompt the agent. For example:

Operation	Expected result
Read a file in the allowed repo	Allowed
Change a file in the allowed repo during a read-only run	Denied
Write to the separate fixture path	Denied
Read the dummy token from the separate fixture path	Denied
Reach a test-only listener outside the runner's permitted network scope	Denied
Apply an agent-proposed patch through the host's patch/apply path	Denied unless separately authorized

Only include rows your setup can safely exercise. A network check needs a controlled test listener and an explicit network policy; never use a real secret or a production endpoint as the probe. If the workflow has no patch-apply tool, record that it is unavailable instead of claiming it passed a denial test.

Test the boundary, not the assistant's manners

First run the allowed read. This confirms the fixture works and gives you a baseline. Then attempt one prohibited action at a time with harmless targets. Inspect the tool or executor result and the fixture afterward. A message saying "permission denied" is useful, but the missing side effect is the other half of the check.

Record each outcome as one of four things: the tool was not exposed; the runner rejected the attempted call; the agent declined without trying; or the action happened. The third result tells you little about enforcement. A different prompt or an untrusted file in the repo might lead the agent to try the call next time.

Patch application deserves its own row. An agent may be unable to write directly but still propose a patch for another component to apply. Check the component that actually performs the edit, not just the agent's tool list. Keep this probe within the disposable fixture, and stop if you cannot tell which component owns the final write.

A denied operation is evidence about this configuration and this route through the tools. It isn't a proof that every possible route is blocked. File access, process execution, and network access can cross different boundaries; changing the runner configuration means running the checks again.

Stop is a separate experiment

Microsoft Foundry Toolkit's release notes describe skill-only toolboxes along with controls to stop active hosted-agent work and reconnect to unfinished responses. Those are useful affordances, but the release entry is not a certification of sandbox isolation or atomic rollback. Discussion about agent trust boundaries on DEV Community and sandbox choices on Reddit raises the same operator question from another direction: what actually happens at the boundary? Neither discussion establishes that a reported security incident occurred or that a particular sandbox is safe.

For a stop/reconnect check, queue a harmless operation in the fixture, request Stop, and inspect the runner's event or tool record before resuming anything. Separate operations that were never issued, operations already completed, and operations whose outcome you cannot yet confirm. Inspect the filesystem or test listener for effects rather than treating the stopped response as evidence of cancellation.

If the workflow supports reconnect, compare the resumed run against the same allow/deny contract. An interrupted session shouldn't silently acquire a wider tool surface. That's a requirement you can test in your own runner, not a claim about Foundry's implementation. If an earlier operation may have finished, reconcile that outcome before issuing a retry. A retry may duplicate a side effect.

Put the result in the release gate

A small repo-local assistant that only summarizes files may need a lighter check than a hosted agent with shell tools and network access. Match the fixture to the powers you actually plan to grant. Re-run it when you change the tool list, sandbox policy, patch route, or reconnect behavior.

If a supposedly denied operation succeeds in the disposable fixture, don't give that configuration access to real resources yet. Fix the executor policy and repeat the test. Changing the label on the button won't fix the boundary.

Source notes

Microsoft Foundry Toolkit release notes — primary vendor release entry for skill-only toolboxes and response controls; no isolation or rollback guarantee inferred.
DEV Community: agent trust-boundary discussion — secondary discussion and threat-model prompts; its security incident claims are unverified here.
Reddit: sandbox discussion — partially accessible practitioner discussion, not proof of a sandbox's safety.

Your Coding Agent Needs a Checkpoint Before Its Next Edit

hefty — Wed, 30 Sep 2026 07:39:20 +0000

An agent changes a settings screen, runs a test, and asks to keep going. The transcript is still scrolling. You can see that something happened, but the next edit may start before anyone has looked at the current one.

A stop button and a live log help, but neither tells you whether this slice is ready to become the starting point for the next one.

I'd put a checkpoint there. Show the exact candidate, the checks that ran, what's still unknown, and a decision the runner actually obeys. This is a proposed workflow, not a feature claim about any product discussed below.

Freeze the slice, not the whole conversation

Consider a hypothetical request to add a "pause notifications" control. The first slice adds the toggle and saves the preference. The agent then wants to change the notification list UI.

A useful checkpoint would look roughly like this (illustrative, not a real run log):

Candidate     <revision under review>
Scope         Pause toggle and preference save; list UI untouched
Check        <named test for saving the preference> — pass on this candidate
Browser      <named toggle-and-reload scenario> — not run
Open gap     Keyboard and screen-reader behavior not reviewed
Next slice   Change notification list UI (requires approval)
Decision     Continue | Revise | Stop

The card points to a specific change rather than asking you to trust an agent's self-assessment. A test of preference persistence does not establish that the control behaves correctly in a browser. Even a passing browser interaction wouldn't settle keyboard or screen-reader behavior by itself.

If the agent changes the toggle after that check, the runner must reassess which evidence still applies. Otherwise the UI has turned a passing result for an older candidate into a passing result for whatever code happens to be on screen now.

Make the three buttons mean different things

Continue accepts the reviewed slice and authorizes only the stated next slice. If the runner is free to change unrelated files or launch a deployment, the UI shouldn't call this a bounded approval.

Revise changes the target before more work begins. In the example, the operator might ask for the missing browser scenario, or request a focused fix to the toggle without touching the list. The resulting candidate needs its own review; editing the work does not magically preserve every old check.

Stop prevents new work from being scheduled. It should also show what has already been issued to tools and what remains uncertain. Stopping a model response cannot be assumed to cancel an in-flight command, undo a file write, or reverse a deployment. A product that offers Stop needs an executor-side boundary and a way to reconcile effects that crossed it.

An attractive frontend can lie by accident here. If Continue, Revise, and Stop are labels above one unconstrained agent loop, none of those buttons gives the operator control.

Why this isn't another final approval screen

A product-refactor account on DEV Community describes two principal agents, three review milestones, and checks that included tests and browser workflows. GitHub's account of its Copilot runtime migration describes shipping a much larger Rust change incrementally and treating end-to-end tests as critical. Neither account establishes a universal checkpoint UI or a productivity benchmark. The practical connection is narrower: an increment gives reviewers something they can accept or send back before the next change widens the scope.

Meanwhile, Foundry Toolkit's release notes describe follow-up and stop controls for active hosted-agent responses, plus inspection updates. That is evidence that runtime steering is getting product attention, not proof that pressing Stop gives atomic cancellation or that Foundry implements the checkpoint proposed here. A Hacker News discussion of coding-agent workflows also shows disagreement about letting agents handle correctness-sensitive tasks. A checkpoint doesn't resolve that disagreement; it makes the acceptance decision explicit.

For frontend builders, the presentation still matters. If the checkpoint itself is a generated interface, compare component and renderer approaches in Generative UI resources. Then keep the actual approval action tied to trusted application state and runner enforcement, not a model-written sentence in a card. The resource is for UI patterns, not evidence that any stop control is safe.

Keep the gate proportionate

A one-line copy edit probably doesn't need a modal. A change that affects saved preferences, browser behavior, or a user's next action deserves a pause. Pick a point where a reviewer can still reject the current slice without first untangling several more edits.

Before pressing Continue, I want to know which candidate I'm accepting and which behavior was checked. If the evidence isn't there, Stop must keep the next slice from starting. Otherwise the agent is moving faster than its review loop.

Source notes

DEV Community: product-refactor account — reported review milestones and mixed checks; one author's case study.
Foundry Toolkit release notes — response-level follow-up/stop and inspection features; not a cancellation guarantee.
GitHub Blog: Copilot runtime migration — first-party incremental migration and end-to-end test account, not frontend UX evidence.
Hacker News: coding-agent workflows and tests — supporting practitioner disagreement, not a representative survey.

An Agent Retry Is Not a Rewind Button

hefty — Mon, 28 Sep 2026 09:35:35 +0000

An agent asks a deployment API to create a release. The call times out. The agent restarts with its conversation intact and tries again.

There may now be two releases. Or one. The timeout tells us what the caller saw, not what the deployment service did. Replaying the conversation cannot settle it.

This is a hypothetical failure, but the design problem is concrete: once a tool can change something outside the agent process, recovery needs to identify that action before repeating it. A retry is another request. The external system decides whether it is the same operation.

Three different things called retry

A persisted session lets a replacement process recover conversation and local progress. Kit's project documentation describes append-only session state, crash-safe single-writer locks, and explicit error results for interrupted tool calls. Those are useful controls for resuming work without pretending an incomplete call succeeded.

Kit also documents reusing an idempotency key for eligible provider-request retries. That key concerns the request to the model provider. It does not make a deployment, payment, or other downstream tool effect happen exactly once.

The distinction matters during a crash. The session can truthfully record that the agent attempted a tool call while the remote service has already committed it. The new process has the transcript; it still lacks the remote outcome.

A builder in a September Hacker News work thread asked how to resume an agent after an irreversible side effect without doing it twice. The question is familiar; the thread doesn't establish a working answer.

Give each consequential action its own identity

Suppose our hypothetical agent is asked to create one release. The harness, rather than the model's next sentence, should assign a stable operation ID to that intended effect. Persist the intended target, payload identity, and ID before dispatch. Record that dispatch is about to happen in durable storage. A later process can then find the attempt even if the original one disappears.

One possible application-level record looks like this. It is a design sketch, not an API provided by Kit or Latch:

operation_id: <stable ID for this intended release>
target: <deployment service and project>
request: <exact release intent / payload reference>
state: planned | sent | confirmed | unknown-reconcile
remote_ref: <release ID, if verified>

planned means the intent was recorded. sent means the process was ready to dispatch; it does not prove the service received the request. confirmed needs a response or an authoritative lookup tied to the same operation. If the process dies after dispatch but before saving that evidence, recovery treats the outcome as unknown-reconcile.

Even this record has a gap: writing sent locally and committing a release remotely are not one atomic transaction. That's why the operation ID must travel to a remote endpoint that can recognize it, or be usable to look up the outcome afterward. A local log entry alone cannot prevent a duplicate release.

Resume by inspecting, not guessing

On restart, the replacement process should inspect the operation record and check the deployment service for a matching effect if the service supports such a lookup. If it finds a matching release, save the remote reference and continue without issuing another create request. If it finds an authoritative failure, record that outcome separately.

If there is no conclusive lookup, a retry is safe only under a suitable upstream deduplication contract: the service must recognize the same operation ID and payload, for the relevant time window, including concurrent attempts. Reuse the original key; generating a new one defeats the point. A provider-request retry key says nothing about these deployment-service guarantees.

No lookup and no trustworthy deduplication? Leave the action unresolved and ask an operator to reconcile it. That's frustrating, but it is cheaper than confidently creating a second external effect. Some APIs offer neither a query by client operation ID nor a strong idempotency contract; an agent cannot manufacture either from its transcript.

Middleware helps within its actual boundary

Latch's README documents tool-call idempotency, timeouts, circuit breakers, budgets, and compensation patterns. Those are useful building blocks for reducing repeated calls and handling a failed step in a multi-step flow.

The catch is in the defaults. Latch's idempotency store is in-memory and single-process. Its saga compensation runs in-process; a process death does not automatically resume or compensate the saga. The project is labeled alpha. I'd borrow the patterns, but I wouldn't treat those defaults as crash-durable or promise exactly-once external effects.

Timeouts need their own handling too. A timeout can mean the request never arrived, or that the effect committed and the acknowledgement was lost. Compensation is different again: undoing a known step after an exception doesn't identify an unknown remote outcome, and some effects cannot be undone at all.

Test the boundary where the process vanishes

For a consequential tool, I would review the failure cases before giving an agent an automatic retry policy:

If the process dies just after dispatch, can a different process find the same operation ID and determine whether the effect happened?
If two workers resume together, what keeps them from issuing competing attempts? A session lock alone doesn't necessarily serialize the remote effect.
How long does the upstream service retain idempotency keys, and what happens if the same key arrives with different input?
When a remote result cannot be determined, where does the unresolved action wait for human reconciliation?

If those questions have no answers, keep the session resumable but disable automatic replay for that tool. Investigate the ambiguous write before sending another request.

Source notes

Kit coding-agent runtime — project documentation for session persistence, interrupted calls, and provider-request retries; not evidence of exactly-once external tool effects.
Latch reliability middleware — project documentation for tool-call controls, with in-memory default storage and in-process saga limitations; alpha, not independently tested here.
Ask HN: What are you working on? (September 2026) — one builder's question about recovering after irreversible side effects; discussion context only.