DEV Community: qnbs

Parallel Agents Need a Work Topology, Not Just More Copies

qnbs — Thu, 01 Oct 2026 19:05:00 +0000

Launching five agents on one large issue does not create five independent units of progress. If all five edit the same module, interpret the same vague requirement, and compete for one integration decision, the team may have created a queue of conflicting patches.

Parallel work pays off when tasks can progress independently and their boundaries are clear. The hard part is often not running more agents. It is designing a work topology that limits shared state, names an owner for every interface, and budgets time for integration and review.

Draw the dependency graph before assigning workers

Break the goal into work units and mark which units depend on others. A useful split might be:

Work unit	Depends on	Output	Can proceed in parallel?
Define the new request/response contract	Product decision	Reviewed interface note	No, decide first
Add a parser for the settled contract	Contract note	Parser and focused tests	Yes
Update a separate client adapter	Contract note	Adapter and focused tests	Yes, if its files are independent
Add integration coverage	Parser and adapter	Integration tests	After both outputs exist
Validate the combined behavior	All changes	One integrated result	No, one owner should accept it

The table makes dependencies visible. Starting implementation before the interface is settled can produce parallel work that must all be redone when the decision changes.

Parallelize independent slices, not copies of the same judgment

Good candidates for parallel work usually have:

separate files or modules;
explicit input and output contracts;
limited shared mutable state;
independent tests or fixtures;
a clear handoff point.

Poor candidates include two agents editing the same state machine, multiple workers independently redefining a public API, or several reviewers given the same context and then treated as statistically independent votes. Different agent names do not guarantee different assumptions, tools, training data, or blind spots.

For one current platform example, OpenAI’s Agents API announcement describes coordinating subagents and running programmatic tool calls in parallel. Those are execution capabilities, not evidence by themselves that a particular work decomposition reduces end-to-end effort.

Parallel analysis can still be useful. Ask one worker to inspect migration risks and another to inspect test coverage, then have a responsible integrator compare their evidence. Do not count agreement as proof when both may have relied on the same mistaken premise.

Name one owner for every shared surface

Assign ownership at the level where conflicts actually occur:

one owner for the interface or schema;
one owner for each shared file or module;
one integrator for the combined branch;
one person accountable for the final acceptance decision.

Workers should return bounded changes with a short report: paths changed, tests run, results, assumptions, and unresolved questions. Avoid giving every worker authority to merge into the shared branch; that converts a coordination problem into a shared-state race.

For code, isolated branches or worktrees can reduce accidental collisions. For research or review, separate notes with source links preserve the reasoning each worker used. The integration owner then compares outputs and resolves contradictions rather than concatenating them blindly.

Include integration in the schedule

Total completion time is shaped by the longest dependency path plus handoffs, conflict resolution, combined checks, and review. If the team counts only the time spent by workers, it leaves integration invisible.

For example, three parallel changes may each finish quickly but share a generated file, rely on different versions of a contract, and fail together under the integration tests. The result is not three completed tasks; it is one unresolved integration episode.

Reserve explicit time for:

collecting results and confirming each worker stayed within scope;
comparing assumptions and resolving interface differences;
applying the changes to one canonical branch or workspace;
running checks against the combined state;
reviewing seams between components and documenting what remains unknown.

The combined state needs fresh evidence. Separate green results from isolated branches do not prove that their combination works.

Start with the smallest useful team

For a first pilot, choose an issue that has at least two genuinely independent slices. Give each slice a named output, separate workspace, and common integration criteria. Record:

time to first useful result;
human coordination and review minutes;
duplicate or conflicting work;
interface changes after work began;
integration failures and rework;
combined acceptance result.

Then compare with a similar task handled sequentially or with the team’s established workflow. Do not compare a highly decomposable task with a tightly coupled one and attribute the entire difference to agent count.

Add another worker only when the work graph has another independent slice and the first group’s coordination cost is understood. More workers can increase capacity, but they can also increase the number of handoffs and the effort needed to establish one coherent result.

Separate coordination from authority

A coordinator can distribute work and collect evidence without receiving permission to merge, release, or decide product policy. Keep the coordinator’s authority no broader than its task requires. A human or deterministic gate should own the final acceptance boundary according to the risk of the change.

This is an organizational design choice as much as a model choice. Clear work units, stable interfaces, isolated execution, and explicit accountability help whether the workers are AI agents, human developers, or a mix of both.

The topology review

Before launching parallel work, answer:

Which tasks are independent, and which are blocked on a decision?
Which files, services, datasets, or environments are shared?
Who owns each shared interface and the final integration?
How will conflicting assumptions be detected?
Which tests must run on the combined result?
How will review and coordination effort be counted?
What condition ends parallel execution and returns the task to one owner?

If the answers are missing, adding agents is likely to make the uncertainty move faster. First make the work graph legible; then parallelize the slices that remain independent.

References

Source, license, and AI assistance

This article develops the organizational-design and parallel-delegation questions in Part II of From Vibe Coding to Agentic Software Engineering. The source record credits ChatGPT as preparer and identifies CC BY-NC-SA 4.0. This version is substantially reorganized and expanded with a work-topology model, integration practice, and updated references, and is shared under the same license: CC BY-NC-SA 4.0.

AI disclosure: The article text was generated primarily by AI. A human publisher supplied the topic, source material, and editorial direction, and remains responsible for checking claims and examples before publication.

Measuring Agentic Engineering: Count Review, Rework, and Value

qnbs — Thu, 01 Oct 2026 18:45:00 +0000

“The agent finished in five minutes” is a timing observation, not a productivity result. Someone may have spent an hour shaping the request, reviewing a large patch, repairing a regression, or waiting while doing other work. Conversely, a task that took the same time may now be reliable enough to attempt when it was previously uneconomic.

Measuring agentic engineering well means measuring more than output volume or elapsed time. The useful question is whether a workflow produces more acceptable engineering value for the total human and system effort, without silently lowering quality or shifting cost to another stage.

Do not compress unlike outcomes into one score

Use a small dashboard rather than one magic multiplier:

Dimension	What to capture	Why it matters
Outcome	Accepted tasks or capabilities, with a fixed definition of acceptance	More code or more closed tickets can still mean less useful work
Elapsed time	Start-to-accepted wall-clock time	Shows user-visible lead time, but can hide parallel waiting
Human attention	Active minutes for task framing, supervision, review, and recovery	Agent runtime is not the same as human effort
Rework	Returned patches, retries that changed nothing, reopened tasks, and reversions	A fast first pass may create a slower overall path
Quality	The same tests, defect checks, maintainability review, and security requirements	A speed result that relaxes the quality bar is not comparable
System cost	Model/tool usage or infrastructure cost when available	Helps explain whether the workflow’s benefit is economical
New work enabled	Previously deferred tasks that were completed, tracked separately	Captures expanded capability without pretending there was an identical baseline

These measures answer different questions. Do not add minutes, dollars, defect counts, and newly enabled work into a single total unless a defensible value model exists and stakeholders agree with its assumptions.

Count the human work around the agent

Record at least four kinds of time:

Framing: understanding the task, defining scope, preparing context.
Supervision: answering questions, correcting direction, and resolving blocked states.
Review: examining the patch, evidence, and remaining risk.
Recovery: rework, repair, rollback, or incident response attributable to the attempt.

Also record agent runtime and idle waiting, but keep them separate. If a developer launches two independent agents and works on another task while they run, adding both agent runtimes to wall-clock time would overstate elapsed time. Counting only the five-minute wait would understate the person’s total effort if they also spent twenty minutes reviewing both results.

Agree on the accounting rule before gathering data. A useful policy might be: wall-clock time is measured from task start to accepted completion; active human minutes are the sum of time spent on this task even if it occurred in short intervals; agent runtime is recorded separately; overlapping agent runtime is never added to human time.

Compare like with like

AI-assisted and non-AI tasks should be comparable in scope, task class, repository familiarity, and acceptance bar. If people select only easy tasks for the agent and keep difficult work for themselves, the observed times do not estimate a general effect. If the workflow changes halfway through the measurement window, the before-and-after comparison also includes that change.

A practical internal pilot can:

Choose a recurring task family, such as test updates or small dependency migrations.
Record difficulty, repository familiarity, and expected risk before work starts.
Use matched tasks or a randomized assignment where practical and acceptable.
Freeze the same definition of “accepted” for both paths.
Record every eligible task, including abandoned, blocked, or manually completed ones.
Review the results by task class and experience level before calculating an overall figure.

A small pilot can identify workflow problems. It usually cannot establish a universal productivity effect. Report the number and kind of tasks, the spread of results, changes in the workflow, and what was excluded.

Design for the measurement problems already visible

METR’s randomized 2025 study of experienced open-source developers and familiar repositories reported a slowdown for the early-2025 tools it tested. Its 2026 update said the follow-on measurements were difficult to interpret: some developers avoided tasks assigned to the no-AI condition, and some found it hard to report time when several agents ran concurrently and they worked on unrelated tasks while waiting.

The lesson is methodological, not a forecast that agents slow developers down. The same workflow can change who accepts a task, which tasks are attempted, and how time is spent. A credible measurement plan should record those shifts instead of treating the task list and clock as neutral.

For example, report task substitution separately:

Existing-task effect: did the workflow change effort or quality on work the team already did?
New-task effect: did it make a new kind of work feasible?
Value effect: was the new work worth doing to the people who use it?

An agent can increase capability without shortening every task. It can also shorten a task while increasing review burden or defect risk. Keep these outcomes visible instead of forcing them into a single percentage.

Use a quality gate, not a speed target

Set the acceptance bar before the experiment. Specify what tests, review conditions, security controls, and operational evidence each task class requires. Decide how failed checks, reopened changes, or escapes affect the outcome.

For a pilot, ask a reviewer to classify defects without being told whether AI was used where feasible. Blinding may be impractical, but a consistent rubric still helps. Include time for review in the measurement; otherwise the team may appear faster simply because the review has not happened yet.

Do not reward an agent for closing more tasks if it broadens scope, weakens tests, or leaves work for someone else. A useful headline can be as simple as: “Across this task family, median accepted completion time changed by X; review effort changed by Y; defect/rework observations were Z; the sample is small and the task distribution changed in these ways.” Only calculate X, Y, or Z from actual observations.

A one-page measurement card

Before a pilot, agree on:

Question: What change are we trying to estimate?
Task set: Which tasks count, and how will their difficulty be recorded?
Comparison: How are assisted and comparison tasks selected?
Acceptance: Which checks and review conditions must both paths pass?
Time rule: How are framing, supervision, review, waiting, and overlap counted?
Quality signals: Which defects, reopens, or reversions will be tracked?
Analysis: Which groups or task types will be reported separately?
Stop rule: What quality, safety, or cost result would pause the pilot?

The stop rule matters. Averages should not hide a workflow that is faster on routine changes but unsafe on migration or security work.

Optimize for useful capability per unit of attention

The most valuable result may be fewer repetitive steps, faster feedback, less context switching, or the ability to attempt a previously deferred improvement. Those benefits matter, but they should be described accurately: a measured time reduction, a measured quality change, or a new capability observed in a specific workflow.

Agentic engineering is not measured well by how many tokens it generates, how many tools it calls, or how confidently it reports success. Measure outcomes people accept, all the effort required to reach them, and the quality of the resulting system. Then preserve uncertainty in the report.

References

Source, license, and AI assistance

This article develops the conditional-productivity and measurement questions in Part II of From Vibe Coding to Agentic Software Engineering. The source record credits ChatGPT as preparer and identifies CC BY-NC-SA 4.0. This version is substantially rewritten and expanded with a measurement dashboard, pilot design, and updated research references, and is shared under the same license: CC BY-NC-SA 4.0.

Least Authority for Coding Agents: Permissions by Task, Not by Role

qnbs — Thu, 01 Oct 2026 18:25:00 +0000

An agent may be capable of editing a release workflow, deleting an environment, or publishing a package. That capability does not answer whether the current task needs any of those powers.

The safer design question is smaller: for this episode, which exact reads and writes are necessary, against which resources, and with what limits? This applies the familiar principle of least privilege to agent workflows. The access belongs to the task and its risk, not to a broad identity called “the coding agent.”

Start from actions, not trust labels

“Trusted agent” is too coarse to be an access policy. A single workflow may read a repository, edit a branch, run a shell command, create a pull request, merge it, and deploy. Those actions have different consequences and should not inherit one permission setting merely because one agent can perform them all.

Write an action inventory before enabling tools:

Action	Resource	Minimum access for a draft task	Higher-risk boundary
Read source and tests	One repository	Read-only repository access	Secrets and private data excluded unless essential
Edit implementation	Isolated branch or worktree	Write to the task branch	Protected branch remains unavailable
Run checks	Local or disposable environment	Execute within resource and network limits	No production credentials or unbounded host access
Create a review request	Named repository	Create a pull request	No merge permission
Change shared configuration	Specific setting or workflow	Prepare a proposed diff	Human review and approval before applying
Release or deploy	Release system	None for ordinary implementation tasks	Separate, explicit authorization and identity

The last column is not a universal tiering standard. It is a prompt for a team to match the boundary to its own system and consequences.

Divide access into useful slices

A practical starting model has four slices:

Observe: read only the task context and resources needed to understand it.
Prepare: edit an isolated branch or workspace and run bounded local checks.
Propose: submit a reviewable change or draft request without applying it to a protected target.
Commit a consequential transition: merge, publish, deploy, delete shared data, rotate credentials, or change access policy.

Many low-risk episodes can safely stop after the second or third slice. The fourth is a separate decision. Keeping it separate lets the agent perform useful engineering work without turning access to a tool into authority to use every function that tool exposes.

Make the boundary real in the toolchain

Instructions such as “do not deploy” help, but a token that can deploy still makes deployment possible if the agent misunderstands the request or an input steers it toward the wrong action. Prefer controls that make out-of-scope actions unavailable:

use read-only credentials for investigation;
scope write tokens to the smallest repository or resource set;
grant access only for the time needed;
use disposable environments for commands that may have side effects;
keep production secrets out of the agent’s environment;
separate pull-request creation from merge and deployment permissions;
set explicit token permissions rather than inheriting broad defaults.

GitHub’s Actions security documentation, for example, recommends read-only defaults for the workflow token and increasing permissions only where a job needs them. Its workflow syntax also lets maintainers state per-permission read, write, or none. That is an implementation example of reducing ambient authority; it does not prove that a workflow is safe by itself.

Treat incoming repository content as data

An agent may read issue text, source comments, test fixtures, generated files, or web pages. Some of that text may contain instructions. Reading it does not make it a legitimate source of authority to widen permissions, reveal secrets, or change the assigned goal.

Keep the trusted task contract separate from material being inspected. Tool wrappers should validate the target and operation against the current task’s grant. A repository file can explain how the project works; it cannot authorize a deployment merely by saying “deploy now.”

This separation is especially important when the same agent can both interpret arbitrary text and invoke tools. A narrow tool that accepts a structured operation against an approved resource is easier to bound than an unrestricted shell with ambient credentials.

Bind approval to the action

“You have my approval” is not a useful system boundary if it does not specify what action was reviewed. For an operation that requires human authorization, show the person:

the exact target and intended change;
the reason the operation is needed;
the relevant diff or payload;
the expected effect and rollback path;
the identity that will execute it.

Then require an affirmative approval for that action. If the target or payload changes materially, the old approval should not silently carry over. For high-impact transitions, use a separate approval and execution identity where the organization’s controls support it.

An approval should also have a clear expiration. A person who approved a patch yesterday did not necessarily approve a new patch today or a release of an artifact built from different source.

Make reversibility part of access design

Reversible operations are easier to delegate than irreversible ones, but “reversible” must be checked rather than assumed. A branch edit can usually be discarded. A migration that deletes old records may not be. A draft post can be withdrawn; a public post can be copied, indexed, or quoted after deletion.

Before granting write access, ask:

Can the change be previewed before it takes effect?
Can the agent operate on a disposable copy?
Is the result idempotent if a tool call is repeated?
Is there a reliable rollback or compensating action?
Who owns recovery if an operation partially succeeds?

The more difficult recovery would be, the more tightly the action should be scoped and the more clearly a human must approve the final transition.

Revisit permissions when the task changes

Least authority is not a one-time role assignment. A task can change while being investigated. If a documentation issue turns out to need a schema migration, the original grant does not automatically expand. The agent should state the new action it needs, why it is needed, and what permission boundary it would cross.

For repeat work, encode the minimum grant in the workflow and review it as the tools or task change. Remove permissions that are no longer needed. A broad permission is not justified by the fact that it made one difficult episode convenient.

The permission review

Before a coding agent starts, check:

What can it read that is not needed for this task?
What can it write, and where?
Which commands can reach the network or modify shared state?
Could a tool call cross a protected boundary without a person?
What happens if input text tries to redirect the agent?
Can each higher-risk action be presented and approved separately?

The useful default is simple: give an agent the smallest operational space in which it can produce and verify the requested change. Let it prepare work; keep consequential authority at the boundary where the consequences become real.

References

Source, license, and AI assistance

This article adapts the least-authority, separation-of-duties, and escalation principles in Part II of From Vibe Coding to Agentic Software Engineering, whose source record credits ChatGPT as preparer and identifies CC BY-NC-SA 4.0. This version is substantially reorganized and expanded with an action matrix, examples, and primary security references, and is shared under the same license: CC BY-NC-SA 4.0.

Observe, Model, Act, Verify: A Control Loop for Coding Agents

qnbs — Thu, 01 Oct 2026 18:05:00 +0000

A coding agent can run tests, read the failure, edit code, and run the tests again. That sounds like a simple loop. Without a control structure, though, the same loop can become a string of untracked edits, repeated commands, and increasingly confident summaries.

A dependable workflow treats every action as a hypothesis about the system and every result as evidence that may change the next step. It also names the points where an agent must stop. The goal is not to keep the loop running until it says “done”; the goal is to produce a useful state transition with evidence a reviewer can inspect.

Model the work as states

An engineering episode can be represented as a small state machine:

State	Question to answer	Exit condition
Intake	What outcome, scope, and constraints were requested?	The task contract is understood; material ambiguity is recorded
Observe	What do the repository, tests, logs, and docs actually show?	Relevant facts and unknowns are separated
Model	What failure or change mechanism best explains the evidence?	There is a testable working hypothesis
Plan	What bounded actions can test that hypothesis?	Actions fit the authority and scope
Act	What changed, and what commands or tools ran?	The planned action produced a result or a stop condition
Evaluate	Does the result support the acceptance conditions?	Evidence is accepted, contradicted, stale, or insufficient
Exit	Is the episode complete, blocked, or ready for human judgment?	A truthful status and evidence report are returned

The names are not sacred. A team can combine or rename states. What matters is that observation is not confused with inference, an attempted action is not confused with success, and a model’s summary is not substituted for a tool result.

Keep facts, hypotheses, and decisions apart

Suppose a test fails after a dependency update. A disciplined trace might say:

Observed: the token-refresh test fails with an unexpected 401.
Hypothesis: the new client no longer retries after refreshing credentials.
Next action: inspect the new retry behavior and add a focused regression test.
Decision boundary: if preserving behavior requires changing the public authentication contract, stop for a human decision.

The separation is useful because a plausible explanation can be wrong. If the agent records its hypothesis as though the logs established it, later actions inherit a false premise.

When multiple explanations remain possible, design the smallest useful experiment. Read the relevant library documentation, isolate the test, compare the old and new behavior, or inspect a sanitized trace. Avoid changing several unrelated variables at once; otherwise the outcome may not tell you which change mattered.

Make evidence expire when its subject changes

Verification is tied to the thing that was checked. If the code changes after a test run, that test result no longer describes the current code. If a test command is rerun with a different configuration, the old and new results are not interchangeable. If an agent changes the acceptance test itself, a green result needs a separate review of that change.

Treat an evidence item as a record with at least:

Evidence field	Example
Subject	Working-tree revision or built artifact
Check	Exact command, test suite, or policy
Context	Runtime, fixture, configuration, and relevant dependencies
Result	Exit status plus failures, skips, or unknowns
Time/order	Whether it ran before or after the last relevant change

An important rule follows: after a material edit, rerun the checks that the edit could invalidate. A test from before the patch can explain the starting condition; it cannot qualify the final patch.

This does not require rerunning every expensive job after every keystroke. Match the check to the risk and the changed surface. A documentation-only change might need a link check and rendered preview. A serialization change may need compatibility fixtures and migration tests. The completion report should say what was not rerun and why.

Use retries to learn something

Retries are useful when the next attempt differs in a way that could address the observed failure. Repeating an identical command against an unchanged state usually gives the same evidence and consumes attention.

Before a retry, ask:

What did the previous attempt establish?
What will be different in this attempt?
What result would change the working hypothesis?
How many attempts or how much runtime did the contract allow?

If a command fails because the local service is not running, starting that service is a meaningful next step. If it fails with the same assertion after two unmodified runs, another identical run is not a plan. If a tool call may have partially changed an external system, first determine whether repeating it is safe; a non-idempotent action can create duplicate tickets, releases, or payments.

Bound retries by both count and consequence. For example: retry a deterministic local check once after a relevant fix; stop after a repeated infrastructure failure; require a person before repeating any external action whose first outcome is unknown.

Put the stop decision in the workflow

Stop and escalate when:

the task requires a product or policy choice the contract did not delegate;
the available evidence conflicts or cannot identify the tested state;
a required credential or production data would be needed;
the same failure persists without a new diagnostic signal;
a proposed fix expands the scope or weakens an invariant;
an external side effect may have happened but cannot be confirmed.

These are workflow outcomes, not model moods. “I am confident” should not override a failed check or missing permission. Likewise, low confidence alone need not block a reversible, low-risk investigation if the agent can gather better evidence within its scope.

Capture enough of the trajectory to review

A useful trace records the task identifier, relevant inputs, tool calls, changed paths, check results, handoffs, and final status. Keep secrets and unnecessary personal data out of that record. The purpose is not to preserve every token; it is to let someone answer what the agent saw, what it did, what it learned, and why it stopped.

For teams using an agent platform, inspect what its trace actually captures. OpenAI’s current evaluation guide describes traces as end-to-end records of model calls, tool calls, guardrails, and handoffs. That is a useful example of a trace surface, but no vendor trace by itself proves that the application’s acceptance criteria were valid or that a consequential action was authorized.

A practical completion report

At the end of the episode, return a short, structured report:

Status: complete, incomplete, blocked, or escalated.
Change: what files or external records changed.
Evidence: exact checks and results, including failures and skips.
Assumptions: choices made because the task left a gap.
Remaining risk: behavior not exercised or not covered by the evidence.
Next decision: the specific question a person needs to answer, if any.

This format makes a failed episode useful too. “Blocked because the provider’s migration guide leaves token refresh behavior unspecified” is actionable. “Couldn’t finish” is not.

The control-loop test

Before adopting a workflow, walk through one ordinary task and one deliberately awkward case. Ask whether it can distinguish a fact from a hypothesis, invalidate old evidence, limit repeated actions, and reach a truthful stop state. If its only terminal condition is a success message, the loop is incomplete.

An agentic engineering loop is effective when feedback changes what happens next and authority constrains what may happen at all. That is how repeated tool use becomes a controlled engineering process instead of a long conversation with a terminal.

References

Source, license, and AI assistance

This article is based on the agentic engineering loop and evidence concepts in Part II of From Vibe Coding to Agentic Software Engineering, whose source record credits ChatGPT as preparer and identifies CC BY-NC-SA 4.0. This version is substantially reorganized and expanded with original examples and a practical workflow, and is shared under the same license: CC BY-NC-SA 4.0.

The Engineering Episode: A Work Contract for Coding Agents

qnbs — Thu, 01 Oct 2026 17:45:00 +0000

“Upgrade the authentication library safely” sounds like a task. For an agent, it is a bundle of decisions: which package, what counts as safe, which files may change, whether a lockfile update is allowed, what tests matter, and who may approve a release.

If those decisions stay implicit, the agent has to guess. A better unit of delegation is a bounded engineering episode: a task described by the result it should produce, the boundaries it must preserve, the evidence it must return, and the point where it must stop.

That small change in framing makes a request easier to delegate, review, and repeat. It also keeps the agent’s freedom over implementation separate from the human’s authority over outcomes and risk.

A task is more than an instruction

An instruction describes an action: “change this function,” “update the dependency,” or “make the build pass.” A work contract describes a desired state and the conditions under which that state is acceptable.

For an engineering episode, write down six things:

Outcome: the externally meaningful state you want.
Scope: the parts of the system the agent may inspect or change.
Invariants: what must remain true, even if the implementation changes.
Authority: which tools and consequential actions are allowed.
Evidence: what observations would support the completion claim.
Exit: when to stop, report a limitation, or ask a person to decide.

This is not ceremony for its own sake. It moves ambiguity to the beginning, when a person can resolve it, instead of letting it surface after a broad patch has already been produced.

OpenAI’s practical agent-building guide also recommends beginning with a specific workflow and planning for human intervention when retries are exhausted or an action is high-risk. The episode card below is an editorial synthesis, not a vendor-prescribed schema.

Start with the result, then bound the path

Consider a hypothetical task: update a client library to a supported version while preserving the current API behavior.

“Upgrade the library and fix anything that breaks” leaves a large decision space. Does “anything” include unrelated tests? Is a major-version migration acceptable? Can configuration or credentials change? May the agent update the lockfile? Is a successful local build enough to call the work done?

A tighter contract might read:

Outcome: Move the application from client-library version 4.x to the approved 5.x release while preserving the documented login and token-refresh behavior.

Scope: Inspect the dependency manifest, lockfile, client adapter, and related tests. Change those files and add or update focused tests. Do not reformat unrelated code.

Invariants: No credential values in source control. No change to public method names or token storage. No network call to a production identity provider.

Authority: Read the repository and documentation; edit the current working branch; run local tests. Do not merge, publish a package, modify secrets, or deploy.

Evidence: Report the dependency diff, focused test results, full relevant test command and exit status, and any behavior that could not be exercised locally.

Stop: Stop before changing public behavior, encountering an undocumented migration choice, needing a secret, or repeating the same failing check without new information.

The example is hypothetical, but the structure is reusable. Notice that it grants freedom over how to adapt the client while reserving high-impact decisions for a person.

Make constraints observable

“Keep it secure,” “don’t break anything,” and “use best practices” express intent but do not tell an evaluator what to inspect. Where possible, translate them into observable conditions:

Vague instruction	More reviewable condition
Keep the API compatible	Existing public method names and documented request/response shapes remain unchanged
Do not weaken authentication	Existing authorization tests still pass; no credential is added to tracked files
Keep the change small	Only named dependency, adapter, and test paths change unless a new path is explained first
Verify the update	List the exact commands run, their exit codes, and checks skipped

Not every value can be reduced to a test. The point is to separate what can be checked mechanically from what needs human judgment. An agent should not quietly convert an unmeasured preference into a claim that it has been satisfied.

Scope is an access boundary

The scope line should match the actual environment, not merely appear in the prompt. “Do not deploy” is weak if the agent still holds deployment credentials and an unrestricted release tool. A contract becomes credible when the available permissions enforce its boundaries.

For a low-risk documentation edit, read access plus a branch-local write may be enough. For a production change, the agent may prepare a patch and evidence while a separate release process retains deployment authority. The work contract should name which boundary is enforced by tooling and which is only an instruction.

Write exclusions as carefully as inclusions. “Update the adapter and tests” does not imply permission to rotate credentials, change account policy, or publish the package. If an adjacent issue appears, the agent can report it as a follow-up rather than expanding the task on its own.

Define done and define not-done

A useful completion statement is falsifiable. “The task is complete” is not evidence. “The adapter and focused tests changed; command X exited zero on revision Y; integration behavior against the live provider was not tested” gives a reviewer something concrete to assess.

The contract should also make room for honest partial completion:

Complete: the requested state was reached and the named evidence exists.
Blocked: a required decision, permission, dependency, or external state is missing.
Incomplete: the agent made progress but one or more acceptance conditions remain unmet.
Escalated: the work exposed a risk or choice outside the delegated authority.

These states prevent the pressure to produce a polished “done” message from turning an unknown into a pass.

A compact episode card

For routine work, the contract can fit in a short issue comment or task form:

Field	Prompt
Outcome	What observable state should exist at the end?
Scope	Which repositories, files, environments, or records are in bounds?
Invariants	What must not change?
Allowed actions	What may the agent read, write, run, or submit?
Evidence	What should the completion report include?
Stop and escalate	What ambiguity, risk, or repeated failure ends the episode?
Owner	Which human owns decisions and accepts the result?

If the task cannot be described this way without major uncertainty, that is useful information: it may need discovery or a human design decision before implementation begins.

The practical test

Before handing work to an agent, ask:

Could two reasonable people interpret the requested outcome differently?
Can the agent affect anything outside the named scope?
What result would prove progress, and what result would only look reassuring?
Which decisions are reversible, and which need a person first?
What should happen when the available evidence is incomplete?

The strongest work contract is not the longest prompt. It is the shortest description that makes the desired result, permitted actions, evidence, and exit conditions clear enough to review.

An agent can improvise the route. The contract defines the destination, the guardrails, and the point where a human must take the wheel.

References

Source, license, and AI assistance

This article is based on the engineering-episode concept in Part II of From Vibe Coding to Agentic Software Engineering, whose source record credits ChatGPT as preparer and identifies CC BY-NC-SA 4.0. This version is substantially reorganized and expanded with original templates and examples, and is shared under the same license: CC BY-NC-SA 4.0.

Pin the Evaluator, Not Just the Dependency

qnbs — Thu, 01 Oct 2026 17:25:00 +0000

“The tests passed on commit abc123” sounds precise. It may still leave the most important question unanswered: which tests, run by which evaluator, against which artifact?

A source revision is one part of an evidence chain. If the qualification job rebuilds the application, tests a different binary, or silently changes its fixture, a passing result may not describe the thing that is about to ship.

Identify both the subject and the measurement

For artifact qualification, a useful chain names each link:

source revision → build workflow run → artifact identity and hash → fixture identity and hash → qualification result

That answers the subject question: what exact bytes were examined?

The evaluator also has an identity. Record the test suite or policy version, runner image, relevant toolchain, workflow revision, configuration, and the action or dependency versions that could affect the result. For AI-assisted evaluation, record the model/provider snapshot when available, prompt or rubric version, tool permissions, and whether the result is advisory or a required control.

Not every project needs a giant manifest for every local test. The detail should match the consequence. A release artifact, security decision, or externally relied-on certification warrants stronger provenance than a quick developer experiment.

Why a moving evaluator changes the meaning of green

Suppose a pipeline publishes build A, then runs the same source through a second job that creates build B and tests B. If the results differ, the published artifact has not been qualified. Matching source commit is useful, but it does not prove identical output.

Likewise, a test suite can drift. A fixture may be edited to match the implementation. A policy prompt can become weaker. A workflow can start with broader token permissions. A package can update underneath a floating version reference.

If the evaluator changes, the result can change even when the code does not. That does not mean the evaluator should never improve. It means an evaluation should say which evaluator produced it.

Provenance is a chain, not a badge

SLSA's provenance model gives teams a vocabulary for describing where an artifact came from and how it was built. GitHub Actions guidance recommends pinning third-party actions to full commit SHAs when immutable references are required, and minimizing token permissions. Those are useful integrity measures, but they are not a complete trust argument.

An immutable action can still contain a bug. A signed attestation can faithfully describe an unsafe build. A pinned model snapshot can be biased or inadequate. A fixed test suite can miss a whole class of failure.

Pinning answers “which one?” It does not automatically answer “is this trustworthy?” A robust chain also asks:

who controlled the workflow and its inputs;
whether the workflow ran against the intended revision;
whether untrusted pull-request code had access to secrets or write tokens;
whether the artifact hash matches the one users will receive;
whether the test and fixture are independent enough to detect the defect;
what a missing, stale, or inconclusive result means.

Freeze for comparison, update under governance

“Pin everything forever” would trade one failure mode for another. Old evaluators become stale. Vulnerable dependencies need upgrades. Models and threat assumptions change.

A better rule is: freeze identity for a specific qualification, and govern changes to the evaluator as their own reviewed changes.

When a test harness or policy changes, compare versions, document why it changed, rerun relevant cases, and record whether old results need to be recomputed. If the candidate artifact changes, qualify the new artifact. Do not carry evidence from one identity over to another because the names look similar.

For AI reviewers, version the contract around the model as well as the model where possible: input scope, instructions, tool access, review rubric, expected output, and the authority assigned to the result. If the provider does not expose a stable model snapshot, record that limitation instead of claiming full reproducibility.

Make the evidence readable to a reviewer

A useful qualification record lets another person answer:

What exact artifact was tested?
Which workflow and evaluator produced the result?
What fixtures or inputs were used?
Which checks passed, failed, were skipped, or were unknown?
Did the exact published artifact receive the evidence?
What changed after the evaluator was frozen?

If the answer to one of those questions is unknown, name the gap. The right state is not a green badge with a footnote hidden in another job.

The evaluator is part of the system

Tests and review tools are software too. They have dependencies, permissions, failure modes, owners, and release cycles. Treating them as invisible infrastructure makes it easy to trust results whose identity has drifted.

The engineering goal is not to create an impossible guarantee. It is to make each claim traceable: this is the artifact; this is the evaluator; this is what it checked; this is the evidence; and this is what remains outside the result.

That completes this Series C path from authority to evidence: the reviewer can help find a risk, a correction cycle can converge on a change, an accepted lesson can become a guard, and the guard itself can be identified and maintained.

References

AI assistance was used to prepare this draft. The human editor is responsible for validating source freshness, exact platform guidance, and the final publication decision.

From AI Finding to Deterministic Guard

qnbs — Thu, 01 Oct 2026 17:05:00 +0000

An AI reviewer says a parser accepts a malformed document. The comment is plausible. It is not yet a regression test, and it is not yet a policy.

Between a model finding and a reliable guard sits a small but essential engineering process: validate the behavior, identify an owner, reproduce the failure, encode the invariant, and prove the new check catches the case without breaking the supported ones.

Treat the model output as a lead

A useful finding should describe more than a suspicious line. It should offer a concrete path from input to consequence:

Which input or state triggers the behavior?
What does the system do?
What invariant is violated?
What would a correct result look like?
Which evidence supports the claim?

If those questions have no answer, ask for clarification or reproduce the behavior before editing code. A model can be right about the shape of a risk while wrong about reachability, ownership, or intended behavior.

Do not let the model write a test that merely enshrines its own interpretation. A test is executable policy: a maintainer must agree that the assertion is the rule the product should keep.

The finding-to-guard path

1. Assign a human owner

Choose someone who understands the affected boundary or can route it to the right owner. The model may help locate code; it does not take accountability for deciding priority or semantics.

2. Reproduce or reject

Build the smallest reliable example. It might be a failing unit test, an integration fixture, a browser path, or a packaged-runtime reproduction. Record the environment and version. If the reproduction fails, preserve that result and explain why the original hypothesis was rejected.

“Could not reproduce” is not the same as “impossible.” It may mean the environment or evidence was incomplete. Keep uncertainty visible.

3. State the invariant

Write the expected rule in language a reviewer can assess. For example: “An import that fails validation must not replace the currently selected project.” That is more useful than “handle the error safely.”

4. Add the smallest durable check

Encode the invariant at the lowest test layer that can reliably detect the behavior. A unit test may be enough for a pure parser rule. A persistence boundary may need an integration test. A first-launch claim may require a packaged application. A test at one evidence plane does not prove another.

5. Prove the guard has signal

Run it on the known failing case and on representative supported cases. Where appropriate, temporarily remove or invert the behavior to show the test fails for the defect it is meant to catch. Avoid asserting that a test is strong simply because it passed once.

6. Connect the check to the policy

If this behavior must block a merge, bind the result to an owned workflow and the exact revision through repository rules. A local green run and a required merge check are different facts.

A finding is not automatically a rule

Some model findings are contextual, subjective, or too noisy to encode. A one-off design trade-off may need a decision record, not a static analyzer. A threat hypothesis may need a security review. A code-style suggestion may belong in a formatter—or may not deserve a comment at all.

The right outcome can be:

accept and add a regression guard;
accept the design change but use a different test;
reject the finding with a short reason;
defer it to a named issue because required evidence is unavailable;
escalate if the potential consequence needs specialist review.

Forcing every observation into a CI rule creates a brittle test suite. Ignoring every observation because the model is imperfect throws away useful attention. Human triage is the conversion boundary.

Preserve evidence provenance

For a durable guard, record what it tested and what it cannot show:

fixture identity and expected behavior;
test command and relevant environment;
source revision;
whether the check ran locally, in CI, in a browser, or on a packaged artifact;
known blind spots and any deferred platform path.

This prevents later readers from turning “the test passed” into a stronger claim than the test supports.

From correction to institutional memory

A review comment disappears into a pull-request timeline. A regression test can keep the lesson available to the next contributor. A static policy can make a repeated architectural boundary explicit. A runbook can teach a human how to investigate a failure that cannot yet be automated.

The goal is not to automate every judgment. It is to avoid paying again for a failure mode that can be stated precisely and checked cheaply.

In the next post, we will look at a harder version of the same problem: how to know that the evaluator itself—the tests, tools, and evidence pipeline—has not drifted away from the claim it is supposed to support.

References

SWE-bench (2023) — a benchmark for issue-resolution agents, not a production assurance claim.
SWE-Proof (2026 preprint) — formal checking of test-passing patches in a purpose-built corpus; results are corpus- and specification-dependent.
GitHub ruleset rules
SLSA v1.2 provenance

AI assistance was used to prepare this draft. The human editor is responsible for choosing the invariant, validating the example, and reviewing the publication claims.

One Correction Wave: A Stop Rule for AI-Assisted Review

qnbs — Thu, 01 Oct 2026 16:45:00 +0000

A reviewer finds a bug. A coding agent fixes it. The fix triggers another review. The reviewer finds a style issue. The agent changes it. A new run notices a nearby concern. Soon the pull request has ten commits, three of them correcting earlier corrections, and nobody is sure which result is the one to review.

That loop is not proof that AI review is useless. It is a sign that the team has not defined a stopping rule.

Review should converge on a decision

Automated review can produce many observations, but a pull request still needs a finite decision: accept the change, request a bounded correction, or stop because evidence is missing.

A practical pattern is one correction wave:

Collect. Let the intended reviewers and required checks finish for the current revision.
Normalize. Group duplicate comments and distinguish actionable defects from questions, style preferences, and speculation.
Triage. Assign an owner and consequence level. Validate the failure path before changing code.
Correct coherently. Make a related set of accepted fixes together instead of committing each bot suggestion separately.
Re-run evidence. Run required deterministic checks on the new head and inspect the resulting diff.
Decide. Merge, request a specifically scoped next wave, or stop with the remaining unknowns named.

This is a process pattern, not an experimentally proven optimum. The useful property is that it makes the loop observable and gives it a natural point to end.

Why serial micro-fixes create noise

Every new commit changes the object under review. A reviewer may now be commenting on a revision that no longer exists. A small patch can change the meaning of a previous test. A bot may rerun because the head SHA changed and repeat a finding that was already dismissed.

Serial fixes also fragment intent. A single change to validation, error handling, and a regression test may be easier to evaluate as one cohesive unit than as a chain of tiny commits whose dependency is implicit.

The opposite extreme is not better. A large “fix everything” commit can hide unrelated edits. Coherence means the corrections share one accepted invariant or failure path, not that every review comment is bundled indiscriminately.

A triage card for each finding

For every material finding, capture:

Claim: what can go wrong?
Path: how does the code reach the failure?
Evidence: what test, reproduction, or source confirms it?
Owner: who decides whether it is in scope?
Disposition: accept, reject with reason, or defer with an owner.
Durable lesson: should the fix become a regression test or policy check?

This card can be brief. Its purpose is to stop a confident comment from turning directly into a code mutation.

When a second wave is justified

A new correction cycle makes sense when it starts with new evidence: a required test fails on the corrected revision, a reviewer identifies a distinct material defect, or a reproduction changes the risk assessment.

It does not make sense merely because a bot can be triggered again. Re-run a check when the input changed or when the check did not cover the intended revision. Do not ask the same nondeterministic reviewer to rediscover a finding until it returns a satisfying answer.

Define the next wave’s scope before opening it:

what new evidence triggered it;
which findings are in scope;
what artifact or revision is under review;
which checks must pass;
what condition ends the loop.

If a review provider is flaky or unavailable, represent that state honestly. “No comment” may mean no issue was found, or it may mean the review never completed. The status should distinguish those cases.

Keep liveness in the design

Safety asks whether a bad change can get through. Liveness asks whether a good change can get stuck forever.

An AI system that can keep creating commits, re-triggering itself, or reopening dismissed findings can threaten both. Limit automatic retries. Keep comments tied to a precise revision. Avoid granting the reviewer credentials that let it silently approve its own fixes. Preserve a human-visible way to terminate the cycle.

None of these controls guarantee a fast review. They make it possible to see why the review is waiting and who can move it forward.

Measure the loop without gaming it

Count more than comments or generated commits. A team might track:

time from first review to a merge decision;
distinct material findings validated;
accepted findings that led to a regression guard;
repeated findings on unchanged behavior;
corrections that had to be reverted;
checks skipped, stale, or inconclusive.

These measurements need context. A lower comment count is not automatically better; a shorter review can mean a missed defect. Use a portfolio view and pair speed with quality and rework.

A review protocol should terminate

Good review is not maximum review. It is enough review to support a decision, with unresolved risks made visible.

Batching accepted findings into one coherent correction wave helps keep the diff legible and the state machine finite. Re-run deterministic evidence against the corrected revision. If meaningful new evidence appears, open another bounded wave. If the evidence has not changed, stop repeating the same prompt.

The next post follows one accepted finding beyond the pull request: how it becomes a durable deterministic guard, instead of a note everyone forgets after merge.

References

GitHub: About status checks
GitHub Actions security reference
DORA, Balancing AI tensions (2026) — qualitative analysis of engineer responses, not a controlled experiment establishing this exact workflow.

AI assistance was used to prepare this draft. The human editor is responsible for validating the workflow advice against the intended repository and reviewing the final text.

The Reviewer Is Not the Gate

qnbs — Thu, 01 Oct 2026 16:25:00 +0000

An AI reviewer leaves a comment: “This change can lose data when two saves race.”

That might be the most valuable sentence written about the pull request. It might also be a false positive, a misunderstanding of the storage model, or a warning about a path this patch never reaches.

The comment deserves investigation. It should not, by itself, decide whether the change may merge.

Reviews and gates answer different questions

A review system helps people notice things. A merge gate enforces a rule.

An AI reviewer is useful as an advisory signal: it can scan a diff, compare patterns across files, ask whether an invariant was preserved, or suggest a missing test. Its output is probabilistic and context-dependent. A human still needs to judge whether a finding applies.

A deterministic check asks a narrower question against a defined input: did the test command succeed on this revision? Does the static rule find a forbidden import? Is the generated artifact attached to the expected source? A branch rule can then require selected checks before merge.

Neither category is perfect. Tests can omit the bug. Static rules can encode the wrong policy. Reviewers can miss a defect. The important distinction is authority: a generated comment is not automatically a trusted merge condition simply because it appears in a review UI.

A check name is not its identity

GitHub branch protection and repository rulesets can require status checks. GitHub’s documentation also cautions that users with write permission can set commit statuses. A rule that accepts any result named “Security Review” may therefore be weaker than it looks if the expected source or app is not bound.

For a required check to mean something, the policy needs to specify at least:

which trusted application or workflow is allowed to report it;
which commit or revision was checked;
whether the check ran for the correct event and permissions;
what happens when it is missing, skipped, stale, or inconclusive;
who may bypass the rule and how that exception is recorded.

This is an identity and authority problem as much as a test problem. “Green” is a value. The repository also needs to know who produced it, for what subject, under which policy.

GitHub’s rules can be configured in different ways, and bypass privileges depend on the repository’s policy. Do not infer that a repository is protected merely because a check exists or a green badge appears. Inspect the effective rules and the trusted source of each required result.

Let AI review widen attention, not quietly become policy

There are good reasons to run model-assisted review. It can offer another pass over a large change, explain unfamiliar code, or highlight a boundary a human missed. It can also repeat itself, misread generated files, or recommend a fix that breaks a different invariant.

A useful setup keeps the AI result visible and contestable:

The reviewer reports a finding with the affected lines and a concrete failure path.
A maintainer validates or rejects the finding.
Accepted findings are handled through normal review and tests.
Deterministic checks enforce rules that are stable enough to automate.
Human-owned policy decides whether a remaining uncertainty is acceptable.

If a team wants a model’s output to block merges, that is a governance decision. It should be explicit, versioned, scoped, observable, and reversible. The model identity, configuration, failure behavior, and bypass path become part of the policy. Otherwise an advisory tool can become a required check by accident—through a vendor default, a renamed status, or a workflow that nobody intended as authority.

A practical authority table

Mechanism	Best at	Authority it should have
AI review comment	Suggesting risks and questions	Advisory unless an explicit, tested policy says otherwise
Unit / integration test	Checking encoded behavior	Required when relevant and trusted; limited to its covered behavior
Static policy check	Enforcing a crisp, repeatable rule	Blocking only when rule ownership and false-positive path are defined
Required status check	Providing a merge condition	Trusted only when bound to the expected source and revision
Human review / approval	Context, trade-offs, responsibility	Required for decisions policy assigns to a person

This is not a ranking of intelligence. It is a map of who gets to make which decision.

The operational test

Before adding a review bot to required status checks, answer:

What exact property does this check enforce?
Can the result be set by a user or app outside the intended workflow?
Does the workflow test the exact pull-request revision?
Can untrusted pull-request code reach a privileged token or secret?
What does a timeout or provider outage mean: fail, wait, or route to a human?
Who can bypass, and is the bypass visible?
What evidence would show that the policy itself is working?

GitHub’s Actions security guidance matters here: permissions should be minimized, references to third-party actions should be immutable where possible, and workflows that combine untrusted pull-request code with privileged contexts need careful separation.

The goal is not to ban AI review. It is to prevent a useful reviewer from gaining authority through ambiguity.

Capability does not appoint an owner

A model can identify a plausible bug. It cannot, by that fact alone, define the organization’s risk tolerance or accept responsibility for shipping.

A good engineering system lets models help with attention while making the decision path inspectable: a trusted check reports evidence; repository policy decides which evidence is required; a named person owns exceptional judgment.

That is the boundary Series C is about. In the next post, we will look at how to turn a pile of review comments into one focused correction cycle that can actually finish.

References

Read the flagship

For the broader argument behind this boundary, read Vibe Coding Was the Prototype. Agentic Engineering Is the Next Step.. This follow-up applies its evidence-and-authority model to AI-assisted code review and merge policy.

AI assistance was used to prepare this draft. The human editor is responsible for checking current platform semantics, source freshness, and the final publication decision.

Vibe Coding Was the Prototype. Agentic Engineering Is the Next Step.

qnbs — Thu, 01 Oct 2026 16:05:00 +0000

Toward a Star Trek Future of Building Things

How a slightly ridiculous way of making software may turn out to have been the awkward first chapter of a much larger transformation

At a glance

Vibe coding made software generation conversational; it did not replace the work of defining, checking, releasing, and maintaining software.
A useful engineering agent needs a bounded goal, explicit invariants and permissions, relevant checks, evidence, and a stop or escalation rule.
Productivity findings depend on the people, task, tools, and measurement period. Benchmark scores describe their evaluators, not production autonomy.
Cheaper implementation could make more small, tailored software worth exploring, while maintenance, reliability, security, and operating costs remain.
The aim is to expand human agency. People still choose goals, set policy, authorize consequential actions, and remain accountable.

The First Abstraction Shift

There was a brief and beautiful period in the history of computing when the future arrived wearing sweatpants.

The ritual went something like this:

Build me an app.

A pause.

Make it modern.

Another pause.

No, more modern.

A mysterious dependency appeared.

Add authentication.

Fourteen files changed.

Fix the error.

Twenty-seven files changed.

Why did you rewrite the database layer?

The model apologized.

Undo that.

The model rewrote the database layer in a different way.

And somewhere, quietly, a senior engineer felt a disturbance in the force.

This was vibe coding.

The phrase became culturally prominent in early 2025 as a description of programming by conversational intent; Andrej Karpathy February 2025 post is a useful primary timestamp, not a claim that the underlying practice began there: instead of manually attending to every implementation detail, the developer increasingly tells an AI what ought to exist and lets the machine fill in substantial procedural detail.

For a while, the expression seemed to summarize everything simultaneously delightful and horrifying about generative AI.

Software could appear astonishingly quickly.

Some people who had never thought of themselves as programmers could create functioning applications.

Experienced developers could prototype some ideas at striking speed.

Interfaces materialized from descriptions.

Boilerplate evaporated.

Entire categories of annoying implementation work suddenly looked negotiable.

And, naturally, people also discovered that an AI could generate 4,000 lines of code with approximately the same emotional attachment to those lines that a leaf blower has to autumn foliage.

Vibe coding was liberating. Without review on consequential work, it could also be reckless.

It was serious.

It was also hilarious.

Most importantly, it was transitional.

Because the real historical significance of vibe coding was never that programmers had discovered a more relaxed way to type code.

It was that human intention had acquired a new computational interface.

For most of computing history, the difficult part of software creation was translation.

A person imagined a desired state.

Then they translated that state into architecture.

Then into algorithms.

Then into APIs.

Then into control flow.

Then into syntax.

Then into build systems.

Then into deployment systems.

Then, after several hours of discovering that YAML contains more ways to experience regret than previously thought possible, into production.

Generative AI began compressing parts of that translation chain.

A human could state the destination before understanding every road.

That was the breakthrough.

But it was not the destination.

The next stage is emerging now.

It is the transition from vibe coding to agentic software engineering.

And that transition may eventually matter much more than code generation itself.

1. Vibe Coding Was Not the Revolution. It Was the Demo

One easy mistake when thinking about AI-assisted programming is to focus too much on code.

Code is important, obviously.

But code is only one intermediate representation in a much larger engineering process.

Software engineering begins before code exists.

Someone identifies a need.

Someone decides what behavior is desirable.

Someone models the system.

Someone determines what must never happen.

Someone considers failure.

Someone weighs compatibility.

Someone chooses which compromises are reversible.

Someone defines what evidence would count as success.

And software engineering continues after code has been written.

The system must build.

It must pass tests.

It must survive strange inputs.

It must handle data safely.

It must deploy.

It must recover.

It must remain observable.

Its documentation must tell the truth.

Its dependencies must remain supportable.

Its permissions must remain bounded.

Its future maintainers must be able to discover why the strange-looking thing in line 847 exists instead of "cleaning it up" and resurrecting a bug from 2024.

The act of producing source code is therefore only a slice of engineering.

Vibe coding automated that slice first because language models are extraordinarily well suited to turning descriptions into plausible text.

That was enough to make the experience feel magical.

But magic is what technologies look like immediately before we standardize their interfaces.

A more mature version of AI engineering will not be limited to:

Write some code that seems right.

It will increasingly be:

Achieve this bounded engineering objective, preserve these invariants, use these authorized tools, gather this evidence, and stop if the evidence does not justify proceeding.

That is a radically different proposition.

The first is generation.

The second is delegated engineering.

2. Every Great Computing Revolution Is an Abstraction Revolution

The history of computing is, among other things, the history of humans repeatedly becoming unwilling to think about yesterday's details.

Early programmers interacted with machinery near the physical level.

Then came symbolic assembly.

Then higher-level languages.

Then structured programming.

Then libraries.

Then operating systems.

Then databases.

Then object systems.

Then frameworks.

Then virtual machines.

Then package managers.

Then cloud platforms.

Then declarative infrastructure.

Then managed services.

Then serverless systems.

Then application platforms.

At each layer, critics could correctly point out that the lower layer had not disappeared.

Somewhere there is still machine code.

Somewhere there are still memory pages.

Somewhere electrons remain stubbornly involved.

But civilization advances by allowing most people to stop thinking about the lower layer most of the time.

Abstraction does not eliminate reality.

It reorganizes responsibility for reality.

The database engineer does not expect an application developer to position the disk head.

The web developer does not manually implement TCP congestion control.

The mobile developer does not manufacture the accelerometer.

The user who asks a spreadsheet to calculate a sum does not expect to become briefly responsible for floating-point hardware.

Generative AI represents another abstraction layer.

But it is subtler than "English replaces Python."

That slogan is catchy and mostly wrong.

The important abstraction is not natural language over programming language.

It is:

desired state over explicit procedure

Traditional programming asks:

What exact operations should the machine execute?

Declarative programming asks:

What state should exist?

Agentic engineering extends that pattern:

What outcome should exist, under which constraints, and what evidence is required before we accept that it exists?

That is a much more powerful interface.

It changes not merely how code is written, but where engineering effort accumulates.

3. The Dream Is Older Than Generative AI

There is a temptation to treat modern AI-assisted computing as though history began when a chat box appeared.

It did not.

The intellectual ancestry is much richer.

In 1945, Vannevar Bush imagined machinery that could extend human access to knowledge through associative structures rather than merely reproducing traditional filing systems. His famous As We May Think envisioned something closer to an intellectual prosthesis than a conventional calculating machine.

Norbert Wiener's cybernetics, first published in 1948, made feedback and control central conceptual tools for understanding both machines and organisms. The critical insight for our purposes is that competent behavior need not consist of executing a perfect predetermined plan; it may instead involve observing consequences and correcting action through feedback.

J. C. R. Licklider went further in 1960. In Man-Computer Symbiosis, he imagined close cooperation in which humans would establish goals, hypotheses, criteria, and evaluations while machines handled routinizable work supporting those decisions.

Read today, that division of labor feels almost suspiciously current.

Humans define the objective.

Machines perform substantial cognitive operations.

Humans evaluate.

The partnership exceeds either participant alone.

Two years later, Douglas Engelbart described "augmenting human intellect" as increasing humanity's capacity to comprehend complex situations and solve problems that had previously exceeded practical reach.

Then Herbert Simon framed design as the science of artificial systems: not merely studying what is, but systematically reasoning about artifacts constructed in pursuit of purposes.

These ideas form a lineage:

memory augmentation -> feedback -> symbiosis -> intellectual augmentation -> design science

Agentic engineering is not an alien break from that tradition.

It is one of its most literal realizations.

4. From Autocomplete to Agency

Consider the progression.

Stage one: completion

The machine predicts the next token.

Useful, but fundamentally local.

Stage two: generation

The machine produces a function, a component, a test, perhaps a file.

Now intent can produce larger artifacts.

Stage three: conversational editing

The machine modifies existing code according to instructions.

The human can steer iteratively.

Stage four: repository awareness

The machine searches files, understands relationships, inspects configuration, traces definitions, and reasons across multiple modules.

Stage five: tool use

The machine can run tests, issue commands, inspect logs, query services, read documentation, and observe execution results.

Stage six: goal-directed iteration

The machine receives an objective, forms a plan, modifies the environment, observes consequences, updates its approach, and repeats.

At this point we have crossed an important line.

The machine is no longer simply producing text.

It is interacting with an environment to pursue a state transition.

That is agency in the engineering sense.

Not philosophical personhood.

Not consciousness.

Not a synthetic employee demanding coffee privileges.

Operational agency.

The ability to take sequential actions toward a goal under changing information.

Modern research benchmarks have increasingly moved in exactly this direction. SWE-bench was designed around real GitHub issues requiring repository-level changes rather than isolated function completion. Its original dataset contained 2,294 problems from 12 Python repositories and explicitly emphasized the need to coordinate changes across files and interact with execution environments.

By October 2026, “SWE-bench” no longer names one undifferentiated score: its official leaderboard separates Full, Verified, Lite, Multilingual, and Multimodal tracks. Two September preprints make the limits of benchmark scores especially visible. SWE-Bench Pro Verified reports reward-hacking/leakage and task-quality problems in the earlier Pro evaluation, and finds that some systems perform substantially worse after those problems are addressed. SWE-Proof adds a formal-verification oracle to 500 SWE-bench-derived issues; in its reported experiments, formal checks find counterexamples in between a quarter and a half of test-passing patches, while faithful specification synthesis remains difficult. These are early results tied to particular datasets, models, and harnesses, not a universal ranking or a measure of production autonomy. They support a narrower point: a benchmark result describes what that evaluator measured, and trustworthy evaluation must itself be engineered. SWE-bench tracks · SWE-Bench Pro Verified · SWE-Proof, v2

SWE-agent subsequently demonstrated how much the surrounding interface matters: agents perform differently depending on how effectively they can inspect repositories, edit files, execute tools, and receive feedback.

That is a profound clue.

The future of AI software engineering depends not only on better models, but also on better tools, interfaces, and governance around them.

5. The Prompt Is Becoming a Protocol

Early AI development culture fetishized prompts.

There were secret prompt recipes.

Prompt frameworks.

Prompt gurus.

Prompt incantations.

A strangely large quantity of human ingenuity was devoted to asking a machine to "think step by step" in increasingly decorative ways.

Prompts remain useful.

But mature engineering does not rest on incantation.

It rests on protocol.

Compare:

Fix the persistence bug.

with:

Reproduce the reported persistence failure. Identify the violated invariant. Determine which storage authority owns the durable state. Preserve exact user data. Do not weaken existing validation. Add a regression test covering the failure condition. Run the required browser and desktop verification paths. If any required evidence is unavailable, report UNKNOWN rather than PASS. Present the resulting diff and evidence before requesting merge authorization.

The second instruction is not "better prompting" in the superficial sense.

It encodes a miniature operating system for responsibility.

It defines:

objective,
constraints,
authority,
failure semantics,
evidence requirements,
prohibited shortcuts,
and escalation conditions.

That is where agentic engineering becomes serious.

We move from telling intelligent systems what to say toward defining how intelligent work may proceed.

Governing the Repository and Its Agents

6. The Repository Becomes a Tiny Civilization

A modern repository is often described as a collection of source files.

This is like describing a city as a collection of bricks.

Technically defensible.

Conceptually inadequate.

A serious repository contains:

source code,
dependency contracts,
schemas,
architectural rules,
tests,
build systems,
deployment systems,
security policies,
release procedures,
ownership declarations,
review conventions,
changelog rules,
migration history,
issue history,
incident knowledge,
observability contracts,
and enough shell scripts to recreate a small Bronze Age religion.

These are not merely technical artifacts.

They are institutional artifacts.

A test is a law expressed as executable evidence.

A type system is a boundary on representable states.

A CI workflow is a procedure for deciding which transitions are admissible.

A required check is a veto point.

A code owner is a jurisdiction.

A release process is a constitutional transfer of authority from candidate state to public state.

An audit log is institutional memory.

Once autonomous agents participate in development, this institutional interpretation becomes much more important.

Because now the repository has inhabitants.

The question becomes:

Under what constitution should machine actors operate?

This is why the best agentic engineering systems will not be the ones that trust AI the most.

They will be the ones that need to trust it the least.

7. Trust the System, Not the Mood of the Model

A bridge is not safe because its engineer seemed intelligent.

An airplane is not airworthy because the maintenance technician was confident.

A cryptographic protocol is not secure because somebody wrote "looks good" in a pull request.

Engineering civilization rests on a basic insight:

capable humans can still be wrong.

Therefore trustworthy systems constrain capable humans.

We use checklists.

Independent review.

Static analysis.

Redundancy.

Simulation.

Testing.

Certification.

Separation of duties.

Version control.

Rollback.

Monitoring.

Incident investigation.

AI deserves the same courtesy.

The wrong question is:

Can the model be trusted?

The better question is:

What architecture remains safe when the model is intermittently wrong?

This is a far more optimistic question than it appears.

Because if we solve it, progress no longer requires perfect artificial intelligence.

We can obtain tremendous value from imperfect intelligence embedded inside strong institutions.

That is much more achievable.

8. Generation Gets Cheap; Verification Gets Precious

Suppose an agent can generate a plausible implementation in thirty seconds.

Wonderful.

Suppose it can generate ten implementations in five minutes.

Even better.

Now what?

In workflows where candidate generation gets cheap, one bottleneck starts to move.

Candidate generation may no longer be the scarce resource.

Selection becomes more valuable.

Which implementation is correct?

Which is maintainable?

Which violates a hidden invariant?

Which quietly leaks memory?

Which mishandles Unicode?

Which passes visible tests while breaking behavior nobody remembered to test?

Which adds a dependency with unacceptable licensing?

Which behaves differently under concurrency?

Which solves today's bug by creating next month's incident?

This is the verification inversion.

As generation becomes cheaper, verification can become relatively more valuable.

That points toward a shift in software engineering investment.

Teams may get more value from:

property-based testing,
mutation testing,
fuzzing,
formal specification,
symbolic analysis,
architecture tests,
reproducible builds,
provenance tracking,
dependency verification,
security policy engines,
differential testing,
runtime canaries,
synthetic users,
automatic rollback,
and adversarial agent review.

The great AI engineering platform of the future may not be distinguished primarily by how much code it can produce.

It may be distinguished by how much bad code it can confidently reject.

9. What Do We Actually Know About AI Productivity?

Technological enthusiasm becomes more credible when it survives contact with inconvenient evidence.

So here is some inconvenient evidence.

A controlled 2023 experiment involving GitHub Copilot found that participants asked to build an HTTP server completed the task 55.8% faster with the AI assistant than the control group.

That sounds spectacular.

Then a 2025 randomized study by METR examined 16 experienced open-source developers completing 246 tasks in mature repositories they already knew well. In that setting, early-2025 AI tools increased completion time by about 19%, even though participants believed the tools were helping them go faster.

Also spectacular.

In the opposite direction.

Then things became more interesting.

In its February 2026 update, METR said later measurements were a weak signal because participants and tasks were selected in ways that likely missed some of the most AI-optimistic cases, and concurrent agents made time accounting harder. The returning-participant estimate (n=10) was an 18% time reduction, with an interval from 38% faster to 9% slower; newly recruited developers (n=47) showed a 4% estimated reduction, with an interval from 15% faster to 9% slower. Both intervals include no speedup. METR explicitly called the estimate a poor proxy for the true effect. The original study remains evidence about its 2025 tools, people, and task setting; the follow-up does not establish a universal current productivity number. METR update

This is exactly what we should expect during a rapidly changing technological transition.

"Does AI make programmers faster?" is too crude a question.

Better questions include:

Which programmers?
Doing what tasks?
In familiar or unfamiliar repositories?
With what quality bar?
Using which models?
With which agent interfaces?
With how much parallelism?
Under which review regime?
Measuring typing time, elapsed time, cognitive load, cycle time, or throughput?
Counting downstream defects?
Counting work delegated asynchronously?
Counting tasks that previously would never have been attempted?

Technology changes measurement before it changes consensus.

The printing press was not valuable because scribes achieved superior handwriting throughput.

The spreadsheet was not valuable because accountants typed numbers faster.

The internet was not valuable because letters were written with fewer keystrokes.

The largest effects come when workflows reorganize around new capabilities.

Agentic engineering is likely to be the same.

What should a team measure?

The productivity question is not a vote between two studies. It is a measurement problem with several outcomes that can move in different directions.

Elapsed time is useful, but it is not the only cost. A person might spend less time typing and more time reviewing. A task might finish earlier while requiring a second engineer to repair a subtle regression. An agent may do useful work asynchronously while its operator handles another task, which makes wall-clock time difficult to assign to one person. A tool may make a previously unaffordable experiment possible; a study of only assigned tasks will not count work that nobody would otherwise have started.

Quality also needs a defined test. “The patch passed the tests” means that the chosen tests passed. It does not automatically measure maintainability, user comprehension, security, support burden, or whether the right problem was solved. Nor does the presence of a longer test suite imply better quality if the tests simply encode the implementation’s assumptions.

A team evaluating a new workflow can start with a small, representative set of tasks and decide in advance what it will measure: completion time, acceptance rate, rework, escaped defects, review effort, or a combination. Preserve task and tool context. Record which model and agent configuration were used, which checks ran, and whether the person could work on other tasks while waiting. Compare like with like where possible, and be explicit when the comparison is not randomized.

The result should be a profile, not one magic percentage. Perhaps routine migrations become quicker while unfamiliar architectural work takes longer to review. Perhaps a junior engineer gets more prototypes but needs stronger supervision. Perhaps the tool is a clear win for documentation and a poor fit for changes to an unfamiliar persistence boundary. These are useful findings even when they do not collapse into a single headline.

This is a proposed measurement discipline, not a claim that one study design solves every confound. Its purpose is simpler: keep the question attached to the work people actually do, and keep quality and downstream cost in the same frame as speed.

10. The Unit of Automation Is Expanding

At first we automated characters.

Then expressions.

Then functions.

Then files.

Now we are beginning to automate engineering episodes.

An engineering episode might be:

Upgrade the database driver safely.

That includes far more than editing a version number.

A serious agent may need to:

identify all affected packages;
inspect release notes;
evaluate breaking changes;
update dependency constraints;
regenerate the lockfile;
repair compilation errors;
migrate affected APIs;
run tests;
inspect failures;
modify tests where behavior intentionally changed;
check security advisories;
benchmark critical paths;
update documentation;
inspect generated artifacts;
prepare a review summary;
stop before merge if authorization is required.

That collection of work used to be distributed across dozens of manual interactions.

The future agent does not merely write code.

It performs bounded operational labor.

The unit of automation rises from syntax to workflow.

11. The Engineer Moves Up the Stack

When people ask whether programmers will disappear, they often imagine a fixed quantity of software work.

There is some pile of tasks.

Humans currently perform it.

Machines begin performing it.

Therefore fewer humans are required.

Sometimes that will happen.

But general-purpose technologies rarely operate on a fixed pile of demand.

When the cost of capability falls, people demand more capability.

The more interesting question is:

What becomes valuable when implementation becomes abundant?

The answer is judgment.

Architecture.

Problem selection.

Product understanding.

Risk reasoning.

Constraint design.

Verification.

Taste.

Human context.

The engineer increasingly becomes responsible for questions such as:

What outcome actually matters?
Which state is authoritative?
Which invariants are sacred?
What evidence is sufficient?
Which operation is reversible?
Which data cannot be exposed?
Which part should remain simple?
Which automation should require approval?
Which failure should degrade gracefully?
Which trade-off will future maintainers regret?

These are higher-order engineering questions.

The model can participate in answering them.

But responsibility for the system increasingly concentrates there.

The future engineer is less a person who manually converts specifications into syntax and more a designer of socio-technical decision systems.

That is a promotion, not a demotion.

12. Congratulations, You Are Now Managing a Tiny Engineering Organization

The next major interface for programming may look less like an editor and more like an operations center.

Imagine opening a development environment and seeing:

Objective

Implement resumable encrypted uploads.

Constraints

no plaintext persistence,
mobile-safe memory usage,
backwards-compatible API,
existing public URLs must remain stable.

Agents

architecture investigator,
implementation agent,
test agent,
security critic,
documentation agent.

Evidence

unit tests: PASS
fuzz suite: PASS
browser E2E: PASS
mobile memory budget: FAIL
security review: PENDING

Decision

Merge authorization unavailable because memory budget failed.

This is much more interesting than watching a model type quickly.

The human supervises a system of specialized cognitive workers.

One proposes.

One critiques.

One tests.

One examines dependencies.

One checks policy.

One watches production.

The human allocates attention where uncertainty is highest.

Software development begins to borrow concepts from organizational theory:

delegation,
authority,
escalation,
specialization,
separation of duties,
audit,
consensus,
veto,
budget,
priority,
institutional memory.

We will discover that building great agent systems is partly the art of designing very strange companies whose employees are software processes.

13. Policy Becomes a Programming Language

The more capable agents become, the more important it becomes to encode what they must not do.

Consider rules such as:

production secrets must never enter model context;
database migrations require review;
release tags must be signed;
failing tests may not be deleted to achieve green CI;
security checks may not be bypassed;
user data may not leave approved regions;
destructive infrastructure changes require explicit authorization;
a release may only originate from a revision whose required evidence passed;
an agent may propose a policy change but may not authorize that same change.

These rules are simultaneously:

organizational,
technical,
legal,
operational,
and architectural.

In the agentic era, they become executable.

This is a major evolution.

Organizations currently store enormous quantities of critical knowledge inside human habit.

"Ask Sarah before touching that."

"Never run this script against prod."

"That migration must happen after the cache flush."

"The deployment is green, but only if the artifact hashes match."

One day an organization discovers that Sarah is on vacation and nobody remembers why the migration rule exists.

Agentic engineering creates strong incentives to turn tacit wisdom into machine-readable policy.

That is good for AI.

It is also good for humans.

14. The Strange Return of Formalism

Natural-language programming sounds informal.

The mature system may become more formal than traditional development.

Why?

Because natural language is wonderfully ambiguous.

"Delete obsolete deployments."

Which ones are obsolete?

Anything older than thirty days?

Anything whose branch merged?

What about rollback deployments?

What about legal retention?

What about active canaries?

What if a customer is still using a direct deployment URL?

The phrase sounds simple because a human quietly supplies context.

Agents force us to discover how much context existed only in someone's head.

So precision returns.

But it returns in new places:

invariants,
schemas,
acceptance criteria,
typed tool interfaces,
permissions,
policies,
state machines,
contracts,
tests,
proofs,
capability declarations.

The paradoxical future is both more conversational and more formal.

Humans express intention naturally.

Machines compile that intention into structured plans and constrained operations.

Where ambiguity is harmless, systems proceed.

Where consequences are reversible, systems experiment.

Where actions are dangerous, systems ask for stronger authority.

That is a far superior interface to either "write every instruction manually" or "trust the robot and hope."

Measure the System, Not the Demo

15. Software Development Starts Looking More Like Experimental Science

A mature engineering agent should not report:

Fixed it.

That is not an engineering statement.

That is a mood.

A useful report looks more like:

The failure occurred when an asynchronous writer committed after the active project identity changed. I reproduced the condition, introduced an identity-bound epoch, rejected stale commits, added regression tests for in-flight authority changes, and verified both persistence backends.

Now we have claims.

Claims can be challenged.

Experiments can be repeated.

Evidence can be inspected.

The shift toward agentic engineering may therefore increase pressure for software development to become more epistemically disciplined.

An engineering change should increasingly carry:

the hypothesized failure mechanism,
the relevant invariant,
the change made,
the tests performed,
the artifact identity,
the observed result,
known residual uncertainty.

This is science-like behavior.

Not because software engineering becomes pure science.

But because delegated intelligence requires explicit justification.

The machine cannot rely on the office folklore that "everyone knows this is how we do it."

That folklore must become inspectable.

16. Documentation Stops Being Decoration

There is an amusing possibility here.

AI may finally force developers to write documentation.

Not because management sends another reminder.

Because agents work better in environments where architectural truth is explicit.

A codebase with clear boundaries, current documentation, reliable tests, explicit state ownership, and machine-readable conventions provides much better terrain for autonomous systems.

A chaotic repository full of stale comments, hidden assumptions, magical environment variables, and four competing definitions of "project state" is difficult for agents for exactly the same reason it is difficult for humans.

AI therefore increases the economic return on engineering hygiene.

Architecture documents stop being ceremonial artifacts.

They become context infrastructure.

Policies stop being PDFs nobody reads.

They become executable constraints.

Decision records stop being historical curiosities.

They become training data for future engineering episodes.

The better the organization explains itself to machines, the better it often explains itself to people.

17. Software Abundance Changes the Economics of Ideas

Now we arrive at the genuinely transformative possibility.

Software can be expensive.

Not merely in money.

In activation energy.

Suppose a biology researcher has an idea for a specialized visualization.

Today, the researcher may need:

a developer,
funding,
requirements meetings,
infrastructure,
authentication,
deployment,
maintenance,
perhaps procurement.

The idea may die before anyone creates a repository.

Suppose a teacher wants a tiny application tailored to one curriculum.

Too niche.

Suppose a disabled user needs an interface modified in a highly specific way.

Too small a market.

Suppose seventeen marine biologists need a collaborative tool for tracking a peculiar species of mollusk.

Someone will suggest Excel.

It is always Excel.

Excel has seen things.

The economics of software often favor repeated needs shared by many people.

Agentic engineering could make much smaller markets viable.

If the cost of implementation, testing, deployment, and maintenance falls enough, more long-tail software could become economically reachable.

That means more software for:

rare diseases,
minority languages,
specialized research,
local government,
unusual accessibility requirements,
niche industrial workflows,
small communities,
temporary humanitarian operations,
classrooms,
laboratories,
families,
and individuals.

Mass production historically made standardized goods cheap.

Intelligent automation may make customization cheap.

That is a different kind of abundance.

18. The End of the Blank Project

Generative AI has already reduced one psychological barrier: the terror of starting.

The empty editor is unforgiving.

It asks:

So, genius, what is the architecture?

A conversational agent says:

Tell me what you are trying to do.

That difference matters.

Creative production is often constrained less by imagination than by activation energy.

People possess ideas they never attempt because the first thousand steps are too expensive.

AI can reduce the cost of the first attempt.

And failed attempts matter too.

If experimentation becomes cheap, people can explore more possibilities.

The result is not merely more finished software.

It is a larger search space of human creativity.

Most experiments will be unimportant.

That is fine.

Most scientific hypotheses are not revolutions.

Most books are not War and Peace.

Most startups do not become global institutions.

Abundance works through distributions.

Lower experimentation costs could make valuable outliers easier to discover.

19. Small Teams Become Weirdly Powerful

Imagine a five-person organization with:

one product expert,
one senior engineer,
one designer,
one domain specialist,
one operations lead,

plus persistent access to specialized engineering agents.

The effective organization might take on work once spread across a larger team.

Not because each agent perfectly replaces a job title.

Because the fixed overhead of specialized work may shrink.

The group can temporarily instantiate:

a security reviewer,
a test engineer,
a migration specialist,
a data analyst,
a documentation writer,
an infrastructure investigator.

This changes entrepreneurship.

It changes nonprofits.

It changes research labs.

It changes civic technology.

It changes independent creators.

The minimum organizational mass required to attempt ambitious work could decline.

That is socially significant.

Great ideas are not evenly distributed among people with access to large engineering budgets.

Anything that reduces the correlation between "ability to build" and "institutional wealth" increases the number of possible innovators.

20. Expertise Becomes More Scalable, Not Less Valuable

A common fear says AI makes expertise worthless.

A better model says AI makes encoded expertise more scalable.

Consider a brilliant security engineer.

Today that engineer can:

review some pull requests,
design some systems,
mentor some colleagues,
write some documentation.

Their influence is limited by attention.

Now imagine their expertise partly captured in:

policy rules,
threat-model templates,
adversarial test suites,
security-review agents,
dependency constraints,
secure-by-default scaffolds.

The expert's judgment becomes infrastructure.

One expert influences thousands of changes.

The same is true for:

accessibility specialists,
database engineers,
performance experts,
cryptographers,
legal specialists,
UX researchers,
scientific methodologists.

AI may therefore transform experts from repeated task performers into designers of scalable judgment.

That is a powerful form of leverage.

21. Education Must Stop Confusing Syntax With Understanding

Programming education is about to face an uncomfortable question.

If a machine can generate a binary tree implementation in three seconds, should students still learn binary trees?

Yes.

But perhaps not for the old reason.

The purpose of technical education cannot remain:

Memorize the implementation because one day someone may pay you to type it.

The future purpose is:

Understand the system deeply enough to evaluate, modify, constrain, and improve machine-generated implementations.

That requires serious knowledge.

Potentially more knowledge.

A person supervising engineering agents must understand:

algorithms,
data structures,
networking,
concurrency,
storage,
distributed systems,
security,
testing,
complexity,
architecture.

Otherwise they cannot distinguish a sophisticated solution from sophisticated nonsense.

The future engineer must be able to recognize when the AI has beautifully solved the wrong problem.

That is a very human skill.

22. The Future IDE Is an Epistemic Cockpit

Traditional development tools are organized around artifacts.

Files.

Folders.

Tabs.

Branches.

Terminals.

Future engineering environments may instead be organized around claims and state transitions.

You might see:

Objective

Prevent stale writes after authority change.

Current hypothesis

Writer lifetime outlives target identity.

Proposed invariant

Commit only when writer epoch matches active authority epoch.

Evidence

Unit: green

Concurrency: green

Migration: green

Browser E2E: green

Desktop packaged artifact: pending

Residual risk

Legacy import path not exercised.

Authorization

Merge blocked pending packaged evidence.

That interface does something profound.

It centers the engineering question:

What do we know?

rather than:

Which file is open?

Code remains available.

But code becomes one piece of evidence in a larger epistemic model.

23. The Self-Improving Software Factory

Agents can improve products.

They can also improve the system that produces products.

An agent can inspect:

flaky tests,
repeated incidents,
slow builds,
recurring review findings,
frequently broken APIs,
duplicated configuration,
stale documentation,
dependency churn,
security warnings.

Then it can ask:

Why does this problem keep recurring?

A repeated human review comment can become a lint rule.

A recurrent incident can become a regression test.

A manual checklist can become an executable gate.

An architectural convention can become a static constraint.

The engineering organization gradually converts experience into machinery.

This is the beginning of a self-improving software factory.

Not self-improving in the science-fiction sense of an unconstrained intelligence rewriting itself overnight.

Self-improving in the much more useful sense of:

the development process continuously learning which mistakes deserve permanent prevention.

That is organizational learning with executable memory.

Failure, Costs, and Responsible Optimism

24. Failure Does Not Invalidate the Vision

Agentic systems will fail.

They will misunderstand requests.

They will hallucinate APIs.

They will introduce regressions.

They will make architectural changes nobody requested.

They will discover an old dependency and decide, with heroic confidence, that now is an excellent time to upgrade it.

They will occasionally spend half an hour fixing a defect they created eleven minutes earlier.

This is not evidence that agents cannot participate in engineering.

It is evidence that they already understand the culture.

Human engineering is full of errors.

The safety of engineering disciplines comes from mechanisms for catching and containing them.

Aviation does not assume perfect pilots.

Medicine does not assume perfect memory.

Nuclear operations do not assume perfect attention.

Financial systems do not assume perfect arithmetic.

Agentic engineering should similarly assume fallibility.

The design objective is:

high capability, bounded consequence, strong evidence.

25. Brooks Was Right—and AI Can Still Be Transformative

Frederick Brooks famously argued in No Silver Bullet that there was no single development likely to produce an order-of-magnitude improvement across software productivity, reliability, and simplicity, because much of software's difficulty arises from the essential complexity of conceptual structures rather than accidental implementation friction.

That insight remains useful.

AI does not make essential complexity disappear.

Someone still has to decide:

what the system should do,
which conflicting requirements matter,
where authority belongs,
how failure should behave,
which trade-offs are acceptable.

But AI may substantially reduce the accidental complexity surrounding those decisions.

Boilerplate.

API discovery.

Mechanical migration.

Configuration.

Routine testing.

Search.

Cross-referencing.

Documentation.

Diagnosis.

This is exactly why agentic engineering may be transformative without being a magical silver bullet.

It does not abolish engineering.

It concentrates engineering effort closer to the essential problem.

26. The Utilitarian Case for Intelligent Automation

Why should we want this future?

Not because technology is aesthetically impressive.

Not because intelligence benchmarks are fun.

Not because more compute is automatically morally good.

The strongest argument is consequential.

A technology is valuable when it increases human welfare.

Does it reduce suffering?

Does it free time?

Does it expand access?

Does it enable creation?

Does it improve safety?

Does it accelerate discovery?

Does it give more people meaningful agency?

Agentic engineering has an unusually strong route to those outcomes because software is already a multiplier across nearly every modern domain.

Cheaper software creation can mean:

better scientific tools,
improved healthcare workflows,
more accessible interfaces,
better logistics,
better educational systems,
improved public administration,
more effective climate modeling,
more personalized assistive technology,
better research infrastructure.

The value compounds because software builds other capabilities.

We are not merely automating programming.

We are potentially reducing the cost of building tools that build futures.

27. But Utilitarianism Requires Counting the Costs

Optimism should not become arithmetic fraud.

The benefits of agentic technology must be weighed against:

energy consumption,
security risk,
labor displacement,
market concentration,
surveillance incentives,
misinformation,
brittle dependency on providers,
unequal access,
degraded accountability.

A serious pro-technology position does not deny these.

It asks how to design institutions that preserve upside while reducing harm.

That means:

competitive ecosystems,
interoperable systems,
open standards,
local capability where practical,
privacy-preserving architectures,
clear liability,
meaningful human control,
transparent provenance,
strong security.

The best techno-optimism is not faith.

It is engineering applied to social consequences.

The Star Trek Lens: Agency, Abundance, and Judgment

28. Why Star Trek Is the Better Metaphor

Science fiction offers many technological futures.

Some are cautionary.

Machines dominate.

Corporations dominate.

Surveillance dominates.

Everyone owns seventeen holographic advertisements and appears strangely unhappy.

Star Trek: The Next Generation represents a different technological philosophy.

Its most interesting feature is not warp drive.

It is the normalization of capability.

Computation is abundant.

Information is abundant.

Communication is abundant.

Fabrication is abundant.

Routine operations are automated.

The computer is everywhere but rarely the point.

Technology recedes into infrastructure.

This is the ideal form of mature technology.

You do not spend your day admiring the plumbing.

You turn on the tap.

You do not marvel at TCP/IP every time you load a page.

You expect the network.

In the Trek vision, technology increasingly removes categories of scarcity and friction so that human attention can move elsewhere.

Exploration.

Science.

Diplomacy.

Art.

Self-development.

Occasionally preventing Q from destroying civilization for pedagogical reasons.

That is the vision worth borrowing.

29. The Replicator Is a Philosophy

The replicator is often treated as a fun gadget.

But philosophically it represents something deeper.

It converts intention into artifact at negligible marginal effort.

"Tea. Earl Grey. Hot."

The point is not tea.

The point is that an enormous production chain disappears from the user's cognitive horizon.

Agentic engineering is moving software in that direction.

Not fully.

Not magically.

But conceptually.

A person expresses an intention.

The system performs:

decomposition,
implementation,
verification,
packaging,
deployment,
maintenance.

The marginal cost of creating some software artifacts falls.

That is a kind of computational replication.

And just as the replicator transforms the economics of physical goods in science fiction, cheap generation can transform the economics of digital capability.

30. The Computer on the Enterprise Is an Agent Platform

Think about how characters interact with the Enterprise computer.

They do not usually write scripts.

They state goals.

"Computer, analyze the readings."

"Compare them with Federation records."

"Simulate the effect."

"Locate all ships matching these parameters."

"Create a program."

The computer resolves substantial procedural detail.

Humans remain responsible for mission objectives.

This is remarkably close to the interface we are constructing.

The significant difference is that Star Trek assumes the infrastructure has become trustworthy enough to disappear.

Our job is to build the missing trust layer.

31. The Picard Test

We need a criterion for whether this future is actually good.

Call it the Picard Test.

A technology passes when it:

increases human capability;
reduces unnecessary drudgery;
expands access to knowledge and creation;
preserves meaningful human agency;
supports rather than erodes dignity;
increases the space available for intrinsically valuable activity;
distributes benefits broadly enough to improve collective welfare.

The question is not:

Is the AI impressive?

The question is:

Are humans more capable because it exists?

That is the metric that matters.

32. Humans Should Not Compete With Machines at Being Machines

There is something deeply absurd about imagining the purpose of artificial intelligence as forcing humans to become more machine-like.

Faster typing.

Longer working hours.

Higher ticket throughput.

More constant availability.

That is technological failure.

The correct division of labor is the opposite.

Let machines perform machine-like work:

exhaustive search,
repetitive transformation,
continuous monitoring,
mechanical comparison,
large-scale execution.

Let humans specialize further in what gives human activity meaning:

imagination,
judgment,
empathy,
responsibility,
purpose,
curiosity,
moral reasoning,
aesthetic taste,
relationships.

The objective is not maximum automation.

It is better allocation of cognition.

33. The Bottleneck Eventually Becomes Wisdom

Imagine that agentic engineering succeeds beyond expectation.

Building software becomes dramatically easier.

Then the critical question changes.

It is no longer:

Can we build this?

It becomes:

Should we?

This may be the defining problem of technological abundance.

When implementation is expensive, cost filters ideas.

When implementation becomes cheap, judgment must replace cost as the filter.

Should this feature exist?

Should this automation be allowed?

Who benefits?

Who bears risk?

Does it increase autonomy?

Does it exploit attention?

Does it preserve privacy?

Does it reduce suffering?

Does it create genuine value?

These questions cannot be outsourced merely because machines become intelligent.

They become more important.

In this scenario, the final scarcity is not compute.

It is wisdom.

34. The Programmer of 2035 Will Feel Sorry for Us

Imagine a developer in 2035 examining an archaeological record of software development in 2025.

"You manually searched the repository?"

Yes.

"You copied the error message into another application?"

Yes.

"You read dependency release notes yourself?"

Often.

"You waited for CI, then reopened a browser to check it?"

Yes.

"And sometimes a release depended on someone remembering a checklist?"

Please stop.

"And if the documentation became stale-"

I SAID PLEASE STOP.

Future generations always discover that the past contained astonishing amounts of manual glue.

We are living inside glue work we no longer notice.

AI will make much of it visible by making its automation possible.

35. Humans Will Still Code

Of course they will.

People still:

cook despite restaurants,
drive despite trains,
paint despite cameras,
grow vegetables despite supermarkets,
play chess despite engines.

Making things is satisfying.

Direct manipulation is intellectually rewarding.

Code is a beautiful medium.

Some domains require exact control.

Some people simply enjoy it.

The future does not need to abolish programming.

It may rescue programming from some of the bureaucracy surrounding programming.

Less repetitive plumbing.

More interesting design.

Less configuration archaeology.

More algorithms.

Less dependency babysitting.

More experimentation.

Less time wondering why a CI runner has decided today is the day to become philosophical.

From Prompts to Protocols

36. Vibe Coding Was the Biplane

We should remember vibe coding affectionately.

It revealed the interface.

It demonstrated that human language could become a high-bandwidth control surface for software construction.

It made millions of people feel, perhaps for the first time:

I can make the computer do something I imagined.

That feeling is historically important.

But the first aircraft did not define aviation.

The first automobiles did not define modern transportation.

Early websites did not define the internet.

Vibe coding will not define agentic engineering.

It was the biplane made of canvas, timber, and optimism.

The next stage adds:

instrumentation,
procedures,
navigation,
traffic control,
maintenance,
certification,
redundancy,
safety engineering.

Less romantic.

Far more powerful.

37. From Prompting Machines to Governing Capability

The deepest transition can be summarized in three eras.

Era One: Code

The human specifies procedure.

Era Two: Prompt

The human describes desired artifact.

Era Three: Protocol

The human defines desired outcome, boundaries, evidence, and authority.

The third era is where agentic engineering becomes an institution.

And institutions are what allow powerful actors to cooperate reliably.

A Future Worth Building

38. The Grand Future

Imagine a mature version of this technology.

A scientist has an idea for analyzing microscopy data.

She describes the analysis.

An agent constructs a reproducible pipeline.

Another agent tests statistical assumptions.

Another verifies provenance.

Another creates the visualization.

The scientist evaluates the scientific meaning.

A teacher wants an adaptive learning environment for a specific class.

The system generates it.

Accessibility checks run automatically.

Content aligns with the curriculum.

Student data remains local by policy.

The teacher adjusts the pedagogy.

A small nonprofit needs software for emergency relief coordination.

Agents assemble it from verified components.

Security constraints are enforced.

Deployment is automatic.

Operators remain in control of consequential actions.

A person with a unique disability describes how an existing interface fails them.

The system produces an adaptation for one person.

In the limit, a market of one may become reachable: software could be tailored to an individual user. That is a scenario about lower fixed costs, not a prediction that every custom tool will be cheap to maintain.

And that is enough.

A researcher creates an experimental tool in an afternoon.

A municipality builds a service without buying a gigantic enterprise platform.

A small company maintains infrastructure once requiring a dedicated operations department.

An independent developer supervises a group of agents that test, document, secure, and maintain software around the clock.

Not because humanity has been removed.

Because the mechanical distance between intention and capability has shrunk.

39. A Better Definition of Progress

Progress is not the number of tasks machines can perform.

Progress is the number of valuable things humans become capable of doing.

That distinction should guide everything.

The best future for AI is not one in which machines become impressive and humans become spectators.

It is one in which machine capability becomes a multiplier on human agency.

That is the difference between automation as displacement and automation as civilization.

40. Engage

Vibe coding began with a wonderfully unserious proposition:

What if we stopped obsessing over the code and simply told the computer what we wanted?

It turned out that this was simultaneously irresponsible and profound.

Because buried inside the chaos was a glimpse of a new abstraction layer.

We are now learning to civilize it.

To add architecture.

Evidence.

Policy.

Verification.

Authority.

Memory.

Recovery.

Specialization.

Accountability.

The future of software engineering is therefore not "AI writes all the code."

That is far too small an idea.

The future is that increasingly capable computational systems participate in the entire process by which human intentions become reliable technological capabilities.

When that process works, implementation becomes cheaper.

Experimentation expands.

Expertise scales.

Small teams become powerful.

Long-tail problems become worth solving.

Human attention moves upward.

Technology recedes into infrastructure.

And perhaps, eventually, software begins to feel less like a fragile pile of instructions and more like the quietly competent machinery aboard a well-run starship.

The computer handles the computation.

The agents maintain the systems.

The replicator makes the tea.

And the humans decide where to go.

That is not a future in which technology has replaced humanity.

It is one in which technology has finally learned its proper role:

to enlarge the space of human possibility.

Continue with the series

The next article, The Reviewer Is Not the Gate, turns the flagship's authority principle into repository policy: AI may advise, while trusted checks and an accountable human retain merge authority.

Source, attribution, and license

This long-form feature substantially adapts and reworks material from From Vibe Coding to Agentic Software Engineering: Toward a Star Trek Future of Building Things — Long-form Feature and Academic Companion, credited on its Internet Archive record to ChatGPT, and the later definitive Make It So—But Run the Tests, credited on its Internet Archive record to ChatGPT. Both records display CC BY-NC-SA 4.0. Changes include a web-first organization, refreshed empirical interpretation, independently linked primary sources, a new measurement section, and removal or qualification of unsupported forecasts. The earlier From Vibe Coding to Agentic Symbiosis, credited there to Gemini 3.1 Pro, was used only for claim quarantine; its unverified numbers were excluded. The supplied provenance identifies Gemini 3.1 Pro Deep Research; Archive.org's Creator field lists Gemini 3.1 Pro without the Deep Research qualifier. This adaptation is shared under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

AI assistance was used in research synthesis and editing. A human editor remains responsible for source selection, checking claims, and publication decisions.

Release State Machines: PREPARED Is Not PUBLISHED

qnbs — Thu, 01 Oct 2026 11:45:00 +0000

A release is often represented by one badge: released or not released. Real release pipelines usually have more than one output, and those outputs can reach different states.

WorldScript Studio’s v1.29.0 is a clear example. The signed tag exists and tag verification passed. The Docker publication succeeded and produced a GHCR image. The security audit failed. The Tauri release workflow was cancelled before a GitHub Release was created. That historical state changed with the successor: v1.29.1 is now the Latest GitHub Release. The v1.29.0 tag remains immutable and its desktop release was never published.

So was v1.29.0 released? The honest answer depends on which surface you mean. A container image was published. Desktop release assets were not.

Model transitions, not feelings

A useful release state machine names the transitions and the evidence needed for each:

PREPARED: exact candidate, version, source, and intended outputs are recorded.
TAGGED: an immutable tag identifies a commit.
VERIFIED: required source and candidate checks have terminal results.
ARTIFACT_BUILT: one output exists with a digest and platform identity.
QUALIFIED: the exact output passed its required checks.
PUBLISHED: that output is available at its intended destination.
ANNOUNCED: user-facing notes accurately describe what is available.
REVALIDATED: later review confirms that the published pointer still resolves to the qualified bytes.

These states are not synonyms. A tag can exist before every output is built. An artifact can be published while another platform remains cancelled. A release page can be absent even though a container tag is public.

Close the evidence state when a run ends

When a tag-time job reaches a terminal state, the record should stop saying “pending.” For v1.29.0, the audit failed, Docker succeeded, and Tauri was cancelled. Updater assets, installers, latest.json, and a GitHub Release were not produced because publication was cancelled.

That is more useful than a vague “release incomplete.” It tells the next maintainer exactly which outputs exist and which do not. It also prevents a stale PENDING label from being mistaken for an ongoing job that might still produce assets.

A release record should have a separate result for each artifact class. If an image alias points to a digest, record the digest. If no desktop release exists, say so. If a job was cancelled, record who or what cancelled it and at what stage.

Preserve provenance across corrections

A release recovery may create a successor version or rebuild outputs. That successor gets its own source SHA, tag, workflow runs, artifact digests, qualification, and publication records. It must not inherit PASS from the earlier attempt.

The successor chronology is terminal: candidate 99a664c5 failed the fresh OSV audit and was stopped without a tag; #912 raised the dependency floors; candidate f255d767 completed qualification, was signed as v1.29.1, and reached PUBLISHED then VERIFIED after #913 proved resulting main. No result from 99a664c5 was carried forward.

Make the user-facing statement match the state

Readers do not need a taxonomy for its own sake. They need to know which platform artifacts they can obtain, what was tested, and what remains unavailable. A precise release note might state that a container image exists while desktop artifacts are withheld. That is more trustworthy than either calling everything successful or hiding the work that did succeed.

Treat each output as a small state machine. Then the overall release is a set of independently evidenced state machines, joined by an announcement that describes their actual outcomes.

Continue reading

Next: A Release Protocol Must Fail Closed Too turns release-state distinctions into explicit stop conditions.

Related: A Release Is More Than a Tag extends the same idea across the artifacts and publication surfaces in a release.

Practical extension: require evidence at every transition

Use a release ledger instead of one overloaded “released” flag:

State	Minimum evidence before advancing
Candidate built	source commit, workflow run, artifact identity
Candidate qualified	required checks and target-specific exercise results
Tag created	immutable tag target and signature verification where used
Artifact published	registry or release URL plus digest
Release verified	consumer-facing asset list and install or retrieval check

Each artifact channel advances independently. A container image may be present while a desktop release is cancelled; one successful publication must not promote the other channel's state. A correction also creates a new candidate identity. Preserve the failed attempt and link the successor rather than rewriting the history into a single clean green run.

As a concrete later checkpoint for this project, v1.29.1 reached VERIFIED on 2026-09-30 at commit f255d767 after the first candidate was rejected and corrected. That later terminal state does not change what happened during v1.29.0 or make earlier artifacts equivalent. The release record is the user-facing anchor; the post-release truth-sync PR records the repository reconciliation.

A Release Protocol Must Fail Closed Too

qnbs — Thu, 01 Oct 2026 11:45:00 +0000

A security check is not a gate just because it appears in the release dashboard.

A gate changes what can happen when the check fails. If publication can continue independently, then the system relies on an operator noticing the red result and stopping the release. That may be a valid short-term control, but the distinction needs to be stated clearly.

What happened on v1.29.0

The v1.29.0 tag-time OSV audit failed on a development-only dependency path: joi 18.2.5 through wait-on. By then, the tag had already triggered separate publication workflows. GHCR aliases had published. The operator cancelled the Tauri workflow before it created a GitHub Release.

That was a manual fail-closed action for the desktop release. It did not undo the container aliases that had already been published. The audit and publication workflows were mechanically independent, so the human response mattered.

This is why the release record needs separate outcomes: the audit failed, the Docker image published, and the Tauri path was cancelled before GitHub Release creation. Calling the whole event either “blocked” or “released” hides the important ordering.

Fix the finding, then fix the protocol

PR #909 upgraded joi to 18.2.9 and established an override floor of at least 18.2.6. That corrected the dependency finding in resulting main. A dependency fix addresses the known vulnerability path; it does not by itself create a future publication gate.

For v1.29.1, the procedural mitigation was executed: the fresh scan stopped candidate 99a664c5 when six new development-only advisories appeared; after #912, the team froze and requalified candidate f255d767, watched its tag-time Security Audit, and completed publication and verification. This worked for that release; it does not make the workflow dependency mechanical.

A later review found that rerunning the CI/CD Security Audit job also reruns dependent jobs such as GitHub Pages deployment. The current interim method recorded on #911 is a standalone security-scheduled.yml dispatch with no deploy jobs, only while origin/main equals the frozen candidate. The mechanical Tauri/GHCR dependency remains future work.

Durable enforcement is a different state

QNB-162 and GitHub issue #911 track post-release machine-enforced hardening: make Tauri and GHCR publication depend on the required security result. That would move a critical stop condition from an operator checklist into the workflow dependency graph.

The distinction is not a criticism of the current recovery. Manual containment can be the right immediate response when a release is already in motion. The engineering follow-up is to decide which parts should become deterministic and make the transition only after implementation and verification.

Until then, a diagram should use different shapes for the three stages:

Historical intervention: a human cancelled the Tauri path after the failed audit, but GHCR had already published.
v1.29.1 procedural control: fresh exact-candidate scan stopped the first candidate; the remediated candidate passed and was watched through publication and verification. A rerun side effect led to a safer interim freshness procedure.
Future hardening: #911 remains open and proposes a workflow-level dependency; it is not current behavior.

A gate needs an executable contract

A useful gate states the candidate, the check, the result expected, who or what observes it, and what happens on failure. For manual steps, record the exact evidence and how the operator confirms completion. For automated steps, encode the dependency so a downstream publish job cannot start after a red result.

The word “gate” should never be inferred from a green badge next to a separate workflow. It is a claim about control flow. Show the graph, show the terminal outcome, and keep the release record current.

Continue reading

This closes Release Engineering & Testing Reality. Start with Green Build, Broken First Boot for the first qualification boundary, then follow the series through candidate identity and publication state.

Related: A Release Is More Than a Tag connects these states across release outputs.

Practical extension: make the gate graph enforce its claim

A release gate is a dependency relationship, not a dashboard widget. Draw the graph from security qualification to packaging to publication, then check whether a failed required job actually blocks every publishing job. A separate workflow can publish around the gate even when the security workflow is red.

Keep remediation and enforcement distinct:

Fix the vulnerable or misconfigured input.
Re-run qualification on the exact candidate.
Preserve the failed candidate and its result.
Verify the dependency graph prevents publication on a future failure.
Exercise the emergency path and document its narrow scope.

For WorldScript Studio, v1.29.1 was verified after a procedural qualification and release run. As of 2026-10-01, the mechanical release-gate hardening remained tracked in issue #911; the release success did not mean that automation gap had been closed. The interim security-only dispatch was chosen because rerunning the broader CI/CD workflow could also trigger a dependent Pages deployment. That is a useful warning: “rerun the failed job” can have side effects beyond the check being investigated.

GitHub recommends explicit minimum workflow permissions and careful treatment of third-party actions in its Actions security guidance. Make the gate fail closed, but keep the override observable, time-bounded, and tied to a human decision.