A production issue landed on our desk recently. A payment provider had slowed down just enough to cross one of our internal timeouts. Our system marked the payment as failed. The provider kept processing it.
The first part was easy to delegate. An AI agent could search logs, correlate timestamps, trace the payment flow, inspect recent deployments, and find similar cases faster than a human clicking through five systems.
The next part was different. Should we change how an ambiguous payment is classified? Should we enable the new behavior in production? Which service should receive it first? How long should we observe it? If a customer may have paid twice, who decides what to refund?
Those are all steps in one incident, and lumping them together under one label ("AI autonomy") hides how differently each one should actually be handled.
AI investigated the issue and prepared code and tests in isolation; independent verification had to sign off before anything moved forward, and humans kept ownership of the rollout and the financial decisions. The incident ran through the same tools throughout, but the authority behind each step kept changing.
That distinction has changed how I think about AI-assisted engineering. It also applies long before an incident exists. Product discovery, problem framing, prototyping, architecture, implementation, and launch all offer different rewards and expose the company to different risks. Collapse all of that into one autonomous product-development task, and the same failure mode just plays out at a much larger scale.
In my previous article, I argued that AI makes implementation cheaper while engineering judgment becomes more valuable. I still believe that. But "humans provide judgment" sounds right without being an operating model: it never says where AI should act, where it should stop, or what evidence it must produce before moving forward.
We needed a better unit than autonomy, so we started thinking in terms of allocation instead.
The principle is straightforward: allocate AI authority according to the reward of the work, the risk of getting it wrong, and the strength of the evidence available at that point in the process. Everything below is the graph we built to turn that policy into something our team actually runs on. Plenty of policies never get past being a slogan repeated in planning meetings.
Trust is assigned per action
Most arguments about AI coding get stuck at the extremes: give the agent a task and let it write the code, run the tests, open the pull request, merge it, and deploy it. Or keep AI to autocomplete with a chat window bolted on, where a human signs off on every action that matters. Both extremes make the same mistake: they hand out one trust level for a pile of actions that carry wildly different risk.
Reading source code carries little risk. Editing it in an isolated branch raises the stakes a notch, and merging raises them again. A disabled feature flag just sits there until someone flips it on for everyone. That's a different action altogether. Same with a rollback plan: harmless on paper, risky the moment someone actually runs it mid-incident.
The reward varies just as much. AI is unusually useful when the work demands breadth: searching sibling code paths, comparing documentation with implementation, correlating production signals, or repeating a focused check for hours. It is less useful when the task is deciding whether a product concept belongs in the system, accepting financial exposure, or choosing between two business outcomes that are both technically valid.
A binary policy wastes this difference. "AI can do everything" takes unnecessary risk. "A human must approve everything" spends human attention on work a machine can verify more cheaply and more consistently.
The policy we ended up with is simple:
Give AI more authority when the reward is high, failure is reversible, and verification is cheap. Add constraints as the blast radius, ambiguity, or irreversibility increases.
We don't run this as a spreadsheet formula. The model below is closer to how we actually think about it:
Opportunity grows with:
leverage × verifiability × reversibility
Exposure grows with:
failure probability × blast radius × irreversibility × detection latency
A task can have enormous potential reward and still require a human decision. Take a production migration: the schema gets inspected, the migration gets drafted and dry-run against a copy, the affected rows get counted, the rollback gets written down. Every one of those can happen without asking a person first. Running the migration against the live database is a different action, one nobody delegates to the agent that drafted it.
Why we moved from a loop to a graph
A good coding agent usually runs a loop:
understand → plan → implement → test → review → correct
↑ │
└────────────────────┘
We invested in making that loop better. Repository instructions supply architectural context, and rules layer in conventions based on whatever files are being touched. Tools handle the rest of the mechanical work (formatting code, running targeted lint checks, regenerating artifacts, protecting files that shouldn't be hand-edited), and a review step checks correctness, completeness, context fit, and consequences before the agent stops.
That works well for a bounded implementation task, but it does not cover the whole product and delivery process.
The work may start with a production problem. It may also start with a customer request, an operational bottleneck, a commercial opportunity, or an idea that sounds good in a planning meeting. Before anyone writes code, the team still has to determine whether the problem is real, which outcome matters, what promise we are willing to make, and which risks we are taking on.
We tried running all of that inside one long agent session first, because it was the obvious thing to try before we went looking for a fancier name for the problem. It didn't hold up: context grew messy and responsibilities blurred, so a coding step ended up inheriting production credentials it never needed. A reviewer saw the implementer's reasoning and got biased toward the same answer, and operational state disappeared the moment the session ended.
A recent term, graph engineering, gives this problem a useful name. The mechanics are not new; state machines and graph-based orchestration existed long before the label. What the term captures is the design work above a single agent loop: defining specialized nodes, the valid routes between them, and the state that moves with the work.
Here's the sanitized version of the incident graph that came out of that redesign:
production signal
│
▼
investigation
│ confirmed with evidence
▼
scoping
│ bounded task and acceptance criteria
▼
implementation
│ code plus tests
▼
independent verification
│
├── insufficient evidence ──→ investigation
├── incorrect or incomplete ─→ implementation
├── scope changed ───────────→ scoping
└── verified
│
▼
human approval
│
▼
progressive rollout
│
▼
production observation
│
├── unexpected signal ─→ rollback or investigation
└── clean window ──────→ complete
Some nodes use AI heavily, others are deterministic tools, and others are human decisions. The graph functions as a control structure, routing the work to whoever or whatever should act next based on its current state.
The graph starts before there is code
That incident graph only covers the reactive half of the work. Product development runs a longer, less certain front half before any of it applies:
customer / business / operational signal
│
▼
product discovery
│ evidence that the problem is real
▼
problem framing
│ desired outcome and constraints
▼
options, prototypes, and trade-offs
│
▼
accountable product decision
│
▼
contracts and architecture boundaries
│
▼
delivery graph decomposition
│
┌───────────┼────────────┐ run in parallel
▼ ▼ ▼
API operations user surface
│ │ │
└───────────┼────────────┘
▼
integration verification
│
▼
launch decision
│
▼
pilot and observation
│
┌───────────┼────────────┐ choose the next route
▼ ▼ ▼
scale revise stop
AI has useful work in almost every node, but its authority changes as the product becomes more concrete.
During discovery, the useful work is searching support conversations, comparing existing behavior against documentation, tracing how adjacent features work, and synthesizing evidence from several systems at once. The reward is highest here precisely because that reading is broad and slow for a person to do by hand. The risk stays manageable as long as the node is read-only and its output is evidence the team still has to weigh before treating anything as settled.
Problem framing needs tighter boundaries. An agent is good at surfacing contradictions, listing who's affected, and drafting several versions of the problem statement side by side. Choosing which outcome we're actually going after stays with a person. "Reduce failed label purchases" and "maximize the number of supported carriers" may point at the same code but produce different products.
Three disposable options cost almost nothing to generate; committing the company to any one of them costs a great deal more. That gap is where AI earns a great deal of leverage in the options node: prototyping competing flows, mapping service boundaries, comparing API shapes, flagging which decisions are still easy to reverse. We want breadth here.
The product decision is an explicit human node. Someone must choose the user promise, the commercial boundary, and the risk the company accepts. An agent can draft the decision record, but the promise itself is the one part of it that can't be delegated.
Once the scope is accepted, the graph decomposes the product into delivery streams. A feature may require an API contract, storage, billing behavior, an operational surface, public documentation, rollout controls, and production measurement. Some streams can run in parallel. Others have hard dependencies. The shared state must preserve those dependencies so that "backend merged" is never confused with "product available to users."
We ran into this on a shipping capability (I'm generalizing the details here to avoid exposing the internal design). It arrived sounding like one feature: let selected customers buy shipping labels through our platform. It wasn't. Underneath that one sentence sat carrier capabilities, rate selection, purchase idempotency, billing, cancellation, eligibility, documentation, and rollout policy. Eight separate problems wearing the name of one feature request.
AI did the legwork of mapping the existing flows and surfacing the gaps between what one layer advertised and what another layer could actually deliver, then went on to draft the contracts, the regression tests, the implementation slices, and the production checks. What it couldn't do was decide which carriers we were willing to promise a customer, how billing should handle a cancellation, who belonged in the first pilot, or what evidence would justify widening it.
That distinction matters because a repository can contain a technically sound backend while the product remains unusable. The API may exist without a user-facing entry point. Documentation may describe a route that production has not deployed yet. A provider may appear in a discovery response while the purchase path cannot serve it. Each component can look complete from inside its own repository.
To keep that from happening again, the graph now carries a fixed bundle of product-level state across every boundary listed below:
problem
scope
user promise
assumptions
open decisions
dependency map
acceptance criteria
rollout gates
observed outcome
Without that state, AI makes local progress and the team assembles the product by memory. Each node only ever checks whether its own task passed, never whether the product moved anywhere.
Each node gets an authority budget
The graph became useful when we stopped defining nodes only by their job and started defining them by their authority.
A node contract includes:
objective
available context
allowed tools
writable state
evidence required
risk boundary
stop condition
escalation condition
recovery path
In the product graph, the discovery node has broad read access but no authority to commit the roadmap. The options node can generate prototypes and compare trade-offs, but its artifacts are disposable until a product owner selects an outcome. The product-decision node may contain less automation than the nodes around it because its output is a promise: what we will build, for whom, and under which constraints.
Investigation reads widely by design: querying logs, inspecting code, comparing production behavior with documentation, looking for related incidents. It's the same kind of breadth that let AI correlate timestamps and deployment logs across five systems during the payment incident, far faster than an engineer clicking through each one by hand. That breadth is what makes the reward high here, and the write access stays minimal for the same reason: production data has to stay untouched by the process investigating it, or the evidence stops meaning anything.
Implementation gets a narrower brief, working in isolation to edit code, add tests, and run repository checks, but it cannot silently expand the task because it discovered an adjacent cleanup. If the scope changes, the correct edge leads back to scoping.
The verifier gets the diff, the acceptance criteria, and the evidence. Where possible, it should be independent from the implementation context. Its job is to find reasons the patch shouldn't ship, independent of how much effort went into writing it.
Deployment ends up with a much smaller toolset and a much higher consequence. What clears it is a clean production observation window inside the pilot cohort it actually watched. A passing test run happens earlier and proves something else entirely. The person who assembles that evidence isn't the person who owns the consequences if the rollout goes wrong.
When I first started using coding agents, I got this backwards: I spent way too much time tweaking prompts and nowhere near enough time deciding what the agent was actually allowed to decide.
Evidence earns authority
For new product work, evidence has to climb a specific ladder before risk-bearing work is allowed to start:
opportunity hypothesis
↓ customer, operational, and market evidence
problem worth solving
↓ options and disposable prototypes
chosen product outcome
↓ contracts, dependency map, and rollout design
committed delivery scope
↓ implementation and integration evidence
pilot candidate
↓ observed customer and operational outcome
scale, revise, or stop
The important transition is from an interesting idea to a product promise. Before that edge, more evidence and more options are just more evidence and more options, and an agent can keep generating them all day. Crossing the edge is different: it commits engineering capacity and can create expectations outside the team, so a person makes that call.
At the beginning, a production symptom is mostly uncertainty. A failed payment may be a provider decline, an internal timeout, a deployment regression, or a monitoring error. Giving an implementation agent a vague issue at this point creates a predictable result: it picks a plausible explanation and starts coding.
Investigation reduces uncertainty, and a reproducing test reduces it again. Independent review may then reveal sibling paths that the first patch missed, while a progressive rollout gives us production evidence without immediately exposing the full system.
hypothesis
↓ production evidence
confirmed defect
↓ bounded scope
implementation candidate
↓ reproducing test
verified change
↓ independent review
approved change
↓ progressive rollout
observed production behavior
That accumulation of evidence earns more authority as work moves forward, and it can also be faked. We once had a monitoring step report that a set of services was clean. The conclusion looked reassuring. Later inspection showed that a shell loop had scanned zero valid targets. The check was green because it had observed nothing.
Telling people to be careful again wouldn't have caught this. The monitoring node needed to report positive coverage instead: how many targets it actually read, how many records it compared, and which time range it covered. A pass/fail flag without coverage just confirms whatever we already hoped was true. It was never built to test that hope against anything real, inside AI-assisted work or out.
Design edges around risk transitions
It's tempting to draw a graph around job titles: researcher, planner, coder, reviewer. That beats one giant agent doing everything, but the real design decision was always the authority attached to each role, whatever it happened to be called.
What actually matters at each edge is a shift in risk. A few of the rules we settled on:
- A product idea earns committed scope only once discovery turns up a problem worth solving and an owner has picked the outcome they're going after.
- Being cheap to generate doesn't make a prototype the public contract.
- Launch needs all three pieces working together: the user surface, the operational path, and the integration behavior. Parallel delivery streams don't converge before that.
- Shipping the implementation and scaling the pilot are different milestones. Scaling needs evidence from real usage; a passing build doesn't count.
- Evidence has to make the scope concrete before a defect moves from investigation into implementation.
- A code change goes to review only once the previously-failing test turns green.
When something goes wrong, the same logic runs in reverse. A reviewer sends work back if the local patch misses another execution path, and unclear ownership holds a production change even when the code is ready. A rollout that behaves differently than predicted returns to investigation. An unrelated defect gets its own record so it can't quietly expand the current change.
These edges prevent a common failure mode in AI-assisted work: momentum becomes its own justification. The agent has already researched the issue, written the code, and explained the solution, so everyone feels pressure to ship it. A graph forces the work to satisfy the next node's contract instead.
The goal is not to route every task through as many boxes as possible. A small refactor with a clear verifier should stay inside one loop. Most of what our team ships in a given week never needs to leave that loop. Graphs earn their overhead when the work crosses specialties, needs independent verification, carries state across long-running steps, or has decisions with different authority levels.
Humans are part of the graph
Some decisions in this graph are human on purpose. Whether to refund a customer, rotate exposed credentials, accept a backward-incompatible API change, or enable a risky feature needs a named owner who answers for the outcome, and that's true no matter how strong the model recommending the action is.
AI makes that decision cheaper to evaluate: gathering evidence, enumerating affected paths, preparing options, explaining what happens under each one, and checking afterward that the approved action ran as intended. What it cannot do is stand in for the owner: the graph has to keep the recommendation and the accountability in different hands, even when the recommendation is excellent.
The human node should not repeat work that upstream nodes already proved. If an engineer must manually rerun every query, rediscover every code path, and rewrite the rollout plan, the graph has failed to preserve useful state. The handoff should contain enough evidence for the owner to focus on the decision.
Human attention is expensive, and terminal-to-terminal copy-paste is one of the cheapest things to take off someone's plate. We'd rather spend that attention on the ambiguity and the consequence.
What we still get wrong
A graph can fail more elaborately than a loop: shared state can drift, two nodes may define "done" differently, and a verifier may end up checking the wrong surface entirely. An automated review can become noisy enough that developers stop reading it, a repository-level check may pass while a required change in another repository never happens, and ownership may stay unclear while two sessions quietly prepare competing changes.
The system can also create false confidence. A detailed issue, a clean diagram, and several green checks feel rigorous, but a clean diagram just means the graph was drawn, not that anyone checked it was watching the right signals.
We now watch for five specific failure patterns:
- A node that can only confirm, never falsify, isn't actually verifying anything. The zero-target monitor from earlier is the textbook version: it had no way to report "something's wrong," so "clean" was the only sentence it knew how to produce.
- Scope creep is sneaky specifically because it doesn't touch the graph: an implementation task quietly turns into a redesign, and because nobody routes it back through scoping, the diagram still claims everything is fine.
- Self-review is real, but it's the weakest kind of review there is. The same context and the same incentives that shaped the patch are still in the room when it gets checked.
- High-risk edges need a name attached to them. We've seen production and financial decisions with no owner, and bolting an AI approval step on top doesn't manufacture accountability out of nothing.
- A rollback plan is just prose until someone runs it, and plenty of the ones we've read sounded solid without ever once being exercised against the actual deployment path.
Graph engineering adds coordination power, and it charges rent for it: state-machine bugs, stale edges, retry loops, operational debt. We budget for that rent by giving the graph the same tests, monitoring, and named owner we'd insist on for any other production system.
A practical allocation model
Four operating modes cover almost everything we route through the graph.
| Work shape | AI authority | Required control |
|---|---|---|
| Low risk, high reward | Execute automatically | Record evidence and stop condition |
| Moderate risk, strong verifier | Execute in isolation | Deterministic checks and independent review |
| High risk, reversible | Prepare and recommend | Human approval and progressive rollout |
| High risk, hard to reverse | Investigate and advise | Accountable human decision |
Discovery research, code search, log correlation, documentation comparison, and disposable prototypes sit in the first row. Ordinary code changes and bounded implementation slices make up most of the second, while feature-flagged production changes, pilot launches, and reversible experiments with a tested rollback fill the third. Destructive data changes, security response, financial actions, public contracts, and the product promises that commit the company to serving a particular need sit in the fourth. Nothing there moves without someone's name attached to it.
The boundaries are not permanent. A task can move to a more autonomous mode when we build a reliable verifier or make the action reversible. That is often a better investment than trying to make the model smarter.
A few years ago, I wrote about designing systems so that more one-way doors become two-way doors. The same principle applies here. Feature flags, isolated workspaces, idempotent operations, explicit state, and tested rollback paths do more than improve reliability; they expand the area where AI can act safely, which is another way of saying they buy a larger autonomy budget.
The SDLC is still there, just made explicit
None of this replaces the SDLC most teams already run. The phases map cleanly: discovery and requirements, design and architecture, implementation, testing and verification, deployment, and operations. What changes is what happens at the handoff between phases. In a normal SDLC, "who signs off before this moves from design to build" or "who's allowed to approve a production deploy" usually lives in a runbook nobody rereads, a team's tribal memory, or whatever the last incident taught someone the hard way. In the graph, that answer is a property of the node itself: its evidence requirement, its risk boundary, its escalation condition.
We didn't stop at drawing the diagram. We built a workflow engine that runs this graph directly, using the same low risk versus high risk logic from the table above. Nodes that qualify as automatable run end to end without a person touching them: an agent does the investigation or implementation, checks its own exit condition, and the engine advances the state on its own. Nodes that need a named human decision stop and open a role-scoped input request instead, addressed to whoever the node's contract says owns that call, not to a generic approval queue.
The practical difference shows up in what a person actually has to do. A ticket tracker asks someone to update a status field by hand at every step, whether or not a human needed to be involved in that step at all. This engine only stops for a person at the points we've deliberately decided require one, and it hands that person exactly the evidence the node promised, not a blank form.
Allocation is the operating model
For many teams, the question is no longer whether AI can write production code. The harder questions concern what happens before and after the code, who controls each transition, and how the system knows when to stop.
Stripping out the checks and handing an agent every available tool would produce more autonomy immediately, and far less visibility into what it's doing with it. We're after something else: as much autonomy as the system can support without losing sight of the risk it's carrying, with AI moving fast where mistakes are cheap, observable, and reversible, demanding evidence before authority increases, and humans staying right on the edges where ambiguity and consequence outweigh execution speed.
Reward, risk, and accountability move at every step along that graph, whatever is doing the work (a person, a service, a deployment job, a model), and the trust attached to that actor depends entirely on the node it's standing in, regardless of who or what it is. The same senior engineer who can approve a production rollback on their own still needs a second signature to touch the billing system; a deployment pipeline runs on that same logic, trusted to automerge one class of change and blocked cold on another. AI fits that pattern instead of breaking it. The actual work is deciding, edge by edge, how much authority each part of the system has earned, and revisiting that as the evidence changes.
Top comments (0)