Your agent triaged the alerts, wrote the patch, opened the pull request, updated the docs, and merged. The bug is in production. The migration ran against the wrong table. The customer got the email with the wrong number. All of it is irreversible. And a human owns every bit of it.
Everyone has deployed agents by now. Almost nobody has written down where an agent's responsibility ends. This article draws that line.
The cost of zero oversight
45% of AI assistant answers about news contain distortions, across 18 countries and 14 languages, according to the EBU/BBC study "News Integrity in AI Assistants" (October 2025). This is not a bug that a better prompt fixes. It is a property of the architecture: large language models do not know what they do not know.
Two public failures already belong in every engineering handbook:
- CNET, January 2023: AI-written articles with gross errors and unattributed borrowing, followed by mass corrections and brand damage (CNN).
- Bloomberg, 2025: at least 36 corrected AI summaries since January (NYT investigation, March 2025).
The pattern is identical in both cases. The agent did "routine" work: summaries, drafts, facts. The error reached the public because a human approval step had been removed. The price of the saved hour was a week of cleanup and a permanent entry in the company's error history.
At the portfolio level the picture is the same. The MIT NANDA report "The GenAI Divide: State of AI in Business 2025" found that about 95% of corporate GenAI pilots show no measurable impact on the P&L (Fortune, August 2025). The report is widely cited but not peer-reviewed, so read it as "pilots without measurable P&L impact", not "AI fails". The usual cause is not the model. It is the missing operating structure around it.
The real criterion is reversibility, not "routine vs creative"
The standard advice "automate the routine, keep the creative work human" breaks on contact with engineering reality.
A dependency bump is routine. It can also take down production on a Friday night, and rolling back a broken deploy with data migrations behind it is not always possible. Writing an architecture proposal is creative work. A bad proposal costs you one meeting and ten minutes of edits.
The working criterion: automate what you can delete tomorrow. Everything that cannot be undone (a deploy, a payment, a public statement, deleted data, a leaked credential) stays with you.
Reversible steps: log triage, alert summaries, test generation, lint and formatting, draft PRs on sandbox branches, documentation drafts, dependency scan reports, research and comparison notes.
Irreversible steps: production access and credentials, spending money, deleting or migrating data, messages sent to customers, public statements, the final call in an incident.
Notice that the split has nothing to do with how hard the task is. It has everything to do with the cost of being wrong.
The three-bucket matrix for an engineering team
| Bucket | What goes in | Who is responsible | Checkpoint |
|---|---|---|---|
| Agent, autonomous | Log triage, alert digests, test generation, lint/format, draft PRs in isolated branches, docs drafts, dependency and vulnerability scan reports | Agent | Automated checks: CI gates, tests pass, output schema valid |
| Agent, under approval | Code changes to shared repos, infra config changes, data migrations with a tested rollback, external communication drafts, any analysis that contains facts | Agent proposes, human approves | Human approves each step before merge, send, or run |
| Human only | Production credentials and access, payments and spend, deleting data, customer-facing sends, public statements, security exceptions, the final incident call | Human, solely | No delegation at all |
A concrete flow. The agent scans the overnight logs, clusters the errors, and files a report. It drafts a fix on a branch and opens a PR with tests attached. A human reads the diff, questions two assumptions, asks for a change, and approves. The agent rebases, waits for green CI, and prepares the release notes. The human presses merge. The agent never holds the deploy key.
No tool names here, only roles and checkpoints. Tools change every quarter. The responsibility boundary does not.
What the market already decided
Augmentation beats automation in practice. The Anthropic Economic Index (February 2025, millions of Claude conversations) shows 57% of usage is augmentation, where the AI works together with a human, versus 43% automation (arXiv, Anthropic). Caveat: this measures Claude users, not the whole economy. Still, the direction is consistent everywhere it is measured: autopilot is marketing, copilot is reality.
Audiences demand a human in the loop. The Reuters Institute Generative AI and News Report (6 countries, 2025) found 12% are comfortable with news made entirely by AI, 43% when a human leads with AI help, and 62% for entirely human-made news (report). Your customers read your changelog, your status page, and your docs with the same eyes.
Newsrooms got there first. In a UK survey of journalists (August to November 2024, published 2025), the most common item in editorial AI policies was "human oversight and control" at 44%, and 60% of respondents said their outlet already has AI protocols (Reuters Institute; UK sample only). An industry built on trust fixed the boundary in writing. Engineering is behind.
Hiding AI use is a losing trade. According to an industry survey by Fractl (1,008 consumers, Q2 2026; methodology not independently verified), 84% want AI-written text labeled and distrust of brands with heavy AI marketing doubled year over year. Treat the numbers as directional, but the direction matches independent research on the AI trust penalty. If a human did not stand behind the output, the market eventually prices that in.
From policy to enforcement: why the boundary must live in the workflow, not in the prompt
Here is the part most teams skip. A delegation matrix in a wiki page changes nothing. Agents do not read wikis. The boundary only works when it is part of the system the agent actually runs in.
We build agent governance tooling, and this is the lesson that cost us the most to learn:
- A boundary you cannot verify is documentation, not control. When we built the Hlinor agent registry, the whole design came down to one question: how do you prove which contract an agent agreed to? The answer was signed YAML contracts with Ed25519 keys, so the allowed scope is a checkable artifact, not a paragraph of prose.
- You cannot draw a boundary around what you cannot see. Our control plane scanner exists because teams consistently underestimate what is connected: agents, MCP servers, forgotten integrations. The scanner inventories them and feeds CI gates, so "what can this agent touch" becomes a question with an answer in the pipeline.
- Scope drift is what happens when the boundary is prose. ScopeGuard, our scope-drift detector, started from a simple observation: agents quietly expand from "summarize the thread" to "answer the thread" unless the allowed scope is explicit and checked. Drift is the default, not the exception.
The same three ideas appear in every audit we run: a declared contract, an inventory of what is actually connected, and a check that reality still matches the contract. The delegation matrix is the human-readable version. Contracts, scanners, and gates are the machine-enforced version. You need both.
An evening checklist: mark your own boundary
- Inventory every agent and what it can touch. Repos, environments, credentials, inboxes, payment rails. If you cannot list it, that is the first finding.
- Sort each capability into the three buckets by reversibility. Delete it tomorrow with no trace? Agent. Needs your judgment? Agent under approval. Cannot be undone? You.
- Install explicit checkpoints at the bucket boundaries. Minimum set: after research, before merge, before send, before spend. A checkpoint is a named human action, not a dashboard.
- Write the rollback rule before you need it. If the agent makes an irreversible mistake, who gets paged, what gets revoked first, and how do you tell the affected customer?
- Write it down as a contract, not a memo. Scope, permissions, and checkpoints in a file the pipeline can check. Review it quarterly, because drift is the default.
The closing line
Gartner forecasts that over 40% of agentic AI projects will be canceled by the end of 2027, citing rising costs, unclear business value, and weak risk controls (Gartner press release, June 2025). This is a forecast, not a fact. But the logic is hard to argue with: agents without a responsibility boundary are a cost center, not an asset.
The difference between the teams that survive the forecast and the teams inside it is simple. The first group drew the delegation boundary before the first incident. The second group drew it after.
Your matrix takes one evening. Draw it tonight, work inside it tomorrow.
Hlinor runs technical risk due diligence on legacy stacks and AI-agent deployments. The open-source tooling behind this article is at hlinor.com/open-source, and a real (anonymised) audit is at hlinor.com/sample-audit.
Top comments (0)