Human oversight under the EU AI Act means a named person can understand what your agent is doing, override or stop it, and prove afterwards that they could. For most agents that comes down to four things you have to build: a capability sheet, a decision boundary enforced in code, a stop control that really halts the system, and a log that records every consequential decision together with who made it. Article 14 only binds high-risk systems, but if you sell into healthcare, HR, lending or the public sector, assume you are in scope and design for it now rather than retrofit it under procurement pressure.
Who this actually applies to
Article 14 sits in Chapter III of Regulation (EU) 2024/1689, the chapter that sets obligations for high-risk AI systems. Whether your agent is high-risk is decided by two annexes. Annex I covers AI that is a safety component of a regulated product, which includes medical devices. Annex III lists use cases: biometrics, critical infrastructure, education, employment and worker management, access to essential private and public services, law enforcement, migration, and the administration of justice.
Translate that into agency terms:
- Not high-risk: a WhatsApp agent that answers product questions and recommends a size. Transparency rules under Article 50 still apply, and we covered those in disclosure rules for AI sales agents.
- High-risk: an agent that triages patient messages, screens job applicants, scores a loan file, or decides eligibility for a public benefit. Same stack, same model, completely different obligations.
Annex III obligations were scheduled to apply from 2 August 2026. The timeline has been the subject of proposed amendments, so check the current consolidated text before committing to a date in a contract. What has not waited for the legal deadline is procurement: hospital and government buyers already ask for oversight documentation in tenders, which is why this matters commercially today.
What Article 14 asks for, in plain language
The article requires that high-risk systems are designed so natural persons can effectively oversee them while in use. The person doing the oversight must be enabled to:
- understand the system's capabilities and limitations, and monitor its operation for anomalies
- stay aware of automation bias, the tendency to over-trust the output
- correctly interpret the output, with tools to do so
- decide not to use the system, or disregard, override or reverse its output
- intervene in its operation, or interrupt it through a stop button or similar procedure
Two things are easy to miss. First, the obligation to build these measures in falls on the provider, which is you or your agency, while the obligation to assign competent people falls on the deployer. Second, the article never says a human must approve every action. The measures must be commensurate with the risk. That proportionality is your design space, and it is where most of the engineering decisions live.
The four things to build
1. A capability sheet
A one-page document, shipped with the product, that states what the agent is for, what it was tested on, what it is known to get wrong, and what it must never do. This sounds like paperwork, but it is also the spec for the next three items. If you cannot write down what the agent must never do, you cannot gate it.
2. A decision boundary enforced in code, not in the prompt
The most common mistake we see in agent demos is a system prompt that says "never send a message without approval" and nothing else. A prompt is a suggestion to a probabilistic model. It is not a permission system, and Anthropic's own reporting on unintended agent actions is a useful reminder that instructions get ignored under enough context pressure.
Split the agent's actions into two lists. Autonomous: fetch, classify, summarise, draft, retrieve. Gated: send to a patient, update a record, reject an application, issue a refund above a threshold. Then enforce the split in the tool layer. The agent gets a request_approval tool and never gets send_message at all. The approval service is ordinary deterministic code with ordinary tests. We go deeper on routing and fallbacks in escalation paths for AI agents, and the rest of the hardening list is in our AI agent security checklist.
3. A stop control that stops
Article 14 is explicit about interruption. A real stop control has three parts: a flag the agent checks before every tool call, cancellation of runs already in flight, and a drain of queued work so nothing resumes on its own when someone flips the flag back. Test it on a schedule, the way you test backups. If stopping the agent today means asking DevOps to scale a deployment to zero, you do not have a stop button, you have an outage procedure.
4. An oversight record that proves something
The log is the part a regulator, an auditor, or a court will actually read. For every consequential output it should let you answer five questions:
- What did the agent see? Inputs, retrieved documents, model version, prompt version.
- What did it propose? The draft, the classification, the recommended decision.
- What did the human do? Approved, edited, rejected, or never looked.
- Who was the human? An identity, not a shared service account.
- When, and for how long? A decision approved in under a second is a rubber stamp, and the record should make that visible.
Store it append-only, with the same discipline we describe in court-ready architecture for healthcare AI.
Our own outreach engine is nowhere near high-risk, but it is built this way because the record is useful regardless of regulation. It scrapes each prospect's site with a self-hosted Firecrawl instance, and a local LLM extracts the relevant facts and writes one tailored email per company in a single call. That single-call pattern beat a multi-step chain on both cost and output quality, and it had a side effect we did not plan for: the facts the model relied on and the draft it produced come out of the same call, so the reviewer sees the evidence next to the conclusion, and there is one input and one output to store per email rather than a chain of intermediate states to reconstruct later.
Automation bias is the requirement people skip
The Act names automation bias directly, and it is the one measure that almost no agent product addresses. If the approve button is the default focus, the reviewer sees only the final answer, and the queue is long, your oversight is theatre and the log will show it.
Three design moves fix most of it:
- Show the evidence, not just the answer. The retrieved passage, the extracted fact, the rule that fired.
- Hide the suggestion for a sample of items. Let the reviewer decide cold on a fraction of cases and compare.
- Track the override rate per reviewer. Zero overrides over weeks is a warning sign, not a success metric.
What this costs and how to scope it
Designed in from the first sprint, oversight is a modest slice of an agent build: the approval service, the flag, and the log schema are small, well-understood components. Bolted on after launch, it is a rewrite of the tool layer and a migration of every historical run you now wish you had recorded. When you scope a high-risk agent, put these four items in the statement of work explicitly rather than hoping they appear under "security".
If you are evaluating an agency for this kind of work, four questions separate a real implementation from a slide:
- Show me the stop control, and show me the last time it was tested.
- Show me one complete log entry for one real decision.
- Which actions are gated, and where in the code is that enforced?
- How do you detect that reviewers have stopped reviewing?
If you are building an agent for a hospital, an HR team, a lender or a public body and you want it to survive the procurement questionnaire and the audit that follows, let's talk.
Originally published on the Pykero blog.
Top comments (0)