This is the next how-to guide in the series. The earlier guides covered individual functions, instrumentation, audit preparation, incident investigation, and selling to enterprises. This one covers a requirement that most teams get wrong in a quiet way: human oversight. Many teams put a person somewhere in the loop and consider the box ticked. Regulators, and your own risk team, will ask a harder question: could that person actually understand what the agent was doing, and could they have stopped it?
It is written for CTOs, heads of risk, and product leads at banks, insurers, and vertical AI companies who deploy agents that influence decisions about people or money.
A note on scope: ZizkaDB records what agents and reviewers do and lets you reconstruct it. The approval gates and escalation rules themselves live in your agent runtime and your operating procedures. This guide shows how to design both so they produce evidence.
Before you start: what oversight actually means
Article 14 of the EU AI Act, which applies to high risk systems, asks for oversight measures that let the people assigned to the task do things like the following:
Understand the system’s capabilities and limitations Monitor its operation and notice anomalies Stay aware of the tendency to over-trust automated output Interpret the output correctly Decide not to use the output, or override it Intervene or stop the system
Article 26 adds that deployers must assign oversight to people with the necessary competence, training, and authority. Whether your agent counts as high risk depends on its use case, so confirm that with legal counsel. The design below is good practice either way, because it is also what customers and auditors tend to ask for.
The key idea is that oversight is only real if it is informed, timely, and effective, and you can only prove those three properties if the review itself leaves a record.
Step 1: Classify actions by consequence
List what each agent can do and sort the actions into tiers:
Tier 1, reversible and low impact: drafting, summarizing, internal lookups. Sampling review is enough. Tier 2, customer visible or moderately consequential: sending a message, changing a limit, recommending a decision. Review before or shortly after, depending on risk. Tier 3, irreversible or high impact: moving money, declining credit, closing an account, filing a regulatory report. A human decision before the action executes.
Write the tiers down and get risk and legal to sign off. This table becomes the backbone of everything else.
Step 2: Choose the oversight mode per tier
Three modes are common, and it is worth being explicit about which applies where:
Approval before action. The agent proposes and a human approves. Use for Tier 3. Intervention window. The agent acts after a short delay unless a human stops it. Use where speed matters and actions can be cancelled. Sampling and after the fact review. Humans review a sample of completed actions. Use for Tier 1 and some Tier 2.
Avoid choosing approval everywhere. Reviewers who see hundreds of trivial requests stop reading them, which defeats the purpose.
Step 3: Make the approval gate a first class event
An approval is a decision, so record it as one. Your agent should emit events when it requests review and when the review is resolved, linked into the same causal chain as the action itself.
python
illustrative pseudocode, check the SDK docs for exact calls
def request_approval(proposal, reviewer_group): req_id = emit("decision", { "action": "approval_requested", "proposal": proposal.summary, "tier": 3, "reviewer_group": reviewer_group, }) return req_id
def resolve_approval(req_id, reviewer_id, outcome, reason): emit("decision", { "action": "approval_resolved", "reviewer": reviewer_id, "outcome": outcome, # approved | rejected | modified "reason": reason, "request_ref": req_id, })
Use the same fixed event types and put the specifics in the payload, so drift baselines stay stable. Capture who decided, what they decided, when, and any reason given. Where the reviewer changes the proposal, record both versions.
Make the gate fail safe: if the approval service is unavailable, a Tier 3 action should wait or be refused, not proceed silently.
Step 4: Give reviewers the context to decide
A reviewer who sees only a recommendation cannot judge it. For each review request, decide what the reviewer sees, and build it from the record:
The agent’s proposal and a short rationale The key inputs and retrieved context behind it, drawn from the causal chain The tool results that mattered Known limitations relevant to this case, such as low retrieval confidence or a tool that returned an error A link to the full session replay for anyone who wants to dig in
Keep the default view short and the depth one click away. The goal is informed review in minutes, not a forensic exercise for every case. Be careful about what you surface: show data the reviewer is entitled to see, and keep tenant and privacy boundaries intact.
Step 5: Assign oversight to named, trained roles
Define who reviews each tier, what training they need, and who has authority to pause or override the agent. Record role assignments with effective dates so you can later show who was responsible on a given day, which pairs naturally with at() for historical state.
Cover cover and escalation: what happens when the reviewer is unavailable, disagrees with the agent repeatedly, or sees something unusual. Write down who can stop an agent entirely and how that is done.
Step 6: Build a stop mechanism and test it
An ability to intervene means little if stopping the agent takes an engineering ticket. Provide a documented way to pause an agent, a tool, or a class of actions quickly, and make sure the pause itself emits an event with who triggered it and why.
Test it as a drill. Pick a scenario, trigger the stop, and measure how long it takes and whether the in-flight sessions end in a clean state. Keep the drill record as evidence.
Step 7: Measure whether oversight is working
Oversight can look healthy while being ineffective. A reviewer who approves 99.9 percent of requests in a few seconds each may be rubber stamping, or the agent may simply be excellent. You need data to tell them apart. Track per reviewer group and per action type:
Approval, rejection, and modification rates Time spent per review How often reviewers open the replay or deeper context Disagreements later proven right, such as an approved action that led to a complaint Queue age, meaning how long requests wait
Watch for warning patterns: near zero rejections combined with very short review times, reviews clustered at the end of shifts, or approval rates that jump after a prompt change. These are signals to investigate, not verdicts. Deliberately inject known test cases, including some the agent gets wrong, to check whether reviewers catch them.
Step 8: Connect oversight to drift monitoring
Oversight and drift watch different things. Reviewers see individual cases, while drift shows shifts in the overall pattern. Use them together:
Review drift after every prompt, model, or tool change, and flag transitions that should never disappear, such as a verification step before a Tier 3 action. When drift appears, temporarily tighten oversight: raise the sampling rate or move a Tier 2 action to approval until you understand the change. When reviewers report that the agent seems different, check the drift view for corroboration.
Remember that drift indicates change, not whether the change is good or bad. Use error rate, session length, and review outcomes alongside it.
Step 9: Close the loop on overrides
Every rejection or modification is a learning signal and a piece of evidence. For each one, review the session with replay, use why() to see what led the agent astray, and classify the cause: bad input, bad context, tool failure, or reasoning error. Feed recurring causes into prompt fixes, validation rules, or evaluation sets.
Keep a log of what changed as a result. Being able to show that human interventions led to concrete improvements is strong evidence that oversight is effective rather than ceremonial.
Step 10: Prepare the oversight evidence pack
When an auditor or customer asks how oversight works, you should be able to hand over a package without starting from scratch:
The action tiers and mode per tier (Steps 1 and 2) Role assignments, training records, and authority to stop the agent (Step 5) Sample review cases with replay and lineage, using synthetic or consented data (Steps 3 and 4) Stop mechanism drill records (Step 6) Reviewer effectiveness metrics and test case results (Step 7) Drift review records and the actions taken (Step 8) Examples of overrides and the resulting improvements (Step 9)
Verify the SHA-256 checksums on any events you include, so you can show the records are unchanged since they were written. Remember that checksums show integrity, not that a decision was correct. The pack supports your technical documentation and compliance reporting but does not replace your risk management system, training obligations, or any formal assessment.
What the business case looks like
These figures are modeled assumptions, not measured results. Replace them with your own.
Assume 500 escalated decisions a month reviewed by humans, at 60 euro per hour for a reviewer. Without connected context, a reviewer spends about 12 minutes per case gathering inputs from several systems, costing 6,000 euro a month. With the proposal, rationale, and replay assembled from the record, review takes about 4 minutes, costing about 2,000 euro a month. That saves roughly 4,000 euro a month, or 48,000 euro a year, and shortens customer wait times as well.
Add audit and customer evidence requests about oversight. Assume 4 requests a year, each needing 2 people for about 3 days to piece together who reviewed what and when, at 100 euro per hour, costing around 4,800 euro per request. With review events already recorded and linked, the same request takes about half a day, costing around 800 euro, a saving of roughly 16,000 euro a year.
The larger value is risk. An oversight process that cannot show it worked is exposed in every serious incident. Under the EU AI Act, breaches of most obligations can bring fines of up to 15 million euro or 3 percent of global annual turnover, whichever is higher, and records that were never captured cannot be recreated later.
Common mistakes to avoid
Putting a human in the loop and stopping there. Presence is not oversight. Reviewers need context, authority, and time.
Requiring approval for everything. Over-review trains people to click through. Reserve approval for actions where it matters.
Not recording the review. If the approval is not an event in the record, you cannot prove it happened or who made it.
Hiding the stop button. An intervention path that takes days to use will not help during an incident.
Never testing reviewers. Without test cases and effectiveness metrics, you cannot tell oversight from rubber stamping.
Treating the tool as the oversight. ZizkaDB supplies the evidence. People, procedures, and authority supply the oversight.
Where to go from here
Start with one agent and its Tier 3 actions. Define the gate, record the approvals, give reviewers a short context view, and run one stop drill. After a month, look at your review times and rejection rates, because they will show you where oversight is real and where it is only ceremony.
If you tell me your sector and the decisions your agents make, I can help tailor the tiers, the reviewer metrics, and the numbers to your case.
Want to test ZizkaDB on your agent? Try our open source version here: https://github.com/ZIZKA-AI-SL/ZizkaDB
Interested in a design partnership? Fill in the form on our site or reach me directly at founder@zizka.ai
Top comments (0)