DEV Community

Cover image for What Is Agentic Governance? (And Why Your AI Team Probably Doesn't Have It)
Waxell profile image Logan
Logan for Waxell

Posted on • Edited on • Originally published at waxell.ai

What Is Agentic Governance? (And Why Your AI Team Probably Doesn't Have It)

#ai #machinelearning #programming #devops

There's a question that doesn't get asked enough in AI engineering circles: once you've shipped your agents into production, who's in charge of them?

Not "who owns the Jira ticket." Who's actually governing the behavior — in real time, at the moment decisions get made?

For most teams, the honest answer is: nobody. Or more precisely, the LLM is, which is a deeply uncomfortable thing to acknowledge once you start thinking about it seriously.

This is the agentic governance problem. And it's not a future problem. If you have agents running in production right now, it's already your problem. Arkose Labs' 2026 Agentic AI Security Report found that 97% of enterprise leaders expect a material AI-agent-driven security or fraud incident within the next 12 months. The seven weeks between late January and mid-March 2026 alone produced an unprecedented concentration of ungoverned-agent security incidents — including the first widely reported case of attackers operating an LLM itself as a post-exploitation tool.

Agentic governance is the set of runtime policies and enforcement mechanisms that define and constrain what AI agents can access, spend, and do — independent of the agent's own reasoning. It operates at three layers: policy definition (what the rules are), runtime enforcement (ensuring those rules are followed in real time), and audit (documenting every governance decision for accountability). Unlike observability, which shows you what your agent did, governance determines what it's allowed to do.

Why Doesn't Observability Equal Governance?

The field has gotten really good at convincing itself that visibility equals control. It doesn't.

You can have a beautifully instrumented tracing setup — every LLM call logged, every tool invocation captured, latency on every hop — and still have zero governance. You're watching the agent do things. That's not the same as governing what it does.

Here's a concrete example. Say your customer support agent has access to your database and can look up account records. With good observability, you know which accounts it looked up, when, and how long the query took. With governance, you've defined and enforced a rule that says: this agent can only retrieve accounts when a verified customer ID is present in the session. Without that rule, the agent will — under the right (wrong) conditions — look up whatever it feels like looking up. It's not being malicious. It's being maximally helpful, which is exactly what you trained it to be.

Observability tells you what happened. Governance determines what's allowed to happen.

The gap between the two isn't theoretical. In June 2026, BerriAI disclosed a wave of LiteLLM proxy vulnerabilities in a single weekend, including an improper-authentication flaw in the MCP Proxy component (CVE-2026-12773) that could let an attacker bypass API key checks entirely and reach proxied AI services without valid credentials — full details in Mercor/LiteLLM: what the breach revealed about AI agent audit trails. Every affected deployment almost certainly had observability: LLM call logs, latency dashboards, error rates. None of that stopped the bypass. Only a policy enforced at the proxy layer — evaluating and blocking the request before it reached a proxied service — would have.

So What Actually Is Agentic Governance?

Agentic governance is the set of policies, controls, and enforcement mechanisms that define and constrain agent behavior at runtime — not at training time, not at prompt time, but in the moment decisions get executed.

It operates across three layers:

Policy definition. What are the rules? This ranges from hard constraints ("this agent may never send an email without human approval") to soft guardrails ("if a single session consumes more than X tokens, alert the operator") to compliance requirements ("no response may include raw PII in customer-facing output"). Policies need to be explicit, versioned, and auditable — not implicit in the system prompt.

Runtime enforcement. Policies mean nothing if they're not enforced. Enforcement happens at three moments: before execution (block an action before it fires), during execution (intercept a call mid-flight and redirect or halt it), and after execution (flag a completed action for review and trigger a remediation workflow). Different risks demand different enforcement timing — the full breakdown is in AI agent policy enforcement: the three-layer framework.

Audit and accountability. Every governance decision — an action allowed, an action blocked, a policy triggered — needs to be captured with enough context to reconstruct what happened and why. "The log says the call was made" is not sufficient. An audit trail for agentic systems needs to capture the full decision context: what state the agent was in, what policies were evaluated, what the outcome was. That documentation is what separates a governance record from a monitoring log — and it's what regulators and auditors ask for. (See what an audit-ready agent deployment looks like.)

That's agentic governance. It's a control layer, not a monitoring layer.

Agentic governance vs traditional AI governance vs observability

Traditional AI Governance Agentic Governance Observability
What it governs Model outputs and decisions Agent actions, tool calls, and runtime behavior Nothing — records only
When it applies At deployment and model evaluation At every step of every execution After actions occur
Primary concern What the model says What the agent does What the agent did
Key mechanism Policy review, red-teaming, output filtering Runtime permission enforcement, scope limits, live kill switches Tracing, logging, dashboards
Observability vs enforcement Observability is sufficient Observability alone is insufficient — enforcement is required N/A
Regulatory surface EU AI Act Annex I (model risk) EU AI Act Annex III + NIST AI RMF (operational risk) Not a compliance instrument

Why Traditional Software Governance Doesn't Transfer

If you're an engineering leader who's shipped traditional software systems, your instinct is probably to reach for familiar tools: RBAC, ACLs, rate limiting, audit logging. These all have analogs in agentic governance, but the mapping isn't clean and the failure modes are different.

Traditional software systems do what they're told. You define the code path, the code executes, you log the result. The system is deterministic given the same inputs.

Agents are not deterministic. The same prompt, the same context, the same tools can produce meaningfully different behavior across runs — and that variance isn't a bug, it's the point. You wanted a system that could reason and adapt. You got one.

This means your governance layer can't assume it knows exactly what the agent will do. It has to be prepared to evaluate behavior as it emerges and apply policies to a moving target.

It also means that the "principal" in your system — the entity making decisions — is no longer a human or a deterministic process. It's a probabilistic model. Designing governance for that requires different mental models than designing access control for a REST API. The OWASP Top 10 for Agentic Applications (December 2025) — the first formal taxonomy of risks specific to autonomous agents — identified goal hijacking, tool misuse, memory poisoning, and identity abuse as distinct attack vectors that standard software governance patterns don't address. OWASP has since followed up with the Agentic AI Security Verification Standard (AISVS) 1.0, giving teams a checklist to test agent deployments against, not just a taxonomy to reference.

What Are the Two Failure Modes Teams Hit Before Governance?

If you're reading this because something went wrong, it probably fits one of two patterns.

Pattern one: the cost explosion. An agent gets into a loop, or a high-traffic moment sends usage through the roof, or a new code path creates unexpectedly expensive chains of calls — and you find out about it when the bill arrives, or when your API rate limit kicks in at 2am. There's no governance layer that set a budget, watched spend, and intervened before it hit your ceiling. (Why agent costs spiral — and how to control them →)

Pattern two: the data incident. A user put a social security number into the input. Or a tool return included a medical record from an adjacent lookup. Or the agent's context window accumulated PII from three different users in a shared session. And it went somewhere it shouldn't have — a log, an API call, a response that got cached. There's no governance layer that was inspecting the data flowing through the system. The Salesforce Agentforce ForcedLeak vulnerability (disclosed September 2025, CVSS 9.4) — in which an indirect prompt injection attack could force an Agentforce deployment to exfiltrate CRM data because the agent had no runtime data handling constraints enforced at the infrastructure layer — was a public example of exactly this failure mode, covered in depth here. (How to keep PII out of your AI agents →)

Both of these are entirely preventable. Both of them require governance, not just better logging.

What Governance-Mature Teams Look Like

The teams that have this figured out — and they're not the majority, not yet — share a few common traits.

They treat agent behavior as something that needs explicit policy, the same way they'd treat data access or financial transactions. They don't assume the model is going to be appropriately cautious because they asked it to be in the system prompt.

They have enforcement that happens before things get expensive. Budget guardrails that fire before a session blows past its allocation, not after. PII detection that runs before data gets sent to the LLM, not after it gets logged in the response.

They can answer a specific question on a bad day: "Show me everything this agent did between 3pm and 4pm yesterday, including every policy evaluation and every tool call that was made or blocked." The answer exists. It's queryable. It doesn't require digging through raw logs.

And they treat governance as a first-class part of their architecture, not an afterthought they'll bolt on once things stabilize. (Spoiler: things don't stabilize. The right moment to add governance is before you need it.) The three-layer model — cognitive architecture, execution infrastructure, and governance plane — is increasingly the industry standard, as covered in The Three-Layer Agentic Architecture Most Teams Build Wrong. This framing is also converging with how the market talks about the category itself — see What Is an AI Control Plane? for how "control plane" and "governance plane" describe the same underlying requirement. In April 2026, Microsoft released the Agent Governance Toolkit as open-source, a framework-agnostic enforcement layer that covers all ten OWASP Agentic AI Top 10 risks — an industry signal that governance-at-the-enforcement-layer is no longer an advanced practice, it's becoming table stakes.

What Does an Agentic Governance Framework Include?

A runtime-capable agentic governance framework covers five operational layers:

1. Identity and authentication — Every agent has a verified identity before execution begins. Credentials are scoped, not inherited from the user or host process. Without this, an agent can impersonate broader permissions than its task requires.

2. Permission scoping — Each agent is granted the minimum permissions needed for its declared task. Scope is set at configuration time and enforced at invocation — not just at deployment. A customer-facing agent has no business writing to a production database, regardless of what its prompt says.

3. Runtime policy enforcement — Policies fire at the moment of action, not after. A policy that says "this agent cannot write to production databases" blocks the write call in real time — it doesn't just log that one occurred. This is the layer that separates governance from observability: one controls, one records. Waxell Runtime enforces this class of policy across 50+ policy categories out of the box, at 0.045ms p95 latency, so the enforcement check never becomes the bottleneck.

4. Audit trail — Every decision, tool call, data access, and output is recorded in a tamper-evident log with full session context. This is what a compliance reviewer or auditor will ask for — and it's what distinguishes a governance record from a monitoring log. (What an audit-ready agent deployment looks like →)

5. Kill switch and escalation — Any agent can be paused or terminated mid-session without disrupting dependent systems. High-stakes actions trigger human review before execution, not after. This is the control that the $47,000 agent loop incident (November 2025) lacked: no mechanism existed to halt the four-agent LangChain system once it entered an infinite loop, and it ran for 11 days before anyone noticed. (Full breakdown: The $47K AI agent loop.)

Why Is AI Agent Governance Becoming a Regulatory Requirement?

This conversation is going to become mandatory — on a timeline that just got clearer. The EU AI Act Annex III high-risk obligations were originally due August 2, 2026. Following the EU's Digital Omnibus on AI — a trilogue agreement reached May 7, 2026 and formally endorsed by the European Parliament on June 16, 2026, with the Council of the EU giving final sign-off on June 29, 2026 — that deadline has been deferred to December 2, 2027, per the European Commission's official AI Act Service Desk timeline. This is a deadline change, not a requirements change: deployer obligations under Article 26, automatic event-logging under Article 19, and human oversight requirements still apply once the deadline arrives, with non-compliance carrying administrative fines of up to €15 million or 3% of worldwide annual turnover, whichever is higher, under Article 99(4). (The higher €35 million / 7% tier in Article 99(3) applies specifically to the Article 5 prohibited-practices category — a narrower, more severe class of violation than most Annex III high-risk obligations.) The NIST AI Risk Management Framework (AI RMF 1.0) is shaping how US enterprises approach internal AI governance. The Colorado AI Act (enforceable June 30, 2026) is the leading edge of US state-level AI regulation, with more states accelerating behind it.

Arkose Labs' 2026 Agentic AI Security Report found that 97% of enterprise leaders expect a major AI agent security or fraud incident within the next 12 months. The gap between threat level and current investment is not sustainable once regulators begin enforcement in earnest — and a 16-month deadline deferral is a longer runway to close that gap, not a reason to deprioritize it.

The teams building governance infrastructure now are doing themselves a favor that compounds: a governance-first architecture is dramatically easier to demonstrate compliance with than a monitoring-first one you're trying to retrofit. Auditors can't audit intentions. They can audit policy records, enforcement logs, and decision trails. For a detailed look at how this plays out at fleet scale — and why only 12% of enterprises currently have centralized governance despite 96% running agents — see 96% of Enterprises Run AI Agents. Only 12% Can Govern Them.

Agentic governance isn't a nice-to-have once your agent fleet gets big enough. It's the thing that lets you keep running agents with confidence — and the thing that protects you when something goes wrong.

That window where you can build it right, before you're under pressure, is open right now — and with the Annex III deadline now in December 2027 rather than this August, most teams have more runway than they think. That's an argument for starting now while the pressure is lower, not for waiting.

How Waxell handles this: Waxell Runtime is the runtime governance layer that sits between your agents and the outside world. You define policies once — spend limits, PII rules, tool constraints, human-in-the-loop gates — and Runtime enforces them across every agent session, across 50+ policy categories, without touching your agent code. Every enforcement decision is recorded in a durable audit trail that distinguishes governance evidence from monitoring logs — the documentation that matters when compliance reviews arrive. No rewrites, no rebuilds. Start free at waxell.dev/signup →

Sources

Frequently Asked Questions

What is agentic governance?
Agentic governance is the set of runtime policies and enforcement mechanisms that control what AI agents can access, spend, and do in production — independent of the agent's own reasoning. It covers policy definition, real-time enforcement, and audit logging, and is distinct from observability, which only shows you what happened after the fact.

How is agentic governance different from AI governance?
Traditional AI governance focuses on model-level controls: output filtering, red-teaming, deployment review, and alignment evaluation. Agentic governance focuses on what agents do at runtime — the tool calls they make, the data they access, the costs they incur. Traditional AI governance applies at the model level; agentic governance applies at the execution layer, in real time, for every action. This is also the essence of what the "AI agent governance framework" phrase points to when people search for it: a runtime framework, not a model-review checklist.

How is agentic governance different from AI observability?
Observability gives you visibility into what your agents did — logs, traces, session records. Governance gives you control over what they're allowed to do. You can have a fully instrumented tracing setup and still have zero governance. The governance layer is what enforces rules in real time; the observability layer is what records the outcomes. The June 2026 LiteLLM proxy vulnerabilities are a clean example: affected deployments had observability, but nothing enforced a block at the proxy layer before the authentication bypass could be used.

What does agentic governance actually cover?
Agentic governance typically covers: cost and token budget enforcement, PII and data handling policies, tool call authorization, human-in-the-loop approval gates, behavioral compliance rules, and the audit trail that documents every governance decision. Together these define the policy envelope within which an agent is permitted to operate.

Why doesn't a system prompt work as a governance layer?
System prompt instructions are suggestions to a probabilistic model. LLMs follow them most of the time — if not all of the time. Under adversarial conditions or distribution shift, compliance with system prompt constraints degrades unpredictably. Governance requires enforcement mechanisms that act outside the model's reasoning process, regardless of what the model decides. The OWASP Top 10 for Agentic Applications (2026) documents the attack vectors — goal hijacking, tool misuse, memory poisoning — that system prompts cannot reliably block.

When should you implement agentic governance?
Before you need it. Governance infrastructure built into an agent deployment from the start costs a fraction of what it costs to retrofit after an incident. If you have agents in production today without explicit policies enforced at the infrastructure layer, you have a governance gap — and the right time to close it is now, not after the first cost explosion or data incident.

What's the difference between agentic governance and traditional software governance?
Traditional software governance assumes deterministic systems: the code does what you wrote, and you control the code. Agents are probabilistic — the same inputs can produce different outputs, and the "principal" making decisions is a model, not a function. This means governance for agents requires policies that evaluate emergent behavior, not just access controls on defined operations.

Does the EU AI Act still require agentic governance if the Annex III deadline moved to 2027?
Yes. The Digital Omnibus deferral changed the compliance timeline, not the requirements themselves — Article 26 deployer obligations, Article 19 automatic logging, and Article 14 human oversight still apply once the deadline arrives on December 2, 2027, and organizations building governed agent deployments now will spend the intervening runway demonstrating compliance readiness rather than scrambling to retrofit it in late 2027.

Top comments (0)

Some comments may only be visible to logged-in visitors. Sign in to view all comments.