DEV Community

t49qnsx7qt-kpanks
t49qnsx7qt-kpanks

Posted on

What open-source governance tooling can't tell you (and why that's a problem)

NOTE: re-routing reply → article because source=devto (comment API deprecated, read-only) and score ≥85

What open-source governance tooling can't tell you (and why that's a problem)

Joao Marques documented five open-source tools for AI agent governance in 2026. The list is solid. The gap is that none of them — Microsoft Agent Governance Toolkit, asqav, Guardrails AI, NeMo Guardrails, AgentMint — answer the question that's actually keeping CISOs up at night: what can my agents do right now that I haven't explicitly authorized?

That's not a tool problem. It's a visibility problem that happens upstream of every governance framework.

Open-source governance tooling generally works in one of two modes. Either it wraps individual agent calls with constraints you've defined — Guardrails AI, NeMo Guardrails — or it surfaces observability data about what agents are doing after the fact. Both are valuable. Both assume you already know what your threat surface looks like.

The problem Joao flags — "agents moving into production faster than governance tooling" — is real, but the mechanism is subtler than deployment speed. The gap isn't that agents are shipped before governance tools exist. It's that agents are shipped before anyone has mapped what the agents can actually reach.

Here's how it plays out: a developer provisions an agent with tool access sized for the initial task. The task expands over two sprints. The tool access expands incrementally — each addition seems small. Six months later, the agent has read/write access to four systems, can call three external APIs, and has a service account with permissions inherited from a 2023 migration that nobody cleaned up. The governance tooling is installed. It's running. But it's enforcing constraints on a permission footprint that no one drew.

The fix before you pick a governance framework: inventory. Not the tools the agent nominally has — the tools it can actually reach, including via tool introspection, inherited service account permissions, and MCP server capabilities that were added during a library update. That's the map every governance framework assumes you have and almost no one actually does.

Three things that surface in that inventory almost every time:

Permissions that survived team changes. An agent was built by an engineer who left. The service account is still active. Its scope was never reviewed.

Tool introspection gaps. MCP servers expose their tool list at runtime. If your governance tooling isn't auditing what the MCP server actually advertises versus what you think it advertises, you have a gap.

Spend authority ambiguity. Agents that can initiate external calls — model inference, API calls, webhook triggers — often have no concept of budget ceiling. The agent knows it can call the endpoint. It doesn't know if this specific request has human approval behind it or if it's extrapolating from a general instruction.

Open-source tooling handles the constraint layer well once you have the map. The audit step that generates the map is the part that's still largely manual — or skipped entirely because teams assume the permissions are what they configured them to be.

The BizSuite AI Audit does the map: a two-hour working session to inventory what your agents can actually reach, surface the gaps between nominal and actual permissions, and hand you a prioritized remediation list in 48 hours. $997. If you're writing about governance tooling in 2026 and want a reference point for what the pre-framework audit actually finds, worth a look: https://getbizsuite.com/ai-audit.html

Top comments (0)