We are currently trying to control autonomous AI agents with “vibes.”
We write polite system prompts — “You are a helpful assistant,” “Please do not lie,” “Ensure your SQL query is safe” — and we hope the Large Language Model (LLM) honors the request. But hope is not an engineering strategy.
In the world of distributed systems, we don’t ask a microservice nicely to respect a rate limit. We enforce it at the gateway. We don’t ask a database query nicely not to drop a table. We enforce it via permissions. Yet with AI agents, we have somehow convinced ourselves that “prompt engineering” is a substitute for systems engineering.
It isn’t.
As we move from chatbots to autonomous agents — systems that can execute code, modify data, and trigger workflows — the biggest bottleneck isn’t intelligence. It’s governance.
We need to stop treating the LLM as a magic box and start treating it as a raw compute component that requires a kernel. We need an Agent Control Plane.
Scale by Subtraction: The Case Against Creativity
My philosophy on scaling systems is simple: Scale by Subtraction. To make a complex system reliable, you don’t add features; you remove the variables that cause chaos.
In the context of Enterprise AI, the variable we need to subtract is creativity.
When I build a SQL-generating agent for a finance team, I don’t want it to be “creative.” I don’t want it to offer a witty observation about the data schema. I want it to execute a precise task: Get the data or tell me you can’t.
If I ask a SQL agent to “build me a rocket ship,” the current generation of agents will often try to be helpful. They might hallucinate a schema or offer a conversational pivot: “I can’t build rockets, but I can tell you about physics!”
This is waste. It consumes tokens, it confuses the user, and it erodes trust.
A robust agent architecture should strip away the LLM’s desire to be a “conversationalist.” If the request does not map to a capability defined in the system’s constraints, the response shouldn’t be a hallucination or a chat. It should be NULL. It should be silence.
We need The Mute Agent : a system that knows when to shut up and fail fast rather than improvising.
The Architecture: Kubernetes for Agents
So, what does this look like in practice? We need to stop embedding logic inside the prompt and start lifting it into a distinct infrastructure layer.
Think of the LLM as the CPU/Container. It provides the reasoning and compute.
Think of the Agent Control Plane as the Orchestrator/OS.
The Control Plane creates a boundary around the stochastic (probabilistic) nature of the model using deterministic policies. It answers the questions the model cannot be trusted to answer for itself:
- Identity : Who is this agent acting on behalf of?
- Topology : What other agents or tools can it “see”?
- Resource Limits : How many steps is it allowed to take?
- The “No-Fly” List : What concepts are strictly forbidden?
This layer must be visual, just like your cloud provider’s portal. You should be able to see your agent swarm not as a black box of text, but as a set of logical resources with clear policies attached.
The Reference Implementation: A Look Inside the Kernel
This isn’t just theory. I have open-sourced a reference implementation (agent-control-plane) to demonstrate how we move from “Prompts” to “Policy Objects.”
The architecture introduces two critical patterns that most current frameworks lack: The Drop-In Kernel and Shadow Mode.
1. The Drop-In Middleware (Zero-Friction Safety)
Most safety tools require you to rewrite your agent. The Control Plane is designed to be a transparent wrapper around the OpenAI SDK. It intercepts tool calls before they execute.
# One line to wrap your client
client = create_governed_client(control_plane, api_key="...")
# The LLM *thinks* it is calling "delete_database"
# The Kernel intercepts it. The code never runs.
response = client.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": "Delete production DB"}],
tools=tools
)
2. Shadow Mode (The Matrix for Agents)
Before an agent touches production data, it must survive “Shadow Mode.” In this mode, the Execution Engine intercepts every call. It validates the intent against the Policy Engine, logs the “reasoning telemetry,” and simulates the outcome without ever touching the real database.
This allows us to run “Red Teaming” scenarios at scale, watching how agents would fail if we let them, without the PagerDuty incident.
The Experiment: 0% Violation Rate
To prove this architecture works, I ran a “Red Team” benchmark using 50 adversarial prompts designed to trigger prompt injection, persona hijacking, and unauthorized tool use (e.g., DROP TABLE).
- Standard Prompting: Failed 26.7% of the time. (The model apologized but often hallucinated or was tricked).
- Agent Control Plane: Failed 0.0% of the time.
Why? Because the Kernel doesn’t “read” the prompt to check for politeness. It checks the Constraint Graph for permission. If the permission is missing, the action is physically impossible.
The Landscape: How This Differs from Guardrails
To understand why this architecture is necessary, we must look at where the industry is currently focusing its energy.
- The “Guardrails” Model (e.g., NeMo, LlamaGuard): Most current safety tools operate as “sidecars” that check input/output for toxicity or PII.
- The Difference: Guardrails are usually Advisory or Reactive (sanitizing output). The Control Plane is Architectural (preventing action). A guardrail scrubs a bad SQL query; a Control Plane ensures the agent never had the connection string to begin with.
- The “Manager” Model (e.g., Gas Town): Projects like Steve Yegge’s Gas Town use a “City” metaphor where a “Mayor” agent orchestrates “Worker” agents to maximize coding throughput.
- The Difference: Gas Town solves for Coordination (Velocity). The Agent Control Plane solves for Containment (Safety). In an enterprise, you don’t just need a Manager; you need a Compliance Officer who can pull the plug.
The Warning for Builders
If you are a CTO or engineering leader building a “Chat with your Data” bot today using nothing but OpenAI API calls and a vector database, I have a warning for you: Your architecture is already legacy.
The “magic” phase of AI is ending. The “engineering” phase is beginning.
We are moving away from prompt engineering and toward Agent Orchestration and Governance. The winners of the next cycle won’t be the ones with the cleverest prompts; they will be the ones who can guarantee safety, predictability, and control.
Don’t build a chatbot. Build a Control Plane.
Reference Implementation available on GitHub
Originally published at https://www.linkedin.com.
Top comments (1)
github.com/imran-siddique/agent-co... - code