Why treating this as an architectural tradeoff is setting your production agents up to fail
I watched a team's agentic remediation system act on a runbook that was fourteen months stale.
The RAG retrieval worked correctly. The document ranked highest by cosine similarity. The agent interpreted that retrieval as authorization to execute. The live environment was meaningfully different from what the document described. The remediation made things worse.
Nobody had explicitly defined where knowing ends and doing begins.
That is the actual production risk hiding inside the "RAG vs. MCP" framing. Not which technology you pick. Whether you have drawn a hard line between your agent's knowledge layer and its execution layer — and whether that line is enforced in code rather than trusted in assumptions.
The Category Error
RAG and MCP don't compete. They answer completely different questions.
RAG answers: what does the agent know?
Runbook context. Historical incident patterns. Service documentation. Architecture decision records. Compliance policy text. Everything the agent needs to reason about a situation before it touches anything.
MCP answers: what can the agent do?
Live metric fetching. System state queries. Remediation execution. API calls against real infrastructure. All the actions that have a blast radius.
These two layers have fundamentally different risk profiles. A RAG query that returns a stale document is a reasoning error. An MCP call that executes against a live system based on a stale document is an incident.
The conflation of these two layers treating "I retrieved something relevant" as equivalent to "I have permission to execute it" removes the safety layer that should sit between them.
What the Conflation Looks Like in Practice
Pattern 1: RAG-to-execution without a gate
An agent retrieves documentation describing how to handle high CPU utilization on a service. The document is from before a major infrastructure migration. The agent, having retrieved relevant context, proceeds directly to execution — restarting pods that no longer exist in the topology described, or modifying configurations that have since moved to a different management plane.
The retrieval confidence was high. The execution authority was never scoped. There was no check between "I know something relevant" and "I will now act on it."
Pattern 2: Knowledge queries routed through execution tooling
The inverse failure. Teams that don't design the boundary explicitly tend to route everything through the heaviest available tool. A query that only needs to retrieve context ends up going through MCP tooling with live-system access — adding latency, consuming rate-limited API quota, and introducing non-determinism into what should be a deterministic knowledge lookup.
The agent has more execution authority than it needs for the task. This is a blast-radius problem waiting for the right failure condition.
The Boundary as a Reliability Control
In SRE practice, we talk about blast radius as a design constraint. You scope the potential damage of any failure before it happens, not after. Circuit breakers, canary deployments, feature flags, staged rollouts — all exist to keep the failure surface bounded.
The RAG/MCP boundary is the same class of control applied to agentic systems.
The design question isn't "RAG or MCP?" It's: at what point in this agent's reasoning flow does read-only context become a candidate for execution authority, and what explicit gate exists at that transition?
That gate needs to be deterministic. It needs to be auditable. And it needs to exist in code, not in the assumption that the model will reason correctly about the distinction every time.
What an Explicit Boundary Looks Like
A well-designed agentic system for infrastructure operations has three distinct layers with explicit handoff points.
Layer 1 — Situational awareness (RAG)
The agent retrieves runbook context, historical incident data, service topology, and policy constraints. This layer is read-only with no execution authority. The output is a structured context package: what the agent knows about the situation, what precedents exist, what the operational constraints are.
Layer 2 — Decision gate
An explicit check before any execution authority is granted. This is where you validate: Is the retrieved context fresh enough to act on? Does the proposed action fall within the pre-approved blast radius? Has a human review been configured for this action class? Is the error budget healthy enough to absorb a remediation attempt?
This layer is where most agentic systems have nothing. The model reasons through it implicitly. That implicit reasoning is not a reliability control.
Layer 3 — Execution (MCP)
Scoped tool calls with explicit permission boundaries. The agent can fetch live metrics, read current system state, and execute approved remediation actions. The scope of what's available in this layer is determined by what passed through Layer 2 — not by what the model decides is appropriate at runtime.
The War Story That Sharpened This for Me
A service I was responsible for had an agentic component handling routine disk pressure events. The pattern: detect high disk utilization → retrieve cleanup runbook → execute cleanup steps.
For eight months it worked correctly. Then we rotated to a new storage backend. The cleanup runbook in the knowledge base hadn't been updated.
The agent retrieved the old runbook with high confidence — service name matched, symptom description matched, top result by similarity score. It executed against paths that no longer existed as described. The cleanup didn't make things catastrophically worse, but it also didn't fix anything, and it consumed enough API quota that the legitimate remediation attempt that followed hit rate limits.
The failure wasn't in the retrieval. The failure was that there was no freshness check between retrieval and execution. No gate asking: is this document current enough to act on?
After that incident we added three architectural controls:
A document freshness gate on any retrieved context informing an execution decision. If document age exceeded a configured threshold for the action class, the agent escalated to a human.
A scope limiter on every MCP tool call, tying the available action set to the specific remediation class that was approved — not to everything the agent had tool access to.
A dry-run mode for remediation actions above a configured blast-radius threshold, staging planned changes as a human-reviewable diff before executing.
None of these are model improvements. All of them are architectural controls. The model's reasoning didn't change. What changed was the system around it.
The Question Worth Asking Your Team
If you're building or operating infrastructure agents right now, the design review question isn't "are we using RAG or MCP?" Both answers are probably yes.
The question is: is the boundary between your knowledge retrieval layer and your execution layer explicitly enforced in code, or is it implicitly trusted in the model's reasoning?
If the answer is "the model handles that," you don't have a boundary. You have an assumption.
Assumptions are not reliability controls.
Drop your approach in the comments — how are you enforcing the knowing/doing boundary in your agentic systems? The more specific the implementation detail, the more useful it is for everyone building in this space.
Tags: #AgenticAI #MCP #RAG #SRE #PlatformEngineering #CloudNative #AIOps #Observability
Top comments (0)