Architectural Evaluation: Why We Re-Engineered AI Safety From Text Guardrails to Infrastructure Control Planes

Introduction

The open-source AI safety ecosystem includes mature tools for structured-output validation, conversational guardrails, sensitive-data detection, and prompt or response filtering. Frameworks such as Guardrails AI, LLM Guard, and NVIDIA NeMo Guardrails address important parts of this problem.

But autonomous AI systems introduce a broader security question.

An agent does not merely generate text. It can select models, retrieve untrusted context, invoke Model Context Protocol (MCP) servers, query databases, modify files, create pull requests, deploy applications, or trigger external business processes.

For these systems, checking whether text is safe is not enough. The platform must also decide:

Is this identity permitted to perform this action, through this connector, against this resource, using these arguments, within this tenant and region—and does the action require approval?

That is the role of an AI infrastructure control plane.

This article explains why we built PolicyAware around policy enforcement, MCP tool governance, risk classification, model routing, evaluation, and audit evidence rather than positioning it as another text-validation wrapper.

---

1. Guardrails and Control Planes Solve Different Problems

Describing every guardrail framework as a basic string filter would be inaccurate. Some support model-based validation, programmable constraints, conversational flows, and structured-output enforcement.

The architectural distinction is primarily about where enforcement occurs and what it governs.

Dimension	Text and conversational guardrails	PolicyAware control plane
Primary responsibility	Validate prompts, responses, conversation flows, or output structures	Govern requests, identities, models, tools, side effects, evaluations, and evidence
Typical inputs	Prompt and model output	Prompt, role, tenant, region, risk, connector, action, arguments, and policy
Common decisions	Pass, fail, retry, repair, or filter	Deny, require approval, allow, conditional allow, or transform
Tool governance	Usually application-specific	Connector/action policy through ToolPolicyEngine and MCP proxying
Protocol awareness	Product- and integration-dependent	Inspects MCP JSON-RPC tools/call requests before forwarding
Model routing	Usually outside validation	Policy-aware routing based on risk, region, provider, cost, and availability
Audit evidence	Validation results and logs	Decisions, reason codes, matched policies, evaluations, and trace IDs
Deployment	Library or framework integration	Embedded SDK, middleware, CLI, MCP proxy, or HTTP sidecar
Runtime implementation	Varies by framework	Local deterministic Python rules in the base package
Performance	Depends on validators and model calls	Must be benchmarked for the deployed policy and integrations

The approaches are not necessarily competitors. A control plane can use specialized guardrail frameworks as optional components while retaining deterministic policy as the final authority.

PolicyAware supports this layered model through optional integrations with Presidio, Transformers-based classifiers, NeMo Guardrails, and Guardrails AI.

---

2. Why Autonomous Agents Need Action-Level Governance

Consider an agent connected to a filesystem MCP server:

filesystem.read_file
filesystem.write_file
filesystem.delete_file

From a language-model perspective, these are tool calls. From a security perspective, they represent substantially different risks.

A production system may require:

• developers can read approved files,
• writes require human approval,
• deletion is denied,
• paths outside an approved directory are blocked,
• sensitive arguments are redacted before forwarding,
• and every decision is associated with an identity and trace ID.

A prompt filter cannot enforce these rules reliably by examining conversation text alone. Enforcement must occur at the tool boundary before the MCP server receives the request.

PolicyAware represents these requirements as deny-by-default policy:

id: mcp_filesystem_policy
schema_version: "0.2"
default: deny

connectors:
  - id: filesystem
    type: mcp
    actions:
      read_file:
        effect: allow
        risk: low
        side_effect: none
        when:
          user.role_in: [developer, security_engineer]

      write_file:
        effect: require_approval
        risk: high
        side_effect: write
        when:
          user.role_in: [developer]

      delete_file:
        effect: deny
        risk: critical
        side_effect: delete

This policy does not ask the model whether deleting a file seems appropriate. It evaluates a deterministic rule using the connector, action, identity, and request context.

---

3. PolicyAware's Runtime Architecture

PolicyAware separates governance into explicit, replaceable components.

A typical request follows this lifecycle:

1. The application supplies the request and structured context.
2. Data-protection checks detect PII, PHI, secrets, and sensitive values.
3. Risk classification assigns a low, medium, high, or critical tier.
4. Policy determines whether to deny, require approval, allow, or transform.
5. Denied and approval-gated requests stop before execution.
6. Allowed requests proceed to a model router or tool-policy engine.
7. Runtime evaluation checks results for leakage, citations, and policy consistency.
8. Audit components record the decision and supporting evidence.

One important semantic property is that transformation does not grant access. A redaction rule may modify an otherwise allowed request, but it cannot convert a denied action into an allowed one.

---

4. First-Class MCP JSON-RPC Governance

PolicyAware includes an MCP policy proxy that evaluates raw JSON-RPC traffic before a tool call reaches the underlying MCP server.

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "filesystem.read_file",
    "arguments": {
      "path": "README.md",
      "query": "Find records for [e-mail]"
    }
  }
}

The proxy can:

• evaluate identity, role, tenant, and agent context,
• inspect tool arguments for sensitive information,
• deny unauthorized actions,
• require approval for high-impact operations,
• redact permitted arguments before forwarding,
• return structured JSON-RPC errors,
• and pass non-tool protocol messages through unchanged.

Test a request from the CLI:

policyaware mcp check policyaware.yaml mcp-request.json

Place a live stdio proxy in front of an MCP server:

policyaware mcp proxy policyaware.yaml \
  --connector filesystem \
  --agent coding_agent \
  --role developer \
  --server-command "python filesystem_mcp_server.py"

When an action is denied, it is not forwarded to the real server. The client receives a structured JSON-RPC error containing the decision, connector, action, and policy context.

---

5. Structured Recovery Instead of Opaque Failure

Policy violations should not necessarily crash an entire agent session. However, claiming that a policy engine automatically reroutes every trajectory would overstate what an enforcement layer can guarantee.

PolicyAware returns structured outcomes that an orchestrator can handle deliberately:

• deny,
• require_approval,
• allow,
• transformed request data,
• reason codes,
• matched policies,
• remediation information,
• and trace identifiers.

In a graph-based agent, the application can convert those decisions into state transitions:

from policyaware import PolicyAwareNodeGuard

guard = PolicyAwareNodeGuard(
    config="policyaware.yaml",
    tool_policy="tool-governance.yaml",
)

def summarize_customer(state):
    return {
        "messages": [
            {
                "role": "assistant",
                "content": "Customer record summarized safely.",
            }
        ]
    }

guarded_node = guard.guard_node(summarize_customer)

A denied state can move to a safe-response node. An approval-required state can pause and enter a human-review workflow. A redacted request can continue with transformed data.

The orchestrator remains responsible for state transitions, suspension, approval persistence, and resumption. PolicyAware supplies the decision and evidence needed to implement that behavior consistently.

---

6. Performance Without Unverifiable Claims

A control plane adds work before and after execution. The objective is not literal zero overhead; it is predictable, measurable overhead appropriate to the protected action.

PolicyAware's base enforcement path is local, deterministic, rules-based, and implemented in Python. The current public package should not be described as using a native C or Rust execution core.

Optional integrations can change runtime characteristics substantially:

• Presidio and spaCy add stronger privacy detection.
• Transformers, Torch, and ONNX add model-based classification.
• NeMo Guardrails and Guardrails AI add external guardrail behavior.
• Remote providers introduce network latency.
• Larger composed policies require additional evaluation work.

The repository includes reproducible benchmarks:

python benchmarks/benchmark_policy_engine.py \
  --requests 1000 \
  --concurrency 1

python benchmarks/benchmark_policy_engine.py \
  --requests 1000 \
  --concurrency 20 \
  --json

The benchmark reports median, p95, and p99 latency, total runtime, and requests per second.

A credible result should document the PolicyAware and Python versions, operating system, hardware, policy size, input distribution, concurrency, enabled integrations, and external calls.

The defensible performance position is:

PolicyAware provides a local deterministic enforcement path, separates heavyweight integrations into optional dependencies, and includes tools for measuring overhead in the target environment.

---

7. Embedded SDK vs. Infrastructure Boundary

PolicyAware supports multiple deployment models, but they do not provide identical security boundaries.

Mode	Primary use	Security consideration
Embedded SDK	Python applications and local checks	Code in the same process may bypass enforcement
Gateway.chat(...)	Central model control, routing, evaluation, and audit	Requests must consistently pass through the gateway
ToolPolicyEngine	Application-owned tool authorization	The application must check policy before execution
MCP stdio proxy	Enforcement before MCP forwarding	Tool permissions still require least privilege
Framework callbacks	Observation and evaluation	Callbacks report; they are not a hard boundary
HTTP sidecar	Polyglot services and process separation	Requires authentication and network controls
Repository scanner	Pre-deployment governance analysis	Static findings do not prove exploitability

For stronger separation, PolicyAware can run as an internal sidecar:

set POLICYAWARE_SIDECAR_TOKEN=replace-with-secret-token

policyaware up \
  --policy policyaware.yaml \
  --tool-policy tool-governance.yaml \
  --require-auth

A sidecar can have a separate process, service identity, deployment lifecycle, private network access, and audit stream.

It should still be combined with IAM, TLS or mTLS, secret management, container isolation, scoped tool credentials, dependency scanning, and centralized monitoring.

Policy authorization decides whether an operation may proceed. It does not replace operating-system sandboxing.

---

8. Selecting the Right Tool

Choose a structured-output or conversational guardrail when:

• schema-conformant output is the primary requirement,
• invalid responses should be retried or repaired,
• conversational flow constraints are needed,
• toxicity or vocabulary filtering is central,
• or the application has no high-impact tool execution.

Choose a conventional AI gateway when:

• provider abstraction is the primary need,
• API-key management is central,
• retries and provider fallback are required,
• routing is based mainly on price, availability, or latency,
• or standard rate limiting is sufficient.

Choose PolicyAware when:

• identities, tenants, regions, and risk must influence decisions,
• agents invoke MCP servers or external tools,
• read, write, delete, deploy, or payment actions need different controls,
• sensitive arguments must be redacted before forwarding,
• high-risk actions require approval,
• model routing must occur after policy evaluation,
• or audit evidence must connect requests, decisions, tools, models, and results.

Use them together when:

• deterministic policy should remain the auditable authority,
• semantic or conversational signals should contribute evidence,
• an AI gateway should handle provider transport,
• and PolicyAware should govern whether an action is permitted.

This layered architecture is stronger than expecting one library to solve every aspect of AI safety.

---

9. Shift Governance Left

Runtime enforcement catches decisions as they happen. Repository scanning catches governance gaps before deployment.

policyaware scan . --format html,json,sarif,markdown

The scanner can identify likely issues such as exposed PII or secrets, direct model calls without governance, unmapped MCP tools, weak tool policies, routing and audit gaps, and invalid policy YAML.

Policy contract checks detect drift between declared actions and application code:

policyaware contract check ./src \
  --policy tool-governance.yaml

For pull requests:

- uses: ktirupati/policyaware-action@v1

These findings support secure review and CI enforcement. Static analysis does not prove exploitability and does not replace runtime testing.

---

10. Current Boundaries and Limitations

PolicyAware is an AI governance framework, not a universal security platform.

It does not replace authentication and IAM, API gateways and WAFs, secret managers, application security testing, container isolation, endpoint security, secure memory, or secure development practices.

Additional boundaries include:

1. The embedded SDK is not a hard boundary against code already running in the same process.
2. An allowed tool is not automatically sandboxed.
3. Callback integrations observe and report but do not provide central execution control.
4. Built-in semantic detection is intentionally lighter than dedicated ML classifiers.
5. Approval decisions require an external durable workflow for suspension, identity verification, expiration, and resumption.
6. Compliance evidence does not constitute legal certification.
7. Performance depends on deployed policies, traffic, integrations, and infrastructure.

Being explicit about these boundaries makes the architecture more credible. PolicyAware governs AI decisions and evidence; the surrounding platform owns identity, credentials, isolation, networking, and execution security.

---

11. Production Evaluation Checklist

Before introducing an AI control plane into production:

1. Model real roles, tenants, regions, and business actions.
2. Keep MCP and tool policies deny-by-default.
3. Separate read-only identities from destructive identities.
4. Require approval for deploy, delete, payment, permission, and export actions.
5. Store approval state durably and authenticate the approver.
6. Use short-lived credentials and keep secrets out of agent state.
7. Sandbox untrusted execution.
8. Run golden datasets for allow, deny, redact, and approval decisions.
9. Benchmark the exact production policy stack.
10. Export decisions and execution outcomes to the same audit system.
11. Test policy and code contracts in CI.
12. Combine deterministic policy with optional semantic detection where required.

---

Conclusion

AI safety is no longer solely a prompt-and-response validation problem.

Once an autonomous system can invoke tools and change external state, governance must operate across identity, context, risk, models, connectors, actions, arguments, approvals, and audit evidence.

Text and conversational guardrails remain valuable. PolicyAware addresses a different architectural layer: the decision point between an AI system's intent and its real-world effects.

The strongest design is not based on claims of zero overhead or mathematical security. It is based on explicit deny-by-default policy, enforcement before side effects, explainable decisions, measurable performance, honest security boundaries, least-privilege execution, and reviewable evidence.

That is the infrastructure-control-plane model PolicyAware is building.

• PolicyAware on GitHub
• Documentation
• PyPI package
• MCP policy proxy
• Security boundaries
• Benchmarks

Comments

[tercel]

The moment you wrote “Is this identity permitted to perform this action, through this connector…” you basically said: this isn’t about text safety anymore, it’s about an authz layer for AI actions.

Your table nails the core implication of that: a “guardrail” that only sees prompts and responses literally has no visibility into connector, action, arguments, tenant, or region. So if a team claims to be “safe” with just prompt filters on top of an agent that can filesystem.delete_file, they’re really running without an action-level access model.

A couple of things I’m curious about in that framing:

• You’ve gone deny-by-default in the filesystem example. For real orgs, do you see people actually authoring policies at the connector.action granularity, or do they drift back to broad “devs can use filesystem” rules because of complexity?
• For MCP proxying, how are you thinking about versioning and change control? If a server adds a new tool or changes args, what stops it from silently bypassing existing assumptions?
• On model routing: once you tie routing to risk and region, doesn’t that effectively turn the control plane into a soft SLO manager too? Do you see that as in scope or an anti-pattern?

Big picture, this reads like “treat agents as distributed systems with side effects, not chatbots with filters.” Curious where you think the minimum viable control plane is for a small team before this becomes worth the integration cost.

[KRISHNA KISHOR TIRUPATI]

Hi @tercel,You absolutely nailed the core message: "treat agents as distributed systems with side effects, not chatbots with filters." I might have to steal that line for my next post!

To quickly address your points:

*Policy Complexity: * Teams usually start broad but rely on deny-by-default to force granular policies the moment a high-impact action (like delete) is introduced.

MCP Schema Changes: Because the framework is strictly deny-by-default, a silent addition of a new tool or unmapped argument will automatically trigger a policy block or fail the CI contract check rather than bypassing security.

The MVCP for Small Teams: The integration cost becomes worth it the exact second your agent is given a tool that can mutate production data or spend money. For a small team, starting with a simple local MCP proxy for human-in-the-loop approvals on write actions is the perfect starting line.

Appreciate the thoughtful pushback and questions. Are you currently building or securing autonomous agents yourself?

[tercel]

Guilty as charged! That exact pain point is what drove me to build apcore—an open-source framework focused on AI-native module standards and agentic protocols (including apcore-mcp for exposing and governing tools safely).

Enforcing deny-by-default at the MCP schema/action boundary has been a game-changer for keeping things secure without slowing down dev velocity. How are you currently handling tool-level authz in your setups?

[KRISHNA KISHOR TIRUPATI]

Love the focus on deny-by-default at the MCP schema boundary! That's exactly the gap we built PolicyAware to fill.In our stack, we handle tool-level authorization by positioning PolicyAware as an intermediary gateway/middleware package. It intercepts the model's intent to call a tool or an MCP server, parses the context (user identity, metadata, and data sensitivity), and validates it against central YAML rules before the action fires. If the authorization check fails or if there's an identity mismatch, the execution is immediately dropped.Balancing that security gate with dev velocity is the ultimate goal—how are you mapping user-level roles down to your apcore-mcp actions?

[tercel]

That’s a clean way to handle things with an external proxy/gateway.

In apcore and apcore-mcp, we tackle this slightly differently by pushing governance and validation directly into the schema-enforced module standard itself.

Rather than intercepting calls solely at a central gateway layer, apcore modules encapsulate their behavior, input/output schemas, and execution constraints at the core level. When apcore-mcp or any other adapter (like CLI, REST, or A2A) projects those modules, user identity, context injection, and validation flow through a unified execution pipeline and context bus.

This means access control, ACL enforcement, and policy evaluation happen deterministically right at the module boundary—regardless of whether the request comes from an LLM via MCP, a command line, or an agent-to-agent protocol.

It decouples the protocol surface from the business logic while keeping the security contract strict and portable. Curious how PolicyAware handles cases where you want the exact same module rules to project cleanly across non-MCP surfaces (like CLI or direct function calls) without maintaining separate rule sets?