Architectural Evaluation: Why We Re-Engineered AI Safety From Text Guardrails to Infrastructure Control Planes

Introduction

The open-source AI safety ecosystem includes mature tools for structured-output validation, conversational guardrails, sensitive-data detection, and prompt or response filtering. Frameworks such as Guardrails AI, LLM Guard, and NVIDIA NeMo Guardrails address important parts of this problem.

But autonomous AI systems introduce a broader security question.

An agent does not merely generate text. It can select models, retrieve untrusted context, invoke Model Context Protocol (MCP) servers, query databases, modify files, create pull requests, deploy applications, or trigger external business processes.

For these systems, checking whether text is safe is not enough. The platform must also decide:

Is this identity permitted to perform this action, through this connector, against this resource, using these arguments, within this tenant and region—and does the action require approval?

That is the role of an AI infrastructure control plane.

This article explains why we built PolicyAware around policy enforcement, MCP tool governance, risk classification, model routing, evaluation, and audit evidence rather than positioning it as another text-validation wrapper.

---

1. Guardrails and Control Planes Solve Different Problems

Describing every guardrail framework as a basic string filter would be inaccurate. Some support model-based validation, programmable constraints, conversational flows, and structured-output enforcement.

The architectural distinction is primarily about where enforcement occurs and what it governs.

Dimension	Text and conversational guardrails	PolicyAware control plane
Primary responsibility	Validate prompts, responses, conversation flows, or output structures	Govern requests, identities, models, tools, side effects, evaluations, and evidence
Typical inputs	Prompt and model output	Prompt, role, tenant, region, risk, connector, action, arguments, and policy
Common decisions	Pass, fail, retry, repair, or filter	Deny, require approval, allow, conditional allow, or transform
Tool governance	Usually application-specific	Connector/action policy through ToolPolicyEngine and MCP proxying
Protocol awareness	Product- and integration-dependent	Inspects MCP JSON-RPC tools/call requests before forwarding
Model routing	Usually outside validation	Policy-aware routing based on risk, region, provider, cost, and availability
Audit evidence	Validation results and logs	Decisions, reason codes, matched policies, evaluations, and trace IDs
Deployment	Library or framework integration	Embedded SDK, middleware, CLI, MCP proxy, or HTTP sidecar
Runtime implementation	Varies by framework	Local deterministic Python rules in the base package
Performance	Depends on validators and model calls	Must be benchmarked for the deployed policy and integrations

The approaches are not necessarily competitors. A control plane can use specialized guardrail frameworks as optional components while retaining deterministic policy as the final authority.

PolicyAware supports this layered model through optional integrations with Presidio, Transformers-based classifiers, NeMo Guardrails, and Guardrails AI.

---

2. Why Autonomous Agents Need Action-Level Governance

Consider an agent connected to a filesystem MCP server:

filesystem.read_file
filesystem.write_file
filesystem.delete_file

From a language-model perspective, these are tool calls. From a security perspective, they represent substantially different risks.

A production system may require:

• developers can read approved files,
• writes require human approval,
• deletion is denied,
• paths outside an approved directory are blocked,
• sensitive arguments are redacted before forwarding,
• and every decision is associated with an identity and trace ID.

A prompt filter cannot enforce these rules reliably by examining conversation text alone. Enforcement must occur at the tool boundary before the MCP server receives the request.

PolicyAware represents these requirements as deny-by-default policy:

id: mcp_filesystem_policy
schema_version: "0.2"
default: deny

connectors:
  - id: filesystem
    type: mcp
    actions:
      read_file:
        effect: allow
        risk: low
        side_effect: none
        when:
          user.role_in: [developer, security_engineer]

      write_file:
        effect: require_approval
        risk: high
        side_effect: write
        when:
          user.role_in: [developer]

      delete_file:
        effect: deny
        risk: critical
        side_effect: delete

This policy does not ask the model whether deleting a file seems appropriate. It evaluates a deterministic rule using the connector, action, identity, and request context.

---

3. PolicyAware's Runtime Architecture

PolicyAware separates governance into explicit, replaceable components.

A typical request follows this lifecycle:

1. The application supplies the request and structured context.
2. Data-protection checks detect PII, PHI, secrets, and sensitive values.
3. Risk classification assigns a low, medium, high, or critical tier.
4. Policy determines whether to deny, require approval, allow, or transform.
5. Denied and approval-gated requests stop before execution.
6. Allowed requests proceed to a model router or tool-policy engine.
7. Runtime evaluation checks results for leakage, citations, and policy consistency.
8. Audit components record the decision and supporting evidence.

One important semantic property is that transformation does not grant access. A redaction rule may modify an otherwise allowed request, but it cannot convert a denied action into an allowed one.

---

4. First-Class MCP JSON-RPC Governance

PolicyAware includes an MCP policy proxy that evaluates raw JSON-RPC traffic before a tool call reaches the underlying MCP server.

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "filesystem.read_file",
    "arguments": {
      "path": "README.md",
      "query": "Find records for [e-mail]"
    }
  }
}

The proxy can:

• evaluate identity, role, tenant, and agent context,
• inspect tool arguments for sensitive information,
• deny unauthorized actions,
• require approval for high-impact operations,
• redact permitted arguments before forwarding,
• return structured JSON-RPC errors,
• and pass non-tool protocol messages through unchanged.

Test a request from the CLI:

policyaware mcp check policyaware.yaml mcp-request.json

Place a live stdio proxy in front of an MCP server:

policyaware mcp proxy policyaware.yaml \
  --connector filesystem \
  --agent coding_agent \
  --role developer \
  --server-command "python filesystem_mcp_server.py"

When an action is denied, it is not forwarded to the real server. The client receives a structured JSON-RPC error containing the decision, connector, action, and policy context.

---

5. Structured Recovery Instead of Opaque Failure

Policy violations should not necessarily crash an entire agent session. However, claiming that a policy engine automatically reroutes every trajectory would overstate what an enforcement layer can guarantee.

PolicyAware returns structured outcomes that an orchestrator can handle deliberately:

• deny,
• require_approval,
• allow,
• transformed request data,
• reason codes,
• matched policies,
• remediation information,
• and trace identifiers.

In a graph-based agent, the application can convert those decisions into state transitions:

from policyaware import PolicyAwareNodeGuard

guard = PolicyAwareNodeGuard(
    config="policyaware.yaml",
    tool_policy="tool-governance.yaml",
)

def summarize_customer(state):
    return {
        "messages": [
            {
                "role": "assistant",
                "content": "Customer record summarized safely.",
            }
        ]
    }

guarded_node = guard.guard_node(summarize_customer)

A denied state can move to a safe-response node. An approval-required state can pause and enter a human-review workflow. A redacted request can continue with transformed data.

The orchestrator remains responsible for state transitions, suspension, approval persistence, and resumption. PolicyAware supplies the decision and evidence needed to implement that behavior consistently.

---

6. Performance Without Unverifiable Claims

A control plane adds work before and after execution. The objective is not literal zero overhead; it is predictable, measurable overhead appropriate to the protected action.

PolicyAware's base enforcement path is local, deterministic, rules-based, and implemented in Python. The current public package should not be described as using a native C or Rust execution core.

Optional integrations can change runtime characteristics substantially:

• Presidio and spaCy add stronger privacy detection.
• Transformers, Torch, and ONNX add model-based classification.
• NeMo Guardrails and Guardrails AI add external guardrail behavior.
• Remote providers introduce network latency.
• Larger composed policies require additional evaluation work.

The repository includes reproducible benchmarks:

python benchmarks/benchmark_policy_engine.py \
  --requests 1000 \
  --concurrency 1

python benchmarks/benchmark_policy_engine.py \
  --requests 1000 \
  --concurrency 20 \
  --json

The benchmark reports median, p95, and p99 latency, total runtime, and requests per second.

A credible result should document the PolicyAware and Python versions, operating system, hardware, policy size, input distribution, concurrency, enabled integrations, and external calls.

The defensible performance position is:

PolicyAware provides a local deterministic enforcement path, separates heavyweight integrations into optional dependencies, and includes tools for measuring overhead in the target environment.

---

7. Embedded SDK vs. Infrastructure Boundary

PolicyAware supports multiple deployment models, but they do not provide identical security boundaries.

Mode	Primary use	Security consideration
Embedded SDK	Python applications and local checks	Code in the same process may bypass enforcement
Gateway.chat(...)	Central model control, routing, evaluation, and audit	Requests must consistently pass through the gateway
ToolPolicyEngine	Application-owned tool authorization	The application must check policy before execution
MCP stdio proxy	Enforcement before MCP forwarding	Tool permissions still require least privilege
Framework callbacks	Observation and evaluation	Callbacks report; they are not a hard boundary
HTTP sidecar	Polyglot services and process separation	Requires authentication and network controls
Repository scanner	Pre-deployment governance analysis	Static findings do not prove exploitability

For stronger separation, PolicyAware can run as an internal sidecar:

set POLICYAWARE_SIDECAR_TOKEN=replace-with-secret-token

policyaware up \
  --policy policyaware.yaml \
  --tool-policy tool-governance.yaml \
  --require-auth

A sidecar can have a separate process, service identity, deployment lifecycle, private network access, and audit stream.

It should still be combined with IAM, TLS or mTLS, secret management, container isolation, scoped tool credentials, dependency scanning, and centralized monitoring.

Policy authorization decides whether an operation may proceed. It does not replace operating-system sandboxing.

---

8. Selecting the Right Tool

Choose a structured-output or conversational guardrail when:

• schema-conformant output is the primary requirement,
• invalid responses should be retried or repaired,
• conversational flow constraints are needed,
• toxicity or vocabulary filtering is central,
• or the application has no high-impact tool execution.

Choose a conventional AI gateway when:

• provider abstraction is the primary need,
• API-key management is central,
• retries and provider fallback are required,
• routing is based mainly on price, availability, or latency,
• or standard rate limiting is sufficient.

Choose PolicyAware when:

• identities, tenants, regions, and risk must influence decisions,
• agents invoke MCP servers or external tools,
• read, write, delete, deploy, or payment actions need different controls,
• sensitive arguments must be redacted before forwarding,
• high-risk actions require approval,
• model routing must occur after policy evaluation,
• or audit evidence must connect requests, decisions, tools, models, and results.

Use them together when:

• deterministic policy should remain the auditable authority,
• semantic or conversational signals should contribute evidence,
• an AI gateway should handle provider transport,
• and PolicyAware should govern whether an action is permitted.

This layered architecture is stronger than expecting one library to solve every aspect of AI safety.

---

9. Shift Governance Left

Runtime enforcement catches decisions as they happen. Repository scanning catches governance gaps before deployment.

policyaware scan . --format html,json,sarif,markdown

The scanner can identify likely issues such as exposed PII or secrets, direct model calls without governance, unmapped MCP tools, weak tool policies, routing and audit gaps, and invalid policy YAML.

Policy contract checks detect drift between declared actions and application code:

policyaware contract check ./src \
  --policy tool-governance.yaml

For pull requests:

- uses: ktirupati/policyaware-action@v1

These findings support secure review and CI enforcement. Static analysis does not prove exploitability and does not replace runtime testing.

---

10. Current Boundaries and Limitations

PolicyAware is an AI governance framework, not a universal security platform.

It does not replace authentication and IAM, API gateways and WAFs, secret managers, application security testing, container isolation, endpoint security, secure memory, or secure development practices.

Additional boundaries include:

1. The embedded SDK is not a hard boundary against code already running in the same process.
2. An allowed tool is not automatically sandboxed.
3. Callback integrations observe and report but do not provide central execution control.
4. Built-in semantic detection is intentionally lighter than dedicated ML classifiers.
5. Approval decisions require an external durable workflow for suspension, identity verification, expiration, and resumption.
6. Compliance evidence does not constitute legal certification.
7. Performance depends on deployed policies, traffic, integrations, and infrastructure.

Being explicit about these boundaries makes the architecture more credible. PolicyAware governs AI decisions and evidence; the surrounding platform owns identity, credentials, isolation, networking, and execution security.

---

11. Production Evaluation Checklist

Before introducing an AI control plane into production:

1. Model real roles, tenants, regions, and business actions.
2. Keep MCP and tool policies deny-by-default.
3. Separate read-only identities from destructive identities.
4. Require approval for deploy, delete, payment, permission, and export actions.
5. Store approval state durably and authenticate the approver.
6. Use short-lived credentials and keep secrets out of agent state.
7. Sandbox untrusted execution.
8. Run golden datasets for allow, deny, redact, and approval decisions.
9. Benchmark the exact production policy stack.
10. Export decisions and execution outcomes to the same audit system.
11. Test policy and code contracts in CI.
12. Combine deterministic policy with optional semantic detection where required.

---

Conclusion

AI safety is no longer solely a prompt-and-response validation problem.

Once an autonomous system can invoke tools and change external state, governance must operate across identity, context, risk, models, connectors, actions, arguments, approvals, and audit evidence.

Text and conversational guardrails remain valuable. PolicyAware addresses a different architectural layer: the decision point between an AI system's intent and its real-world effects.

The strongest design is not based on claims of zero overhead or mathematical security. It is based on explicit deny-by-default policy, enforcement before side effects, explainable decisions, measurable performance, honest security boundaries, least-privilege execution, and reviewable evidence.

That is the infrastructure-control-plane model PolicyAware is building.

• PolicyAware on GitHub
• Documentation
• PyPI package
• MCP policy proxy
• Security boundaries
• Benchmarks