DEV Community: KRISHNA KISHOR TIRUPATI

Architectural Evaluation: Why We Re-Engineered AI Safety From Text Guardrails to Infrastructure Control Planes

KRISHNA KISHOR TIRUPATI — Mon, 14 Sep 2026 19:44:20 +0000

Introduction

The open-source AI safety ecosystem includes mature tools for structured-output validation, conversational guardrails, sensitive-data detection, and prompt or response filtering. Frameworks such as Guardrails AI, LLM Guard, and NVIDIA NeMo Guardrails address important parts of this problem.

But autonomous AI systems introduce a broader security question.

An agent does not merely generate text. It can select models, retrieve untrusted context, invoke Model Context Protocol (MCP) servers, query databases, modify files, create pull requests, deploy applications, or trigger external business processes.

For these systems, checking whether text is safe is not enough. The platform must also decide:

Is this identity permitted to perform this action, through this connector, against this resource, using these arguments, within this tenant and region—and does the action require approval?

That is the role of an AI infrastructure control plane.

This article explains why we built PolicyAware around policy enforcement, MCP tool governance, risk classification, model routing, evaluation, and audit evidence rather than positioning it as another text-validation wrapper.

1. Guardrails and Control Planes Solve Different Problems

Describing every guardrail framework as a basic string filter would be inaccurate. Some support model-based validation, programmable constraints, conversational flows, and structured-output enforcement.

The architectural distinction is primarily about where enforcement occurs and what it governs.

Dimension	Text and conversational guardrails	PolicyAware control plane
Primary responsibility	Validate prompts, responses, conversation flows, or output structures	Govern requests, identities, models, tools, side effects, evaluations, and evidence
Typical inputs	Prompt and model output	Prompt, role, tenant, region, risk, connector, action, arguments, and policy
Common decisions	Pass, fail, retry, repair, or filter	Deny, require approval, allow, conditional allow, or transform
Tool governance	Usually application-specific	Connector/action policy through `ToolPolicyEngine` and MCP proxying
Protocol awareness	Product- and integration-dependent	Inspects MCP JSON-RPC `tools/call` requests before forwarding
Model routing	Usually outside validation	Policy-aware routing based on risk, region, provider, cost, and availability
Audit evidence	Validation results and logs	Decisions, reason codes, matched policies, evaluations, and trace IDs
Deployment	Library or framework integration	Embedded SDK, middleware, CLI, MCP proxy, or HTTP sidecar
Runtime implementation	Varies by framework	Local deterministic Python rules in the base package
Performance	Depends on validators and model calls	Must be benchmarked for the deployed policy and integrations

The approaches are not necessarily competitors. A control plane can use specialized guardrail frameworks as optional components while retaining deterministic policy as the final authority.

PolicyAware supports this layered model through optional integrations with Presidio, Transformers-based classifiers, NeMo Guardrails, and Guardrails AI.

2. Why Autonomous Agents Need Action-Level Governance

Consider an agent connected to a filesystem MCP server:

filesystem.read_file
filesystem.write_file
filesystem.delete_file

From a language-model perspective, these are tool calls. From a security perspective, they represent substantially different risks.

A production system may require:

developers can read approved files,
writes require human approval,
deletion is denied,
paths outside an approved directory are blocked,
sensitive arguments are redacted before forwarding,
and every decision is associated with an identity and trace ID.

A prompt filter cannot enforce these rules reliably by examining conversation text alone. Enforcement must occur at the tool boundary before the MCP server receives the request.

PolicyAware represents these requirements as deny-by-default policy:

id: mcp_filesystem_policy
schema_version: "0.2"
default: deny

connectors:
  - id: filesystem
    type: mcp
    actions:
      read_file:
        effect: allow
        risk: low
        side_effect: none
        when:
          user.role_in: [developer, security_engineer]

      write_file:
        effect: require_approval
        risk: high
        side_effect: write
        when:
          user.role_in: [developer]

      delete_file:
        effect: deny
        risk: critical
        side_effect: delete

This policy does not ask the model whether deleting a file seems appropriate. It evaluates a deterministic rule using the connector, action, identity, and request context.

3. PolicyAware's Runtime Architecture

PolicyAware separates governance into explicit, replaceable components.

A typical request follows this lifecycle:

The application supplies the request and structured context.
Data-protection checks detect PII, PHI, secrets, and sensitive values.
Risk classification assigns a low, medium, high, or critical tier.
Policy determines whether to deny, require approval, allow, or transform.
Denied and approval-gated requests stop before execution.
Allowed requests proceed to a model router or tool-policy engine.
Runtime evaluation checks results for leakage, citations, and policy consistency.
Audit components record the decision and supporting evidence.

One important semantic property is that transformation does not grant access. A redaction rule may modify an otherwise allowed request, but it cannot convert a denied action into an allowed one.

4. First-Class MCP JSON-RPC Governance

PolicyAware includes an MCP policy proxy that evaluates raw JSON-RPC traffic before a tool call reaches the underlying MCP server.

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "filesystem.read_file",
    "arguments": {
      "path": "README.md",
      "query": "Find records for jane@example.com"
    }
  }
}

The proxy can:

evaluate identity, role, tenant, and agent context,
inspect tool arguments for sensitive information,
deny unauthorized actions,
require approval for high-impact operations,
redact permitted arguments before forwarding,
return structured JSON-RPC errors,
and pass non-tool protocol messages through unchanged.

Test a request from the CLI:

policyaware mcp check policyaware.yaml mcp-request.json

Place a live stdio proxy in front of an MCP server:

policyaware mcp proxy policyaware.yaml \
  --connector filesystem \
  --agent coding_agent \
  --role developer \
  --server-command "python filesystem_mcp_server.py"

When an action is denied, it is not forwarded to the real server. The client receives a structured JSON-RPC error containing the decision, connector, action, and policy context.

5. Structured Recovery Instead of Opaque Failure

Policy violations should not necessarily crash an entire agent session. However, claiming that a policy engine automatically reroutes every trajectory would overstate what an enforcement layer can guarantee.

PolicyAware returns structured outcomes that an orchestrator can handle deliberately:

deny,
require_approval,
allow,
transformed request data,
reason codes,
matched policies,
remediation information,
and trace identifiers.

In a graph-based agent, the application can convert those decisions into state transitions:

from policyaware import PolicyAwareNodeGuard

guard = PolicyAwareNodeGuard(
    config="policyaware.yaml",
    tool_policy="tool-governance.yaml",
)

def summarize_customer(state):
    return {
        "messages": [
            {
                "role": "assistant",
                "content": "Customer record summarized safely.",
            }
        ]
    }

guarded_node = guard.guard_node(summarize_customer)

A denied state can move to a safe-response node. An approval-required state can pause and enter a human-review workflow. A redacted request can continue with transformed data.

The orchestrator remains responsible for state transitions, suspension, approval persistence, and resumption. PolicyAware supplies the decision and evidence needed to implement that behavior consistently.

6. Performance Without Unverifiable Claims

A control plane adds work before and after execution. The objective is not literal zero overhead; it is predictable, measurable overhead appropriate to the protected action.

PolicyAware's base enforcement path is local, deterministic, rules-based, and implemented in Python. The current public package should not be described as using a native C or Rust execution core.

Optional integrations can change runtime characteristics substantially:

Presidio and spaCy add stronger privacy detection.
Transformers, Torch, and ONNX add model-based classification.
NeMo Guardrails and Guardrails AI add external guardrail behavior.
Remote providers introduce network latency.
Larger composed policies require additional evaluation work.

The repository includes reproducible benchmarks:

python benchmarks/benchmark_policy_engine.py \
  --requests 1000 \
  --concurrency 1

python benchmarks/benchmark_policy_engine.py \
  --requests 1000 \
  --concurrency 20 \
  --json

The benchmark reports median, p95, and p99 latency, total runtime, and requests per second.

A credible result should document the PolicyAware and Python versions, operating system, hardware, policy size, input distribution, concurrency, enabled integrations, and external calls.

The defensible performance position is:

PolicyAware provides a local deterministic enforcement path, separates heavyweight integrations into optional dependencies, and includes tools for measuring overhead in the target environment.

7. Embedded SDK vs. Infrastructure Boundary

PolicyAware supports multiple deployment models, but they do not provide identical security boundaries.

Mode	Primary use	Security consideration
Embedded SDK	Python applications and local checks	Code in the same process may bypass enforcement
`Gateway.chat(...)`	Central model control, routing, evaluation, and audit	Requests must consistently pass through the gateway
`ToolPolicyEngine`	Application-owned tool authorization	The application must check policy before execution
MCP stdio proxy	Enforcement before MCP forwarding	Tool permissions still require least privilege
Framework callbacks	Observation and evaluation	Callbacks report; they are not a hard boundary
HTTP sidecar	Polyglot services and process separation	Requires authentication and network controls
Repository scanner	Pre-deployment governance analysis	Static findings do not prove exploitability

For stronger separation, PolicyAware can run as an internal sidecar:

set POLICYAWARE_SIDECAR_TOKEN=replace-with-secret-token

policyaware up \
  --policy policyaware.yaml \
  --tool-policy tool-governance.yaml \
  --require-auth

A sidecar can have a separate process, service identity, deployment lifecycle, private network access, and audit stream.

It should still be combined with IAM, TLS or mTLS, secret management, container isolation, scoped tool credentials, dependency scanning, and centralized monitoring.

Policy authorization decides whether an operation may proceed. It does not replace operating-system sandboxing.

8. Selecting the Right Tool

Choose a structured-output or conversational guardrail when:

schema-conformant output is the primary requirement,
invalid responses should be retried or repaired,
conversational flow constraints are needed,
toxicity or vocabulary filtering is central,
or the application has no high-impact tool execution.

Choose a conventional AI gateway when:

provider abstraction is the primary need,
API-key management is central,
retries and provider fallback are required,
routing is based mainly on price, availability, or latency,
or standard rate limiting is sufficient.

Choose PolicyAware when:

identities, tenants, regions, and risk must influence decisions,
agents invoke MCP servers or external tools,
read, write, delete, deploy, or payment actions need different controls,
sensitive arguments must be redacted before forwarding,
high-risk actions require approval,
model routing must occur after policy evaluation,
or audit evidence must connect requests, decisions, tools, models, and results.

Use them together when:

deterministic policy should remain the auditable authority,
semantic or conversational signals should contribute evidence,
an AI gateway should handle provider transport,
and PolicyAware should govern whether an action is permitted.

This layered architecture is stronger than expecting one library to solve every aspect of AI safety.

9. Shift Governance Left

Runtime enforcement catches decisions as they happen. Repository scanning catches governance gaps before deployment.

policyaware scan . --format html,json,sarif,markdown

The scanner can identify likely issues such as exposed PII or secrets, direct model calls without governance, unmapped MCP tools, weak tool policies, routing and audit gaps, and invalid policy YAML.

Policy contract checks detect drift between declared actions and application code:

policyaware contract check ./src \
  --policy tool-governance.yaml

For pull requests:

- uses: ktirupati/policyaware-action@v1

These findings support secure review and CI enforcement. Static analysis does not prove exploitability and does not replace runtime testing.

10. Current Boundaries and Limitations

PolicyAware is an AI governance framework, not a universal security platform.

It does not replace authentication and IAM, API gateways and WAFs, secret managers, application security testing, container isolation, endpoint security, secure memory, or secure development practices.

Additional boundaries include:

The embedded SDK is not a hard boundary against code already running in the same process.
An allowed tool is not automatically sandboxed.
Callback integrations observe and report but do not provide central execution control.
Built-in semantic detection is intentionally lighter than dedicated ML classifiers.
Approval decisions require an external durable workflow for suspension, identity verification, expiration, and resumption.
Compliance evidence does not constitute legal certification.
Performance depends on deployed policies, traffic, integrations, and infrastructure.

Being explicit about these boundaries makes the architecture more credible. PolicyAware governs AI decisions and evidence; the surrounding platform owns identity, credentials, isolation, networking, and execution security.

11. Production Evaluation Checklist

Before introducing an AI control plane into production:

Model real roles, tenants, regions, and business actions.
Keep MCP and tool policies deny-by-default.
Separate read-only identities from destructive identities.
Require approval for deploy, delete, payment, permission, and export actions.
Store approval state durably and authenticate the approver.
Use short-lived credentials and keep secrets out of agent state.
Sandbox untrusted execution.
Run golden datasets for allow, deny, redact, and approval decisions.
Benchmark the exact production policy stack.
Export decisions and execution outcomes to the same audit system.
Test policy and code contracts in CI.
Combine deterministic policy with optional semantic detection where required.

Conclusion

AI safety is no longer solely a prompt-and-response validation problem.

Once an autonomous system can invoke tools and change external state, governance must operate across identity, context, risk, models, connectors, actions, arguments, approvals, and audit evidence.

Text and conversational guardrails remain valuable. PolicyAware addresses a different architectural layer: the decision point between an AI system's intent and its real-world effects.

The strongest design is not based on claims of zero overhead or mathematical security. It is based on explicit deny-by-default policy, enforcement before side effects, explainable decisions, measurable performance, honest security boundaries, least-privilege execution, and reviewable evidence.

That is the infrastructure-control-plane model PolicyAware is building.

Overcoming the Performance Tax in AI Safety: Engineering a Low-Latency Control Plane for Autonomous Agents

KRISHNA KISHOR TIRUPATI — Mon, 14 Sep 2026 19:08:58 +0000

Abstract

As enterprise AI systems move from single-turn retrieval-augmented generation (RAG) toward agents that plan, call tools, and modify external state, safety can no longer be treated only as text filtering.

This article presents a practical control-plane approach using PolicyAware, an open-source Python framework for deny-by-default policy, PII and secret handling, MCP tool governance, model routing, runtime evaluation, repository scanning, and audit evidence.

The goal is not to promise "zero overhead." The goal is to keep the common enforcement path local, deterministic, measurable, and separate from optional heavyweight ML integrations. Teams should benchmark median, p95, and p99 latency in their own request path before setting production objectives.

1. Why Text Guardrails Are Not Enough

First-generation AI safety tools often treat security as an input/output problem: inspect the prompt, validate the response, and block suspicious text.

That remains useful, but an autonomous workflow has a wider attack surface. An agent can:

choose a model or provider,
retrieve untrusted context,
call an MCP server,
read or write files,
execute database operations,
create pull requests,
deploy software,
or trigger an external business process.

The important question is therefore not only "Is this text safe?" It is also:

Is this actor allowed to perform this action, against this resource, in this tenant and region, with these arguments, at this risk level, and does it require human approval?

A control plane answers that question before the side effect occurs.

2. Text Validation vs. an AI Control Plane

Dimension	Text-focused guardrail	PolicyAware control-plane approach
Primary role	Prompt/output inspection	Request, model, tool, evaluation, and audit governance
Decision context	Mostly text	Role, tenant, region, risk, connector, action, arguments, and policy
MCP awareness	Usually application-specific	JSON-RPC `tools/call` inspection and connector/action mapping
Decisions	Commonly allow or block	Deny, require approval, allow, conditional allow, and transform
Data handling	Detection or rejection	Detect and optionally redact PII, PHI, secrets, and sensitive fields
Deployment	Inline validator	Embedded SDK, middleware, CLI, MCP proxy, or sidecar
Runtime foundation	Product-dependent	Local, deterministic Python rules in the base package
Heavy detection	Often bundled	Optional Presidio, Transformers, NeMo Guardrails, and Guardrails AI extras
Performance claim	Frequently generalized	Benchmark locally; do not assume a universal latency number

At the time of writing, the repository declares PolicyAware version 0.4.4 and Python 3.10+. The base package is intentionally lightweight, with Pydantic, PyYAML, Typer, and Rich as core dependencies.

3. The Runtime Architecture

PolicyAware separates governance into explicit engines:

AI App / RAG Pipeline / Agent
              |
              v
PolicyAware SDK / CLI / Middleware / Callback
              |
              v
Data Protection -> Risk Classification -> Policy Decision
                                           |       |
                                      deny/approval |
                                                   v
                                      Model Routing or Tool Governance
                                                   |
                                                   v
                                      Runtime Evaluation
                                                   |
                                                   v
                                      Audit Trace / Evidence

The request lifecycle is:

Construct a request with identity, tenant, application, region, task, and risk context.
Inspect prompts and arguments for PII, PHI, secrets, and sensitive categories.
Assign a deterministic risk tier: low, medium, high, or critical.
Evaluate deny, approval, allow, and transform rules.
Stop denied or approval-gated work before model or tool execution.
Route allowed model requests or authorize a connector/action pair.
Evaluate outputs for leakage, citations, and policy consistency.
Emit traceable reason codes, matched policy IDs, decisions, and audit evidence.

A key semantic rule is that transform rules do not grant access. A redaction transform modifies an otherwise permitted request; it does not turn a denied request into an allowed one.

4. Quick Start: Govern a Raw Model Call

Install the base package:

pip install policyaware

Create a gateway and inspect a prompt before sending it to a provider:

import policyaware
from openai import OpenAI

gateway = policyaware.Gateway.from_policy_file("policyaware.yaml")
client = OpenAI()

safe_prompt, metadata = gateway.inspect_and_mutate(
    prompt="Email jane@example.com about claim ACME-42.",
    context={
        "user_role": "billing_admin",
        "session_id": "99x-delta",
        "risk": "low",
    },
    app="claims-assistant",
)

print(metadata["decision"])
print(metadata["actions"])

response = client.responses.create(
    model="gpt-4.1-mini",
    input=safe_prompt,
)

print(response.output_text)

inspect_and_mutate(...) fails closed with PermissionError for denied or approval-required requests. If policy permits redaction, it returns the transformed prompt and audit metadata through the same runtime path.

5. Governing MCP Tool Calls Before Execution

MCP servers can expose powerful operations. A filesystem server, GitHub connector, database connector, or deployment tool should not receive a request until policy has evaluated the intended action.

A deny-by-default MCP policy can distinguish read, write, and delete operations:

id: mcp_proxy_policy
schema_version: "0.2"
default: deny

connectors:
  - id: filesystem
    type: mcp
    actions:
      read_file:
        effect: allow
        risk: low
        side_effect: none
        when:
          user.role_in: [developer, security_engineer]

      write_file:
        effect: require_approval
        risk: high
        side_effect: write
        when:
          user.role_in: [developer]

      delete_file:
        effect: deny
        risk: critical
        side_effect: delete

PolicyAware can inspect a raw JSON-RPC request:

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "filesystem.read_file",
    "arguments": {
      "path": "README.md",
      "query": "email jane@example.com"
    }
  }
}

Use the CLI for a deterministic policy check:

policyaware mcp check policyaware.yaml mcp-request.json

Or place a live stdio proxy in front of an MCP server:

policyaware mcp proxy   policyaware.yaml   --connector filesystem   --agent coding_agent   --role developer   --server-command "python filesystem_mcp_server.py"

For a denied tool call, the proxy returns a structured JSON-RPC error to the client instead of forwarding the request to the real server. Non-tool protocol messages such as initialize pass through normally.

Repository guide: MCP Policy Proxy

6. LangGraph-Style Node and Tool Governance

PolicyAware includes a dependency-free PolicyAwareNodeGuard for graph-style agent workflows:

from policyaware import PolicyAwareNodeGuard

guard = PolicyAwareNodeGuard(
    config="policyaware.yaml",
    tool_policy="tool-governance.yaml",
)

def summarize_customer(state):
    return {
        "messages": [
            {
                "role": "assistant",
                "content": "Customer record summarized safely.",
            }
        ]
    }

guarded_node = guard.guard_node(summarize_customer)

The guard can inspect graph state before a node runs, block denied states, return approval-required decisions for high-risk work, evaluate MCP-style tool calls, and attach governance metadata back to state.

This is more useful than throwing an opaque exception because the orchestrator receives a structured decision that can be logged, displayed, or routed to a durable approval workflow.

Runnable example: LangGraph Agent Governance

7. Performance Engineering Without Benchmark Theater

Policy enforcement adds work. The engineering objective is to make that work predictable and proportionate to risk.

PolicyAware's base path is local and rules-based. Policy loading can be cached so YAML is not parsed on every request. Optional ML and external guardrail engines are intentionally separated because Presidio, Transformers, Torch, ONNX, NeMo Guardrails, and Guardrails AI can add model-loading time, container size, and request latency.

The repository includes reproducible local benchmarks:

python benchmarks/benchmark_policy_engine.py   --requests 1000   --concurrency 1

python benchmarks/benchmark_policy_engine.py   --requests 1000   --concurrency 20   --json

The benchmark reports:

median latency in microseconds,
p95 latency,
p99 latency,
total runtime,
and requests per second.

Repository scanning can also be measured:

python benchmarks/benchmark_scan.py . --iterations 3

A responsible latency statement should include the PolicyAware version, Python version, hardware, operating system, policy size, input distribution, concurrency, enabled extras, and whether external services were called.

Do not copy a single development-machine number into a universal production SLA.

Benchmark guide: Lightweight Benchmarks

8. Choose the Right Enforcement Boundary

Mode	Best use	Important boundary
Embedded SDK	Fast adoption in Python apps and prototypes	Code executing in the same process may bypass local checks
`Gateway.chat(...)`	Central model request control, routing, evaluation, and audit	The application must send controlled requests through the gateway
Tool policy / MCP proxy	Connector and action enforcement before side effects	Approved tools still need OS-level isolation and least privilege
LangChain/LlamaIndex callbacks	Observation, streamed-token accounting, and reporting	Callbacks observe; they are not a hard execution boundary
HTTP sidecar	Polyglot services and stronger process separation	Requires authentication, private networking, and operational ownership
`policyaware scan`	Local and CI detection of governance gaps	Static analysis does not prove runtime exploitability

For stronger enterprise separation, run PolicyAware as an internal sidecar or gateway:

set POLICYAWARE_SIDECAR_TOKEN=replace-with-secret-token
policyaware up   --policy policyaware.yaml   --tool-policy tool-governance.yaml   --require-auth

A sidecar can have separate process memory, service identity, deployment lifecycle, audit stream, and network policy. It should still be combined with TLS or mTLS, IAM, secret management, and least-privilege tool credentials.

9. Shift Governance Left with Repository Scanning

Runtime checks are only one layer. PolicyAware also includes an offline governance linter:

policyaware scan . --format html,json,sarif,markdown

The scanner can identify likely risks such as:

PII, PHI, and secret exposure,
direct model calls without governance,
unmapped MCP tools,
weak or missing tool policies,
routing and audit gaps,
and invalid policy YAML.

For pull requests, the repository links an official GitHub Action:

- uses: ktirupati/policyaware-action@v1

Static findings are signals, not proof of exploitability. They should feed secure code review, policy tests, and CI decisions rather than replace them.

10. Production Checklist

Before adopting an AI control plane in production:

Model real roles, tenants, regions, resources, and business actions in policy.
Keep tool governance deny-by-default.
Separate read-only identities from write, delete, deploy, and payment identities.
Require approval for high-impact side effects.
Store approval state durably and verify the approver's identity before resuming.
Use short-lived credentials and keep secrets out of prompts and agent state.
Sandbox untrusted execution with containers, Wasm, gVisor, Firecracker, or Kubernetes isolation.
Run golden datasets for allow, deny, redact, and approval outcomes.
Benchmark the exact production policy stack and optional integrations.
Export policy decisions and execution results to audit, metrics, SIEM, or GRC systems.
Run policy contract checks so YAML actions do not drift from tool function signatures.
Treat prompt-injection defense as layered: deterministic policy, restricted tools, optional semantic signals, and human review.

11. What PolicyAware Is—and Is Not

PolicyAware is designed for LLM applications, RAG pipelines, MCP workflows, autonomous agents, and AI governance scans.

It is not a replacement for:

IAM and authorization,
WAF or API gateway controls,
application security testing,
secrets management,
container or process isolation,
endpoint security,
or secure software development practices.

The embedded Python SDK is not a secure-memory boundary. PolicyAware can decide whether a tool call should run, but it does not itself sandbox an approved command. Optional semantic classifiers can improve detection, but deterministic policy remains the auditable enforcement base.

These limitations are a strength when stated clearly: the framework owns AI governance decisions and evidence, while the platform continues to own identity, isolation, credentials, networking, and execution security.

Read the full security boundaries and limitations.

12. Conclusion

Text filtering remains part of AI safety, but autonomous systems require action-level governance.

A useful AI control plane should answer:

who is acting,
what model or tool is being requested,
which data and resources are involved,
whether the action is allowed,
whether sensitive values must be transformed,
whether a human must approve,
and what evidence must be retained.

PolicyAware provides an open-source implementation of that pattern through a lightweight Python package, deny-by-default YAML policy, MCP JSON-RPC interception, model routing, evaluation hooks, repository scanning, and audit traces.

The credible path to low latency is not an unmeasured "zero-overhead" claim. It is a local deterministic fast path, selective use of heavier detectors, cached policy state, reproducible benchmarks, and continuous measurement in the real deployment.

GitHub: ktirupati/policyaware
Documentation: PolicyAware docs
PyPI: policyaware
Examples: Runnable examples

Stopping Runaway AI Loops: Implementing Enterprise FinOps and Observability with PolicyAware

KRISHNA KISHOR TIRUPATI — Fri, 31 Jul 2026 00:23:23 +0000

Autonomous agents don't just fail loudly—they fail expensively. A single misconfigured retry loop between an agent and an LLM can generate thousands of redundant tool calls and API requests before anyone notices, turning a minor logic bug into a five-figure cloud bill. PolicyAware is built to be the operational safety net that catches this class of failure before it reaches your finance team's dashboard.

1. The Recursive Agent Crisis

Every SRE and platform engineer who has run agentic workloads in production has a version of this story. An agent is wired to call an LLM, interpret the response, and take an action—often invoking another tool, which produces output that gets fed straight back into the same LLM. Under normal conditions this loop terminates in a few steps. Under a bad prompt, a malformed tool response, or a subtle logic error, it doesn't.

The agent gets stuck reasoning in circles: it calls a tool, receives an ambiguous or malformed result, decides the task is incomplete, and calls the LLM again to "retry." Each retry consumes tokens, each tool call hits a downstream API, and there is no natural circuit breaker unless one has been explicitly engineered. Within minutes, a single stuck session can produce:

Thousands of duplicate or contradictory API calls to internal and third-party services.
Sustained LLM token consumption that dwarfs normal daily usage.
Cascading load on downstream systems that were never designed for machine-speed request volume.

By the time monitoring dashboards catch the anomaly—if they catch it at all—the damage is already done: a runaway bill, a rate-limited API partner, or a compromised production database from thousands of unchecked write attempts. Traditional APM tools tell you a service is under load; they don't tell you an autonomous agent is the one generating that load, or why.

This is why the recursive agent crisis is fundamentally a governance problem, not just a monitoring problem. Rate limits and cost alerts fire after the money is spent. What's needed is a control layer that understands agent intent and enforces limits before the damage compounds—which is exactly the role PolicyAware plays in an enterprise AI stack.

2. Pre-Deployment Auditing with PolicyAware

The cheapest place to catch a runaway-loop risk is before it ships. PolicyAware includes a local static scanning tool designed to run directly in CI/CD, auditing a codebase for the structural issues that lead to recursive disasters and compliance gaps.

Running the scanner locally or in a pipeline step is a single command:

policyaware scan .

This command walks the repository and flags:

Unshielded MCP tool definitions that expose destructive or high-cost operations without a corresponding PolicyAware policy attached.
Unbudgeted API routes—endpoints an agent can call that have no token, rate, or spend ceiling defined anywhere in the codebase.
Missing termination conditions in agent loop logic, such as retry blocks without a max-iteration guard.
Compliance gaps relative to your organization's baseline policy set, so a tool added by one team doesn't silently bypass governance rules enforced elsewhere.

A typical CI integration adds PolicyAware as a required check before merge:

# .github/workflows/policyaware-scan.yml
name: PolicyAware Audit
on: [pull_request]

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install PolicyAware
        run: pip install policyaware
      - name: Run PolicyAware scan
        run: policyaware scan . --fail-on-critical

With --fail-on-critical, PolicyAware blocks the pull request outright if it detects an unshielded tool definition or an unbudgeted route reachable by an autonomous agent. This turns agent governance into a build-time gate rather than a production incident, giving SREs and platform teams the same shift-left guarantees they already expect from security and dependency scanning.

3. Runtime Cost Controls

Static scanning catches structural risk before deployment, but recursive loops are a runtime phenomenon—so PolicyAware also operates as a live infrastructure gateway, sitting in front of your LLM and tool endpoints and evaluating every request against a global budget in real time.

Instead of trusting each agent instance to self-limit, PolicyAware enforces token and request budgets centrally, at the proxy layer, so a single misbehaving session cannot silently exceed organizational limits. A typical runtime configuration looks like this:

# policy.yaml (runtime cost section)
budgets:
  global:
    max_tokens_per_minute: 50000
    max_requests_per_minute: 200

  per_session:
    max_tokens_per_session: 20000
    max_tool_calls_per_session: 50
    max_retries_per_task: 3

  circuit_breaker:
    enabled: true
    trigger:
      identical_call_repeated: 5
      window_seconds: 30
    action: terminate_session
    reason: >
      Session terminated by PolicyAware: repeated identical
      tool calls detected, indicating a recursive loop.

With this configuration, PolicyAware enforces three layers of protection simultaneously:

A global ceiling on tokens and requests per minute across the entire fleet of agents, preventing any combination of sessions from overwhelming shared infrastructure.
A per-session budget that caps how much a single agent instance can consume before it is forced to stop and escalate to a human.
A circuit breaker that detects the specific signature of a recursive loop—the same tool call repeated in a tight window—and terminates the session immediately, before it reaches the per-session ceiling.

Because PolicyAware sits at the gateway layer rather than inside application code, these budgets apply uniformly across every agent, framework, and team using the platform. A new service doesn't need to reimplement cost controls; it inherits them automatically the moment it routes through PolicyAware.

4. Enterprise Observability Traces

Budgets and circuit breakers stop the bleeding, but SRE teams also need forensic visibility into what happened, when, and why. PolicyAware addresses this with native OpenTelemetry hooks that emit structured JSON telemetry on every prompt execution and tool invocation it mediates, with no custom instrumentation required.

Each trace emitted by PolicyAware captures the fields DevOps and compliance teams actually need during an incident review:

{
  "timestamp": "2026-07-30T23:41:12Z",
  "trace_id": "pa-8f2c1e9a",
  "session_id": "agent-run-4471",
  "tool": "db.execute_sql",
  "decision": "denied",
  "policy_rule": "block_destructive_sql",
  "tokens_consumed": 812,
  "cumulative_session_tokens": 19875,
  "budget_remaining_pct": 0.6,
  "latency_ms": 42
}

Because these traces follow the OpenTelemetry spec, they plug directly into the observability stack most platform teams already run:

Stream traces into Datadog for real-time cost dashboards and anomaly alerting tied to specific agents or teams.
Export metrics to Prometheus for budget-remaining and denial-rate gauges that feed existing SRE alerting rules.
Visualize session-level token burn and policy denials in Grafana, correlated against the same infrastructure metrics used for every other production service.

This native telemetry turns PolicyAware from a silent enforcement layer into an auditable system of record. When finance asks why a token budget was exceeded, or compliance asks which sessions attempted a destructive database operation, the answer is a query away rather than a forensic reconstruction from scattered application logs.

For any enterprise running generative AI or RAG architectures in production, this combination—pre-deployment scanning, runtime budget enforcement, and structured observability—is not an optional add-on. It is the baseline operational requirement for keeping autonomous agents financially and operationally accountable, and PolicyAware is purpose-built to be the utility that delivers all three from a single control plane.

Securing Autonomous AI Agents: Why PolicyAware is the Essential Control Plane for MCP Tool Execution

KRISHNA KISHOR TIRUPATI — Thu, 30 Jul 2026 23:59:13 +0000

Autonomous AI agents are no longer confined to sandboxes—they now execute real actions against databases, shells, and file systems in production. In that world, PolicyAware is the missing control plane for MCP-based tool execution, giving engineering and security teams a way to govern agentic behavior without slowing down innovation.

1. The Shift to Agentic AI

For most of the last decade, AI integration meant a static text chatbot bolted onto an app: prompt in, text out, no persistent state, no external capabilities. That model was low-risk because it was also low-power.

Anthropic's Model Context Protocol (MCP) changes this equation. MCP formalizes how LLMs invoke external tools—databases, HTTP APIs, shell commands, file systems, internal services—through structured, typed tool calls. An MCP-enabled agent doesn't just generate SQL as text; it issues an explicit execute_sql tool invocation with arguments that run directly against production systems.

This shift introduces three structural changes:

Agents become orchestration layers, deciding which tools to call, when, and with what parameters.
Tool executions become programmable, repeatable, and logged as first-class events.
The blast radius of a single compromised session now extends to everything those tools can reach.

In this environment, PolicyAware introduces an architectural separation most stacks currently lack: an independent Agent Control Plane sitting between MCP agents and the tools they invoke. Instead of trusting the LLM to behave, PolicyAware inspects, approves, or blocks every tool call at the argument level before it touches infrastructure.

2. The Action-Level Vulnerability

Most teams experimenting with MCP still think in terms of classic prompt security: jailbreak detectors, content filters, safe-completion policies. These operate on unstructured text and try to prevent the model from generating problematic output. Useful, but blind to the new failure mode agent tooling introduces.

With MCP, the critical risk surface is not the text the LLM prints—it's the actions it takes. A compromised or misaligned agent can:

Construct a valid DELETE FROM customers WHERE ... statement and pass it to a database tool.
Invoke a run_shell tool with arguments like rm -rf /var/lib/data or a piped remote script execution.
Write files into a repository via a write_file tool, introducing poisoned dependencies or leaked secrets.

Traditional prompt firewalls rarely see these actions in a meaningful way. They operate at the prompt-plus-completion boundary, not at the tool-plus-arguments boundary where actual damage occurs. If an LLM accepts a hidden prompt injection, it can respond with entirely innocuous text while still issuing a dangerous tool call underneath.

Consider a realistic chain of events: a user opens a long-running MCP session; a retrieved web page contains a hidden instruction telling the agent to run a destructive query whenever it's later asked to "optimize a report"; the agent internalizes this; a legitimate request later arrives; the agent silently emits a tool call carrying a destructive SQL statement. To most observability stacks, this looks like a normal, syntactically valid MCP invocation. The vulnerability lives in the semantics of the action, not in the conversation text—which is exactly the gap PolicyAware exists to close by enforcing zero-trust, deny-by-default policy over tool arguments themselves.

3. PolicyAware as the Shield: Zero-Trust Agent Control Plane

PolicyAware is deliberately not another LLM safety filter layered on top of prompts. It is an Agent Control Plane deployed alongside your MCP tools so that every agent-mediated action passes through a hardened, auditable approval path.

Architecturally, PolicyAware runs as a standalone proxy sitting between your MCP tool server and the real services those tools reach. Rather than an agent calling tools directly, it calls through PolicyAware, which parses tool arguments at the protocol level, evaluates them against policy-as-code rules, and then forwards, modifies, or blocks the call with a structured error.

Getting started is straightforward:

pip install policyaware

Once installed, spin up the standalone PolicyAware proxy:

policyaware up --config policy.yaml --port 8080

This launches a PolicyAware control plane process bound to localhost:8080, reading enforcement rules from policy.yaml. MCP tools like db.execute_sql or system.shell are then pointed at this proxy instead of directly at the underlying database or shell. From the agent's perspective nothing changes—it still issues ordinary tool calls—but every invocation now flows through PolicyAware before touching anything sensitive.

Several properties make PolicyAware particularly suited to enterprise AI architectures:

Zero-trust by design: PolicyAware assumes all agent requests are untrusted and requires explicit allow-listing of safe operations.
Protocol-aware parsing: it distinguishes SELECT from DELETE, ls from rm, read-only HTTP calls from mutating writes.
Policy-as-code workflow: security and DevOps teams manage rules in version-controlled YAML, reviewed with the same rigor as infrastructure-as-code.
Standalone proxy topology: PolicyAware deploys into an existing microservice mesh without modifying the LLM itself.

By elevating PolicyAware to the control plane for MCP tool execution, teams decouple what an agent wants to do from what the system will actually allow—the core principle of zero-trust architecture, and the only realistic path to securing autonomous agents as they gain broader authority in production.

4. Policy-as-Code Example: Deny-by-Default Topology

The heart of PolicyAware is its policy-as-code model: allowed and denied agent behaviors are described in YAML and enforced by the proxy at runtime, giving security teams a concrete, reviewable contract for agent actions instead of relying on opaque model settings.

A typical pattern for MCP-based stacks is deny-by-default: all tools and operations are blocked unless explicitly allowed, read-only operations are gated but easy to permit, and mutating operations require narrowly scoped, reviewed policies.

Here is a simplified policy.yaml showing how PolicyAware enforces this topology for a db.execute_sql tool and a system.shell tool:

version: 1
mode: deny-by-default

tools:
  db.execute_sql:
    allowed_verbs:
      - select
      - show
      - explain
    rules:
      - id: block_destructive_sql
        match:
          query_contains_any:
            - "delete "
            - "drop "
            - "truncate "
            - "update "
        action: deny
        reason: >
          Destructive SQL verbs are not allowed for autonomous agents.
          Use a supervised maintenance workflow instead.

  system.shell:
    allowed_commands:
      - "ls"
      - "cat"
      - "pwd"
    rules:
      - id: block_wildcard_rm
        match:
          command_regex: "rm\\s+-rf\\s+(/|\\.)"
        action: deny
        reason: >
          Recursive delete commands are blocked in agentic workflows.

      - id: block_remote_scripts
        match:
          command_contains_any:
            - "curl "
            - "wget "
        action: deny
        reason: >
          Downloading and executing remote scripts is not permitted
          for autonomous agents.

This configuration expresses several guarantees enforced by PolicyAware:

The global mode: deny-by-default ensures any tool, verb, or command not explicitly allowed is automatically blocked.
For db.execute_sql, PolicyAware parses the SQL and only permits queries whose leading verb is SELECT, SHOW, or EXPLAIN.
A dedicated block_destructive_sql rule denies any query containing destructive verbs like DELETE, DROP, TRUNCATE, or UPDATE regardless of surrounding context.
For system.shell, PolicyAware restricts the agent to a small set of introspection commands and denies recursive deletes or remote script execution outright.

Because PolicyAware operates as a proxy, these rules apply uniformly across every agent and session using MCP tools. Teams can evolve policy.yaml exactly as they evolve Terraform or Kubernetes manifests—through code review, CI checks, and staged rollouts. When a new tool is introduced, it can start locked down and only gain specific, audited capabilities over time.

For senior engineers and AI architects, this is the right level of control: PolicyAware doesn't try to out-argue the model at the prompt level—it simply refuses to execute unsafe actions. For DevOps and security teams, it's a familiar pattern: centralize authorization and enforcement in a control plane, define security posture as code, and instrument everything for observability and incident response. As MCP-based agents take on more operational responsibility, PolicyAware is positioned as the essential, open-source control plane standing between agentic intent and irreversible action.

Scan Your AI App for PII, Secrets, Tool Risks, RAG Gaps, and LLM Governance Issues with PolicyAware

KRISHNA KISHOR TIRUPATI — Thu, 23 Jul 2026 14:46:26 +0000

Introduction: AI features ship fast, governance arrives late

Most teams don't start their AI journey with a governance plan -- they start with a demo. They wire in an LLM SDK, add some RAG, experiment with tools and agents, and only later realize that sensitive data, unapproved calls, and missing audit trails are scattered across the codebase.

By that point, AI features are in production or close to it, and it's hard to answer basic questions like:

Where do we send PII or PHI in prompts?
Which files call providers like OpenAI or Anthropic directly without going through our gateway?
Which tools and agents can take risky actions without approval or budget limits?
Where does RAG return answers without citations or grounding checks?
How much of our LLM usage is actually covered by our policies and audit requirements?

PolicyAware exists to make those questions answerable. It's an open-source, policy-aware AI gateway and governance framework for LLM, RAG, MCP/tool, and AI-agent applications, with a CLI and Python API you can run locally in your own environment.

This article focuses on one capability: local code scanning with policyaware scan, which helps detect AI governance gaps in your repository without calling models or external services.

What `policyaware scan` actually does

policyaware scan is a fast, local governance and compliance scanner for AI application repositories. It scans source code, policy files, notebooks, infrastructure configuration, and data pipeline code for risks that matter in LLM, RAG, MCP/tool, and agent-based systems.

Important design choices:

It does not call external services.
It does not call models.
It does not execute your project code.

You can run it safely on a laptop, CI runner, or secure build agent without changing how your application behaves.

Governance areas it checks

The scanner looks for issues across multiple AI-specific governance categories, including:

PII: prompts or config containing emails, phone numbers, SSNs, or credit-card-like values that should be redacted before reaching models, tools, logs, or evaluators.
PHI: patient identifiers, medical record language, diagnosis or prescription text that may require regulated-domain policies and redaction.
Secrets: hardcoded API keys, tokens, bearer tokens, and secret assignments that should be moved into secret managers or environment variables and rotated.
Direct LLM calls: direct usage of SDKs and frameworks such as OpenAI, Anthropic, Bedrock, Vertex AI, LiteLLM, Ollama, LlamaIndex, DSPy, and Semantic Kernel that bypass the PolicyAware gateway routing.
Provider governance gaps: model and provider usage that isn't routed by approved provider, role, risk level, region, cost, or availability policy.
MCP/tool governance risks: function calls, LangChain tools, MCP connectors, and other tools that can take impactful actions without connector or action policies and approval requirements.
Autonomous agent risks: patterns like while True, agent.run, or auto-execution loops that don't enforce human approval, max iterations, budgets, or audit logging.
RAG grounding and citation gaps: retrieval, vector store and embedding usage where responses are not tied to sources, metadata, or citation checks.
Data residency concerns: external endpoints, regions, API bases, and cloud locations that may move regulated data outside approved boundaries without tenant or region constraints.
Cost governance gaps: model or agent calls without clear token, timeout, rate, retry, or budget limits at request and workflow levels.
YAML policy quality: policy schema errors and missing default: deny rules that weaken deny-by-default behavior.
Configuration governance: .env, YAML, JSON, Terraform, Docker, CI/CD files that hold credentials or sensitive values instead of secret manager references.
Audit gaps: model and tool calls that lack trace/audit language, making it harder to reconstruct decisions, routes, evaluation results, and response metadata later.

From a platform or security perspective, this helps teams see AI-specific risks that traditional SAST or generic secret scanners often miss, especially around RAG behavior, tool orchestration, agent loops, and policy coverage.

Installation

PolicyAware is published on PyPI, so you can install it like any other Python tool:

pip install policyaware

If you're working directly from the PolicyAware repository, there's also a development install option documented in the wiki, but for most enterprise users pip install policyaware is enough to get started with the CLI.

Quick start: scan a local folder and open the HTML report

To run a basic local scan against an AI application folder:

policyaware scan ./mylocalfolder

By default, this generates an HTML report:

policyaware-scan-report.html

You can tell the CLI to open the report automatically in your browser:

policyaware scan ./mylocalfolder --open

This is a good "first touch" workflow:

Run the scan against a sample or production-like project.
Open the HTML report.
Review the executive summary and severity counts.
Click into the findings by file and category to see what the scanner is actually flagging.

For AI platform, security, and compliance engineers, this is a low-friction way to surface governance issues without introducing latency into production traffic or forcing teams to adopt a full gateway orchestration on day one.

Multiple report formats for humans, CI, and automation

policyaware scan supports multiple output formats so you can serve developers, reviewers, and automated workflows with the same scan:

policyaware scan . --format html,json,sarif,markdown

Here's how those formats are typically used:

HTML: for humans -- developers, policy engineers, security reviewers, and compliance teams reviewing findings, remediation recommendations, and coverage.
JSON: for automation -- pipelines that push findings into internal dashboards, ticketing systems, or governance workflows.
SARIF: for GitHub code scanning and similar platforms that understand SARIF and can show findings inline in pull requests and repository views.
Markdown: for review notes, compliance summaries, and documentation PRs where you want to include finding lists and remediation checklists in plain text.

In practice, teams often generate all four formats in CI and let different personas consume whichever format fits their workflow.

CI usage: fail builds on high-risk AI governance gaps

You can use policyaware scan in CI to enforce a minimum governance bar before merging AI changes. For example, failing the pipeline when the scan finds high-severity issues:

policyaware scan . --fail-on high

A GitHub Actions workflow that wires this into pull requests and pushes to main looks like this:

name: PolicyAware Scan

on:
  pull_request:
  push:
    branches: [main]

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - name: Install PolicyAware
        run: python -m pip install policyaware
      - name: Run scan
        run: policyaware scan . --format html,json,sarif,markdown --fail-on high

This setup supports governance review by catching high-risk PII, PHI, secrets, LLM routing gaps, unapproved tools, and agent risks before they land in main, while still letting lower-severity findings through for future remediation.

Example findings PolicyAware Scan can flag

To make this concrete, here are some example patterns the scanner can flag in an LLM/RAG/agent repository:

Hardcoded API keys: secrets like OPENAI_API_KEY = "sk_live_..." or bearer tokens in Python, JS, YAML, or .env files.
Prompts containing PII: system or user prompts that include email addresses or phone numbers, especially in templates reused across multiple flows.
Direct OpenAI or Anthropic SDK calls: imports and calls to provider SDKs where requests bypass the PolicyAware gateway and its routing policies.
LangChain tools without an approval policy: tool definitions that can make HTTP calls, read/write files, or trigger external systems without connector or action policies.
RAG response paths without citation checks: retrieval and response code that surfaces answers to users without verifying sources or enforcing citation formatting.
Agent loops without budgets or max iterations: while True loops or agent-run patterns that can incur unbounded cost or perform repeated, unreviewed actions.
Missing audit traces: model or tool calls where no logging, tracing, or audit metadata is attached, making investigations and compliance reviews harder.
YAML policies without default: deny: policy files that allow actions by default instead of requiring explicit allow rules, which can weaken your governance posture.

Each finding in the HTML report includes redacted evidence and suggested remediation, so developers can see what triggered the finding and how to address it without exposing sensitive values in the report itself.

Why the HTML report matters for developers and reviewers

The HTML report is designed to be readable by both engineers and governance stakeholders. It includes:

Executive summary and overall risk.
Severity counts and finding categories.
Compliance area counts and compliance-area filters.
Top recommendations and remediation checklist.
Findings grouped by file, with redacted evidence and suggested fix snippets.
Guardrails orchestration findings and policy coverage score.
Governance reviewer summary and links to documentation.

This makes it possible for:

AI platform engineers to triage findings and prioritize fixes across services.
Security engineers to focus on secrets, provider routing, and guardrails.
Compliance engineers to map findings to frameworks like SOC 2 or internal AI governance standards.
Enterprise architects to see where policies are enforced versus where they're missing.

Instead of handing someone a raw JSON or SARIF file, you can share a single HTML report that surfaces the most important issues and gives teams a structured remediation path.

Python API: use LocalCodeScanner directly

If you prefer to control scanning from Python instead of the CLI, PolicyAware exposes a LocalCodeScanner and ScanConfig class you can use from your own scripts or orchestration tools:

from pathlib import Path
from policyaware import LocalCodeScanner, ScanConfig

config = ScanConfig.from_file(Path("examples/policyaware-scan.yaml"))

report = LocalCodeScanner(workers=4, config=config).scan(
    Path("."),
    out=Path("policyaware-scan-report.html"),
    json_out=Path("policyaware-scan-report.json"),
    sarif_out=Path("policyaware-scan-report.sarif"),
    markdown_out=Path("policyaware-scan-report.md"),
)

print(report.overall_risk)
print(report.category_counts)
print(report.compliance_counts)

This pattern lets you:

Embed scans into internal tooling or developer portals.
Customize concurrency (workers) and configuration.
Generate multiple report formats in one call.
Programmatically inspect overall_risk, category counts, and compliance counts to drive additional automation.

Example scan config YAML

You can customize what policyaware scan looks at and which categories it enables by adding a YAML scan configuration file. A simple example looks like this:

scan:
  include_extensions:
    - .py
    - .yaml
    - .json
    - .ipynb
    - .tf

  exclude_dirs:
    - tests/fixtures
    - generated

  enabled_categories:
    - PII
    - PHI
    - Secrets
    - LLM Governance
    - MCP/Tool Governance
    - RAG Governance
    - Guardrails Integration
    - Audit Gaps
    - Configuration Governance

  max_file_size: 1mb
  max_findings_per_file: 30

This configuration focuses the scanner on Python, notebook, config, and Terraform files, skips fixtures and generated content, and enables governance categories that matter for many enterprise AI applications. You can tune this for your environment, including severity overrides, disabled categories, ignore patterns, and baselines as documented in the Local Code Scan wiki page.

Where PolicyAware Scan fits relative to SAST and security scanners

PolicyAware Scan is not a replacement for SAST tools, generic secret scanners, or cloud security platforms, and it does not guarantee compliance or replace full security review. Instead, it focuses specifically on AI governance and LLM application risks.

A practical way to think about it:

Tool type	Primary focus
SAST	General code vulnerabilities (injections, unsafe APIs, insecure patterns).
Secret scanners	Credentials and tokens in code, config, and history.
Cloud security tools	Infrastructure misconfigurations, network exposure, IAM policies, runtime drift.
PolicyAware Scan	AI governance gaps around LLMs, RAG, tools, agents, policies, audit, and compliance.

In other words:

Keep using SAST for core application security.
Keep using secret scanners and cloud security tools for infrastructure, dependencies, and credentials.
Use PolicyAware Scan to add an AI governance lens across your AI-specific code, policies, and orchestration.

This layered approach gives platform and security teams better coverage without overclaiming what any single tool can do.

Links and further reading

If you want to go deeper or integrate this into your own platform, the key links are:

PyPI: https://pypi.org/project/policyaware/
GitHub: https://github.com/ktirupati/policyaware
Docs: https://ktirupati.github.io/policyaware/
Local Code Scan Wiki: https://github.com/ktirupati/policyaware/wiki/Local-Code-Scan

These include additional examples, ready-to-use YAML templates, CLI reference, and governance guidance for LLM, RAG, tools, agents, and audit.

Closing

Local governance scanning won't solve every AI risk in one shot, but it does give your team visibility into where sensitive data, provider calls, tools, agents, and policies show up in the code -- and where your current governance posture has gaps.

Try it on a local AI app, open the HTML report, and fix the highest-risk governance gaps first.

PolicyAware vs Guardrails vs AI Gateways vs Model Routers: The Comparison Every AI Engineer Needs to Read

KRISHNA KISHOR TIRUPATI — Sat, 23 May 2026 15:43:16 +0000

I've been building AI-powered features for a while now, and the hardest conversations I have with my team are never about which model to use. They're always about the same thing: what is this system actually allowed to do, and how do we prove it?

That question pushed me to build PolicyAware - an open source Python control plane that sits in front of your models, tools, and retrieval systems. Before I explain what it does, I want to walk through why the tools most teams reach for first - guardrails, AI gateways, and model routers - are genuinely useful but leave a critical gap wide open.

The landscape right now

If you search for "AI safety" or "LLM governance" you will find three categories of tools coming up again and again:

Guardrail libraries - validate prompts and outputs against safety rules
AI gateways - proxy your requests to model providers, centralize API keys
Model routers - pick the cheapest or fastest model for each request

All three are useful. None of them alone answers the governance question.

Here is the mental model I use: a guardrail checks what the model says. A gateway manages where the request goes. A router decides which model handles it. But none of them ask the most important question first: should this request be allowed to run at all, under this user's role, for this tenant, in this region, given this risk level?

That is the gap PolicyAware fills.

Side-by-side comparison

Capability	Guardrails	AI Gateway	Model Router	PolicyAware
Block unsafe prompts before execution	Sometimes	Sometimes	No	Yes
Redact PII / PHI / secrets pre-execution	Sometimes	Sometimes	No	Yes
Decisions using role, tenant, region, risk	Limited	Limited	Limited	Yes
Deny-by-default posture	Usually no	Usually no	No	Yes
Govern MCP / agent tool calls	Usually no	Sometimes	No	Yes
Require human approval for risky actions	Usually no	Sometimes	No	Yes
Route across providers after policy approval	No	Yes	Yes	Yes
Evaluate RAG citation, grounding, leakage	Sometimes	Limited	No	Yes
Emit audit traces with reason codes	Limited	Sometimes	Limited	Yes
Generate compliance evidence artifacts	Usually no	Usually no	No	Yes

The right column is not a flex. It is a description of what enterprise AI systems actually need once they move beyond read-only chat and start touching real data, real tools, and real business workflows.

When each tool is the right call

Use a guardrails library when your only need is response formatting, toxicity filtering, or structured output validation. If you do not need RBAC, tenant rules, approval flows, or audit evidence, a guardrail is lighter and faster.

Use an AI gateway when your main problem is juggling provider keys, rate limits, and fallback routing. Gateways are great infrastructure. They are just not governance.

Use a model router when you are optimizing for cost, latency, or quality tradeoffs across providers. A router does not decide whether a request should run - only which model would run it.

Use PolicyAware when your AI system touches sensitive data, calls external tools, operates under regional compliance rules, or takes actions with financial or operational consequences. If you need to explain a decision to a security team six months from now, you need a control plane, not just a proxy.

How the architecture fits together

Here is the pattern I use in production. The key rule is: nothing reaches a model, retriever, or tool until the control plane has made an explicit decision.

+-----------------------------+
| Application Layer           |
| (web app / API / workflow)  |
+-------------+---------------+
              |
              v
+-----------------------------+
| PolicyAware Control Plane   |
|                             |
|  1. Identity + context      |
|  2. Deny-by-default check   |
|  3. PII / PHI detection     |
|  4. Risk classification     |
|  5. Approval gate (if high) |
|  6. Provider routing        |
+--------+----------+---------+
         |          |
         v          v
  +----------+  +----------+
  | RAG Layer|  | Tools /  |
  | retrieval|  | MCP      |
  | citation |  | payments |
  +----+-----+  +----+-----+
       |              |
       +------+-------+
              |
              v
       +-------------+
       | Model Layer |
       | (local/SaaS)|
       +------+------+
              |
              v
       +-------------+
       | Evaluations |
       | leakage     |
       | grounding   |
       | audit trace |
       +-------------+

Every arrow in that diagram has a policy decision attached to it. That is the entire point.

A real example: the $500 refund prompt

Let us make this concrete. A customer-support copilot gets this message:

Email jane@example.com and refund the customer $500.

Here is what different tools do with it:

A guardrail might check whether the output looks safe
A gateway forwards the request to your provider of choice
A router picks GPT-4.1 because it is the best model for support tasks
PolicyAware stops and works through the full decision tree before any of that happens

Code: policy-first middleware

from dataclasses import dataclass, field
from enum import Enum
import re
from typing import List, Optional

class Decision(str, Enum):
    ALLOW = "allow"
    DENY = "deny"
    REQUIRE_APPROVAL = "require_approval"

@dataclass
class RequestContext:
    user_id: str
    role: str
    tenant: str
    region: str
    task_type: str
    prompt: str
    tools: List[str] = field(default_factory=list)

@dataclass
class PolicyResult:
    decision: Decision
    risk_tier: str
    redacted_prompt: str
    reason_codes: List[str]
    required_approver: Optional[str] = None

EMAIL_RE = re.compile(r"[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}")

ALLOWED_TOOLS = {
    "support_agent": {"knowledge_search", "draft_email"},
    "finance_manager": {"knowledge_search", "draft_email", "issue_refund"},
}

def evaluate_policy(ctx: RequestContext) -> PolicyResult:
    reason_codes = []
    allowed = ALLOWED_TOOLS.get(ctx.role, set())

    for tool in ctx.tools:
        if tool not in allowed:
            return PolicyResult(
                decision=Decision.DENY,
                risk_tier="high",
                redacted_prompt=EMAIL_RE.sub("[REDACTED]", ctx.prompt),
                reason_codes=[f"tool_not_permitted:{tool}"],
            )

    redacted = EMAIL_RE.sub("[REDACTED]", ctx.prompt)
    if EMAIL_RE.search(ctx.prompt):
        reason_codes.append("pii_detected")

    is_high_risk = "refund" in ctx.prompt.lower() or "issue_refund" in ctx.tools
    if is_high_risk:
        reason_codes.append("high_risk_financial_action")
        return PolicyResult(
            decision=Decision.REQUIRE_APPROVAL,
            risk_tier="high",
            redacted_prompt=redacted,
            reason_codes=reason_codes,
            required_approver="finance_supervisor",
        )

    reason_codes.append("policy_allow")
    return PolicyResult(
        decision=Decision.ALLOW,
        risk_tier="low",
        redacted_prompt=redacted,
        reason_codes=reason_codes,
    )

# Try the refund prompt
ctx = RequestContext(
    user_id="u-1001",
    role="support_agent",
    tenant="acme-corp",
    region="us-east",
    task_type="customer_support",
    prompt="Email jane@example.com and refund the customer $500.",
    tools=["draft_email", "issue_refund"],
)

result = evaluate_policy(ctx)
print(result.decision)      # Decision.DENY
print(result.reason_codes)  # ['tool_not_permitted:issue_refund']

The support agent gets denied before the prompt ever reaches a model. The reason code is logged. The redacted prompt is stored. That is the audit trail your security team will ask for.

Code: compliant model routing

Routing still matters - but it should only happen after policy approves the request.

@dataclass
class RouteDecision:
    provider: str
    model: str
    reason: str

COMPLIANT_MODELS = {
    "us-east": [("azure_openai", "gpt-4.1"), ("local_vllm", "llama-3.1-70b")],
    "eu-west": [("azure_openai_eu", "gpt-4.1"), ("local_vllm_eu", "llama-3.1-70b")],
}

def route_after_policy(result: PolicyResult, ctx: RequestContext) -> RouteDecision:
    if result.decision != Decision.ALLOW:
        raise PermissionError(f"Cannot route - decision is {result.decision}")

    options = COMPLIANT_MODELS.get(ctx.region, [])
    if not options:
        raise RuntimeError(f"No compliant providers for region: {ctx.region}")

    provider, model = options[0]
    return RouteDecision(provider, model, "policy-approved compliant route")

A traditional router asks: which model is fastest? This asks: which model is allowed? The order of those questions changes everything about your compliance posture.

Code: audit trace

This is the piece most teams skip - and regret during their first security review.

from datetime import datetime
import json

def emit_audit_trace(ctx: RequestContext, result: PolicyResult, route=None):
    trace = {
        "timestamp": datetime.utcnow().isoformat() + "Z",
        "user_id": ctx.user_id,
        "tenant": ctx.tenant,
        "region": ctx.region,
        "task_type": ctx.task_type,
        "decision": result.decision.value,
        "risk_tier": result.risk_tier,
        "reason_codes": result.reason_codes,
        "tools_requested": ctx.tools,
        "route": None if route is None else {
            "provider": route.provider,
            "model": route.model,
        },
        "prompt_preview": result.redacted_prompt[:200],
    }
    print(json.dumps(trace, indent=2))

Sample output for the denied refund request:

{
  "timestamp": "2026-05-23T15:00:00Z",
  "user_id": "u-1001",
  "tenant": "acme-corp",
  "region": "us-east",
  "task_type": "customer_support",
  "decision": "deny",
  "risk_tier": "high",
  "reason_codes": ["tool_not_permitted:issue_refund"],
  "tools_requested": ["draft_email", "issue_refund"],
  "route": null,
  "prompt_preview": "Email [REDACTED] and refund the customer $500."
}

Every denied request, every approval gate, every route choice - all replayable. That is the evidence layer.

Start using PolicyAware today

PolicyAware is open source, MIT licensed, and published as a Python package. You do not need a SaaS contract. You do not need to rip out your existing stack. Drop it in as a middleware layer in front of your LLM calls.

pip install policyaware

Simplest integration pattern:

from policyaware import evaluate_policy, RequestContext

ctx = RequestContext(
    user_id=current_user.id,
    role=current_user.role,
    tenant=current_user.tenant,
    region=current_user.region,
    task_type="customer_support",
    prompt=user_message,
    tools=requested_tools,
)

result = evaluate_policy(ctx)

if result.decision == "allow":
    response = call_your_llm(result.redacted_prompt)
elif result.decision == "require_approval":
    request_human_approval(ctx, result)
else:
    return {"error": "Request denied", "reason": result.reason_codes}

One function call between your application and your model. Policy first. Everything else second.

The bottom line

Guardrails make your outputs safer. Gateways make your infrastructure cleaner. Routers make your model spend smarter. But none of them govern the full execution path.

If your AI system is making decisions that touch real people, real money, or real compliance boundaries - you need a control plane that runs policy before execution and produces evidence after it.

That is exactly what PolicyAware is built for. Star the repo, install the package, and let me know what governance problems you are running into - I am actively building this out in the open.

GitHub: https://github.com/ktirupati/policyaware

pip install policyaware

Build a Policy-Aware AI Gateway in Python: Data Protection + Policy Enforcement with policyaware

KRISHNA KISHOR TIRUPATI — Mon, 18 May 2026 04:33:30 +0000

Most AI apps ship without any real governance layer. Prompts flow raw to models, sensitive data ends up in logs, and nobody finds out until a compliance audit or a breach. I built policyaware to fix that — a Python-first package that gives you data protection and policy enforcement in front of any AI system.

This article is a hands-on technical walkthrough. Every section has working code. By the end you will have a pattern you can wire into any AI gateway or agent pipeline today.

Quick Install

!pip install policyaware

GitHub: https://github.com/ktirupati/policyaware
Wiki: https://github.com/ktirupati/policyaware/wiki

Part 1 — Data Protection

What the engine detects

The DataProtectionEngine scans any string and returns a structured DataFindings object. It classifies content into three buckets:

Bucket	What it catches
PII	email, phone, SSN, credit card
PHI	medical record, patient ID, diagnosis, medication
Secrets	API keys, bearer tokens, private keys

Inspecting a prompt

from policyaware import DataProtectionEngine

text = "Hi, I'm Jane. Reach me at jane@example.com or 212-555-7890."

engine = DataProtectionEngine()
findings = engine.inspect(text)

print(findings.contains_pii)        # True
print(findings.contains_phi)        # False
print(findings.contains_secrets)    # False
print(findings.contains_sensitive)  # True  (aggregate flag)
print(findings.categories)          # ['email', 'phone']
print(findings.redactions)          # 2

DataFindings field reference

Field	Type	Description
`contains_pii`	bool	email, phone, SSN, credit card detected
`contains_phi`	bool	medical record, diagnosis, medication detected
`contains_secrets`	bool	API key, bearer token, private key detected
`contains_sensitive`	bool	True if any of the above is True
`categories`	list	e.g. `['email', 'phone', 'ssn']`
`redactions`	int	Total number of matches found
`redacted_text`	str	Sanitised text returned by `.redact()`

Part 2 — Policy Enforcement

Data protection tells you what is in the request. Policy enforcement tells you what to do about it. The PolicyEngine loads a YAML file and evaluates every request against your rules, returning a structured PolicyDecision.

The four decision outcomes

Decision	Meaning
`allow`	Request passes through, apply any transforms
`deny`	Request is blocked outright
`conditional_allow`	Passes but triggers follow-up checks
`require_approval`	Routes to a human-in-the-loop flow

The engine is deny-by-default. If no rule explicitly grants access, the request is blocked. No silent pass-throughs.

Writing your first policy YAML

Rules reference DataFindings fields directly via the data root:

# support_policy.yaml
id: support_policy
schema_version: "0.2"
default: deny

rules:

  # Rule 1: Block anything containing secrets (API keys, tokens)
  - name: deny_secret_leakage
    effect: deny
    when:
      data.contains_secrets: true

  # Rule 2: Redact PII for standard users, but not for compliance officers
  - name: redact_pii_standard_users
    effect: transform
    action: redact
    when:
      data.contains_pii: true
      user.role_not_in:
        - privacy_admin
        - compliance_officer

  # Rule 3: Allow support agents in US for low/medium risk requests
  - name: allow_support_agents
    effect: allow
    when:
      user.role_in:
        - support_agent
        - support_manager
      request.region: us
      risk.tier_in:
        - low
        - medium

Enforcing the policy at runtime

Load the YAML, build a GatewayRequest, inspect the prompt, then call decide:

from policyaware import DataProtectionEngine, GatewayRequest, PolicyEngine

# Load policy from YAML file
policy = PolicyEngine.from_file("support_policy.yaml")

# Build the request context
request = GatewayRequest(
    tenant="acme-corp",
    app="support-copilot",
    user={"role": "support_agent", "id": "u_001"},
    context={"region": "us", "risk": "low"},
    messages=[{"role": "user", "content": "Email jane@example.com, urgent!"}],
)

# Step 1: inspect the prompt
findings = DataProtectionEngine().inspect(request.prompt_text)

# Step 2: evaluate policy
decision = policy.decide(request, findings)

# Step 3: act on the decision
print(decision.decision.value)   # 'allow' / 'deny' / 'conditional_allow' / 'require_approval'
print(decision.actions)          # ['redact']
print(decision.matched_rules)    # ['redact_pii_standard_users', 'allow_support_agents']
print(decision.violated_rules)   # []
print(decision.reason)           # Human-readable explanation
print(decision.reason_codes)     # Machine-readable codes for logging
print(decision.risk_score)       # Numeric risk score
print(decision.risk_tier)        # 'low' / 'medium' / 'high' / 'critical'
print(decision.remediation)      # Suggested fix if blocked

PolicyDecision field reference

Field	Type	Description
`decision`	enum	`allow`, `deny`, `conditional_allow`, `require_approval`
`actions`	list	Transforms to apply e.g. `['redact']`
`matched_rules`	list	Rules that matched the request
`violated_rules`	list	Rules that were violated (for audit logs)
`reason`	str	Human-readable explanation
`reason_codes`	list	Machine-readable codes for dashboards
`risk_score`	float	Numeric risk score
`risk_tier`	str	`low`, `medium`, `high`, `critical`
`remediation`	str	Suggested fix when request is blocked

Policy Context Roots

Inside every when clause you can reference these roots:

Root	Example usage	What it covers
`tenant`	`tenant: acme`	Customer or team identifier
`app`	`app: support-copilot`	Calling application or service
`user`	`user.role_in: [support_agent]`	Role, ID, department attributes
`request`	`request.region: us`	Region, task type, autonomy level
`data`	`data.contains_pii: true`	Output from `DataProtectionEngine`
`risk`	`risk.tier_in: [low, medium]`	Risk score and tier
`ml`	`ml.prompt_injection.detected: true`	Optional ML classifier signals

Validate Policies Before Production

Ship broken policies and you get silent misses or unintended blocks. policyaware ships a schema validator and CLI to catch issues early.

Python validator:

import yaml
from policyaware import PolicySchemaValidator

with open("support_policy.yaml", "r", encoding="utf-8") as f:
    policy = yaml.safe_load(f)

PolicySchemaValidator().validate(policy)  # raises on schema errors

CLI commands:

# Validate the YAML schema
policyaware policy validate support_policy.yaml

# Explain how a specific request flows through your rules
policyaware policy explain --request sample_request.json

The explain command is especially useful in CI/CD pipelines — you can run policy checks against a suite of sample requests before merging.

Optional: ML-Assisted PII Detection with Presidio

Regex-based rules miss things like names and addresses. For those, policyaware supports an optional Microsoft Presidio integration:

pip install "policyaware[presidio]"

from policyaware import PresidioPIIClassifier

classifier = PresidioPIIClassifier(score_threshold=0.5)

assessment = classifier.classify(
    "Jane Doe lives at 120 Main St and her phone is 212-555-7890."
)

print(assessment.model_dump())
# Returns detected entities with type, value, and confidence score

The Presidio findings feed back into the same data and ml roots in your YAML, giving you deterministic + ML detection in one framework.

TL;DR — What You Get in One Package

Capability	How
Detect PII, PHI, Secrets	`DataProtectionEngine().inspect(text)`
Redact sensitive content	`DataProtectionEngine().redact(text)`
Enforce access policies via YAML	`PolicyEngine.from_file("policy.yaml")`
Rich audit-ready decisions	`PolicyDecision` with reason, risk, remediation
ML-assisted detection	`PresidioPIIClassifier` (optional extra)
Validate policies before shipping	`PolicySchemaValidator` + CLI

Get Started Now

!pip install policyaware

Here is the fastest path to seeing value:

Install the package
Run DataProtectionEngine().inspect() on one real prompt from your app
Write a 3-rule YAML that reflects your actual governance needs
Call policy.decide(request, findings) and log the full PolicyDecision

That four-step experiment is enough to understand whether policyaware fits your stack.

I am the author and sole maintainer of this package. I built it because every AI project I worked on had the same gap — no structured layer between raw user input and the model. If you run into anything unexpected, have a governance pattern not covered yet, or want to contribute, I want to hear from you.

GitHub: https://github.com/ktirupati/policyaware
Wiki & Docs: https://github.com/ktirupati/policyaware/wiki

If this was useful, drop a like, share it with your team, and star the repo. Every bit of feedback helps make policyaware better for everyone building serious AI systems in Python.

I Built an AI Agent That Remembers My Entire Codebase (So I Don't Have To)

KRISHNA KISHOR TIRUPATI — Tue, 28 Apr 2026 22:34:01 +0000

Ever spent 20 minutes digging through a legacy module just to remember how a specific utility function handles null pointers? We've all been there. Modern codebases are growing at a rate that outpaces human memory. That's why I decided to build a "Second Brain" for my development workflow: a Retrieval-Augmented Generation (RAG) based AI Agent.

The Problem: Context Switching is a Productivity Killer

As developers, we spend more time reading code than writing it. When you're juggling microservices, custom hooks, and complex database schemas, the cognitive load becomes immense. I wanted something that didn't just "guess" based on general training data (looking at you, vanilla GPT-4), but actually knew my specific implementation details.

The Architecture: How It Works

The core of this system is a RAG pipeline optimized for source code. Here’s the high-level flow:

Ingestion: A Python script crawls the repository, ignoring files in .gitignore.
Parsing: It breaks the code into logical chunks (functions, classes, or modules).
Embedding: These chunks are converted into vector representations using OpenAI's text-embedding-3-small.
Storage: The vectors are stored in a Pinecone database.
Retrieval: When I ask a question, the agent finds the most relevant code snippets.
Reasoning: An LLM (GPT-4o) uses that retrieved context to provide a precise answer.

Show Me the Code!

Here is a simplified version of the ingestion logic using LangChain:

from langchain_community.document_loaders import GenericLoader
from langchain_community.document_loaders.parsers import LanguageParser
from langchain_text_splitters import Language

# Load your local codebase
loader = GenericLoader.from_path(
    "./my-awesome-project",
    glob="**/*",
    suffixes=[".py", ".js"],
    parser=LanguageParser(language=Language.PYTHON, parser_threshold=500)
)
docs = loader.load()

# Split and Embed (Simplified)
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import Chroma

embeddings = OpenAIEmbeddings()
vectorstore = Chroma.from_documents(docs, embeddings)

Why This is a Game Changer

Since integrating this into my local CLI, I’ve noticed:

Instant Onboarding: I can point it at a new library and ask "How is authentication handled?" and get a breakdown in seconds.
Better Debugging: I can paste an error trace and ask "Which part of our business logic could cause this?"
Consistency: It helps ensure I'm using existing patterns instead of reinventing the wheel.

Final Thoughts

Building an AI agent that remembers your codebase isn't about replacing the developer; it's about augmenting them. It removes the "grunt work" of searching and lets you focus on architectural decisions and problem-solving.

Are you using any custom AI tools in your workflow? Let's discuss in the comments!

Vibe Coding in 2026: How AI Tools Like Cursor, Replit, Claude, and GitHub Copilot Are Changing the Way We Build Software

KRISHNA KISHOR TIRUPATI — Mon, 27 Apr 2026 22:22:35 +0000

Remember when writing code meant typing every line, every bracket, every semicolon? That world is fading fast. In 2026, we are living in what many call the era of vibe coding, where describing what you want in plain English can get you most of the way to working code.

I have been building with these tools every day, and the shift is real. This is not about replacing developers. It is about changing how we work.

What Is Vibe Coding

Vibe coding means you describe your intent in natural language, and the AI writes the implementation. You focus on the what and why, while the AI handles the how.

Instead of writing:

def calculate_fibonacci(n):
    if n <= 1:
        return n
    return calculate_fibonacci(n-1) + calculate_fibonacci(n-2)

You tell Cursor or Claude: Create a function that calculates the nth Fibonacci number using recursion.

The AI writes it, you review it, you ship it.

The Main Players in 2026

GitHub Copilot

Copilot lives inside your editor. You type a comment, it suggests the next few lines. You accept, modify, or reject.

It works in VS Code, JetBrains, Neovim. It is the most widely adopted tool because it fits into existing workflows without forcing you to change editors.

Example workflow:

// Function to fetch user data from API and cache it

Copilot suggests:

async function fetchUserData(userId) {
  const cacheKey = `user-${userId}`;
  const cached = await cache.get(cacheKey);

  if (cached) {
    return JSON.parse(cached);
  }

  const response = await fetch(`/api/users/${userId}`);
  const userData = await response.json();

  await cache.set(cacheKey, JSON.stringify(userData), 3600);
  return userData;
}

You review it. Maybe you change the cache TTL. Maybe you add error handling. But the structure is there.

Cursor

Cursor is an AI-first editor built on VS Code. It understands your entire codebase, not just the file you are editing.

You can ask it: Find everywhere we are making API calls without proper error handling.

It scans your repo, shows you every instance, and can fix them all at once.

Cursor has two modes:

Chat mode for asking questions
Composer mode for making changes across multiple files

I use Cursor when I need to refactor or when I am working on a feature that touches many files.

Claude and ChatGPT

These are not IDEs. They are reasoning engines.

You paste your code, describe the problem, and they help you debug, refactor, or architect.

They excel at:

Explaining complex code
Suggesting architectural improvements
Writing test cases
Converting code between languages

Example:
You: This Python script is slow when processing large CSV files. How can I optimize it?

Claude analyzes your code and suggests using pandas chunking, multiprocessing, or switching to Polars.

Replit

Replit is a browser-based IDE with AI built in. You can go from idea to deployed app without leaving the browser.

It is especially good for prototypes, learning, and small projects where you do not want to set up a local environment.

Real Architecture Example

Let me show you how I built a real feature using vibe coding.

Task: Add a rate limiter to our API endpoints.

Here is how the conversation with Cursor went:

Me: I need to add rate limiting to all our Express routes. Use Redis for storage. Limit to 100 requests per minute per IP.

Cursor generated:

const redis = require('redis');
const client = redis.createClient();

const rateLimiter = async (req, res, next) => {
  const ip = req.ip;
  const key = `ratelimit:${ip}`;

  const requests = await client.incr(key);

  if (requests === 1) {
    await client.expire(key, 60);
  }

  if (requests > 100) {
    return res.status(429).json({
      error: 'Too many requests'
    });
  }

  next();
};

module.exports = rateLimiter;

Then I asked: Add logging for when users hit the limit.

It updated:

if (requests > 100) {
  console.log(`Rate limit exceeded for IP: ${ip}`);
  return res.status(429).json({
    error: 'Too many requests',
    retryAfter: 60
  });
}

Final step: Apply this to all routes.

Cursor modified the main app file:

const rateLimiter = require('./middleware/rateLimiter');
app.use(rateLimiter);

Total time: 3 minutes.

The Architecture of AI Coding Tools

Here is how these systems work under the hood:

┌─────────────────────────────────────────────────────────┐
│                    Your Editor/IDE                      │
│  (VS Code, Cursor, Replit)                             │
└────────────┬────────────────────────────────────────────┘
             │
             ▼
┌─────────────────────────────────────────────────────────┐
│              AI Coding Assistant Plugin                 │
│  • Captures your code context                          │
│  • Sends prompts + context to AI                       │
│  • Receives suggestions                                │
└────────────┬────────────────────────────────────────────┘
             │
             ▼
┌─────────────────────────────────────────────────────────┐
│                   AI Model Layer                        │
│  (GPT-4, Claude, Codex)                                │
│  • Processes code context                              │
│  • Generates suggestions                               │
│  • Returns formatted code                              │
└────────────┬────────────────────────────────────────────┘
             │
             ▼
┌─────────────────────────────────────────────────────────┐
│                  Your Codebase                          │
│  • Changes applied                                      │
│  • Context updated                                      │
│  • Ready for next iteration                            │
└─────────────────────────────────────────────────────────┘

The key is context. The more the AI knows about your project, the better its suggestions.

Cursor indexes your entire repo. GitHub Copilot looks at open files and recent edits. Claude works with what you paste.

Workflow Example: Building a REST API

Let me walk through building a simple REST API for a task manager using vibe coding.

Step 1: Define the structure

Me to Cursor: Create an Express API with routes for creating, reading, updating, and deleting tasks. Use MongoDB for storage.

Cursor generates:

const express = require('express');
const mongoose = require('mongoose');
const app = express();

app.use(express.json());

const taskSchema = new mongoose.Schema({
  title: String,
  description: String,
  completed: Boolean,
  createdAt: { type: Date, default: Date.now }
});

const Task = mongoose.model('Task', taskSchema);

app.post('/tasks', async (req, res) => {
  const task = new Task(req.body);
  await task.save();
  res.json(task);
});

app.get('/tasks', async (req, res) => {
  const tasks = await Task.find();
  res.json(tasks);
});

app.put('/tasks/:id', async (req, res) => {
  const task = await Task.findByIdAndUpdate(req.params.id, req.body, { new: true });
  res.json(task);
});

app.delete('/tasks/:id', async (req, res) => {
  await Task.findByIdAndDelete(req.params.id);
  res.json({ message: 'Task deleted' });
});

mongoose.connect('mongodb://localhost/taskmanager');
app.listen(3000);

Step 2: Add error handling

Me: Add try-catch blocks and proper error responses.

Cursor updates all routes:

app.post('/tasks', async (req, res) => {
  try {
    const task = new Task(req.body);
    await task.save();
    res.json(task);
  } catch (error) {
    res.status(500).json({ error: error.message });
  }
});

Step 3: Add validation

Me: Validate that title is required and completed defaults to false.

Cursor modifies the schema:

const taskSchema = new mongoose.Schema({
  title: { type: String, required: true },
  description: String,
  completed: { type: Boolean, default: false },
  createdAt: { type: Date, default: Date.now }
});

Total development time: under 10 minutes.

The Data Flow

Here is how data moves through a vibe coding session:

Developer Intent
      |
      v
  Natural Language Prompt
      |
      v
  AI Model (with code context)
      |
      v
  Generated Code Suggestion
      |
      v
  Developer Review & Edit
      |
      v
  Final Implementation

What Works Best

After months of daily use, here is what I have learned:

Be specific in your prompts
- Bad: Make this faster
- Good: Optimize this loop using a hash map instead of nested iteration
Give context
- Instead of: Write a login function
- Say: Write a login function that checks credentials against our PostgreSQL users table, returns a JWT token, and logs failed attempts
Iterate in small steps
- Do not ask for an entire feature at once
- Build piece by piece, testing as you go
Review everything
- AI makes mistakes
- It might use deprecated methods
- It might miss edge cases

The Limits

Vibe coding is not magic. It struggles with:

Complex business logic that requires domain expertise
Performance optimization for specialized use cases
Architectural decisions that involve tradeoffs
Debugging production issues that need deep system knowledge

You still need to understand what the code does. You still need to think like an engineer.

My Setup

Here is my current workflow:

Cursor for feature development and refactoring
GitHub Copilot for autocomplete while editing
Claude for architecture discussions and code review
Replit for quick prototypes and experiments

I spend less time typing, more time thinking.

Final Thoughts

Vibe coding is not replacing developers. It is changing what we focus on.

Instead of remembering syntax, we think about architecture.
Instead of writing boilerplate, we design systems.
Instead of debugging typos, we solve real problems.

The tools are here. The question is: are you using them?

What has your experience been with these AI coding tools? Are you skeptical, excited, somewhere in between? Let me know in the comments.

Designing and Deploying Agentic AI Systems in Production Using Azure OpenAI

KRISHNA KISHOR TIRUPATI — Sun, 26 Apr 2026 02:27:32 +0000

Designing and deploying agentic AI systems on Azure OpenAI is ultimately a software engineering problem, not just a prompt engineering exercise.

Introduction

Agentic AI on Azure OpenAI combines large language models with tools, memory, and orchestration so systems can perceive context, reason about goals, and act through APIs or workflows. In enterprise environments, these agents sit inside existing architectures, integrate with business systems like CRMs or ERPs, and must meet stringent requirements for reliability, security, observability, and governance.

Architecture Overview

At a high level, an Azure OpenAI agent in production is a composition of model, orchestration layer, enterprise services, and platform capabilities from Azure.

Typical Layers

1. Experience Layer
This includes chat widgets, web and mobile apps, IVR, or line-of-business front ends that capture user inputs and display responses. They communicate with a backend agent API over HTTPS and often stream partial responses for better perceived latency.

2. Orchestration and Agent Runtime
This is usually implemented as a microservice or set of services running on Azure Kubernetes Service, Azure Container Apps, or App Service. It handles dialogue state, calls Azure OpenAI for reasoning, invokes tools via function calling, manages retries, and applies business rules such as guardrails or approval workflows.

3. Azure OpenAI Service
This provides deployed models such as GPT-4 class models, responses or chat APIs, function/tool calling, and system-level safety settings. You configure deployments per region and SKU, define capacity, and integrate them with your orchestration tier through the standard REST or Python/Java/.NET SDKs.

4. Enterprise Tools and Data
Agents rely on tools that wrap internal systems: REST APIs, databases, search endpoints, and workflow engines. For retrieval augmented generation, you usually add Azure AI Search or vector indexes, while for workflow automation you integrate with Logic Apps, Power Automate, or internal microservices.

5. Cross-Cutting Services
Governance, observability, and security come from services like Azure Monitor, Application Insights, Log Analytics, API Management, Key Vault, and Entra ID (Azure AD). These ensure authentication, authorization, quota management, rate limiting, metrics, tracing, and auditing.

Core Components of an Azure OpenAI Agent

An agent is more than a single prompt; it is usually composed of several cooperating elements.

Policy and Role Definition
The agent's role defines its scope, allowed tools, and tone via system prompts and configuration. You specify what it may do, what data it may touch, and which escalation paths it must follow for sensitive actions.

Memory and Context
Short-term memory is the conversation history and state for the current session, while long-term memory comes from knowledge bases and logs. On Azure this is often implemented with Azure AI Search, Cosmos DB, or SQL, combined with embeddings produced by Azure OpenAI models.

Tooling Interface
Functions are exposed to the model using Azure OpenAI function or tool calling: you define function schemas, arguments, and natural-language descriptions, then let the model choose when to call each tool. The orchestration layer executes the selected tool, captures the results, and feeds them back to the model as messages.

Safety, Guardrails, and Filters
You apply content filters, allow/deny lists, and input/output validation before and after every model call. For high-risk domains, human-in-the-loop review and approval are added as explicit steps in the workflow.

How an Agent Behaves in Real Enterprise Scenarios

In production, agent behavior is shaped by business rules, data access patterns, and organizational risk tolerance. Below are practical scenarios that show how this plays out with Azure OpenAI.

Support Automation
A customer opens a support chat in a portal. The frontend sends the message to an agent API that enriches it with user profile data and recent tickets from a CRM tool. The agent uses Azure AI Search to retrieve relevant knowledge articles and internal runbooks, then asks Azure OpenAI to draft a response via the responses or chat API with function calling. If the issue exceeds certain risk thresholds, the agent routes the conversation to a human agent, attaching a summarized context and proposed reply for faster handling.

Decision Support
A portfolio manager asks, "How will this product change impact our quarterly margin?" The orchestration layer calls financial and sales data APIs to fetch current numbers, then passes structured summaries to the model through tools. The agent runs scenario analysis through multiple calls: one to generate assumptions, one to compute summaries over metrics, and one to explain trade-offs in business language. Outputs include narrative explanation plus structured justification, which can be stored for audit.

Workflow Automation
An internal user requests, "Create a change request for updating this microservice and notify the owners." The agent uses tools to create a work item in Azure DevOps or ServiceNow, update a change calendar, and send notifications via email or Teams connectors. It returns a summary with links, IDs, and the steps it performed, giving transparency into actions.

End-to-End Agent Workflow Example

Consider a support automation agent deployed on Azure OpenAI and fronted by a web chat in a corporate portal.

Step 1: User Request and Intake
The user types: "My invoice shows the wrong amount, can you fix it?" The frontend passes this text, session identifiers, and user ID to a backend API along with any client-side telemetry such as locale and device type. Basic validation, rate limiting, and authentication via Entra ID occur at API Management or the gateway.

Step 2: Context Assembly
The agent service fetches user profile details and recent invoices through internal APIs exposed as tools. It queries Azure AI Search using an embeddings-based index over billing policies and knowledge articles, returning several relevant passages. The service then constructs a prompt for Azure OpenAI that includes system instructions, conversation history, retrieved documents, and structured invoice data.

Step 3: Reasoning and Tool Selection
Using the responses or chat API with function calling enabled, the model decides that it must call a "get_invoice_details" tool because the user is referencing a specific invoice. The orchestration layer executes that tool by calling the billing service, then posts the result back as a tool response, prompting the model again. The model now checks for mismatched line items and determines that a partial credit is appropriate per policy.

Step 4: Action and Validation
The agent calls another tool, "create_credit_memo," but this time the orchestration code applies an extra guard: for credits above a certain amount, it requires human approval instead of automatic execution. The tool either executes or records the request in a queue for human review and returns the status to the agent. The orchestration layer logs all inputs, decisions, and tool outputs into Application Insights and Log Analytics for observability and audit.

Step 5: Response Generation and Streaming
The agent calls Azure OpenAI one more time with all updated context to generate a user-friendly explanation of what was done and what the user should expect next. Streaming is enabled so the frontend can display tokens as they arrive, which significantly improves perceived latency even if the overall response generation takes a few seconds. The final message is persisted to a conversation store along with structured metadata such as outcome status and tags for analytics.

This pattern repeats across messages, giving the agent a dialog loop where each turn includes intake, context building, reasoning, tool use, and output.

Implementation Approach on Azure OpenAI

A robust agent implementation emerges from a staged approach that moves from problem definition to production hardening.

Defining the Use Case
Start with one or two focused journeys where agents can deliver measurable value, for example first-line support or internal request automation. Define clear success metrics such as deflection rate, handle time reduction, or user satisfaction, and translate them into model-level KPIs like answer accuracy or escalation rates.

Designing Agent Workflows
Map the current process step by step, then identify which decisions can move to the agent and which must remain with humans. Translate this into an orchestration design that uses patterns such as sequential flows, concurrent calls, or handoff flows. For complex environments, adopt a multi-agent design where specialized agents handle retrieval, planning, or domain-specific tasks, coordinated by a higher-level controller.

Prompt and Policy Engineering
Author precise system messages that describe role, boundaries, and tone, and include examples of desired behavior and red lines. Use few-shot examples for tricky reasoning steps, and add structured instructions that explain how to decide whether a tool is required. Encode non-negotiable business rules outside the prompt in actual code, so the agent can propose actions but cannot bypass compliance logic.

Tool Integration
Wrap each enterprise system in a well-typed function definition with clear names and human-readable descriptions that help the model choose correctly. Keep tool schemas small; large or rarely used tools can be loaded conditionally via a higher-level tool search step to keep the active tool set manageable. Implement timeouts, retries with backoff, and circuit breakers per tool to avoid cascading failures when downstream systems are slow or unavailable.

Deployment and Operations
Deploy the orchestration runtime to Azure Kubernetes Service or Azure Container Apps with proper horizontal scaling policies tied to CPU, memory, or QPS. Expose APIs through Azure API Management to control access, apply request throttling, and centralize authentication with Entra ID. Configure Azure Monitor, Application Insights, and Log Analytics for metrics, traces, and logs that capture every agent call, tool invocation, and error. For secrets and configuration such as API keys and connection strings, rely on Azure Key Vault and managed identities rather than environment variables or embedded secrets.

Production Challenges and How to Handle Them

Putting agents into production surfaces a set of recurring engineering challenges that go beyond prompt tuning.

Reliability
API failures, timeouts, and model-side rate limits are common when systems operate at scale. You address this by using exponential backoff retries, circuit breakers, graceful degradation strategies, and careful quota management through Azure resource planning and API Management. For critical actions, implement idempotent operations and compensating transactions so repeated tool calls do not corrupt state.

Latency
The main contributors to latency are network overhead, tool call cascades, and token generation within the model. Effective strategies include response streaming, reducing prompt and response length, batching where possible, and parallelizing independent tool calls. Model choice also matters: using smaller or more efficient deployments where appropriate can significantly improve latency and throughput.

Cost Management
Cost scales with total tokens and call volume, especially in multi-call agent workflows. You can control cost by pruning unnecessary context, compressing history into summaries, capping max tokens, and routing low-value traffic to cheaper models. Monitoring per-feature and per-tenant consumption and applying quotas ensures no single consumer overwhelms the budget.

Debugging and Observability
Debugging agents is difficult because behavior emerges from prompts, model weights, tools, and data working together. Rich logging of prompts, tool calls, and outputs, combined with correlation IDs across services, makes it possible to replay problem sessions and iteratively refine prompts and workflows. Telemetry dashboards that track hallucination reports, escalation rates, tool error rates, and user feedback are essential to continuous improvement.

Scalability
Scaling requires both the model side and the orchestration side to handle higher load with predictable performance. On the model side, that means provisioning sufficient capacity, using multiple deployments, and sometimes applying multi-region strategies for resilience. On the application side, it means stateless or externally stateful services, asynchronous processing for long-running actions, and autoscaling policies that respond to traffic patterns.

Governance and Security
Enterprises need strong control over who can invoke agents, what data they can access, and how their actions are audited. Azure provides a foundation through Entra ID for identity, RBAC for resource access, private networking, and customer-managed keys for encryption at rest. You augment this with fine-grained policy at the application level, including role-based access to tools, PII redaction, data minimization, and retention controls. For regulated workloads, systematic logging and human-in-the-loop review for high-risk tasks provide additional assurance.

Conclusion

Agentic AI on Azure OpenAI is most successful when treated as an engineered system that combines models, tools, data, and governance rather than a single intelligent component. By starting with clear use cases, designing explicit workflows, investing in observability and guardrails, and using Azure's platform capabilities for scaling and security, organizations can deploy agents that deliver meaningful automation and decision support while staying within enterprise risk boundaries.

DEV Community: KRISHNA KISHOR TIRUPATI

Architectural Evaluation: Why We Re-Engineered AI Safety From Text Guardrails to Infrastructure Control Planes

Introduction

1. Guardrails and Control Planes Solve Different Problems

2. Why Autonomous Agents Need Action-Level Governance

3. PolicyAware's Runtime Architecture

4. First-Class MCP JSON-RPC Governance

5. Structured Recovery Instead of Opaque Failure

6. Performance Without Unverifiable Claims

7. Embedded SDK vs. Infrastructure Boundary

8. Selecting the Right Tool

Choose a structured-output or conversational guardrail when:

Choose a conventional AI gateway when:

Choose PolicyAware when:

Use them together when:

9. Shift Governance Left

10. Current Boundaries and Limitations

11. Production Evaluation Checklist

Conclusion

Overcoming the Performance Tax in AI Safety: Engineering a Low-Latency Control Plane for Autonomous Agents

Abstract

1. Why Text Guardrails Are Not Enough

2. Text Validation vs. an AI Control Plane

3. The Runtime Architecture

4. Quick Start: Govern a Raw Model Call

5. Governing MCP Tool Calls Before Execution

6. LangGraph-Style Node and Tool Governance

7. Performance Engineering Without Benchmark Theater

8. Choose the Right Enforcement Boundary

9. Shift Governance Left with Repository Scanning

10. Production Checklist

11. What PolicyAware Is—and Is Not

12. Conclusion

Stopping Runaway AI Loops: Implementing Enterprise FinOps and Observability with PolicyAware

1. The Recursive Agent Crisis

2. Pre-Deployment Auditing with PolicyAware

3. Runtime Cost Controls

4. Enterprise Observability Traces

Securing Autonomous AI Agents: Why PolicyAware is the Essential Control Plane for MCP Tool Execution

1. The Shift to Agentic AI

2. The Action-Level Vulnerability

3. PolicyAware as the Shield: Zero-Trust Agent Control Plane

4. Policy-as-Code Example: Deny-by-Default Topology

Scan Your AI App for PII, Secrets, Tool Risks, RAG Gaps, and LLM Governance Issues with PolicyAware

Introduction: AI features ship fast, governance arrives late

What policyaware scan actually does

Governance areas it checks

Installation

Quick start: scan a local folder and open the HTML report

Multiple report formats for humans, CI, and automation

CI usage: fail builds on high-risk AI governance gaps

Example findings PolicyAware Scan can flag

Why the HTML report matters for developers and reviewers

Python API: use LocalCodeScanner directly

Example scan config YAML

Where PolicyAware Scan fits relative to SAST and security scanners

Links and further reading

Closing

PolicyAware vs Guardrails vs AI Gateways vs Model Routers: The Comparison Every AI Engineer Needs to Read

The landscape right now

Side-by-side comparison

When each tool is the right call

How the architecture fits together

A real example: the $500 refund prompt

Code: policy-first middleware

Code: compliant model routing

Code: audit trace

Start using PolicyAware today

The bottom line

Build a Policy-Aware AI Gateway in Python: Data Protection + Policy Enforcement with policyaware

Quick Install

Part 1 — Data Protection

What the engine detects

Inspecting a prompt

DataFindings field reference

Part 2 — Policy Enforcement

The four decision outcomes

Writing your first policy YAML

Enforcing the policy at runtime

PolicyDecision field reference

What `policyaware scan` actually does