<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: vatandeepai</title>
    <description>The latest articles on DEV Community by vatandeepai (@vatandeepai).</description>
    <link>https://dev.to/vatandeepai</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4200077%2Fa863cd13-4521-4525-9425-e8eba2eb738d.png</url>
      <title>DEV Community: vatandeepai</title>
      <link>https://dev.to/vatandeepai</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/vatandeepai"/>
    <language>en</language>
    <item>
      <title>AI Guardrails, Explained: Protecting Inputs and Outputs</title>
      <dc:creator>vatandeepai</dc:creator>
      <pubDate>Mon, 16 Nov 2026 05:32:55 +0000</pubDate>
      <link>https://dev.to/vatandeepai/ai-guardrails-explained-protecting-inputs-and-outputs-21h1</link>
      <guid>https://dev.to/vatandeepai/ai-guardrails-explained-protecting-inputs-and-outputs-21h1</guid>
      <description>&lt;p&gt;A language model will do almost anything you ask it to. That flexibility is the reason the technology is useful, and it is also the reason it is risky to put in front of real users with real data. A model has no built-in sense of which inputs are dangerous, which outputs would leak a secret, or which requests cross a policy line your organization cannot cross. Left to its own devices it will happily echo a social security number, repeat an injected instruction, or generate content your compliance team would never approve. Guardrails are the layer that supplies the judgment the model lacks.&lt;/p&gt;

&lt;p&gt;A guardrail is a programmable check that sits between your application and the model and decides whether a given piece of text is allowed to pass, needs to be changed, or must be blocked. The most effective place to run these checks is the &lt;a href="https://www.truefoundry.com/ai-gateway" rel="noopener noreferrer"&gt;AI gateway guardrails&lt;/a&gt; layer that already intercepts every request, because a control that lives there applies consistently to every application without each team having to reimplement it. This article explains how guardrails work, where they run, and how to deploy them without breaking the experiences your users depend on.&lt;/p&gt;

&lt;h3&gt;
  
  
  Two sides of every conversation
&lt;/h3&gt;

&lt;p&gt;The single most important idea in guardrails is that a model interaction has two sides, and each needs its own protection. There is the input, the text your application sends to the model, and the output, the text the model sends back. These carry different risks and call for different checks.&lt;/p&gt;

&lt;p&gt;Input guardrails run before the model sees the request. Their job is to catch problems in what the user or application is about to send: a prompt-injection attempt, a block of personal data that should never leave your boundary, a request whose content violates policy. Catching these on the way in means the model never processes the dangerous material in the first place.&lt;/p&gt;

&lt;p&gt;Output guardrails run after the model has produced a response but before that response reaches the user. Their job is to inspect what the model generated: a leaked credential, a piece of personal data the model reconstructed, unsafe code, content that fails a moderation standard. The model can produce problems even from a clean input, which is why inspecting the output is not optional.&lt;/p&gt;

&lt;p&gt;When models call external tools, as agentic systems do through protocols like MCP, the same logic extends further. Checks can run pre-tool, before a tool is invoked, and post-tool, on what the tool returns, so that the guardrail boundary wraps not just the model but every action it takes.&lt;/p&gt;

&lt;h3&gt;
  
  
  Validate or mutate
&lt;/h3&gt;

&lt;p&gt;When a guardrail finds something, it can respond in one of two modes, and the distinction shapes the user experience.&lt;/p&gt;

&lt;p&gt;The first mode is validate. A validating guardrail makes a pass-or-fail judgment: the text is acceptable, or it is not. If it fails, the request is blocked. This is the right behavior for checks where any violation is unacceptable, such as a hard policy boundary.&lt;/p&gt;

&lt;p&gt;The second mode is mutate. A mutating guardrail does not simply reject; it rewrites. The canonical example is PII redaction, where the guardrail finds personal data in the text and replaces it with a placeholder, allowing a sanitized version to continue. Mutation is powerful because it keeps the interaction alive instead of failing it, but a mutating guardrail can still block when the content is beyond repair. The choice between validate and mutate is really a choice about whether a problem can be fixed in place or must stop the request.&lt;/p&gt;

&lt;h3&gt;
  
  
  How strictly to enforce
&lt;/h3&gt;

&lt;p&gt;Knowing what to check is only half the design. The other half is deciding what happens operationally when a check triggers, and this is where many deployments go wrong by being too aggressive too early. A mature guardrail system offers graded enforcement levels.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Audit&lt;/strong&gt; runs the guardrail and records what it would have done, without affecting the request. Nothing is blocked or changed; you simply collect data on how often the guardrail fires and whether its judgments look right.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Enforce But Ignore On Error&lt;/strong&gt; applies the guardrail's decision when the check runs cleanly, but if the guardrail itself errors, for example a dependency times out, the request is allowed through rather than failed. This protects you from a guardrail outage taking down your product.&lt;/li&gt;
&lt;li&gt;*&lt;em&gt;Enforce *&lt;/em&gt; applies the decision strictly, including failing closed when the guardrail cannot run.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The recommended path is a staged rollout: begin in Audit to understand behavior and tune for false positives, move to Enforce But Ignore On Error once you trust the judgments but want resilience, and graduate to full Enforce for the checks that must never be bypassed. This progression lets you deploy protection without the painful surprise of a new guardrail blocking legitimate traffic on day one.&lt;/p&gt;

&lt;h3&gt;
  
  
  A streaming caveat worth knowing
&lt;/h3&gt;

&lt;p&gt;There is one technical detail that trips up teams the first time they combine guardrails with streaming responses. When a model streams its answer token by token, with stream: true, the full output does not exist as a single object until the stream completes, so output guardrails are skipped for streaming responses. Input guardrails always run regardless, because the input is fully available before the model is called. If you need output checks to apply, you set stream: false so the complete response is available to inspect before it is returned. Understanding this trade-off between streaming responsiveness and output inspection is essential to designing a safe experience.&lt;/p&gt;

&lt;h3&gt;
  
  
  Applying guardrails: policies and headers
&lt;/h3&gt;

&lt;p&gt;There are two complementary ways to decide which guardrails apply to a given request. The first is centralized: gateway policies that match on attributes of the request, such as the calling application, the target model, or the tool being invoked, and apply the appropriate guardrails automatically. This is how a security team enforces a baseline across the whole organization without depending on individual developers to opt in.&lt;/p&gt;

&lt;p&gt;The second is per-request, using a header that specifies the guardrails for a single call. This gives application developers fine-grained control where they need it. A header-based configuration names the input and output guardrails to run:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;{&lt;br&gt;
  "llm_input_guardrails": ["my-group/pii-redaction"],&lt;br&gt;
  "llm_output_guardrails": ["my-group/secrets-detection"]&lt;br&gt;
}&lt;br&gt;
Passed on an actual request, that looks like the following, where a scope header controls how many of the most recent messages the guardrail examines:&lt;br&gt;
curl -X POST "{GATEWAY_BASE_URL}/chat/completions" \&lt;br&gt;
  -H "Authorization: Bearer YOUR_API_KEY" \&lt;br&gt;
  -H "Content-Type: application/json" \&lt;br&gt;
  -H 'X-TFY-GUARDRAILS: {"llm_input_guardrails":["my-group/pii-redaction"]}' \&lt;br&gt;
  -H "X-TFY-GUARDRAILS-SCOPE: 10" \&lt;br&gt;
  -d '{"model":"openai/gpt-4o","messages":[{"role":"user","content":"Hello"}]}'&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The scope value controls which messages are inspected: all of them, only the last, or a specific number of recent ones. A useful default is that system prompts are excluded from guardrail inspection, so your own carefully written instructions are not flagged as if they were untrusted user input.&lt;/p&gt;

&lt;h3&gt;
  
  
  Why the gateway is the right home for guardrails
&lt;/h3&gt;

&lt;p&gt;A guardrail is only as valuable as the consistency with which it is applied. If each application team implements its own PII check, the organization ends up with a patchwork: slightly different rules, different coverage, and inevitable gaps where a team simply forgot. A new service that no one remembered to protect becomes the weak link, and an attacker needs only one. Running guardrails in the gateway solves this by making protection a property of the shared infrastructure rather than of each application's discipline. Every request that routes through the gateway inherits the baseline, and a security team can update a policy in one place and have it take effect everywhere at once.&lt;/p&gt;

&lt;p&gt;This centralization also separates concerns cleanly. Application developers focus on building features, while the people responsible for safety and compliance own the guardrail policies. Neither has to depend on the other remembering to do the right thing on every request, which is exactly the kind of fragile coordination that security boundaries are meant to eliminate.&lt;/p&gt;

&lt;h3&gt;
  
  
  What guardrails can check
&lt;/h3&gt;

&lt;p&gt;The checks themselves cover a broad range of risks. Content Moderation screens for unsafe or policy-violating material. PII and PHI Detection finds personal and health data. Prompt Injection detection looks for attempts to hijack the model's instructions; these three more computationally involved checks are often available in a hosted form. Secrets Detection catches credentials and keys. A Code Safety Linter inspects generated code, and a SQL Sanitizer guards against dangerous database statements. Regex Pattern Matching lets you enforce your own custom patterns. For agentic systems, policy engines such as Cedar, which can default to denying MCP tool calls unless explicitly allowed, and OPA provide fine-grained authorization over what tools an agent may use. Beyond these built-ins, a gateway typically integrates external guardrail providers and supports fully custom checks written in Python or built with frameworks like Guardrails AI, so you are never limited to a fixed menu.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Key takeaways&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Guardrails supply the judgment a model lacks, deciding whether text may pass, must be rewritten, or must be blocked.&lt;/li&gt;
&lt;li&gt;Every interaction has two sides: input guardrails run before the model and output guardrails run after, with pre-tool and post-tool checks extending the boundary to agentic tool calls.&lt;/li&gt;
&lt;li&gt;Validate mode blocks on failure; mutate mode rewrites, as in PII redaction, and can still block when content cannot be repaired.&lt;/li&gt;
&lt;li&gt;Stage enforcement from Audit to Enforce But Ignore On Error to Enforce so new guardrails never surprise production traffic.&lt;/li&gt;
&lt;li&gt;Output guardrails are skipped for streaming responses unless you set stream to false; input guardrails always run, and system prompts are excluded by default.&lt;/li&gt;
&lt;li&gt;Apply guardrails centrally through gateway policies or per request through headers, drawing on built-in, external, and custom checks.&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>ai</category>
      <category>llm</category>
      <category>security</category>
    </item>
    <item>
      <title>Best Semantic Caching Solutions for a Governed LLM Gateway in 2026</title>
      <dc:creator>vatandeepai</dc:creator>
      <pubDate>Thu, 22 Oct 2026 10:58:28 +0000</pubDate>
      <link>https://dev.to/vatandeepai/best-semantic-caching-solutions-for-a-governed-llm-gateway-in-2026-29li</link>
      <guid>https://dev.to/vatandeepai/best-semantic-caching-solutions-for-a-governed-llm-gateway-in-2026-29li</guid>
      <description>&lt;p&gt;Caching is easy to add and hard to govern. A standalone semantic cache will happily shave tokens off your bill, but once it sits in front of production traffic it inherits every hard question your platform already faces: who is allowed to hit it, whose budget a served response counts against, whether a cached answer leaks sensitive data, and how you prove any of it to an auditor. That is why the most durable place for a cache in 2026 is inside a governed gateway, where caching, routing, and policy are enforced together rather than layered on afterward. Bolting a cache onto an ungoverned stack tends to create blind spots exactly where regulators and finance teams look first. The solutions below all offer caching, but they differ sharply in how much governance comes with it, and in how much of that governance you have to assemble yourself.&lt;/p&gt;

&lt;h2&gt;
  
  
  What to look for in a governed caching gateway
&lt;/h2&gt;

&lt;p&gt;Begin with the policy model, not the cache. Can you cap spend and enforce it in the request path per team, app, and user, or do budgets only surface after the money is gone? Can cache behavior be scoped per route, with freshness and similarity thresholds under your control? Then check the safety path: a cache should respect the same guardrails for PII, secrets, and prompt injection as a live call. Finally, demand unified observability, so cost, latency, and hit rates are visible by model, team, and user, and every decision leaves an audit trail.&lt;/p&gt;

&lt;h2&gt;
  
  
  The best governed caching solutions in 2026
&lt;/h2&gt;

&lt;h3&gt;
  
  
  TrueFoundry
&lt;/h3&gt;

&lt;p&gt;TrueFoundry starts from governance and treats caching as something enforced within it. Response and semantic caching run at the gateway, but they sit beside per-team, per-app, and per-user budgets that are applied in the request path, not reconciled after spend. The same layer carries guardrails with no external credentials, including PII and PHI detection, secrets detection, content moderation, and prompt-injection checks, plus Cedar and OPA policies, so a cached response is subject to the same rules as a fresh one. Observability is unified across cost, latency, and errors by model, team, and user, with full request tracing and no instrumentation required. Architecturally, a central control plane holds routing and config while a regional gateway plane sits alone in the request path, which is how TrueFoundry keeps caching, routing, and policy coherent as a single system. You can see how it fits together on the &lt;a href="https://www.truefoundry.com/" rel="noopener noreferrer"&gt;LLM gateway&lt;/a&gt;. The trade-off is that it is a platform commitment, heavier than teams wanting only a cache library will want.&lt;/p&gt;

&lt;h3&gt;
  
  
  Portkey
&lt;/h3&gt;

&lt;p&gt;Portkey is a well-rounded gateway that bundles caching with routing and guardrails, and it is a reasonable fit for teams wanting governance without building their own proxy. Its caching is dependable and the gateway abstractions are clean. The limitation is operational: reaching a tightly enforced, multi-team policy posture still takes meaningful configuration and ongoing management on your side, so governance is available but not quite turnkey at scale.&lt;/p&gt;

&lt;h3&gt;
  
  
  Martian
&lt;/h3&gt;

&lt;p&gt;Martian focuses on intelligent model routing, directing each request to the model best suited to it, which can meaningfully lower cost and improve quality. For teams whose primary pain is picking the right model per request, that specialization is valuable. The limitation is breadth: it is a router first, so comprehensive caching, budget enforcement, and the full guardrail and audit surface of a governance platform are not its center of gravity, and you may need to supplement it.&lt;/p&gt;

&lt;h3&gt;
  
  
  LiteLLM
&lt;/h3&gt;

&lt;p&gt;LiteLLM is a popular open-source proxy that unifies many providers behind one API and includes caching and basic budget features. It is flexible, transparent, and free to run, which makes it a favorite for teams that want control and are comfortable operating infrastructure. That ownership is the cost: production-grade governance, deep observability, advanced guardrails, and high-availability operations are yours to build and maintain, which is real engineering effort over time.&lt;/p&gt;

&lt;h2&gt;
  
  
  The verdict
&lt;/h2&gt;

&lt;p&gt;If you only need caching, a library will do. But the question this list answers is what to use when caching has to coexist with routing and governance, and there the answers diverge. Martian is the specialist router, LiteLLM the flexible open-source proxy you operate yourself, and Portkey the capable gateway that still asks you to configure policy carefully. TrueFoundry is the strongest fit when governance is the starting point rather than an afterthought, because budgets, guardrails, observability, and semantic caching are enforced together in one control plane. For regulated or multi-team environments, that integration is what keeps a cache from quietly becoming a compliance gap.&lt;/p&gt;

</description>
    </item>
  </channel>
</rss>
