<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: sandhil kypa</title>
    <description>The latest articles on DEV Community by sandhil kypa (@sandhil_kypa_b7cd7c3d5f39).</description>
    <link>https://dev.to/sandhil_kypa_b7cd7c3d5f39</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4257097%2F6f61f3a0-b48c-49f6-9f84-40a5bb96127a.png</url>
      <title>DEV Community: sandhil kypa</title>
      <link>https://dev.to/sandhil_kypa_b7cd7c3d5f39</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/sandhil_kypa_b7cd7c3d5f39"/>
    <language>en</language>
    <item>
      <title>I built an open-source AI gateway for Kubernetes: budgets, kill switch and evals in CI</title>
      <dc:creator>sandhil kypa</dc:creator>
      <pubDate>Sat, 21 Nov 2026 10:09:19 +0000</pubDate>
      <link>https://dev.to/sandhil_kypa_b7cd7c3d5f39/i-built-an-open-source-ai-gateway-for-kubernetes-budgets-kill-switch-and-evals-in-ci-5fhf</link>
      <guid>https://dev.to/sandhil_kypa_b7cd7c3d5f39/i-built-an-open-source-ai-gateway-for-kubernetes-budgets-kill-switch-and-evals-in-ci-5fhf</guid>
      <description>&lt;p&gt;Every team wants AI. Then three questions arrive: What does it cost? Where are the keys? Can we stop it?&lt;/p&gt;

&lt;p&gt;I'm a platform engineer, and I built SanVi to answer all three on Kubernetes.&lt;/p&gt;

&lt;h2&gt;
  
  
  The gateway
&lt;/h2&gt;

&lt;p&gt;Apps call one OpenAI-compatible endpoint (LiteLLM engine) with their own virtual key. Each key has its own models, budget and rate limit. No key → 401. Over the rate limit → 429. Over budget → blocked. Every call is logged with team, model, tokens and cost in Postgres.&lt;/p&gt;

&lt;h2&gt;
  
  
  Keys never in code
&lt;/h2&gt;

&lt;p&gt;The provider key lives only in Vault and syncs in through External Secrets. I rotated it: about 70 seconds, no redeploy.&lt;/p&gt;

&lt;h2&gt;
  
  
  The kill switch
&lt;/h2&gt;

&lt;p&gt;Leaked key, runaway agent loop, spend spike: &lt;code&gt;kill_switch.sh block &amp;lt;team&amp;gt;&lt;/code&gt; returns 401 in seconds, and &lt;code&gt;unblock&lt;/code&gt; restores it. Nothing is deleted.&lt;/p&gt;

&lt;h2&gt;
  
  
  AI access as code
&lt;/h2&gt;

&lt;p&gt;Teams, models and budgets live in &lt;code&gt;access/teams.yaml&lt;/code&gt;. Changes go by pull request; CI checks policy (a greedy $50 budget was blocked); Argo CD applies on merge.&lt;/p&gt;

&lt;h2&gt;
  
  
  Evals decide, not opinions
&lt;/h2&gt;

&lt;p&gt;The Right-Size Advisor runs the same eval suite on each model. On my workload Haiku matched Sonnet: 95% cheaper, 40% faster. Any change that drops below 80% fails CI.&lt;/p&gt;

&lt;h2&gt;
  
  
  Human in the loop
&lt;/h2&gt;

&lt;p&gt;The deployment agent turns one sentence into a validated Dockerfile and Helm chart, but it can only open a pull request.&lt;/p&gt;

&lt;h2&gt;
  
  
  Audit-ready
&lt;/h2&gt;

&lt;p&gt;A compliance script checks 12 controls (no secrets in Git, least-privilege access, audit trail, kill switch, quality gate...) mapped to ISO 27001, SOC 2 and EU AI Act themes, on every change.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try it
&lt;/h2&gt;

&lt;p&gt;Repo, Quickstart and Helm chart: &lt;a href="https://github.com/sandhilk/sanvi-gateway" rel="noopener noreferrer"&gt;https://github.com/sandhilk/sanvi-gateway&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;It's a personal project, so blunt feedback is welcome. What would you need before running this in your company?&lt;br&gt;
&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fs74ni75bs1psektaprsl.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fs74ni75bs1psektaprsl.png" alt=" " width="799" height="316"&gt;&lt;/a&gt;&lt;br&gt;
&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fgxiny56mcjx0colqbjkt.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fgxiny56mcjx0colqbjkt.png" alt=" " width="799" height="316"&gt;&lt;/a&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>kubernetes</category>
      <category>devops</category>
      <category>opensource</category>
    </item>
    <item>
      <title>Every team wants AI. Then three questions arrive: What does it cost? Where are the keys? Can we stop it?

I'm a platform engineer, and I built SanVi to answer all three on Kubernetes.

## The gateway
Apps call one OpenAI-compatible endpoint (LiteLLM engine</title>
      <dc:creator>sandhil kypa</dc:creator>
      <pubDate>Sat, 21 Nov 2026 10:05:43 +0000</pubDate>
      <link>https://dev.to/sandhil_kypa_b7cd7c3d5f39/every-team-wants-ai-then-three-questions-arrive-what-does-it-cost-where-are-the-keys-can-we-4682</link>
      <guid>https://dev.to/sandhil_kypa_b7cd7c3d5f39/every-team-wants-ai-then-three-questions-arrive-what-does-it-cost-where-are-the-keys-can-we-4682</guid>
      <description></description>
      <category>ai</category>
      <category>devops</category>
      <category>infrastructure</category>
      <category>kubernetes</category>
    </item>
  </channel>
</rss>
