DEV Community

Lakshmi Sravya Vedantham
Lakshmi Sravya Vedantham

Posted on

My AI Agent Leaked an API Key, Burned $47, and Looped 200 Times — So I Built It a Bodyguard

#ai #rust #security #opensource

Here is what happens when you deploy an AI agent without safety rails:

  • It sends the same prompt 200 times in a loop
  • It leaks your API key inside a prompt to another LLM
  • It burns through $47 before you notice
  • It hits 5xx errors and keeps retrying into an error spiral

I have seen all of these. So I built llm-guard — a configurable safety proxy that catches these before they cause damage.

What is llm-guard?

A single Rust binary that sits between your code and any LLM API. It checks every request against configurable rules and either blocks or warns.

Your code / agent
        |
   http://localhost:4002
        |
    ┌──────────┐
    │ llm-guard │  ← checks rules before forwarding
    └─────┬────┘
          |
    LLM API
Enter fullscreen mode Exit fullscreen mode

Zero code changes. Swap one environment variable:

export OPENAI_BASE_URL=http://localhost:4002/v1
Enter fullscreen mode Exit fullscreen mode

6 Safety Rules, Each Configurable

Rule Detects Default
loop_detector Same prompt sent 3+ times in a session block
cost_limiter Session spend exceeds threshold block
error_spiral 3+ consecutive errors (5xx/4xx) block
sensitive_data API keys, emails, SSNs, credit cards in prompts block
token_bomb Single request exceeds token limit warn
slow_response Response latency exceeds threshold warn

Each rule can be set to block, warn, or off independently.

Quick Start 

git clone https://github.com/LakshmiSravyaVedantham/llm-guard.git
cd llm-guard
cargo build --release
cp config.example.toml config.toml
./target/release/llm-guard start
Enter fullscreen mode Exit fullscreen mode

What Gets Blocked

When a rule fires with action = "block", llm-guard returns HTTP 403:

{
  "error": {
    "message": "llm-guard: Request blocked by loop_detector -- same prompt sent 4 times",
    "type": "guard_block",
    "rule": "loop_detector"
  }
}
Enter fullscreen mode Exit fullscreen mode

The request never reaches the LLM. Your agent gets a clear, structured error it can handle gracefully.

Configure Per Rule 

[[rules]]
name = "loop_detector"
action = "block"
max_repeats = 3

[[rules]]
name = "sensitive_data"
action = "block"
patterns = ["api_key", "email", "ssn", "credit_card"]

[[rules]]
name = "cost_limiter"
action = "block"
session_limit_usd = 1.00

[[rules]]
name = "token_bomb"
action = "warn"    # just log, do not block
max_tokens = 50000
Enter fullscreen mode Exit fullscreen mode

Set any rule to "off" to disable it entirely.

Sensitive Data Detection

llm-guard scans every outgoing prompt for:

  • API keys — OpenAI (sk-), AWS (AKIA), GitHub (ghp_), Slack tokens
  • Email addresses
  • Social Security Numbers
  • Credit card numbers

All pattern matching happens locally with regex. Nothing leaves your machine.

Live Dashboard

Run llm-guard dash for real-time event monitoring:

┌─ llm-guard dashboard (q to quit) ──────────────────┐
│ Total events: 15   Blocked: 3    Warnings: 12       │
└─────────────────────────────────────────────────────┘
┌─ Recent Events ─────────────────────────────────────┐
│ 14:23:01  BLOCK  loop_detector   Repeated prompt    │
│ 14:22:45  WARN   slow_response   8.2s latency       │
│ 14:20:12  BLOCK  sensitive_data  API key in prompt   │
│ 14:18:30  WARN   cost_limiter    Session at $0.82    │
└─────────────────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

The Full Stack

llm-guard is the third tool in a trilogy. Each is standalone, but they chain together:

Your code -> llm-guard:4002 -> llm-lens:4001 -> llmux:4000 -> Provider
                  |                  |                |
            safety rules      records traces    caches + failover
Enter fullscreen mode Exit fullscreen mode

Three binaries, zero code changes, complete AI agent infrastructure.

Tool Purpose Repo
llmux Gateway: failover, caching, cost tracking Done
llm-lens Flight recorder: session replay Done
llm-guard Safety proxy: rule-based protection This post

Why Rust?

  • Sub-millisecond rule evaluation overhead
  • Single binary, no runtime dependencies
  • Thread-safe concurrent request handling
  • Regex matching without external services
  • ~5MB binary, ~15MB RAM

Try It 

git clone https://github.com/LakshmiSravyaVedantham/llm-guard.git
cd llm-guard && cargo build --release
Enter fullscreen mode Exit fullscreen mode

Star it if useful: github.com/LakshmiSravyaVedantham/llm-guard

llm-guard is MIT licensed. Built with Rust, axum, tokio, ratatui, regex, and rusqlite.

Top comments (0)