DEV Community

karmendra pandey
karmendra pandey

Posted on

Argus: a security layer for every AI model you call

#opensource #python #security #ai

Your app talks to AI models. Who's watching what goes in and out?

Last week I watched an AWS secret key sail through a prompt to a third-party model in a demo. Nobody noticed. That's when I stopped treating AI security as a checklist item and built Argus — an open-source security layer that sits in front of every model you call.

The problem in one picture

Right now, most apps call models like this: app → API key → model. There's nothing in between. So:

  • A developer pastes an AWS key into a prompt. It leaves your network.
  • A user's email and phone number ride along in the conversation history.
  • "Ignore all previous instructions" arrives in user input, and nobody flags it.
  • When something leaks, there's no record of what happened.

Gateways like LiteLLM solve routing and budgets. Nobody owns the security in between. That's the gap Argus fills.

What Argus does

Argus is an OpenAI-compatible gateway. You point your app at it with one base_url change, and every request passes through a security policy — on the way in and on the way out — before any model sees it. OpenAI, Anthropic, Bedrock, vLLM, Ollama, anything with an OpenAI-shaped endpoint.

pip install argus-gateway
argus   # serves on localhost:4000
Enter fullscreen mode Exit fullscreen mode 
import openai
client = openai.OpenAI(api_key="<your-virtual-key>", base_url="http://localhost:4000/v1")
Enter fullscreen mode Exit fullscreen mode

The security machinery

Four pluggable detectors scan every request and response. Detectors find; a policy engine decides — allow, redact, block, or flag:

Detector Catches Default
Secrets AWS keys, private keys, API tokens, password = ... Block — never reaches a model
PII emails, phones, SSNs, credit cards Redact — [REDACTED:EMAIL], request continues
Prompt injection "ignore previous instructions", role hijacks, system-prompt probes, jailbreaks Flag — allowed, but marked
Blocklist your own patterns Block

Every decision lands in an audit log — timestamp, key, detector, action, and a redacted snippet. Raw PII and secrets never touch the trail. That's your compliance story: GET /admin/audit.

Here's what a blocked secret looks like in practice:

POST /v1/chat/completions
{"messages": [{"role": "user", "content": "my aws key is AKIA..."}]}

→ 400 blocked by security policy: secrets:AWS_KEY
Enter fullscreen mode Exit fullscreen mode

And every successful response tells you what the layer did:

"gateway": {
  "security": {"redactions": ["pii:EMAIL"], "flags": []},
  "cost_usd": 0.000004, "tier": "small", "degraded": false
}
Enter fullscreen mode Exit fullscreen mode

Cost intelligence rides along

Security is the product, but the gateway also thinks about money — because the same layer that sees every request is the right place to price it:

  • Tiered routing — requests are scored for difficulty and sent to the cheapest tier that can handle them
  • Graceful budgets — blow your budget and requests fall back to the cheapest tier instead of dying with a 429 (hard-stop mode if you prefer)
  • Per-task attribution — pass X-Task-Id and a whole agentic run's spend is ledgered as one task

This comes from my earlier work on tokenecon and the paper behind it.

Try it, break it

It's early — v0.1.0, MIT licensed, and I'd rather have people poke holes in it now:

On the roadmap: streaming, Bedrock/Gemini/Mistral adapters, semantic caching, OpenTelemetry export, and an ML-based injection detector (the detector interface is pluggable — contributions welcome).

If you've ever wondered what your prompts actually carry out the door — point Argus at your traffic for a day and check the audit log. You might be surprised. I was.

Top comments (0)