DEV Community

Cover image for Stoke: A kill switch for runaway AI agents
Paw from Oz
Paw from Oz

Posted on • Originally published at stokegate.com

Stoke: A kill switch for runaway AI agents

#ai #llm #showdev #rust

I've been building with AI coding agents daily — Claude Code, OpenCode, custom agents. They're productive, but they have a failure mode that costs real money: they retry, loop, and fan out on metered API keys. Billing alerts fire hours after the damage is done.

Observability tools (Langfuse, Helicone, Portkey) tell you what happened. None of them stop it from happening.

Stoke

Stoke is a single Rust binary (~5.5 MB, zero runtime dependencies) that sits between your AI agents and model providers. It enforces policy before any provider is called:

  • Hard budget caps — per-API-key spend limits in USD. Over the limit = 429 Budget exceeded, not an alert. Streamed responses accrue spend too.
  • Loop detection — exact prompt-hash matching plus optional semantic similarity. 5 similar requests from one key in 60 seconds = blocked for 120 seconds. Catches agents stuck in retry loops.
  • Rate limiting — per-key requests-per-minute over a sliding window.

The key insight: enforce, don't just observe.

What else it does

  • Node-aware routing — polls your Ollama nodes, places requests on warm machines, balances by live load, fails over automatically
  • Federation — one Stoke can front another across machines, with a hop guard that prevents cycles
  • Auto-routing — send "model": "auto" and Stoke scores every (model, node) candidate on cost, predicted latency, and your preferences. Every response carries a receipt explaining the decision.
  • Anthropic Messages API — point Claude Code at Stoke with ANTHROPIC_BASE_URL. Same enforcement, then passthrough.
  • Plugin system — built-in PII redaction, audit logging, prompt harness + HTTP webhooks in any language + JS/TS via deno_core
  • Fail-closed auth — no keys configured and no dev flag = every request rejected. There is no accidentally-open state.

Why Rust

Every Python proxy (LiteLLM, OpenRouter) adds latency and memory overhead — 500MB+ RAM, Postgres, Redis. Stoke is 5.5 MB, starts in milliseconds, uses ~10 MB RAM, and has zero runtime dependencies. It's a single binary you can curl-pipe-sh install.

Multi-machine setup

If you have Ollama running on two machines (say a MacBook and a Mac Studio), Stoke routes between them automatically — preferring the machine where the model is already loaded in RAM, balancing by in-flight count, and measuring real time-to-first-token per node.

Or use federation: run a Stoke on each machine, keep Ollama on 127.0.0.1, and front them with a type = "stoke" provider. Stoke is the only network-facing door, and it requires auth.

Quickstart 

curl -sSf https://stokegate.com/install | sh -s -- --version nightly
Enter fullscreen mode Exit fullscreen mode

Minimal config:

routing = "single"
default_model = "qwen3:8b"

[server]
host = "127.0.0.1"
port = 8787

[[providers]]
name = "local"
base_url = "http://127.0.0.1:11434/v1"
tier = "local"
Enter fullscreen mode Exit fullscreen mode

Start it:

STOKE_API_KEYS=stk-mykey ./target/release/stoke
Enter fullscreen mode Exit fullscreen mode

Point any OpenAI-compatible client at http://localhost:8787/v1 — OpenCode, Cursor, custom apps. Budget caps, rate limits, and loop detection apply to every request.

Status

Pre-release, MIT licensed, EU-based. Dogfooded daily on a real two-Mac + Ollama setup. Built in the open at github.com/Ozperium/stoke.

The smoke scripts (./scripts/smoke.sh, ./scripts/smoke_federation.sh) run the full system end-to-end with mock Ollama nodes — no Ollama required to verify the claims.

What's next

  • Anthropic to OpenAI translation (so Claude Code can run against local models)
  • Homebrew formula and Docker images
  • Per-key loop-detection thresholds
  • OpenTelemetry traces and Prometheus metrics

Feedback welcome. The repo is github.com/Ozperium/stoke.

Top comments (0)