Stop Juggling 7 LLM API Keys: One BYOK Router That Ranks, Streams & Compacts for You

SentioGate is a production-ready BYOK (Bring Your Own Key) routing layer for LLM APIs. Supply your provider keys with each request, and the router picks the best provider for that task, streams OpenAI-compatible responses back, retries and fails over automatically, and — when sessions get long — compresses context by 85–95% without losing continuity.

One OpenAI-compatible endpoint. No per-token markup. Drop-in SDKs for Python and JavaScript/TypeScript in three lines.

The 3-line drop-in

Python

pip install ai-auto-router-client

from ai_auto_router_client import AutoRouterClient

router = AutoRouterClient(api_key="sg_live_...")
resp = router.chat(
    messages=[{"role": "user", "content": "Explain raft consensus in 2 sentences"}],
    provider_keys={"openai": "sk-...", "anthropic": "sk-ant-..."}
)
print(resp["choices"][0]["message"]["content"])

JavaScript / TypeScript

npm install @hushamdev/ai-auto-router-sdk

import { AutoRouter } from '@hushamdev/ai-auto-router-sdk';

const router = new AutoRouter({ apiKey: process.env.SENTIOGATE_API_KEY });
const res = await router.chat({
  messages: [{ role: 'user', content: 'Explain raft consensus in 2 sentences' }],
  providerKeys: {
    openai: process.env.OPENAI_API_KEY,
    anthropic: process.env.ANTHROPIC_API_KEY
  }
});
console.log(res.choices[0].message.content);

Both SDKs are built purely on the standard library — no httpx, no axios, no heavy runtime. Apache-2.0 licensed.

What it does

1. Multi-provider routing with automatic failover

Route across OpenAI, Anthropic, Google Gemini, Groq, Mistral, xAI, DeepSeek, OpenRouter — and your own self-hosted endpoints — through a single OpenAI-compatible API. The router continuously scores each provider on cost, latency, reliability, capacity and quality, and automatically fails over when a provider rate-limits or errors out. You don't write the retry loop.

2. Hardened transport

• Wall-clock request deadlines so a hung provider can't stall your app
• Transient-error retries with exponential backoff
• Rate-limit-aware capacity scoring read from provider response headers

3. Streaming, OpenAI-style

Pass stream: true and receive standard OpenAI-style SSE chunks plus a final usage/cost event — so your front-end keeps working even when the router picks a different backend mid-stream.

4. Semantic context compression

For long agent / chat sessions, the router detects when your message tree approaches the model's context window and automatically compresses older history into a compact restore point, cutting input tokens by 85–95% while letting a different model pick up the conversation seamlessly.

5. Custom providers — route to any OpenAI-compatible endpoint

Add Ollama, vLLM, LM Studio, or a private LLM appliance at runtime. They participate in the same scoring / failover pipeline as the cloud providers. Your local models, routed intelligently next to your frontier keys.

6. Observability & governance

Prometheus metrics, structured logging, health checks, request tracing, and a built-in encrypted key vault so provider keys stay in scope, in memory, and never on disk.

When to reach for it

• You're running multiple LLM providers and want one stable API for your app
• Your agent / long-chat products are burning through token budgets
• You want local models (Ollama, vLLM) in the same routing pool as the cloud
• You need a BYOK layer so your customers' keys never touch your servers
• You want to route on real cost/latency/health instead of hard-coding a provider

Links

• Python SDK (Apache-2.0): ai-auto-router-client
• JS/TS SDK (Apache-2.0): @hushamdev/ai-auto-router-sdk
• Product & pricing: sentiogate.app · Pricing

If you're running OpenAI + a fallback today, or your token bill is out of control on long agent sessions, the SDKs are two install commands away.

Planning a follow-up on the **semantic compaction algorithm* and rate-limit-aware scoring in detail — comment below and I'll prioritize whichever you want first.*