I built BrighTO-Router as a free open-source LLM gateway/router written in Rust.
The goal is simple: give a team one self-hosted API endpoint for AI traffic, while keeping the hot path small, fast, and understandable.
What it does
- OpenAI-compatible chat, completions, and embeddings
- Anthropic Messages routing
- Local/custom OpenAI-compatible endpoints
- Rerank adapters
- ASR/transcription proxying
- Team API keys and token budgets
- PostgreSQL usage metadata ledger
- Round-robin and weighted Model Groups for model load balancing
- One Docker install
Why Rust
LLM gateways sit on the request path. The router should add as little latency and operational complexity as possible. Rust gives predictable memory use, fast async networking, and a deployment model that fits one Docker image.
Benchmarks
The project publishes deterministic mock-backend benchmarks up to 1M-token payloads. These measurements report router overhead, not model inference speed, so the results are repeatable and honest.
Links
Repo: https://github.com/thusinh1969/BrighTO_Router
Docs: https://thusinh1969.github.io/BrighTO_Router/
Docker: thusinh1969/brighto_airouter:v1
Top comments (0)