The latest articles on DEV Community by Nguyễn Anh Nguyên (@nguyn_anhnguyn_f7d9e52). https://dev.to/nguyn_anhnguyn_f7d9e52 https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F1561505%2F57e30c2e-5de3-4c50-bfe8-90c1f7e96669.png https://dev.to/nguyn_anhnguyn_f7d9e52 en Nguyễn Anh Nguyên Sun, 27 Sep 2026 02:41:02 +0000 https://dev.to/nguyn_anhnguyn_f7d9e52/brighto-router-an-open-source-rust-llm-gateway-for-fast-self-hosted-ai-traffic-ice https://dev.to/nguyn_anhnguyn_f7d9e52/brighto-router-an-open-source-rust-llm-gateway-for-fast-self-hosted-ai-traffic-ice <p>I built BrighTO-Router as a free open-source LLM gateway/router written in Rust.</p> <p>The goal is simple: give a team one self-hosted API endpoint for AI traffic, while keeping the hot path small, fast, and understandable.</p> <h2> What it does </h2> <ul> <li>OpenAI-compatible chat, completions, and embeddings</li> <li>Anthropic Messages routing</li> <li>Local/custom OpenAI-compatible endpoints</li> <li>Rerank adapters</li> <li>ASR/transcription proxying</li> <li>Team API keys and token budgets</li> <li>PostgreSQL usage metadata ledger</li> <li>Round-robin and weighted Model Groups for model load balancing</li> <li>One Docker install</li> </ul> <h2> Why Rust </h2> <p>LLM gateways sit on the request path. The router should add as little latency and operational complexity as possible. Rust gives predictable memory use, fast async networking, and a deployment model that fits one Docker image.</p> <h2> Benchmarks </h2> <p>The project publishes deterministic mock-backend benchmarks up to 1M-token payloads. These measurements report router overhead, not model inference speed, so the results are repeatable and honest.</p> <h2> Links </h2> <p>Repo: <a href="https://github.com/thusinh1969/BrighTO_Router" rel="noopener noreferrer">https://github.com/thusinh1969/BrighTO_Router</a><br> Docs: <a href="https://thusinh1969.github.io/BrighTO_Router/" rel="noopener noreferrer">https://thusinh1969.github.io/BrighTO_Router/</a><br> Docker: <code>thusinh1969/brighto_airouter:v1</code></p> <p><a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F036r3ig8zr4kk1y0j7ge.png" class="article-body-image-wrapper"><img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F036r3ig8zr4kk1y0j7ge.png" alt=" " width="800" height="267"></a></p> backend llm opensource rust