The latest articles on DEV Community by Ayush Dwivedi (@ayushships). https://dev.to/ayushships https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4008460%2Feab9e659-35c2-427e-8eb7-4cf645006d53.png https://dev.to/ayushships en Ayush Dwivedi Mon, 29 Jun 2026 15:54:07 +0000 https://dev.to/ayushships/i-accidentally-built-a-production-llm-router-by-running-it-for-3-months-on-my-own-projects-2kmg https://dev.to/ayushships/i-accidentally-built-a-production-llm-router-by-running-it-for-3-months-on-my-own-projects-2kmg <p>I didn't set out to build an LLM router. Genuinely didn't.</p> <p>I was just tired of paying per-token rates every time I tested something. My own projects were burning through API credits doing nothing special, just running experiments, pipelines, small automations. The math didn't make sense anymore.</p> <p>So I built a proxy layer in front of multiple providers. Kept it private. Used it myself. Didn't think much of it.</p> <p>Three months later I looked at the logs and saw 7 to 8 billion tokens routed through it. That's when I realized this thing was actually production-grade.</p> <h2> What's running through it </h2> <p>The model mix I use day to day: Llama 70B, DeepSeek, Qwen3, MiniMax M3, StepFun 3.7 Flash, Cerebras, and a few others. All open source, no GPT-4o, no Claude. If open source models cover your use case, the pricing difference vs per-token billing is significant.</p> <h2> What I actually learned </h2> <p><strong>Provider reliability is not equal.</strong> Some providers quietly drop requests under load. You won't notice until you're in a late-night run and half your completions are just gone. Failover saved me more times than I expected.</p> <p><strong>Cerebras is genuinely fast.</strong> I added it half expecting it to be just another provider. Ended up making it the default for anything latency-sensitive. The difference is real.</p> <p><strong>Routing logic matters more than model selection.</strong> Just pointing to "the best model" isn't enough. You need fallback chains, timeout handling, and something tracking which providers are actually responding. That's where most of the actual engineering went.</p> <h2> Opening it up now </h2> <p>The product is called Janux. I'm opening it up with a founding price for the first 50 users.</p> <p>Plans:<br> $5/month flat<br> $29/year<br> BYOK (bring your own key) is completely free</p> <p>First month free if you pre-register now.</p> <p>Only open source models. If that works for your stack, the pricing is a lot better than paying per token.</p> <p>janux.studiotrx.in</p> <p>Would genuinely love feedback from anyone building with LLMs seriously. What providers are giving you trouble? What's your current setup?</p> llm opensource showdev webdev