Guvna
Most LLM gateways sit at two extremes: TypeScript services eating hundreds of MB
of RAM to forward HTTP, or enterprise platforms that need a database, a dashboard,
and an ops budget. For a single developer using free-tier providers, both are wrong-sized.
This post introduces Guvna — a ~16MB static
Go binary that gives you one OpenAI-compatible endpoint with automatic provider failover,
running in 15–40MB of RAM.
The problem: free tiers fail constantly
Rate limits, expired keys, provider outages — usually mid-stream.
Your app shouldn't know or care. Tried deploying 9router and omniroute on my vps but the amount of ram they took felt so overwhelming. So did my research but the enterprise built new-route didn't click in for me either. So why not just make my own?
Chains: failover as configuration
Show the curl from the README creating myfree with groq → mistral steps,
then the chat call. Explain: clients call chain names; real model names never
leak into your apps; failed providers cool off 60s→10min and recover on their own.
Key pools: one dead key never burns the pool
Rotation strategies (round-robin / least-used / sequential) + class-based quarantine.
Auth failures bench a key hard; transient failures get retried later.
When you don't need a chain
Provider-prefixed passthrough: "model": "groq/llama-3.3-70b-versatile".
Running it
Docker Compose quickstart + go install alternative + env-only secrets, YAML config-as-code.
What it deliberately isn't
Single-user. No multi-tenancy, no web dashboard, no database. That's the feature —
it's why the binary is 16MB. If you need team features, LiteLLM et al are great. But in the mean while I'm having ideas that they all those functionality can be developed on top of the lean smart core, maybe as a plugin or even shipped inside lazy-loaded only on demand!
Try it / feedback
Repo link, issues welcome, README quickstart is 4 commands.
Top comments (0)