Two days ago, I open-sourced Kilovolt—a zero-copy Rust reverse proxy designed to act as a strict financial circuit breaker for OpenAI endpoints. The community feedback was phenomenal, but one recurring theme immediately surfaced: we need granular visibility.
If you are running a multi-agent system, a global budget cap protects your wallet, but it leaves you flying blind. You need to know exactly which agent is burning tokens and how much it is costing you in real-time.
So, I executed a midnight sprint to ship v1.2.0.
📊 The Single Pane of Glass
I didn't want to break the "drop-in replacement" ethos of the project. Forcing users to spin up a secondary container just for a React frontend was a non-starter.
Instead, I embedded a lightweight HTML/Tailwind dashboard directly into the Rust binary. It is served natively on the /dashboard route with absolutely zero external dependencies.
Here is what the telemetry matrix looks like now:
⚙️ The Engineering Telemetry
By decoupling the main.rs monolith into a highly modular architecture, we implemented strict financial auditing without sacrificing our bare-metal performance:
- Granular Cost Auditing: The system now tracks individual Request IDs, token burns, and calculates the exact USD cost per transaction right before your eyes.
- The 15MB Flex: Even while polling and serving this live frontend matrix, the Rust container is maintaining a steady memory footprint of ~15.00 MB.
- Sub-Millisecond Overhead: The p99 routing latency remains entirely negligible.
If you want to protect your runway and get total observability over your LLM pipeline, the multi-arch container is live.
docker pull yodsarun/kilovolt-proxy:v1.2.0
Top comments (0)