Why Your Coding Agent Doesn't Need an Electron App: A Minimalist LLM Proxy in Go

TL;DR (the bottom line, up front)

Your coding agent (Claude Code, Cursor, Codex, ...) doesn't need a dashboard. It needs a transparent, self-healing, low-overhead pipe between its SDK and a pool of LLM providers — because at 3 AM, when the agent is running unattended, nobody is there to click a button.

That's why we built VMR: a local, single-binary LLM router in Go.

• ~31K lines of Go, 4 direct dependencies, ships as one static binary (~12–15MB)
• Zero database, zero web UI, zero Node/Python runtime
• Byte-faithful pass-through: requests arrive upstream byte-identical to a direct connection
• Session-sticky cache affinity: keeps long conversations on the same endpoint, protecting upstream Prompt Cache (we observe 85–95% hit rates, ~50–70% cost reduction on cache discounts — self-measured, not third-party audited)
• From our benchmark (Apple M1): routing overhead +0.3–0.9ms p50; RSS 15MB; cold start ~60ms

This post is the design reasoning behind the minimalist bet. No hype, just the decision chain.

1. The problem: agent traffic is server traffic, not desktop traffic

Two nights pushed us here.

Night one was the bill. After wiring Claude Code / Cursor to multiple providers (Anthropic, DeepSeek, MiniMax, OpenRouter...), I started with the naive approach — hardcode different Base URLs per provider. It fell apart fast. Agent sessions are long-context, high-frequency round-trips. The moment a subagent switch slightly perturbs the request body, the provider's Prompt Cache key misses, and the cost climbs visibly. I was bouncing between "one provider's bill" and "another provider's queue time."

Night two was the outage. A batch refactor was supposed to run for ~an hour unattended. An upstream key hit 429, the proxy wedged, and by morning the task had died with progress lost.

Both nights pointed the same direction: the proxy layer has to be designed for unattended operation — self-healing, self-retrying, and always leaving evidence — not for human dashboard-watching.

2. What we looked at (and why we walked away)

I'm not going to pretend we didn't consider "fuller" solutions. We did, and they're genuinely good at what they do:

• LiteLLM-class translation gateways: huge coverage, but also huge complexity. The "all protocols → internal format → target protocol" translation layer is something you maintain forever as upstream event formats evolve. For a solo maintainer, that's a real tax.
• Electron/desktop control planes (e.g. Claude Code Router): deep, well-made — agent profiles, tool markets, Web UI. For our use case it was a drinking fountain built to get a glass of water. We wanted the invisible, non-invasive layer only.
• Hardcoded Base URLs + manual switching: simplest, but no failover, no health probing, no audit trail. At 3 AM nobody is switching anything manually.

They serve different audiences. The question we kept asking, which became VMR's standing review rule:

Every new feature, ask: does this add capability, or does it add complexity?

By that test, dashboards, user management, billing, prompt management, plugin systems, and MCP frameworks all got rejected at the door. Useful? Yes. Complexity-adders for the "be a transparent base for coding agents" mission? Also yes.

3. Three constraints that pushed us to Go + a single binary

Not "Go for the sake of Go." Three constraints stacked:

1. Unattended = server-traffic semantics. High concurrency, long sessions, zero supervision. The proxy must be low-latency, high-throughput, and self-healing. This workload is closer to a reverse proxy / load balancer than a desktop GUI. Go's goroutine model, its server-grade GC behavior, and single-file cross-compilation make it the cheapest fit.

2. Byte-level fidelity keeps you compatible with upstream. LLM providers ship new params and event types fast. If your proxy does JSON.parse → mutate → stringify on every request, any unknown field is theoretically at risk of being reordered, escaped, or dropped. The only way to make "new upstream features work the day they ship, without waiting for a VMR release" is to not touch any byte we don't need to touch. Go's json.RawMessage makes this clean.

3. Solo maintenance = fewer dependencies = safer. Four Go module deps vs a 50+ npm package tree + Electron + React. Supply-chain surface, audit cost, build time — all orders of magnitude apart. For a solo project, "I can write this with only the standard library" is a survival advantage.

4. Byte-faithful pass-through (not "good-enough" pass-through)

"Byte-faithful" is not marketing copy in VMR. Three concrete mechanisms:

// 1) json.RawMessage: keep the whole request body raw; parse only model/stream
type CanonicalRequest struct {
    Model  string
    Stream bool
    Raw    json.RawMessage // every other byte passes through untouched
    Header http.Header
    Facts  RequestFacts
}

// 2) MarshalNoEscape: refuse Go's default HTML escaping.
// Default json.Marshal turns "<div>" into "\u003Cdiv\u003E" — semantically
// equal, byte-different. VMR cares about bytes.
func MarshalNoEscape(v any) ([]byte, error) {
    var buf bytes.Buffer
    enc := json.NewEncoder(&buf)
    enc.SetEscapeHTML(false)
    enc.Encode(v)
    return bytes.TrimSuffix(buf.Bytes(), []byte("\n")), nil
}

// 3) Byte-level model rewrite: scan bytes, replace the "model" key's value.
// No JSON unmarshal, no object round-trip. Key order, whitespace, unknown
// fields stay exactly as the client sent them.
func RewriteModel(body []byte, realModel string) []byte { /* byte-level state machine */ }

VMR never unpacks the request body into an object. json.RawMessage is a zero-copy byte container; the model substitution is pure string manipulation. That's why go.mod stays at four deps:

github.com/fsnotify/fsnotify  → config hot-reload
github.com/klauspost/compress → zstd audit-log compression
golang.org/x/image            → image downscale
gopkg.in/yaml.v3              → config parsing

No ORM, no web framework, no DI container, no logging library. HTTP uses net/http; logging uses log. ~31K lines of Go, one static binary.

5. Why we refuse cross-protocol translation

VMR exposes both an OpenAI and an Anthropic entrance, but routes each within its own protocol family — it never translates between them. This is a stance, not a gap:

• SSE event semantics can't be losslessly mapped. Anthropic's message_start/content_block_delta/tool_use stream vs OpenAI's choices[0].delta.tool_calls[] chunks — every new event type upstream adds is another patch to the translation layer.
• Forward compatibility breaks. New upstream event type → translation layer doesn't know it → dropped or error. Byte pass-through is natively forward-compatible.
• Mainstream providers already expose both faces. MiniMax, DeepSeek, OpenRouter offer OpenAI and Anthropic compatible surfaces. The translation layer no longer creates value.

We could do it — we've tested both protocol faces against real providers. We just think the complexity isn't worth it. (This is also the crux of the difference from LiteLLM-class gateways, where the translation layer is a core feature.)

6. Agent-first, not human-first

Two design decisions that only make sense if the user is an agent, not a person:

• Session-Sticky affinity: keep a long conversation pinned to one physical endpoint so the provider's Prompt Cache stays warm. Observed hit rates 85–95%, ~50–70% cost reduction on cache discounts (self-measured).
• Error-class-aware cooldowns: rate-limit (429) honors Retry-After; auth failure (401) gets a long cooldown; content-filter rejection (403) switches provider without penalizing the endpoint — the provider is healthy, it's just doing its compliance job; 5xx gets exponential backoff. Most gateways treat all failures the same. VMR doesn't.

Plus the flight recorder: per-request JSONL audit (client↔VMR, VMR↔upstream) grouped by Session → Task → Turn — built to answer "which model did this agent session use, for how many tokens, and where did it go off the rails," not "what does this chat look like in a table."

7. Honest tradeoffs

• No dashboard, no web UI. Observability is vmr report / vmr story on the CLI. Deliberate, but it does filter out users who want a graphical view.
• Providers are config, not built-in. Currently validated: MiniMax, DeepSeek, OpenRouter. Adding one is near-zero code (a config block), but you verify compatibility yourself.
• Strategy dimensions are partial. priority works; weight/round_robin are "in progress."
• Two known architectural tradeoffs: image downscaling happens at the server layer (can't be un-done after failover), and the concurrency gate is global, not per-model.
• The real one: the project is young and small (community, iteration velocity are nothing like CCR's). If you pick it, accept you may write your own provider configs and verify compatibility. The upside: 4 deps and 31K lines mean even if development stalled, handover cost is low.

8. Try it

If you're running your own coding agents and want a transparent, self-healing, evidence-keeping data plane between them and your LLM providers, the code is on GitHub: github.com/bigfatsea/vmr. We're actively working on the scheduling dimensions (weight/round_robin) and the vmr story diff-and-replay tooling. Issues and PRs welcome.