Do Not Add an LLM Router Until You Can Defend Each Downgrade

Cost optimization is not a model-picker problem

When an LLM bill grows, the tempting answer is a blanket model downgrade. That is usually the wrong first move. A support classification, a retrieval rewrite, and a complex planning step do not have the same failure cost.

The useful question is: which calls have evidence that they can be cheaper?

Frugon is an MIT-licensed, Python 3.10+ CLI that analyzes OpenAI-compatible request/response JSONL logs. Its stated purpose is to compare candidate models, estimate costs, and propose a split between calls that might move and calls that should remain on the baseline. See the README, project configuration, and v0.2.4 release.

The useful design choice: analysis before automation

The interesting part is not a headline saving. It is converting a bill into a reviewable hypothesis:

1. Collect a bounded, sanitized sample of real calls.
2. Identify call groups that are candidates for a cheaper model.
3. Run a task-specific regression suite against those groups.
4. Automate routing only for groups that pass.

That sequence is boring in the best way. It creates an artifact an engineering team can challenge, reproduce, and roll back.

A local capture still has security boundaries

Frugon also includes frugon capture. Its capture implementation runs a local HTTP server, forwards OpenAI-compatible completion requests to the configured upstream, and stores canonical JSONL records locally.

This deserves normal production scrutiny. Logs can contain prompts and responses; retention and redaction are still your responsibility. Optional quality measurement uses your own provider keys, so it is a real provider-boundary decision, not a free offline proof. The source also validates upstream schemes and strips sensitive headers on cross-origin redirects—good defensive detail, but not a substitute for your own review.

Where it fits

Frugon looks most useful for teams that already retain compatible call logs and need a defensible downgrade shortlist. It is less useful if you have no representative task set, because a cost estimate cannot prove output quality.

Not tested / not run: this is a public-documentation and source review only. I did not install, execute, benchmark, or validate Frugon in a production workload. The project’s MIT license and public repository are linked for further review.