Knowing which model openrouter/auto actually ran (and what it cost)

If you route through OpenRouter with openrouter/auto, you've traded a decision for a convenience. The router picks a model per request based on price, latency, and availability, and most of the time it picks well. The catch is that "most of the time" is doing a lot of work in that sentence, and you don't find out which model handled a given request until you go looking.

This matters more than it used to. Agents like OpenClaw and a growing pile of internal tools default to auto precisely because they don't want to hard-code a model. That's reasonable. But it means a single agent loop can fan out across three or four different models in an afternoon, each with its own per-token price, and your only signal is an aggregate bill at the end of the month. When the bill is surprising, "which model, on which request, for which developer" is exactly the breakdown you don't have.

Here's how to get it back without rearchitecting anything.

The data is already there

OpenRouter records the model it selected on every completion, even when you asked for auto. You don't need to intercept traffic to see this. The platform exposes it two ways.

The first is per-request, at generation time. A chat completion response includes a model field with the concrete model that ran — not openrouter/auto, but the actual anthropic/claude-... or google/gemini-... string it resolved to. If you control the call site, log that field alongside the generation id:

resp = client.chat.completions.create(
    model="openrouter/auto",
    messages=messages,
)
print(resp.model)  # e.g. "anthropic/claude-3.5-sonnet", not "openrouter/auto"

The second is after the fact, which is the one that scales. OpenRouter has an activity/usage endpoint that returns historical generations with the resolved model, token counts, and cost per row. Polling that endpoint on a schedule gives you a complete ledger without touching your application code at all:

curl https://openrouter.ai/api/v1/activity \
  -H "Authorization: Bearer $OPENROUTER_API_KEY"

You can also fetch a single generation by id from the generation endpoint if you want to enrich a specific log line. The shape varies, so check the current OpenRouter docs for exact field names before you build against it — but the resolved model and the cost are both in there.

Turning rows into per-model, per-developer truth

Raw generation rows are not an answer. The work is in the grouping, and the grouping is only as good as your key hygiene.

The single most useful thing you can do is give every developer (and every agent) their own OpenRouter API key. Not for security theater — for attribution. If everyone shares one key, the usage endpoint hands you a pile of generations with no way to say who incurred them. One key per identity turns an anonymous stream into rows you can GROUP BY.

With that in place, a daily poll and a few lines of aggregation get you the table you actually want:

from collections import defaultdict

rows = fetch_activity()  # list of generation records
by_model_dev = defaultdict(lambda: {"cost": 0.0, "calls": 0})

for r in rows:
    key = (r["owner"], r["model"])  # owner derived from which API key
    by_model_dev[key]["cost"] += r["usage"]["cost"]
    by_model_dev[key]["calls"] += 1

Now auto stops being a black box. You can see that one developer's agent quietly resolved to an expensive model for 80% of its calls last Tuesday, while everyone else stayed on the cheap tier. That's a finding. You can't act on a number you can't attribute.

Setting a threshold that catches the bad day

A ledger tells you what happened. To catch a runaway agent the same day, you need a tripwire, and the cheapest useful one is a per-developer baseline.

Compute each developer's mean daily spend and standard deviation over a trailing window — two or three weeks is plenty. Then flag any day that crosses mean + 3σ. Three sigma is deliberately boring: it ignores normal variance and only fires when something is genuinely off, which is what you want from an alert you'll actually keep enabled. A retry loop with no backoff, an agent stuck re-reading the same 200KB file, a test harness someone left running over lunch — these don't creep, they spike, and a 3σ band on a per-developer baseline catches them on the day they happen rather than in next month's reconciliation.

Wire that check into your daily poll, and post the exceedances somewhere people look. The whole thing is maybe forty lines of code plus a cron entry.

Why polling beats a proxy here

The instinct is often to put a gateway in front of OpenRouter so you can see everything. For cost visibility specifically, that's more machine than the job needs. A proxy is now in your critical path: it can add latency, it can go down and take your inference with it, and it sees every prompt and completion your developers send — a meaningful expansion of what you're storing and securing. The usage API gives you the same cost and model data after the fact, read-only, with none of that exposure. If the question is "what did we spend and on which model," you don't need to stand in the middle of the conversation to answer it.

That's the whole approach: one key per identity, poll the usage endpoint, group by model and developer, and put a 3σ tripwire on each baseline. It's a quiet afternoon of work and it pays for itself the first time auto does something expensive you didn't expect.

If you'd rather not maintain the poller, the keys, and the alerting yourself, that's what we built Reckon for — read-only usage-API polling (no proxy, no SDK, it never sees your prompts), KMS-encrypted keys, per-developer and now per-model OpenRouter visibility in realtime, Slack digests with same-day anomaly alerts, a /spend command, and a Linear integration. It's free up to three developers and $19 per developer per month on Pro. Either way — build it or don't — the visibility is worth more than the bill it saves you.