Short answer: Use one API key across model providers only behind an attempt ledger that attributes every retry, token count, route, and fallback to the right fintech tenant.
The cheapest route is the one you can explain per tenant after a difficult review, not the one with the lowest advertised token price. For a fintech code-review service using one logical credential across OpenAI, Claude, and Gemini, I would make attribution and a bounded fallback path mandatory before optimizing model selection. A slightly more expensive review that is traceable beats a cheap review that silently lands on the wrong tenant's bill.
I have been woken by alerts that meant nothing and missed the one that mattered, so I distrust a dashboard that reports only aggregate tokens. The operational question is narrower: which tenant caused this request, which provider actually served it, what did the input and output cost, and did a retry charge the tenant twice?
Why the cheapest review is not the cheapest attempt
The bounded production scenario is a pull request containing payment-ledger changes. The service receives a repository identifier, tenant identifier, commit range, and review policy. It selects a model, requests structured findings, streams progress to the caller, and records usage. The caller sees one API surface; the ledger must still preserve the provider, model, attempt, token counts, and pricing snapshot.
The failure is easy to miss. A provider timeout triggers a retry, the retry succeeds, and the final review looks normal. An aggregate chart then shows one completed review while the cost ledger contains two attempts. If the original request had no durable idempotency key, a second worker can repeat the same work. The model was not the only variable. Accounting was part of the request path.
The invariant is simple: every billable attempt gets one immutable usage record linked to a tenant and a review, even when the attempt fails after the provider accepts the request. The final answer may be one review; the cost history may contain several attempts. Those are different facts and should not be collapsed.
The ledger is the incident boundary.
Where a one-key boundary loses evidence
A single application credential can simplify rotation and keep provider credentials out of tenant-facing code. It must not erase provider identity from internal records. The router boundary should normalize request intent and response shape, while preserving the evidence needed to audit the decision.
I first treated the adapter as a formatting layer. That was too optimistic. I use a narrow internal contract instead: a review request, a provider-neutral finding schema, an attempt identifier, and a usage event. The adapter for each provider translates that contract. OpenAI, Claude, and Gemini expose different request details and response conventions; pretending they are interchangeable makes the boundary look clean while moving ambiguity into retries, parsers, and invoices.
A structured response is useful here because downstream accounting and review publishing should not parse prose. The schema should reject missing severity, file location, or explanation fields before a result becomes a published finding. It does not make model output correct. It makes invalid output visible.
type ReviewRequest struct {
\tTenantID string
\tReviewID string
\tAttemptID string
\tDiff []byte
}
type UsageEvent struct {
\tTenantID string
\tReviewID string
\tAttemptID string
\tProvider string
\tModel string
\tInputTokens int64
\tOutputTokens int64
\tStatus string
\tPriceVersion string
}
func recordUsage(event UsageEvent) error {
\tif event.TenantID == "" || event.ReviewID == "" || event.AttemptID == "" {
\t\treturn fmt.Errorf("missing attribution identity")
\t}
\treturn appendImmutableUsage(event)
}
The important detail is PriceVersion, not a hard-coded rate in application code. Pricing changes, discounts have conditions, and token definitions can differ at the boundary. Store the pricing reference used for the calculation, then calculate a derived amount from the recorded token counts. Keep the raw counts.
How should one API key across providers compare token cost?
The page should not be “token spend is high.” That is a symptom with no owner. Page on a bounded condition such as an unexplained tenant-attributed spend deviation, a missing usage event for a completed review, or a retry count crossing its policy limit. The alert payload should answer what page fired: tenant, review, attempt, provider, model, status, and the correlation identifier.
A streaming response complicates this. Server-sent events can deliver progress to a client, but a completed stream is not proof that the accounting write completed. I commit the usage event independently and make publication wait for an accounting state that is durable enough for the service's recovery model. If the client disconnects, the review attempt still has an owner.
type AttemptState string
const (
\tAttemptStarted AttemptState = "started"
\tAttemptAccepted AttemptState = "accepted"
\tAttemptRecorded AttemptState = "usage_recorded"
\tAttemptPublished AttemptState = "published"
\tAttemptFailed AttemptState = "failed"
)
func canPublish(state AttemptState) bool {
\treturn state == AttemptRecorded
}
This is deliberately conservative. It can delay a user-visible result when accounting is unavailable. That is the tradeoff: a financial workflow gets a slower answer in exchange for a record that can be reconciled. It does not apply to every application: a disposable experiment may accept approximate attribution, and a non-billable local run does not need the same ledger. The limitation is intentional; the decision should follow the consequence of being wrong.
How do I compare providers without lying to the tenant ledger?
Compare the complete attempt, not a headline input-token number. The useful unit is a tenant's review workload: prompt tokens, repository context, output tokens, retries, rejected structured output, latency, and accepted result rate. A model with a lower nominal rate can lose that advantage if it needs more context, produces unusable output, or causes a second attempt.
I would run a fixed evaluation set drawn from representative review changes, with sensitive code removed and expected findings reviewed by engineers. Record provider and model as dimensions, but do not rank them by a single score. For routing, define a rule that can be inspected: for example, use the preferred class for a policy category, reject a result that fails schema validation, and allow one explicitly recorded fallback. The rule is more valuable than a magic “cheapest” switch because it explains why a tenant was charged.
The three provider families are operationally different even when they sit behind one adapter. OpenAI's structured-output documentation describes schema-constrained output behavior; Anthropic's API documentation describes its own message interface; Google's Gemini API documentation describes its own model and pricing surface. Their limits, token accounting, availability, and supported controls can change independently. The neutral engineering response is an adapter plus a current policy table, not a claim that one universal price comparison stays true. The limitation is that no comparison remains authoritative without a dated evaluation set and a current pricing reference.
| Decision field | Record it | Why it matters |
|---|---|---|
| Identity | tenant, review, attempt | Separates retries and ownership |
| Execution | provider, model, region if relevant | Explains the route actually taken |
| Usage | input and output tokens, status | Supports reconciliation |
| Policy | route rule and fallback reason | Makes selection reviewable |
| Pricing | effective date or version | Prevents stale calculations |
What should a budget policy record before routing?
First, make the usage event append-only and idempotent on the attempt identifier. Second, test timeout, client disconnect, malformed output, provider rejection, and worker restart as accounting cases, not only as transport cases. Third, sample complete traces while retaining inexpensive aggregate metrics for normal operations. A dashboard can show a trend; a trace can explain a bill.
Then establish a per-tenant budget policy with a clear behavior at the boundary: queue, reject, or use an approved fallback. Do not silently switch models because a budget threshold was crossed. That creates an invoice that cannot be explained to the tenant and a review result that cannot be reproduced by support.
I would choose one logical key only after these records exist. The key reduces credential handling at the application boundary; it does not solve cost allocation, correctness, or incident response. Once attribution is durable, model price becomes one input among several, and routing can be changed without rebuilding the audit trail.
One retry is a policy decision, not a free safety net.
The durable lesson from this incident pattern is cost visibility is a correctness property. Optimize token cost after every attempt has an owner, a reason, and a recoverable record.
Top comments (0)