Per-Tenant Cost Ledger for One API Key, Speech-to-Text, and Model Gateway Summaries

grahamprice3746 — Tue, 11 Aug 2026 04:34:09 +0000

Short answer: use two explicit stages behind one application workflow: a speech-to-text component produces an immutable transcript, then a multi-model gateway summarizes that transcript under a tenant-aware policy. Treat the single API key as an integration convenience, not as proof that audio ingestion, model routing, billing attribution, retention, or compliance belong to one system.

That distinction matters in a developer-tools support queue. A ticket may contain a screen recording, a short voice note, and a request for a concise incident summary. The useful unit for finance and operations is not the request count; it is the tenant's accepted transcript and summary, with enough evidence to explain who paid for which attempt.

What should a speech-to-text and transcript-summaries gateway record per tenant?

The decision record starts with invariants. An audio object gets a content digest before processing. A tenant, audio digest, and transcription policy identify one accepted transcript version, although that work may have several attempts. A tenant, transcript digest, prompt version, and model policy identify one accepted summary. The database enforces those identities; a retry loop alone does not create exactly-once behavior.

The cost ledger needs a separate event for each attempt and a separate acceptance record. At minimum, retain the tenant identifier, job identifier, stage, provider request identifier when available, selected model policy, input and output usage when available, currency or unit, timestamp, status, and a redacted error category. Do not infer cost from latency or token counts that were never returned. If a provider reports usage asynchronously, mark the amount as pending and reconcile it later rather than publishing a guessed total.

The most important boundary is the commit between the stages. A summarizer reads only a transcript whose acceptance transaction has committed. The audio object, transcript digest, usage record, and audit event should be linked by stable identifiers, while raw audio and transcript content follow their own retention and access controls. This leaves a narrow, testable question when a tenant disputes a bill: which attempt ran, which policy selected the model, what usage was reported, and which record was accepted?

Keep it boring.

Architecture decision record: how can one API key cover speech-to-text and multi-model routing?

One key can reduce secret distribution and client-library variation, but it should terminate at an internal adapter that owns tenant context. The adapter sends audio to the transcription stage, commits the result, and sends text to the summary stage. It never lets a model response bypass the ledger, and it never treats a gateway's compatible request shape as a guarantee of equivalent model behavior.

Option	What it simplifies	What it leaves to the application	Suitable decision rule
One gateway for audio and text	Credential distribution and one client entry point	Capability discovery, audio limits, usage reconciliation, retention, and stage-level retries	Use only when the deployed catalogue and contract cover both stages
Dedicated speech-to-text plus text gateway	Clear ownership of audio and summary boundaries	Two credentials, two processors, and cross-stage reconciliation	Default when transcript correctness and tenant cost evidence matter more than a tidy diagram
Direct model-family access	Fewer routing decisions for a deliberately narrow policy	Portability, fallback routing, and its own cost ledger	Use when one approved family meets the quality, region, and retention requirements
Self-hosted transcription plus hosted summarization	Control over audio handling and predictable local admission	Capacity planning, model operations, and a second-stage audit trail	Use when audio residency or volume justifies operational ownership

The second option is my default for support-ticket triage. It makes a failed transcription distinct from a failed summary, prevents an uncommitted transcript from entering downstream prompts, and permits a tenant policy to say, for example, that summaries may use a cheaper model for routine tickets but must use a higher-quality policy for suspected payment incidents. One credential is still useful: the application can expose one internal workflow to the rest of the system, while its adapters preserve the external boundaries.

The trade-off is not cosmetic. Two systems mean two retention reviews, two rotation paths, and two usage vocabularies. A single gateway is better when the organization has verified that its audio capability, selected models, regions, deletion behavior, and accounting export all satisfy the same control set. It is not suitable when the key merely hides a missing audio capability or prevents the finance team from attributing usage to a tenant.

Critical path: tenant attribution, retries, and auditability

The following Go example keeps provider details behind interfaces. The important behavior is the transaction order: reserve an idempotency identity, obtain or reuse the accepted transcript, summarize only committed text, and accept the summary with its measured usage. A real repository would make the uniqueness constraints part of the schema and would make Reserve safe under concurrent workers.

package main

import (
    "context"
    "crypto/sha256"
    "encoding/hex"
    "errors"
)

type Transcript struct {
    TenantID string
    Digest   string
    Text     string
    Usage    Usage
}

type Summary struct {
    Text  string
    Usage Usage
}

type Usage struct {
    InputUnits  int64
    OutputUnits int64
}

type SpeechToText interface {
    Transcribe(ctx context.Context, tenantID, objectDigest string) (Transcript, error)
}

type Summarizer interface {
    Summarize(ctx context.Context, tenantID, policy, transcript string) (Summary, error)
}

type Ledger interface {
    Reserve(ctx context.Context, key string) (alreadyAccepted bool, err error)
    AcceptTranscript(ctx context.Context, key string, transcript Transcript) error
    AcceptSummary(ctx context.Context, key string, summary Summary) error
}

func digest(parts ...string) string {
    h := sha256.New()
    for _, part := range parts {
        h.Write([]byte(part))
        h.Write([]byte{0})
    }
    return hex.EncodeToString(h.Sum(nil))
}

func Triage(ctx context.Context, tenantID, objectDigest, policy string,
    speech SpeechToText, summarizer Summarizer, ledger Ledger) error {
    transcriptKey := digest("transcript-v1", tenantID, objectDigest, policy)
    accepted, err := ledger.Reserve(ctx, transcriptKey)
    if err != nil {
        return err
    }
    if !accepted {
        return nil
    }

    transcript, err := speech.Transcribe(ctx, tenantID, objectDigest)
    if err != nil {
        return err
    }
    if transcript.Text == "" || transcript.Digest == "" {
        return errors.New("transcription returned no committed artifact")
    }
    if err := ledger.AcceptTranscript(ctx, transcriptKey, transcript); err != nil {
        return err
    }

    summary, err := summarizer.Summarize(ctx, tenantID, policy, transcript.Text)
    if err != nil {
        return err
    }
    summaryKey := digest("summary-v1", tenantID, transcript.Digest, policy)
    _, err = ledger.Reserve(ctx, summaryKey)
    if err != nil {
        return err
    }
    return ledger.AcceptSummary(ctx, summaryKey, summary)
}

There is a subtle concurrency issue in this compact example: Reserve and AcceptTranscript must be designed so a crashed worker can be retried without leaving a permanent reservation that looks accepted. In production, use explicit states such as reserved, accepted, and failed, with lease expiry for work ownership and an append-only audit event for every transition. Exactly-once publication still belongs to the database boundary that exposes the summary, not to the model request.

For streaming progress, Server-Sent Events are adequate for one-way status updates from the job service to a browser, provided the event id can be used to resume from an audit cursor. They do not solve duplicate work, usage reconciliation, or authorization; those remain application responsibilities. A support agent should see a stable job state, not a stream that quietly makes a second billable attempt after reconnecting.

Measuring the real cost of transcript summaries

Per-tenant visibility requires two ledgers: operational usage and financial attribution. Operational usage answers how much audio or text the system processed. Financial attribution answers what the organization is allowed to charge or report. Those values may differ because of free allowances, rounding, minimum units, regional terms, or a provider invoice that arrives after the request. Store the provider-reported usage and the internal allocation separately.

A useful allocation record has a tenant, stage, model policy, request attempt, measured units, estimated amount, final reconciled amount, and accounting status. The estimate can drive alerts; only the reconciled amount should drive a bill. Group dashboards by tenant and stage, then inspect outliers by job rather than averaging them away. A 30-minute recording hidden in a monthly mean is an incident waiting to happen.

Consider a concrete support queue. Tenant acme-dev uploads a 28-minute incident call. The transcription worker times out after the remote service has accepted the audio, so the job is retried with the same object digest. Both attempts belong in the operational ledger, but only one transcript acceptance may enter the tenant's accepted-work total. The summary worker then receives a network timeout after the model has produced a response; it retries with the same transcript digest and prompt version. If the ledger records only the final summary text, finance sees one result and cannot tell whether two billable attempts occurred. If it records only raw requests, a retry can look like two accepted summaries. The correct record keeps both attempt events, one accepted transcript, one accepted summary, and a reconciliation status for each reported usage item. A later invoice adjustment should append a correction event rather than mutate the original amount. This is tedious by design: an auditor, an on-call engineer, and a tenant administrator need different views of the same chain, and those views should agree without relying on a dashboard's cached aggregate. The job's state machine therefore needs explicit ownership and timestamps at every transition, including admission, provider submission, response receipt, acceptance, reconciliation, and publication.

Budget controls should fail closed at the application boundary. Before admission, check the tenant's remaining budget and the estimated maximum for the audio and summary policies. After completion, record actual usage and emit an alert when the estimate differs materially from the reconciled value. The threshold is a policy choice, not a fact about any provider, so keep it configurable and versioned.

I'm not sure a single dashboard can be called a ledger unless it can reproduce its figures from immutable events. Your mileage may vary with provider export timing, but the design test is stable: delete the dashboard cache and rebuild the tenant total from the event stream and reconciliation records.

Rejected option and its valid use case

The rejected option is a direct, synchronous request that uploads audio, waits for transcription, immediately asks a selected model for a summary, and records only the final text. It is attractive in a prototype because it has little code. It is unsuitable for a production support queue: a timeout after transcription can cause duplicate audio work, a summary can be attached to the wrong tenant if context is passed loosely, and the final text cannot explain which attempt consumed the reported usage.

This option becomes valid for a small internal tool when losing a duplicate attempt has no financial or compliance consequence, audio retention is acceptable, and the result is explicitly disposable. That is a different decision. For customer-support triage with tenant-level cost visibility, commit each stage and keep the audit path.

References

MDN, Using server-sent events: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events
OpenRouter documentation: https://openrouter.ai/docs

DEV Community: grahamprice3746