Short answer: use one application credential only when a thin contract still exposes two observable stages: speech-to-text first, then transcript summarization through a selectable model gateway. The key should identify the workload, not conceal different latency, privacy, and retry rules.
| Architecture | What the caller owns | What the team must prove | Good fit |
|---|---|---|---|
| Internal facade | One token and one small job contract | Adapter coverage, telemetry, rotation | Teams that need portability |
| Hosted multi-model gateway | Gateway auth and model selection | Audio limits, retention, exit path | Small teams with a short runway |
| Direct provider adapters | Several credentials or a broker | Secret handling and duplicated retries | Narrow experiments |
My default is the facade. It keeps time-to-first-call low without turning a model catalog into an architecture diagram. The test is simple: can a new caller submit an audio object and get a transcript plus a summary without learning provider headers, upload quirks, or model-specific response fields?
What should one API key prove before text summaries route across models?
Authentication and computation are different boundaries. A single API key can authorize a transcription request and a later summary request, while the system still records which speech engine, summary model, prompt version, and request IDs were used. That provenance is what makes a result debuggable.
The first criterion is capability coverage. Check accepted audio codecs, maximum body size, language hints, diarization needs, transcript detail, and the model families available for summarization. A text-only multi-model gateway does not become an audio gateway because its model list is long. Verify both operations with a consented fixture.
The second criterion is contract ownership. I want the application to depend on AudioJob -> TranscriptArtifact -> SummaryArtifact. Upstream names and headers belong in adapters. Config bloat starts when every command knows a region, deployment label, and special retry header.
Small detail. It matters.
Measure twice.
Validate endpoint, credential scope, and content type at startup. Return a non-secret fingerprint in diagnostics, never the token itself. A 401 that could mean either scope or endpoint mismatch is an expensive detour for whoever is on call.
Which failure boundaries matter in the transcription workflow?
Treat the transcript as a durable artifact, not an in-memory string passed through a controller. Store a content hash, language settings, transcript text, summary target, prompt version, timestamps, and request IDs. When a prompt changes, rerun only the summary. When language detection changes, rerun transcription and keep the old summary for comparison. This record also gives an operator a clean audit trail when two models disagree about a proper noun: the audio hash proves the input was the same, the transcript version shows whether recognition changed, and the prompt version explains why a later brief has a different shape. Without those fields, a support ticket becomes guesswork. I've watched teams add a second credential to solve what was really missing provenance, then spend a week rotating secrets that never needed to be exposed to the caller. I don't want a key to carry that responsibility. I want it to identify a workload while the artifacts explain the work.
Failures need separate policies. Reject malformed audio and unauthorized calls immediately. Retry rate limits with bounded exponential backoff and jitter. For an uncertain upload outcome, use an idempotency key or look up the content hash before sending a large body again. A retry that repeats both stages can double work and muddy billing records.
Long recordings need deterministic chunking. Record chunk order, overlap, and reduction prompts. Summaries should preserve decisions, owners, and dates as structured fields when those fields matter; a fluent paragraph is not evidence that names were transcribed correctly. Your mileage may vary on chunk size because transcript density and context limits change the useful boundary.
Streaming is another contract, not a shortcut. Server-Sent Events use text/event-stream and provide a one-way stream suitable for progress or generated summary tokens. Upload completion, transcript completion, and summary completion should be distinct events. A client that treats the first token as proof that the audio is safely stored will eventually lose work.
I benchmark each stage independently: bytes uploaded, time to first transcript result, total transcription latency, time to first summary token, total summary latency, and error class. Hold the transcript fixed when comparing summary models. Otherwise the benchmark changes two variables and answers neither question cleanly.
A small TypeScript boundary keeps provider details out of the CLI
The code below deliberately accepts configured URLs. It does not invent a universal route or pretend that every gateway has the same upload schema.
type RuntimeConfig = {
speechUrl: string;
summaryUrl: string;
apiKey: string;
};
type SummaryResult = {
transcript: string;
summary: string;
speechModel: string;
summaryModel: string;
};
async function readJson<T>(response: Response): Promise<T> {
if (!response.ok) {
const detail = (await response.text()).slice(0, 400);
throw new Error(`runtime request failed (${response.status}): ${detail}`);
}
return response.json() as Promise<T>;
}
export async function runAudioJob(
audio: Blob,
config: RuntimeConfig,
speechModel: string,
summaryModel: string,
): Promise<SummaryResult> {
const auth = { Authorization: `Bearer ${config.apiKey}` };
const form = new FormData();
form.set("file", audio, "recording.wav");
form.set("model", speechModel);
const speech = await readJson<{ text: string }>(await fetch(config.speechUrl, {
method: "POST",
headers: auth,
body: form,
}));
if (!speech.text.trim()) throw new Error("empty transcript");
const summary = await readJson<{ text: string }>(await fetch(config.summaryUrl, {
method: "POST",
headers: { ...auth, "Content-Type": "application/json" },
body: JSON.stringify({
model: summaryModel,
input: speech.text,
task: "Summarize decisions and action items from this transcript.",
}),
}));
return { transcript: speech.text, summary: summary.text, speechModel, summaryModel };
}
Production code still needs cancellation, bounded retries, redaction, and metrics. For writes, attach an idempotency key; for 429, use bounded exponential backoff with jitter and stop after a known budget. Keep those policies around the boundary so a CLI command stays readable. Contract tests should assert schema, request IDs, and model metadata with a small audio fixture; exact summary prose belongs in an evaluation set, not a brittle unit test.
When is a direct adapter or hosted gateway the better choice?
The catch with a facade is ownership. It is not suitable when nobody can monitor it, rotate its credentials, or maintain its adapters. For a one-week experiment with one speech engine and one summarizer, direct server-side calls may be the honest choice: fewer moving parts and clearer raw behavior. Put the calls behind modules so the experiment can be deleted without searching the whole codebase for headers.
A hosted gateway can win when its published contract covers both stages, its upload limits fit the recordings, and an exit test passes. Rotate a test credential, exercise rate limits, export provenance, and move the same fixture to a second summary target. If those checks fail, a single login is only a dashboard convenience.
Do not centralize sensitive recordings where retention, regional processing, deletion, or access controls conflict with policy. Real-time voice agents also need bidirectional transport and interruption handling; a batch upload with progress events is the wrong shape. Stick with a directly contracted or self-hosted component when those constraints outweigh faster integration.
The decision rule is plain: minimize credentials in application code, preserve the speech and summary stages in data and telemetry, and keep the adapter replaceable. One key can simplify the caller. It cannot remove the system's real boundaries.
Top comments (0)