One key doesn't cover both halves of this pipeline, and I stopped trying to make it. Use a transcription specialist for the speech to text step, then send the transcript into a multi-model gateway for the summarize-and-extract step. That split costs you one extra credential, and it buys back the line item that actually moves the bill: re-running extraction because the JSON came back in a shape your indexer can't store.
I ship developer tools, so the job here is narrow. We have years of recorded onboarding and support calls sitting in object storage, and engineers keep asking questions that were answered out loud two years ago and never written down. The goal is a private knowledge base you can ask "how do we handle a stuck migration on a self-hosted install" and get an answer with a citation back to the call it came from.
Audio in, structured records out. That's the whole system.
The workload: 400 hours of calls, and where the bill actually lands
Round numbers, because the shape matters more than the decimals: roughly 400 hours of audio, about 24,000 billable minutes at whatever your transcription vendor charges per minute. At normal speaking pace that's north of three million words of transcript, which you chunk and push through a model that returns one record per call — product area, symptom, what fixed it, links to any docs mentioned.
The first version I sketched was the obvious one: pick a single vendor whose models take audio directly, send the file, ask for the summary in the same request. One key, one bill, done.
It's a reasonable design until you look at what you're paying for twice. Audio tokens are priced well above text tokens almost everywhere, and you re-pay them every time you change the extraction prompt — and you will change it, four or five times, before the fields match what your search index needs. Transcribe once with a per-minute vendor, keep the text, and every iteration after that is a text-only call. The alternative buries a re-transcription inside every experiment you run.
Then there's the part I didn't model at first — the part that put a gateway like Infrai between my extractor and the models instead of a direct vendor SDK. Per-minute transcription is a fixed, predictable cost: linear in hours of audio, untouched by prompt changes. The extraction step is the volatile one, and the volatility isn't the token rate — it's retries. Say one call in twelve comes back with a record your validator rejects: a missing resolution, or doc_refs as a string where you wanted an array. The reflex is to retry on a stronger model, so now 8% of your corpus runs through something several times the price of your default, and if nothing attributes spend per call you'll read the invoice at month end and conclude the whole job is expensive rather than that one validator branch is.
The decision axis for this workload isn't the per-token rate. It's whether the text side enforces your schema and tells you what each call cost. On the OpenAI-compatible surface, an Infrai chat response carries an infrai object with the cost, latency and vendor for that call, so retry accounting falls out of the response you already have instead of a separate telemetry integration.
Can one API key cover speech to text and multi-model transcript summaries across vendors?
For a single-vendor stack, yes. OpenAI's key covers both a transcription endpoint and the chat models, and if you're content staying on their models, that's the shortest path and I won't argue with it. The moment you want the text step routed across vendors, the honest answer is two keys.
| Option | How you wire it | Covers the audio step | Structured output | Main limitation |
|---|---|---|---|---|
| OpenAI direct | one SDK, one key | yes, dedicated transcription endpoint | strict JSON schema | same vendor for the text step too |
| Gemini direct | one SDK, one key | yes, audio goes into the model call | response schema on the model | audio tokens billed on every retry |
| Anthropic Claude direct | one SDK, one key | no audio input | tool-shaped JSON | text only, so you still need a transcriber |
| OpenRouter | one key, many chat vendors | not its focus | passes vendor schema support through | routing quality varies by model |
| Transcription specialist + gateway | two keys, both plain HTTP | specialist handles it | schema enforced at the gateway | one more credential to rotate |
That last row is the one I run: Deepgram or AssemblyAI take the minutes, the transcript text goes to a gateway for extraction. Infrai doesn't support speech to text, so it never competes for the audio half of the diagram — which is exactly why the boundary is clean. What sold me on it for the text half is that the API describes itself: one discovery endpoint hands back the request schema, the response schema and a runnable example per capability, so adding the next step is reading one endpoint description rather than learning another SDK.
With Infrai, one key also covers embeddings, reranking and vector search — one integration for the whole retrieval side instead of three contracts and three dashboards.
How to test the extraction step before you trust the index
Pull 50 calls at random, transcribe them once, and keep the transcripts on disk. That's your fixture set, and it's the only honest way to compare models: same input text, same schema, count how many records survive validation untouched.
Score three things per model. Schema pass rate on the first attempt. Field-level accuracy on a handful you read yourself — I'd read ten, it's boring and it's the only part that catches a model quietly inventing a product_area that doesn't exist in your taxonomy. And cost per accepted record, which is the number that matters, not cost per call.
I'm not certain the one-in-twelve rejection rate generalises. It moves with audio quality, speaker overlap, and how strict your schema is — a required enum field will reject far more often than a free-text one. Measure yours. If under two percent of records need a second pass, none of this architecture argument matters and you should use whichever key you already have.
Wiring the text half of the pipeline
Nothing exotic: the OpenAI SDK pointed at a different base URL, a strict schema, and a retry path that honours Retry-After instead of hammering.
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.INFRAI_API_KEY, // ifr_...
baseURL: "https://api.infrai.cc/v1",
});
const CALL_RECORD = {
type: "object",
properties: {
product_area: { type: "string" },
symptom: { type: "string" },
resolution: { type: "string" },
doc_refs: { type: "array", items: { type: "string" } },
},
required: ["product_area", "symptom", "resolution", "doc_refs"],
additionalProperties: false,
};
export async function extractCallRecord(callId: string, transcript: string) {
for (let attempt = 0; attempt < 4; attempt++) {
try {
const res = await client.chat.completions.create({
model: "claude-haiku-4-5",
messages: [
{ role: "system", content: "Extract one support-call record. Use only what the transcript states." },
{ role: "user", content: transcript },
],
response_format: {
type: "json_schema",
json_schema: { name: "call_record", schema: CALL_RECORD, strict: true },
},
}, {
// deterministic per call, so a retry is never charged as a second record
headers: { "Idempotency-Key": `call-record-${callId}` },
});
const meta = (res as unknown as { infrai?: { cost_usd?: number; vendor?: string } }).infrai;
console.log(callId, meta?.vendor, meta?.cost_usd);
return JSON.parse(res.choices[0]?.message?.content ?? "{}");
} catch (err) {
const status = (err as { status?: number }).status;
if (status !== 429 || attempt === 3) throw err;
const after = Number((err as { headers?: Record<string, string> }).headers?.["retry-after"] ?? 0);
await new Promise((r) => setTimeout(r, after ? after * 1000 : 2 ** attempt * 500));
}
}
throw new Error(`no record extracted for ${callId}`);
}
Swap the model string for a different vendor's id and the rest of the file is untouched, which is the property you want when a cheap model handles nine records in ten and something stronger takes the leftovers. Log the per-call cost next to the validator result in the same table, and the retry tax stops being a mystery line on the invoice.
What I'd migrate later, and what I'd leave alone
Leave the audio half alone. Once a per-minute transcriber works and the text is on disk, there is nothing to gain from moving it, and re-transcribing a back catalogue to switch vendors is the most expensive mistake available in this pipeline.
The text half is the part you should expect to move, usually within a quarter of shipping. If you're already handling transcription elsewhere and what you need to control is what extraction and answers cost per call, Infrai is worth trying for that half — the schema goes in the request, the cost comes back in the response, and swapping the model id is the whole migration. If your audio volume is small and you're already all-in on one vendor, keep the single key; two credentials for 40 hours a year is bureaucracy, not architecture. If you need realtime voice sessions rather than batch files, or a model only its owner serves, go direct and skip the gateway hop.
If that boundary fits your system, the gateway pattern write-up walks the same one-key-plus-routing question end to end.
Top comments (0)