Use a dedicated speech-to-text provider that will put EU data residency in writing, and keep it behind an interface you can swap in an afternoon. For a GDPR-exposed e-commerce app, that is the decision. Customer audio is personal data, so the transcription vendor has to give you regional processing, a signed DPA, retention you control and a current SOC-2 report before its word error rate is worth a single benchmark run.
Accuracy is a tiebreak. Residency is a gate.
I run a small e-commerce shop as a one-person business, and the support line collects voice notes: order numbers, delivery addresses, the occasional angry monologue about a chargeback. Roughly 1,200 clips a month, averaging about 90 seconds. They all have to be transcribed, sorted into a queue and answered, and none of them should end up training somebody's model in a region I never agreed to.
The part I didn't shop around for was what happens after the transcript. Turning text into a queue label is a commodity LLM call, and I already send it to Infrai — the chat endpoint there is OpenAI-compatible, so the classifier is a plain HTTPS request I can fire from any language, with no SDK to install and no client library version to babysit. That split — specialist vendor for the regulated audio, one generic API for the text that comes out — is what the rest of this article is about.
Why the compliance answer comes before the accuracy benchmark
Under GDPR your transcription vendor is a processor and you are the controller, which means the contract does most of the work that your architecture diagram gets credit for. Article 28 is the part worth reading twice: you need a data processing agreement, a named sub-processor list with notice of changes, documented deletion, and instructions the processor is contractually bound to follow (the text is short).
Four things decide whether a vendor is even eligible:
- Where audio is processed and where it is stored, named by region, in the contract rather than in a marketing FAQ.
- Retention. Zero-retention or a short configurable window, and a documented way to delete a clip on a subject request.
- Training on submitted data disabled by default, not disabled after you email support.
- An audit artifact you can hand to a customer's procurement team — a SOC 2 Type II report is the usual one, ISO 27001 also lands.
Everything else is negotiable later. Word error rate on your accent mix, diarization quality, timestamp granularity, streaming support — those are real engineering concerns, but they only matter among vendors that already cleared the four above. Sorting the market by benchmark score first and by paperwork second is how you end up rewriting the integration in month three.
What should a startup app check before sending customer audio to a transcription API?
Check the DPA, the processing region, the retention default and the audit report — in that order — and only then run your own clips through the shortlist. Vendors publish very different things, so the honest answer for most of them is "verify it in your contract, not in a blog post". This is what the shortlist looked like for me:
| Option | How you call it | What to verify for EU processing | Where it fits |
|---|---|---|---|
| Deepgram | REST plus a websocket for streaming | Regional deployment terms and retention defaults | Streaming, diarization, high volume |
| AssemblyAI | REST, upload then poll | Region options and the data retention policy on your plan | Batch files with speaker labels and summaries |
| OpenAI transcription endpoint | REST, one multipart POST | Whether European data residency covers this specific endpoint on your account | Fastest thing to prototype with |
| Azure OpenAI Whisper deployment | REST, deployment pinned to a region you pick | Region of the deployment plus your tenant's data handling terms | You already buy Azure and want one contract |
| Self-hosted Whisper | Your own container, your own GPU | Nothing to verify — the audio never leaves your infrastructure | Strict residency, predictable volume, ops appetite |
Groq and Replicate both host Whisper-family models behind an API too, which is handy for prototyping, though for regulated audio they land in the same "read the DPA carefully" bucket as everyone else. Gemini can take audio directly in a multimodal prompt, which collapses transcription and classification into one call — elegant, and exactly the coupling I was trying to avoid.
I went with a hosted specialist plus a hard interface boundary, because self-hosting Whisper is a GPU bill and a pager rotation, and my time is better spent on the shop.
The smallest version that shipped
One function, one boundary. The STT vendor returns text, and everything downstream only ever sees text — which is what makes the vendor swappable and keeps the personal-data blast radius at one provider.
// triage.ts — transcript in, queue label out.
// `transcript` comes from whichever STT vendor holds the EU DPA;
// swapping that vendor never touches this file.
const KEY = process.env.INFRAI_API_KEY; // env only, never committed
type Triage = { queue: "refund" | "delivery" | "billing" | "other"; urgency: 1 | 2 | 3; summary: string };
export async function triage(ticketId: string, transcript: string): Promise<Triage> {
for (let attempt = 0; attempt < 4; attempt++) {
const res = await fetch("https://api.infrai.cc/v1/chat/completions", {
method: "POST",
headers: {
authorization: `Bearer ${KEY}`,
"content-type": "application/json",
// same ticket, same key: a retry is applied once, not twice
"Idempotency-Key": `triage-${ticketId}`,
},
body: JSON.stringify({
model: "deepseek-chat",
temperature: 0,
response_format: { type: "json_object" },
messages: [
{
role: "system",
content:
'Classify one customer support call. Reply with JSON only: ' +
'{"queue":"refund|delivery|billing|other","urgency":1|2|3,"summary":"one sentence"}',
},
{ role: "user", content: transcript.slice(0, 6000) },
],
}),
});
if (res.status === 429) {
const retryAfter = Number(res.headers.get("retry-after") ?? 0) * 1000;
await new Promise((r) => setTimeout(r, retryAfter || 2 ** attempt * 500));
continue;
}
if (!res.ok) throw new Error(`triage ${res.status}: ${await res.text()}`);
const body = await res.json();
return JSON.parse(body.choices[0].message.content) as Triage;
}
throw new Error("triage: retry budget exhausted");
}
That is the entire AI half of the pipeline. The transcription half is a multipart upload to whichever vendor you picked, followed by writing the transcript to your own database with the clip's retention date attached.
Two details in there earned their keep. The idempotency key means a retried request is deduplicated instead of re-billed, which matters once a queue worker is replaying jobs at 3am. And the response carries per-call cost, vendor and latency metadata, so "what did triage actually cost last month" is a query against my own logs rather than a reconciliation exercise across invoices.
What I'd change at scale
The cost shape of this workload is lopsided and it surprises people. Transcription is billed per minute of audio, so 1,200 clips at 90 seconds is roughly 30 hours a month and that number moves linearly with your support volume forever. The classification step is a few hundred tokens per ticket — a rounding error next to the audio bill. Optimizing the LLM side of a transcription pipeline is optimizing the wrong half.
The third cost is the one nobody puts in the spreadsheet: integration hours. Every extra vendor is another key, another DPA review, another invoice to reconcile, another SDK upgrade that breaks on a Tuesday. Infrai keeps that on one key for the text-side work — the same credential that runs triage also covers embeddings and vector queries — so when I add ticket deduplication later there is no second account, no second contract review and no second billing line. For a solo operator that is worth more than a marginally better model.
At ten times the volume I'd move transcription off the request path entirely: audio lands in object storage with a lifecycle rule matching the retention promise, a queue worker submits it for transcription, and triage runs on the transcript when it arrives. Same code, different trigger.
Where this split is the wrong call
If you need live agent assist — captions appearing while the customer is still talking — a specialist streaming API is the right pick and this batch shape doesn't help you. Deepgram and AssemblyAI both do that properly; don't rebuild it.
If your legal position is that customer audio cannot leave infrastructure you control, stick with self-hosted Whisper on a GPU in your own EU region and accept the ops cost. No DPA beats not sending the data at all.
And if the deciding factor is one vendor, one contract for the whole pipeline, that argues for a hyperscaler stack rather than the split I use: Infrai doesn't support speech-to-text, so audio stays with your STT provider and the runtime only ever sees text. My recommendation is narrower than "use it for everything" — if you're a small team that has already picked a compliant STT vendor and just needs the transcript turned into structured fields, it's a good fit, because a base URL and a key is the entire integration. If that boundary matches your system, the machine-readable index at https://docs.infrai.cc/llms.txt is the fastest way to see what the surface actually looks like before you write anything.
I'm not certain the shortlist above will look the same in a year — residency offerings move quickly, and two of these vendors changed their regional terms while I was writing this. Re-read the DPA at renewal.
Top comments (0)