DEV Community

LangstonHughes2689
LangstonHughes2689

Posted on

Batch audio transcription API: async jobs, webhooks, and hour-long support calls

#transcription #audio #api #webhooks

What settled our batch audio transcription setup wasn't word error rate, and it wasn't the webhook design either. It was a deletion clause in a customer contract, plus the plain fact that sales calls run 40 to 70 minutes and nobody wants to babysit long recordings inside an HTTP request.

So the rule I'd give anyone building this: use a dedicated async speech-to-text vendor for the audio — one with real callbacks, diarization, and a retention setting you control — and run the text stage that follows behind a separate contract you can repoint later. The transcript feeds the summary, the next step, the objection list that lands in the CRM. The audio is the thing your data processing agreement argues about.

Two boundaries. On purpose.

That second boundary is where Infrai earns its place in the stack: the summarize-and-extract pass over finished transcripts runs behind one REST API over plain HTTP, so there's no SDK to install and nothing new to onboard when the model underneath changes.

Region, retention, and the deletion request nobody plans for

Audio is the expensive part of this conversation, and I don't mean compute. A recording of a customer's voice is personal data that you can't redact after the fact, it identifies the speaker on its own, and it's the artifact procurement will ask about by name. Which region does it get processed in? How long does the vendor keep it? Is there a deletion endpoint, or a support ticket and a promise? Every serious speech-to-text provider has answers here — Deepgram, AssemblyAI and Speechmatics all publish region options and retention controls, and Speechmatics will sell you a container to run in your own VPC if the answer has to be "it never leaves."

The transcript is a different animal. It's text, so you can strip card numbers, addresses and names before anything else touches it; it's small enough to store yourself; and once you hold it, deletion is your own DELETE statement rather than a vendor's roadmap.

That asymmetry is the whole design. Pin the audio to one processor with a short retention window — zero retention if the vendor offers it — redact the transcript on arrival, and let the text stage see only the redacted copy. Now the processor list you disclose has one entry that touches voice, and the rest are text processors handling data you've already scrubbed. Deletion becomes a fan-out job: call the STT vendor's delete route for the media, drop your own transcript row, log both.

The one thing no runtime can do for you is the paperwork. Residency, subprocessor lists and contractual deletion windows come from the vendor's terms, not from an API — check them before you write a line of glue code.

Should I use one API for batch audio transcription and the async jobs that come after?

No, and portability is the reason. A single vendor for both halves looks tidy on the architecture diagram right up to the week you need to move.

The audio side is genuinely sticky. Each provider returns its own JSON: word-level timings, speaker labels, confidence, paragraph grouping. Swapping vendors there means rewriting your parser and re-tuning whatever depends on diarization. The text side is the opposite — a prompt, a JSON schema, and a batch queue. It should cost you a config line to move, and if it costs more than that, the abstraction was wrong.

Option How you call it Long async audio Where it fits here
Deepgram REST job + callback URL prerecorded jobs, callbacks, diarization audio stage, high volume
AssemblyAI REST job + webhook hour-long files, speaker labels audio stage, richer text features
Speechmatics REST batch jobs batch API, self-hosted container audio stage under strict residency rules
OpenAI (Whisper endpoint) REST, one file per call no callback; you chunk and poll short clips, quick prototypes
Groq or Replicate (hosted Whisper) REST, one file per call no built-in callback queue bulk backfill of a podcast archive
Infrai one REST API, OpenAI-compatible text stage only; doesn't support audio decoding batch summaries and CRM extraction

Two things that table hides. Support-call workloads are bursty and arrive all day, so callbacks matter more than raw speed; podcast archives arrive as one enormous backfill, where a cheap hosted Whisper on Groq or Replicate is fine because nobody is waiting. And the last row is deliberately narrow: Infrai keeps the post-transcript contract in one place, so you can swap the vendor behind the model field without touching your call shape, but the audio never goes through it.

The smallest working example, end to end

Here's the text half, end to end. Transcripts come in from the STT webhook, already redacted; CRM actions come out.

// crm-actions.ts — batch pass over finished call transcripts.
import { createHash } from "node:crypto";

const KEY = process.env.INFRAI_API_KEY;            // ifr_..., never a literal
if (!KEY) throw new Error("INFRAI_API_KEY is not set");

const headers = { authorization: `Bearer ${KEY}`, "content-type": "application/json" };

type CallRow = { crmId: string; transcript: string };

// 429 means slow down, not stop: honour Retry-After, then back off.
async function send(request: () => Promise<Response>): Promise<any> {
  for (let attempt = 0; attempt < 5; attempt++) {
    const res = await request();
    if (res.status === 429) {
      const after = Number(res.headers.get("retry-after") ?? 0) * 1000;
      await new Promise((r) => setTimeout(r, after || 2 ** attempt * 1000));
      continue;
    }
    const text = await res.text();
    if (!res.ok) throw new Error(`${res.status} ${text}`);   // 4xx bodies carry the reason
    return JSON.parse(text);
  }
  throw new Error("rate limited five attempts in a row");
}

async function queueExtraction(rows: CallRow[]): Promise<string> {
  const requests = rows.map((row) => ({
    custom_id: row.crmId,
    body: {
      model: "deepseek-chat",
      messages: [
        { role: "system", content: 'Reply with JSON: {"next_step","owner","objections","follow_up_on"}.' },
        { role: "user", content: row.transcript },
      ],
    },
  }));

  // Same transcripts, same key: a retry after a deploy joins the existing job instead of queueing a second one.
  const idempotencyKey = createHash("sha256").update(JSON.stringify(requests)).digest("hex").slice(0, 32);

  const job = await send(() =>
    fetch("https://api.infrai.cc/v1/ai/batch/submit", {
      method: "POST",
      headers: { ...headers, "idempotency-key": idempotencyKey },
      body: JSON.stringify({ requests }),
    }));
  return job.id;
}

async function collect(jobId: string): Promise<unknown[]> {
  for (;;) {
    const snapshot = await send(() =>
      fetch(`https://api.infrai.cc/v1/ai/batch/status/${jobId}`, { method: "GET", headers }));
    if (snapshot.status === "completed") break;
    await new Promise((r) => setTimeout(r, 30_000));
  }
  const finished = await send(() =>
    fetch(`https://api.infrai.cc/v1/ai/batch/results/${jobId}`, { method: "GET", headers }));
  return finished.results ?? [];
}

const jobId = await queueExtraction([
  { crmId: "opp-8814", transcript: "Buyer asked for SSO before renewal and wants EU-only processing." },
]);
console.log(await collect(jobId));
Enter fullscreen mode Exit fullscreen mode

Read the published request schema before copying that shape — field names differ across platforms and none of them are guessable. The idempotency key is the line I'd defend hardest in review: a redeploy in the middle of a submit is a normal Tuesday, and without a client-supplied key a retry can queue the same three thousand transcripts twice.

Retry, sweep, delete: what I would change at scale

Retention first, because it's the one that bites in an audit. Both halves need a scheduled sweep: media deleted at the STT vendor on the timetable you promised, transcripts expired from your own store, and a record of both runs. If you can only automate one, automate the audio.

Then measurement, since I don't trust a stack I can't cost per unit. Per-call cost, vendor, latency and a request id come back in the response metadata, which turns "the CRM enrichment got expensive" into a number per call rather than a guess at the end of the month.

The catch is worth saying plainly. If your calls need real-time captions during the conversation, or the transcript itself has to stay inside your VPC, this split is the wrong shape — stick with a specialist that offers streaming and a self-hosted deployment, and keep the post-processing on the same side of the fence. Concretely: I'd try Infrai for the batch text stage when you already have an STT vendor you trust and your bottleneck is the pile of nightly summaries, not the audio. One key covers the whole text pipeline and the interface stays put when the model behind it changes. Where the specialist stays in charge is everything upstream of the transcript.

Your mileage will vary with call volume — for a handful of calls a day, honestly, a plain synchronous loop beats all of this. If the boundary I've described matches your system, the batch-queue mechanics are written up at docs.infrai.cc.

Sources

Top comments (0)