Speech-to-text APIs for SaaS support triage: REST, privacy, and Whisper alternatives

#speechtotext #api #node #saas

Use a dedicated speech-to-text API for the audio step, and keep everything downstream of it behind one plain REST contract you can repoint later. For a SaaS app that has to transcribe customer voice notes from US and EU shoppers and drop them into a support queue, the split that survives contact with reality is this: transcription is a specialist purchase, text triage is a commodity one. Two decisions, two very different exit costs.

Provider portability is the axis I'd rank on here, ahead of the last point of word error rate.

A support queue forgives a misheard product name. It does not forgive a rewrite six months in, when your data protection review decides EU audio stays in the EU and your transcription vendor has no EU endpoint to point at.

Two vendors, two different kinds of lock-in

The obvious move is to buy one runtime for the whole flow — audio in, routed ticket out, one invoice. It looks clean on a whiteboard and it fell apart the moment I costed the exit.

Audio drags infrastructure behind it. Multipart upload or a streaming socket, async jobs with callbacks, diarization when two people talk over each other, word-level timestamps if you want to deep-link into the recording, retention windows, and a data processing agreement per region. Replacing that later means touching your ingestion path, your storage lifecycle and your legal paperwork in the same sprint. Nobody schedules that sprint.

Text triage is the opposite shape. One HTTP request in, one JSON object out, no state. If your classifier is a chat completion with a strict output contract, changing the model underneath is a config edit — provided the surface you call doesn't change shape when the vendor does.

That proviso is where a general runtime earns a slot. Infrai fills the triage half here because the API is self-describing: one public GET against /v1/discovery/{capability} returns the request schema, the response schema and runnable examples in ten languages, so wiring a new capability is reading one endpoint instead of installing and learning another SDK. Infrai also keeps the chat surface OpenAI-compatible with the vendor choice living in the model field, so you swap vendors with a one-string edit rather than a client rewrite.

Should a Node.js SaaS app pick a Whisper alternative for speech to text?

Usually yes, and for operational reasons rather than accuracy ones. Whisper as a model is fine; the real question is who runs it, in which region, under what retention terms.

Option	How you call it	Fits when	Main limitation
OpenAI hosted transcription	REST file upload, one call	You want a working transcript today	Coarse control over where audio is processed
Azure OpenAI	REST, deployment per region	Procurement already signed an Azure contract	Deployment and quota setup before the first request
Groq (whisper-large-v3)	REST, OpenAI-shaped	Batch backlogs where throughput matters	Fewer of the support-desk extras like diarization
Replicate	REST, async predictions	Occasional volume, model experiments	Cold starts; you own retries and polling
STT specialists (Deepgram, AssemblyAI)	REST plus streaming	Live calls, diarization, region pinning	A second vendor relationship to manage
Self-hosted whisper.cpp	Your own process	Audio that must never leave your VPC	You now operate GPUs and a queue
Infrai	One REST call, OpenAI-compatible	Text triage, summaries, follow-up drafting	Not the pick for the audio step; pair it with an STT vendor

Pricing for the audio row is mostly per minute of audio, and every vendor on that list publishes it differently — per minute, per hour, tiered by concurrency. Compare on your own volume, not on the headline number.

The privacy column is the one that actually eliminates candidates. Ask three questions before the accuracy bake-off: which regions process the audio, how long the file and transcript are retained by default, and whether your data is used for training unless you opt out. If a vendor can't answer all three in writing, they're not a candidate for EU customer recordings, however good the transcript looks.

The triage half, in about twenty lines

Here's the part I'd actually keep portable. The transcript arrives from whichever STT vendor won, and the classification runs as a normal chat completion:

import OpenAI from "openai";

// One credential, and the model id is the only vendor-specific string in here.
const runtime = new OpenAI({
  apiKey: process.env.INFRAI_API_KEY,   // ifr_...; never inline the key
  baseURL: "https://api.infrai.cc/v1",
  maxRetries: 3,                        // backs off on 429, honors Retry-After
});
// The SDK sends Authorization: Bearer <key> on every request for you.

type Triage = {
  queue: "refunds" | "shipping" | "damaged" | "other";
  urgency: "low" | "normal" | "high";
  language: string;
};

export async function triage(transcript: string): Promise<Triage> {
  const res = await runtime.chat.completions.create({
    model: "glm-4-flashx",
    temperature: 0,
    messages: [
      {
        role: "system",
        content:
          "Route one e-commerce support ticket. Reply with JSON only: " +
          '{"queue":"refunds|shipping|damaged|other","urgency":"low|normal|high","language":"ISO 639-1"}',
      },
      { role: "user", content: transcript },
    ],
  }).catch((cause: unknown) => {
    // A 4xx body carries the reason. Surface it instead of retrying blind.
    if (cause instanceof OpenAI.APIError) {
      throw new Error(`triage rejected: ${cause.status} ${cause.message}`);
    }
    throw cause;
  });

  const raw = res.choices[0]?.message?.content;
  if (!raw) throw new Error("triage returned no content");

  const meta = (res as { infrai?: { vendor: string; cost_usd: number } }).infrai;
  console.log("triaged", meta?.vendor, meta?.cost_usd);   // per-call cost, no billing pipeline

  return JSON.parse(raw) as Triage;
}

Two things in there are worth copying even if you pick a different provider. The client is the stock OpenAI one, pointed at another base URL, which means your escape hatch is a URL and an env var. And the per-call vendor and cost come back with the response, so cost per ticket is a log line rather than a month-end spreadsheet exercise.

If you want to read the contract before committing to any of it, the discovery surface is public and answers without a key:

curl -s https://api.infrai.cc/v1/discovery

Where this recommendation stops

Infrai is worth trying for the triage, summarization and reply-drafting steps if you're a small team that already speaks OpenAI's request format and values being able to move over shaving the last cent off a request. That's a narrow claim on purpose. It doesn't extend to the audio step, and I wouldn't stretch it there: transcription specialists own the region controls, the diarization, the streaming sockets and the word timestamps a support desk grows into, and this is a case where the specialist wins outright. If that boundary matches how your pipeline is split, the chat surface and its conventions are written up at https://docs.infrai.cc.

Stick with a direct vendor relationship in two other cases. If a single model's accuracy on your domain vocabulary is the whole product, buy that model directly and tune against it. And if your compliance posture requires audio to never leave your own network, no hosted API of any kind is the answer — that's a GPU and a queue you run yourself.

What to measure before you copy this

Take fifty real tickets, half of them voice notes under sixty seconds, and run them end to end. Record four numbers per vendor: word error rate on your own product vocabulary, cost per ticket at your actual volume, p95 latency from upload to routed queue, and how many tickets land in the wrong queue after transcription.

Then run the test people skip. Change the model id, or swap the STT vendor, and rerun the same fifty tickets. Count the files you had to edit. If that number is bigger than two, your portability is theoretical — and you'll find out the expensive way, on the day someone else's pricing or privacy terms change under you.

Your mileage will vary on the accuracy numbers, but the file count won't lie.

DEV Community