Short answer: Put a provider-neutral adapter in front of the speech-to-text API, classify why Whisper is unavailable, and send only normalized, reviewable text into catalog enrichment.
A speech-to-text fallback in a healthtech catalog pipeline should protect the catalog's data contract, not pretend that every recognizer produces interchangeable text. My default design is a small transcription boundary: accept one job shape, normalize provider responses into one internal result, and route the same audio through an ordered set of adapters when a primary capability is unavailable. The enrichment stage then works from that normalized result and records uncertainty instead of silently inventing product attributes.
The adapter is the product boundary.
The distinction matters for messy descriptions. A pharmacist's voice note might say, "sterile 10 milliliter saline flush, luer lock, latex-free," while a catalog description contains abbreviations, dosage units, and punctuation that a generic text parser can misread. Speech recognition is only the first transformation. The fallback must preserve enough provenance for a reviewer to tell what was heard, what was normalized, and what was inferred.
How should Node.js speech-to-text API integration handle a Whisper fallback?
Treat unavailability as a typed routing decision. A timeout, an unsupported audio encoding, an empty transcript, and a provider rejection do not mean the same thing. Retrying all of them creates duplicate work and can turn a temporary outage into a queue storm. Unsupported media should go to preprocessing; an overloaded service can be retried with bounded backoff; an authentication or policy error should be surfaced and quarantined.
I initially treated a fallback as a second HTTP call. That was the wrong boundary: it made provider errors visible in catalog code and made a late primary response capable of overwriting a completed fallback. I use a result envelope that keeps the original job id, language hint, audio checksum, adapter name, and a confidence state. The exact confidence vocabulary depends on the application, but it should be stable across adapters. Here is the boundary in Python because the data shape is easier to inspect than a framework-specific client:
from dataclasses import dataclass
from typing import Optional
@dataclass
class Transcript:
text: str
adapter: str
status: str
audio_sha256: str
error_code: Optional[str] = None
def normalize(text: str, adapter: str, audio_sha256: str) -> Transcript:
cleaned = " ".join(text.split())
if not cleaned:
return Transcript(cleaned, adapter, "empty", audio_sha256, "empty_transcript")
return Transcript(cleaned, adapter, "ready", audio_sha256)
The production implementation can live behind a Node.js interface, but the contract should remain boring. The caller should not know which adapter ran, except through metadata and policy decisions. Keep the original bytes or a content-addressed reference, subject to retention and patient-data rules. A transcript without its input identity is hard to audit. That extra metadata costs storage and review effort, but dropping it makes a clinical correction impossible to explain later.
Do not use a fallback to hide semantic uncertainty. If the first recognizer returns a plausible sentence with the wrong unit, a second transcript is not automatically correct. The catalog enrichment step should validate units, required fields, and vocabulary against a healthtech-specific schema, then send contradictions to review.
Which failure classes belong in the adapter boundary?
The adapter should classify failures before the queue decides what to do. That gives operations a useful signal and prevents every downstream component from learning provider-specific error names. A compact internal vocabulary is enough: temporary, unsupported, policy, auth, quota, invalid_input, and empty. Include a retry-after value only when the upstream response supplies one or your own policy defines it. Do not manufacture a precise delay from guesswork.
Audio preparation deserves its own step. WebRTC can deliver media with browser-dependent characteristics, and a recording that plays in a browser is not necessarily in the format or duration range accepted by a transcription service. Validate MIME type, byte length, duration, sample rate, and channel count before choosing an adapter. Rejecting a bad file early is cheaper than sending it through two recognizers and then debugging a blank catalog field. MDN's WebRTC documentation is a useful standards-level reference for the media side of this boundary.
For long recordings, make jobs resumable and idempotent. Store a deterministic job key from the audio checksum plus the requested language and segmentation options. The queue can then retry the same work without creating a second enrichment event. Keep chunking policy outside the provider adapter, because otherwise a provider switch changes both latency and text segmentation at once.
How do the common provider shapes change the design?
The comparison belongs after the contract. Three widely used choices illustrate different boundaries, not a winner. OpenAI's speech-to-text API exposes hosted transcription models and accepts documented audio inputs; its boundary is a remote API contract and account policy. Google Cloud Speech-to-Text offers synchronous, asynchronous, and streaming modes; that broader mode set changes how a queue models completion. Amazon Transcribe provides batch and streaming transcription, with feature and language coverage that must be checked against the selected region and input. Azure AI Speech is another hosted option with its own SDK, region, and feature boundaries; treating it as interchangeable without checking those details is a design error.
Those differences matter in a clinical catalog workflow. A batch path can attach a stable source object and wait for a final transcript. A streaming path may emit interim text that must never update the canonical product record before finalization. A hosted API may have a different retention, regional-processing, or data-processing contract than a self-hosted recognizer. The adapter should expose these as capabilities and policy metadata, rather than leaking three incompatible SDKs into enrichment code.
A capability record can stay small:
CAPABILITIES = {
"primary": {
"streaming": True,
"batch": True,
"languages": {"en-US"},
"max_seconds": 3600,
},
"secondary": {
"streaming": False,
"batch": True,
"languages": {"en-US", "es-US"},
"max_seconds": 1800,
},
}
The values above are application policy, not claims about any particular service. Populate them from the selected service's current documentation and deployment configuration. Capabilities change. That is why they belong in configuration with tests, not in a hard-coded chain of optimistic assumptions.
There is also a privacy boundary. Product descriptions can contain patient or prescriber information even when they should not. Redact or reject sensitive content according to the organization's policy before external processing, and log the policy decision without logging the raw transcript. Security review should include audio storage, temporary files, encryption, region, deletion, and access controls. A hosted recognizer is a poor fit when the organization cannot approve the processing region or retention terms; a self-hosted path is the more controllable choice, at the cost of operating model downloads, capacity, patching, and quality evaluation. This is operational work, not a checkbox added after integration.
What does a rollout look like without corrupting the catalog?
Start in shadow mode: run the fallback path for a sampled set, compare normalized transcripts and schema validation outcomes, and keep the canonical catalog unchanged. Measure field-level acceptance rather than word-level similarity. A transcript that differs in punctuation but preserves a 10 mL unit may be useful; a fluent transcript that changes a concentration is dangerous. This is a real trade-off: word error rate is easy to chart, while a correct concentration and package count require domain validation that takes longer to build.
Then gate writes by state. ready can proceed to deterministic parsing, needs_review can create a work item, and rejected can retain a reason plus the source checksum. Attach the adapter and policy version to every enrichment event. When a reviewer corrects "10 mg" to "100 mg", that correction should be traceable to the source audio and parser version without exposing the audio in an ordinary application log.
I would test the routing table with four fixtures before enabling automatic writes: a transient upstream failure, an unsupported codec, an empty result, and a transcript containing a unit boundary. Add replay tests using scrubbed recordings. Also test that a late response from the primary cannot overwrite a completed fallback result. That race is easy to miss because both requests look successful in isolation.
SSE is useful for delivering job status to an operator dashboard when the browser needs one-way updates; MDN documents the event-stream format and reconnection behavior. It should report states such as queued, transcribing, review, and complete, not partial clinical attributes. The catalog writer remains the authority for committed data.
Portability is earned through this separation, but the more immediate win is controlled change. A recognizer can be replaced, a codec policy can be tightened, or a compliance rule can move without rewriting the product parser. The catalog receives a stable event contract either way.
Sources
- https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events
- https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API
- https://platform.openai.com/docs/guides/speech-to-text
- https://cloud.google.com/speech-to-text/docs
- https://docs.aws.amazon.com/transcribe/latest/dg/what-is.html
Top comments (0)