The constraint that changes this decision is structured output correctness: a media moderation system cannot quietly turn a missing transcript into a plausible-looking chat answer. Short answer: treat 404, 501, and available=false as capability signals, inspect the model catalog, then route audio through a dedicated speech-to-text component before asking a chat model to classify the report.
This is an experiment note from the notebook-to-prod boundary. The simple approach was to post every recording to /v1/audio/transcriptions, retry a 404 or 501, and let the chat model fill in when speech-to-text was unavailable. The chosen approach checks capability first, preserves the audio job as an explicit state, and evaluates the transcript-to-JSON contract separately. Measure on your own recordings before copying either design.
Designing the transcript-to-report boundary
For a media report, define the output before selecting an audio model. A useful record might contain report_id, transcript, language, confidence_notes, decision, and reason_codes. The classifier should be instructed to return the schema you test, and the service should validate the result before it reaches human review. Invalid JSON is a product failure, not a cosmetic formatting issue.
Here is the narrow evaluation shape I would start with:
from dataclasses import dataclass
@dataclass
class ModerationCase:
report_id: str
expected_decision: str
expected_reason_codes: set[str]
audio_path: str
def score_report(case: ModerationCase, report: dict) -> dict:
decision_ok = report.get("decision") == case.expected_decision
reasons = set(report.get("reason_codes", []))
return {
"report_id": case.report_id,
"decision_correct": decision_ok,
"reason_codes_recall": len(reasons & case.expected_reason_codes)
/ max(1, len(case.expected_reason_codes)),
"schema_valid": isinstance(report.get("transcript"), str),
}
The hard part is not the happy-path transcript. Test clipped speech, background music, multiple speakers, code-switching, empty audio, and long recordings. Track word or character error against a labelled set, but also track the fields that drive review routing: wrong decision, missing reason code, and invalid schema. Prompt cost belongs in the same report because a longer transcript changes downstream token use even when recognition quality is stable.
One short warning: a passing transcription sample proves very little. It has to survive the distribution your moderators actually see.
What does an audio transcription API 404 or 501 mean when ASR is available?
An HTTP path and a usable model are different facts. /v1/audio/transcriptions can be part of the documented surface while the relevant ASR entry in the model catalog reports available=false. A 404 may describe the selected route or deployment; a 501 may describe a capability that the deployment does not expose. Neither response is a good reason to spend a retry budget.
I use three classifications in the worker:
-
available: the catalog contains a suitable speech-to-text model, so the audio job may proceed. -
unavailable: the catalog knows about the capability, but it cannot serve this request; select the configured ASR path or pause the feature. -
unknown: the catalog response does not establish capability; stop before upload and emit an operator-facing diagnostic.
That distinction matters in media moderation. A ten-minute clip can consume queue time and storage before a late transcription failure becomes visible. It also matters for observability: retryable transport failures, unsupported capability, malformed audio, and invalid structured output should be different counters.
How should a Node.js builder check audio transcription availability?
Start with discovery, not the multipart upload. The relevant catalog surfaces are /v1/models and /v1/models/{id}; the audio request is /v1/audio/transcriptions. Read the model identifier, capability, availability, and any region metadata returned by the catalog. Keep the route strings in one configuration object so a startup check and a request builder cannot drift apart.
The following Python check is deliberately small enough to paste into an eval harness. It consumes a saved catalog response and makes no claim about an undocumented response field beyond the data, capability, and available values used by the decision.
import json
import sys
TRANSCRIPTION_PATH = "/v1/audio/transcriptions"
ASR_CAPABILITIES = {"asr", "speech-to-text"}
def classify_catalog(catalog: dict) -> str:
models = catalog.get("data", [])
matches = [
model for model in models
if model.get("capability") in ASR_CAPABILITIES
]
if any(model.get("available") is True for model in matches):
return "available"
if matches:
return "unavailable"
return "unknown"
catalog = json.load(sys.stdin)
print(json.dumps({
"path": TRANSCRIPTION_PATH,
"state": classify_catalog(catalog),
}))
In a Node.js service, the same result should gate queue submission. The language of the worker is incidental; the state transition is the contract. Do not call a chat model with raw audio just because its interface accepts a message. A transcript, a moderation decision, and a structured moderation report are three different artifacts.
How do chat models, self-hosted ASR, and managed speech services compare?
The alternatives have different ownership boundaries. A self-hosted recognizer gives a team control over audio movement and deployment, but the team owns GPU capacity, model updates, language coverage, and evaluation. A managed speech service reduces inference operations, while adding account, region, quota, retention, and data-processing decisions. A chat model with audio input can be useful for a bounded multimodal experiment, but audio acceptance does not imply a stable transcription contract with timestamps, speaker labels, or predictable formatting.
For the moderation pipeline, compare options on the same labelled clips and the same report schema. Ask: can the system process the required languages in the US and EU regions? What happens at the maximum recording length? Can an operator replay a failed job safely? Which fields are guaranteed, and which are merely present in a sample? The answer should be evidence from tests and public specifications, not a feature checklist copied into a README.
There is a real trade-off here. A single general API can simplify credentials and integration when one application needs several backend capabilities, while a dedicated ASR service can make speech-specific behavior easier to reason about. Your mileage may vary. The boundary that matters is ownership of the transcript contract, not the number of endpoints in a product brochure.
When is a chat-model fallback the wrong choice?
The catch is that a fallback can hide the original failure. If the classifier receives a generated paraphrase instead of a transcript, a reviewer may never know that recognition was unavailable. Keep transcription_unavailable distinct from transcription_failed, and require an explicit provider-selection decision before a job is submitted. Retry only conditions your system has classified as transient, such as a rate-limit policy you can observe; do not retry a catalog state that says the capability is unavailable.
Choose dedicated ASR or self-hosting when verbatim text, timestamps, speaker separation, residency controls, or repeatable quality are release requirements. Use a chat model after transcription when the task is classification, extraction, or summarization and the report schema has tests. Stay with the simpler catalog-gated architecture when the feature is still exploratory, but keep the eval harness and structured validation from the first notebook.
Before shipping, record transcript quality, structured-output validity, moderation recall, queue latency, region behavior, rejection reasons, and prompt cost. Those measurements tell you whether the alternative is fit for the job. A retry count does not.
Top comments (0)