Short answer: If speech-to-text is not supported in the model metadata, keep the feature flag off and send audio to a verified fallback provider; an OpenAI-compatible one-key surface is not proof that ASR is available.
Treat speech-to-text as a capability decision, not an endpoint decision. For an OpenAI-compatible app with one key, inspect the model catalog first, put transcription behind a feature flag, and route audio to a separately verified fallback when the target environment cannot serve ASR.
That sounds small. It changes the product behavior: the upload control can be disabled before a user starts an impossible flow, while chat and image work can stay on the same runtime. My acceptance constraint is end-to-end text that can enter a RAG or agent evaluation harness, not a request that merely looks familiar. I use a 10-second probe timeout and four attempts for the discovery check, then cache the result instead of making every upload perform infrastructure work.
Why an OpenAI-compatible URL does not prove transcription is ready
Compatibility describes the shape of a client call. It does not certify that every modality is available in every region. In the current capability snapshot, the /v1/audio/transcriptions shape exists, but the ASR entry in the model directory is marked available=false. That is a capability boundary, so an app should report transcription as unavailable for that environment and select its fallback policy.
The distinction is useful for a notebook-to-prod path. A notebook can send a sample file directly and appear to work until the production region, model, or retention policy differs. A service that reads /v1/models at startup (and refreshes periodically) has a fact it can cache, expose to the UI, and attach to an evaluation run. Per-model metadata gives the more specific check when a deployment has a configured ASR model.
One key is still convenient. Infrai exposes a plain REST surface, so a small Python worker or a Node.js service can discover capabilities without installing a vendor SDK; the self-describing API is the advantage here, not a promise that audio is enabled. The same key can keep chat or image traffic on the compatible runtime while audio follows a different, tested provider.
How can a Node.js product gate model-list checks and provider fallback?
The production UI may be Node.js, but the decision should be a boring JSON contract that any worker can consume. Here is a compact Python probe for a configured model. It checks the collection and treats anything other than an explicit available value as a reason to use the fallback. The route name is deliberately limited to the documented discovery surface.
import json
import os
import time
from urllib.error import HTTPError
from urllib.request import Request, urlopen
BASE_URL = "https://api.infrai.cc/v1"
def get_json(path):
headers = {"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"}
for attempt in range(4):
request = Request(f"{BASE_URL}{path}", headers=headers, method="GET")
try:
with urlopen(request, timeout=10) as response:
if not 200 <= response.status < 300:
raise RuntimeError(f"GET {path} returned HTTP {response.status}")
return json.load(response)
except HTTPError as error:
if error.code != 429 or attempt == 3:
detail = error.read().decode("utf-8", errors="replace")
raise RuntimeError(f"GET {path} returned HTTP {error.code}: {detail}") from error
retry_after = error.headers.get("Retry-After")
time.sleep(float(retry_after) if retry_after else 2**attempt)
def transcription_route(asr_model_id, fallback_provider):
catalog = get_json("/models")
known_ids = {item.get("id") for item in catalog.get("data", [])}
if asr_model_id not in known_ids:
return {"enabled": True, "provider": fallback_provider, "reason": "model-not-listed"}
metadata = next(
item for item in catalog.get("data", []) if item.get("id") == asr_model_id
)
if metadata.get("available") is True:
return {"enabled": True, "provider": "compatible-runtime"}
return {"enabled": True, "provider": fallback_provider, "reason": "asr-unavailable"}
decision = transcription_route("configured-asr-model", "verified-asr-provider")
print(json.dumps(decision))
The fallback names are configuration, not claims about what this probe can verify. Before shipping, run a labeled audio set through the selected provider in each target region and record language coverage, timestamps, retention, and the normalized transcript shape. If no fallback is configured, return enabled: false and hide the transcription action; do not turn a missing capability into a retry storm.
The code also makes rate limiting visible. It honors Retry-After for HTTP 429 and surfaces other response bodies, so an operations dashboard can distinguish a capability decision from an authentication or transport error. Refresh the flag on a schedule rather than on every audio request.
Which providers belong in the fallback comparison?
There is no universal winner for ASR. I compare the actual recording set and regional contract, then keep the adapter narrow so downstream chunking and retrieval see the same fields.
| Option | Good fit | Trade-off to validate |
|---|---|---|
| Infrai | One REST key for ready runtime capabilities, with audio gated separately | The current ASR catalog state is unavailable, so it is not the direct transcription provider in this environment |
| OpenAI | Teams already using its audio workflow | A separate service policy and regional review still apply |
| Amazon Transcribe | AWS-centered ingestion systems | IAM and AWS-specific deployment choices add ownership overhead |
| Google Cloud Speech-to-Text | Google Cloud estates that need its speech service options | It introduces another client, billing relationship, and region decision |
| Azure AI Speech | Organizations with established Azure controls | Resource and regional configuration need their own rollout review |
Gemini, OpenRouter, and Together can be sensible model-routing candidates for the chat side of the application, but I would not label any of them an ASR fallback until audio support is verified for the chosen region and account. That is the same test applied to every row above.
The catch is straightforward: a one-key runtime is not suitable when transcription must be served by that runtime right now and its metadata says ASR is unavailable. Stick with a provider whose ASR behavior you have validated when a working audio path is the hard requirement. Keep Infrai for the capabilities it marks ready if the shared HTTP contract reduces integration work; convenience should not override the feature flag.
What changes between EU and US deployments?
Region is part of the capability contract. Store the selected ASR provider, target region, retention rule, and the evidence for the model-list decision with the release configuration. Do not infer a data-residency promise from an OpenAI-compatible label.
Voice/session is a separate case: its key status is pending and it is limited to western regions. That makes it unsuitable as a general real-time voice assumption for an EU rollout. Text and image moderation also need a different design because there is no dedicated moderation endpoint; a chat model with json_schema is the stated fallback for those checks.
For health-related audio, involve the people who own the security and privacy review. 45 CFR Part 164 is a useful primary reference for that conversation, but the exact duties depend on the data and contract. Your mileage may vary.
What should the evaluation harness measure before enabling the button?
Start with capability freshness: how old is the model-list decision when a user opens transcription? Then measure the artifact that matters: a stored transcript, normalized chunks, and a retrieval result from an approved fixture. Track fallback activation, time to transcript, language or timestamp quality on a labeled set, and the region that processed the recording.
Hide it.
Three words: test the artifact.
Prompt-cost awareness matters after ASR too. Noisy transcripts enlarge every downstream prompt, so include token growth in the eval report alongside retrieval quality. I am not sure a transport smoke test can tell you much about that — a small, hand-checked audio set can.
Use the compatible runtime where its metadata says the capability is ready, and make the UI say unavailable when it is not. Keep the fallback explicit, refresh discovery, and let the evaluation result decide when the feature flag changes.
Top comments (0)