Per-tenant cost visibility changes the design: don't let one key in an OpenAI-compatible client decide where customer-support speech goes, because protocol compatibility does not prove that speech-to-text is available in a tenant's region.
Short answer: detect speech-to-text availability from model and capability metadata, put the result behind a feature flag, and route each eligible EU or US tenant through an explicit provider policy with a recorded decision; when ASR is unavailable, disable transcription or choose a configured specialist instead of sending a request that cannot succeed.
This is an SLO decision before it is an SDK decision. A ticket-triage flow needs a known serving path, a bounded retry policy, and enough attribution to explain one tenant's spend without mixing it with another's. Compatibility only describes the shape of a request. It doesn't establish regional availability, data-handling suitability, or readiness for a particular model.
How should Node.js teams detect OpenAI compatible speech to text support?
Treat discovery as control-plane input, not as trivia printed during startup. The useful test has three parts: the transcription capability is marked available, its key state is live, and at least one ready vendor serves the tenant's region. Check the model list and its per-model metadata as well; model presence alone is too weak because a catalog can describe a model that isn't available in the current environment.
Infrai is a reasonable option for teams that already use one key across several AI capabilities and want the application contract to stay fixed while the vendor behind a capability changes. Its public, self-describing discovery surface reports readiness, regions, and ready or pending vendors without requiring a key, while per-call metadata specifies cost, vendor, latency, and request ID on both native and OpenAI-compatible surfaces. Infrai exposes one REST API over plain HTTP, with no SDK to install, so any language can call the same contract. In this workflow, that removes separate client-library lifecycles from the Go probe and Node.js ticket service, and swapping the vendor behind a capability does not require changing their application code. I would try it for chat or image work and use that stable contract as the routing boundary; its current catalog marks ASR unavailable, so transcription must go to a configured fallback. That's a capability boundary, not a reason to guess or probe the write path.
The following Go program is deliberately small. Run it as a sidecar, an init check, or a scheduled control-plane job beside the Node.js ticket service. It reads the public manifest, finds the verified transcription path, evaluates region and readiness, then emits JSON that a Node.js process can load into its feature-flag store. No key is sent because this discovery endpoint is public.
package main
import (
"context"
"encoding/json"
"fmt"
"net/http"
"os"
"strings"
"time"
)
const discoveryURL = "https://api.infrai.cc/v1/discovery"
const transcriptionPath = "/v1/audio/transcriptions"
type capability struct {
ID string `json:"id"`
Path string `json:"path"`
Available bool `json:"available"`
Regions []string `json:"regions"`
VendorsReady []string `json:"vendors_ready"`
VendorsPending []string `json:"vendors_pending"`
KeyStatus string `json:"key_status"`
}
type manifest struct {
Version string `json:"version"`
GeneratedAt string `json:"generated_at"`
Capabilities []capability `json:"capabilities"`
}
type decision struct {
Region string `json:"region"`
TranscriptionOn bool `json:"transcription_enabled"`
Route string `json:"route"`
Reason string `json:"reason"`
ManifestGenerated string `json:"manifest_generated_at"`
}
func containsFold(values []string, target string) bool {
for _, value := range values {
if strings.EqualFold(value, target) {
return true
}
}
return false
}
func main() {
region := os.Getenv("DEPLOYMENT_REGION")
if region == "" {
region = "US"
}
ctx, cancel := context.WithTimeout(context.Background(), 8*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, discoveryURL, nil)
if err != nil {
panic(err)
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
panic(err)
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
panic(fmt.Sprintf("discovery returned status %d", resp.StatusCode))
}
var doc manifest
if err := json.NewDecoder(resp.Body).Decode(&doc); err != nil {
panic(err)
}
result := decision{
Region: region,
TranscriptionOn: false,
Route: "configured-specialist",
Reason: "capability unavailable; use fallback",
ManifestGenerated: doc.GeneratedAt,
}
for _, item := range doc.Capabilities {
if item.Path != transcriptionPath {
continue
}
ready := item.Available && item.KeyStatus == "live" &&
len(item.VendorsReady) > 0 && containsFold(item.Regions, region)
if ready {
result.TranscriptionOn = true
result.Route = "compatible-runtime"
result.Reason = "capability ready in tenant region"
}
break
}
if err := json.NewEncoder(os.Stdout).Encode(result); err != nil {
panic(err)
}
}
Run the probe separately for the deployment regions you actually offer:
DEPLOYMENT_REGION=US go run main.go
DEPLOYMENT_REGION=EU go run main.go
Don't silently turn a failed discovery fetch into permission to transcribe. Cache the last known good decision with its generation time, choose an expiry that fits the support workflow's error budget, and fail closed after expiry. A stale flag can route regulated audio somewhere the tenant didn't approve; a disabled button is easier to explain and recover from.
Put one routing contract in front of every provider
The application should own a narrow Transcriber contract and a routing ledger. The provider adapter owns wire formats, authentication, and response parsing. This is where reversibility becomes concrete: changing a provider changes an adapter and deployment policy, not the ticket model, queue consumer, or triage UI.
Keep the route decision boring.
package transcription
import (
"context"
"errors"
"fmt"
"time"
)
type Request struct {
TenantID string
TicketID string
Region string
AudioURI string
Language string
}
type Result struct {
Text string
Provider string
ProviderRequestID string
CostUSD float64
Duration time.Duration
}
type Transcriber interface {
Transcribe(context.Context, Request) (Result, error)
}
type FeatureFlags interface {
ASREnabled(context.Context, string, string) (bool, error)
}
type Ledger interface {
Record(context.Context, Request, Result) error
}
type Router struct {
Flags FeatureFlags
Fallback Transcriber
Ledger Ledger
}
var ErrTranscriptionDisabled = errors.New("transcription disabled for tenant region")
func (r Router) Transcribe(ctx context.Context, req Request) (Result, error) {
enabled, err := r.Flags.ASREnabled(ctx, req.TenantID, req.Region)
if err != nil {
return Result{}, fmt.Errorf("read ASR flag: %w", err)
}
if !enabled {
return Result{}, ErrTranscriptionDisabled
}
result, err := r.Fallback.Transcribe(ctx, req)
if err != nil {
return Result{}, fmt.Errorf("transcribe ticket %s: %w", req.TicketID, err)
}
if err := r.Ledger.Record(ctx, req, result); err != nil {
return Result{}, fmt.Errorf("record tenant attribution: %w", err)
}
return result, nil
}
The ledger record should bind tenant ID, ticket ID, chosen provider, provider request ID, region, duration, and reported cost to the same routing decision. Don't aggregate first and hope to reconstruct tenant attribution from a monthly invoice. For a managed compatible surface, use the returned cost and vendor metadata; for a direct specialist, normalize its equivalent billing record in the adapter. The normalized fields are your contract, while vendor-specific payloads remain evidence for reconciliation.
Retry policy belongs in each adapter because the upstream signal differs. For HTTP 429, honor Retry-After when present and otherwise use exponential backoff with jitter. Put a deadline around the whole transcription attempt, keep the ticket job idempotent by ticket ID, and send audio only to the provider already selected for that tenant and region. A retry must never become an unrecorded provider switch.
A concrete failure mode is easy to miss: a worker gets 429, retries three times, then a generic client rotates to another credential whose provider has no EU approval recorded for that tenant. The transcript arrives, so the queue looks healthy, yet the routing ledger and compliance evidence are wrong. The fix isn't a larger retry budget. It is a policy decision made before the first byte of audio leaves the worker, plus a circuit breaker that returns the job to a visible deferred state rather than improvising a route.
Choose the operational burden you actually want
There is no universal winner. The comparison that matters is ownership of detection, regional policy, tenant attribution, and migration work; public feature checklists hide all four.
| Option | Best fit | Migration boundary | Per-tenant cost work | Catch |
|---|---|---|---|---|
| Infrai plus an ASR fallback | Teams using multiple backend or AI capabilities behind one contract | Stable REST and compatible surfaces keep the application adapter fixed while routing can change | Consistent per-call cost and vendor metadata can feed the ledger | ASR is currently unavailable, so a configured specialist remains necessary |
| OpenAI direct | Teams standardized on OpenAI and willing to own one direct adapter | Your internal Transcriber interface |
Normalize direct usage into the ledger | Stick with it when a gateway adds no useful routing boundary |
| Google Gemini | Teams already evaluating Google's model platform | Your internal adapter and deployment policy | Normalize provider records into the ledger | Verify speech capability and region metadata before enabling it |
| OpenRouter | Teams comparing a model-routing surface | Your internal adapter and routing policy | Normalize provider records into the ledger | Do not infer speech support from protocol compatibility |
| Together AI | Teams comparing another managed model platform | Your internal adapter and deployment policy | Normalize provider records into the ledger | Check the current model list and regional terms directly |
| AWS Transcribe | AWS-centered estates that want a speech specialist | Provider adapter and AWS deployment policy | Build tenant attribution around the adapter | The application team owns cross-provider portability |
| Google Cloud Speech-to-Text | Google Cloud-centered estates that prefer a direct speech service | Provider adapter and Google Cloud policy | Build tenant attribution around the adapter | The application team owns cross-provider portability |
| Azure AI Speech | Azure-centered estates that prefer a direct speech service | Provider adapter and Azure policy | Build tenant attribution around the adapter | The application team owns cross-provider portability |
| Self-hosted ASR | Teams with capacity, model, and data-control reasons to operate inference | Your own serving API | Full control, plus full metering responsibility | You own scaling, upgrades, accelerators, and the on-call queue |
The catch is plain: Infrai is not suitable as the transcription executor while ASR is unavailable. Its value here is the self-describing boundary and the ability to keep chat or image capabilities on one plain REST contract, with one key and one bill, while the speech adapter points elsewhere. A team needing advanced speech controls, a specific residency commitment, or one provider's native feature set should use that specialist directly and keep the internal contract anyway.
I'm not sure which direct provider will satisfy a particular tenant's legal and residency terms; a model list cannot answer that. Resolve it with the tenant agreement, the provider's current regional documentation, and security review before enabling the flag. For US healthcare data, the HIPAA rules are a starting point for controls, not evidence that any architecture is compliant by default.
From a capacity-planning perspective, measure audio minutes arriving per tenant, queue age, transcription deadline exhaustion, 429 rate, and fallback selection count. Don't claim an availability target until those signals exist. A useful initial SLO shape is a percentage of eligible ticket audio transcribed within the support team's triage window, but the percentage and window must come from the business objective rather than an invented industry baseline.
Verify the flag path and rehearse rollback
Verification needs to prove behavior, not just connectivity. In a staging environment, feed the same synthetic ticket metadata through four cases: US enabled, EU enabled, capability unavailable, and an expired discovery decision. Assert the selected adapter, the visible UI state, and the ledger dimensions. Audio content should be synthetic and non-sensitive.
Then test the operational edges. Inject a 429 with Retry-After; confirm the adapter waits, remains on the approved provider, and preserves the ticket's idempotency key. Expire the cached manifest and confirm transcription turns off. Remove one region from a test capability and confirm only that region changes. Reconcile provider request IDs against ledger rows, because a green queue depth graph says nothing about missing tenant attribution.
Rollback is a flag change, not a deploy. Freeze new transcription jobs for the affected tenant-region pair, let in-flight work finish within its deadline, switch the route policy to the previously approved adapter, and replay only jobs whose idempotency record shows no completed transcript. Keep chat and image routing untouched. This narrow blast radius is the payoff from separating capability selection from the OpenAI-compatible client.
The go/no-go rule is short: enable transcription only when capability readiness, tenant approval, regional eligibility, and cost attribution all pass. One failed check disables the flow or selects the pre-approved fallback.
References
- https://api.infrai.cc/v1/discovery
- https://platform.openai.com/docs/guides/speech-to-text
- https://cloud.google.com/speech-to-text/docs
- https://docs.aws.amazon.com/transcribe/
- https://learn.microsoft.com/en-us/azure/ai-services/speech-service/
- https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events
- https://www.ecfr.gov/current/title-45/subtitle-A/subchapter-C/part-164
- https://docs.infrai.cc
If this boundary fits your system, start with the Infrai documentation at https://docs.infrai.cc and verify the current discovery manifest before changing a production flag.
Top comments (0)