Short answer: route production supplier-invoice audio to an external speech-to-text provider whenever the model catalog does not advertise an available ASR model, and record that routing decision against the tenant before attempting an upload. A 404, 501, or available=false is a capability signal, not a reason to keep retrying.
For a marketplace, transcription is only the first stage. The real output is an auditable chain from an audio attachment to extracted invoice fields, with every attempt attributable to a tenant, region, provider, and immutable request ID. The least complex design is therefore a small ASR port backed by a catalog check and a tenant usage ledger; the invoice extractor never needs to know which provider heard the audio.
What belongs inside the provider boundary?
Suppose suppliers leave spoken corrections to invoices: “purchase order 18472, quantity 16, tax 7.5 percent.” The marketplace needs text before it can extract the purchase order, quantity, and tax fields. It also needs to answer a less glamorous question at month-end: which tenant caused each transcription charge? A shared API key and an unlabelled total are insufficient for reconciliation, even if the transcription itself is accurate.
Make one logical operation ID at ingestion and carry it through audio storage, transcription, field extraction, review, and posting. The ID should be stable across retries. A separate attempt ID should change on every network call. This distinction gives the system exactly-once posting semantics without pretending that HTTP delivery itself is exactly once: duplicated attempts may exist, but only one accepted result can advance the invoice state.
The provider boundary can remain small:
package main
import (
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
)
type TranscriptionRequest struct {
OperationID string
TenantID string
Region string
Audio []byte
}
type TranscriptionResult struct {
ProviderRequestID string
Text string
}
type Transcriber interface {
Transcribe(context.Context, TranscriptionRequest) (TranscriptionResult, error)
}
func operationID(tenantID, invoiceID string, audio []byte) string {
h := sha256.New()
fmt.Fprintf(h, "%s\x00%s\x00", tenantID, invoiceID)
h.Write(audio)
return hex.EncodeToString(h.Sum(nil))
}
func main() {
audio := []byte("example audio bytes")
fmt.Println(operationID("tenant-us-042", "invoice-18472", audio))
}
The hash is an idempotency identity, not a security boundary. Keep authorization tenant-scoped, encrypt the original audio, and define retention independently for source audio, transcript, and extracted fields. In regulated payment workflows, deletion obligations and audit retention can pull in opposite directions; legal and compliance owners must resolve that policy rather than letting an SDK default decide it.
One subtle point matters here. “Exactly once” belongs at the state transition that accepts a transcript or posts invoice fields, not at the speech request. If a timeout hides a successful provider response, another attempt may be necessary. The ledger must preserve both attempts while a uniqueness constraint on operation_id protects the accepted result.
Auditability survives.
How should a backend handle an audio transcription API with 404, 501, or available=false?
Check the model catalog before building a multipart upload or a retry queue. If no available ASR entry is present, stop at admission control and select the configured external provider. Do not classify that state as transient. Exponential backoff helps with 429 responses and some network failures; it cannot create a capability that the catalog says is unavailable.
This distinction prevents a surprisingly expensive failure mode: 10,000 invoice recordings enter a queue, each receives a non-serviceable response, and each is retried several times while the useful work remains at zero. No amount of jitter repairs the decision. Fail fast.
The following Go program performs the catalog gate against the documented model route. It uses an environment variable for the key, sets the method explicitly, handles 429 with Retry-After or bounded exponential delay, checks every status, and treats absence of an available ASR model as a routing result rather than an exception. It is intentionally conservative about model naming because catalogs evolve.
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"os"
"strconv"
"strings"
"time"
)
type Model struct {
ID string `json:"id"`
Capability string `json:"capability"`
Available bool `json:"available"`
}
type ModelCatalog struct {
Object string `json:"object"`
AvailableOnly bool `json:"available_only"`
Count int `json:"count"`
Data []Model `json:"data"`
}
func retryDelay(resp *http.Response, attempt int) time.Duration {
if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
return time.Duration(seconds) * time.Second
}
return time.Duration(1<<attempt) * time.Second
}
func fetchModels(ctx context.Context, client *http.Client, baseURL, key string) (ModelCatalog, error) {
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, strings.TrimRight(baseURL, "/")+"/v1/ai/models", nil)
if err != nil {
return ModelCatalog{}, err
}
req.Header.Set("Authorization", "Bearer "+key)
resp, err := client.Do(req)
if err != nil {
return ModelCatalog{}, err
}
if resp.StatusCode == http.StatusTooManyRequests {
resp.Body.Close()
time.Sleep(retryDelay(resp, attempt))
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
defer resp.Body.Close()
return ModelCatalog{}, fmt.Errorf("model catalog returned status %d", resp.StatusCode)
}
defer resp.Body.Close()
var catalog ModelCatalog
if err := json.NewDecoder(resp.Body).Decode(&catalog); err != nil {
return ModelCatalog{}, err
}
return catalog, nil
}
return ModelCatalog{}, errors.New("model catalog rate limit retry budget exhausted")
}
func hasASR(catalog ModelCatalog) bool {
for _, model := range catalog.Data {
name := strings.ToLower(model.ID + " " + model.Capability)
if model.Available && (strings.Contains(name, "asr") || strings.Contains(name, "transcription")) {
return true
}
}
return false
}
func main() {
key := os.Getenv("INFRAI_API_KEY")
if key == "" {
panic("INFRAI_API_KEY is required")
}
baseURL := os.Getenv("AI_API_BASE_URL")
if baseURL == "" {
panic("AI_API_BASE_URL is required")
}
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
catalog, err := fetchModels(ctx, &http.Client{Timeout: 12 * time.Second}, baseURL, key)
if err != nil {
panic(err)
}
if !hasASR(catalog) {
fmt.Println("route=external_asr reason=no_available_asr_model")
return
}
fmt.Println("route=catalog_asr reason=available_model")
}
Run it only after setting INFRAI_API_KEY and AI_API_BASE_URL; keys must never enter source control. A JavaScript or Node.js service should implement the same state machine even though this article uses Go: catalog says available, proceed; catalog says unavailable, route externally; transport is rate-limited, retry within budget; authorization or request validation fails, surface the response and stop.
I'm not sure which provider will satisfy every marketplace's residency and retention obligations, because contract terms, enabled regions, and tenant agreements differ. Resolve that uncertainty with a documented data-flow review and the current provider terms. Do not infer compliance from a region label alone.
How does a tenant ledger make ASR cost visible?
Provider invoices are reconciliation inputs, not the primary application ledger. Record an estimated or pending usage entry before dispatch, then attach the provider request ID and final billed amount when those become available. Never manufacture a cost from audio duration unless the selected provider's current billing contract defines that calculation.
A practical record needs operation_id, attempt_id, tenant_id, invoice_id, region, provider, model, audio_seconds, status, provider_request_id, currency, and cost_minor_units. Keep cost_minor_units nullable until reconciliation. Store timestamps for admitted, dispatched, completed, accepted, and reconciled states. The accepted transcript should reference the winning attempt, while rejected duplicates remain in the audit trail.
This is where a stable capability contract is valuable. Infrai exposes one REST API under one key and provides per-call cost, vendor, latency, and request metadata; its strategic advantage is that the provider behind a capability can change without forcing the application contract to change, while the same key and billing surface reduce reconciliation joins across backend capabilities. For this ASR workflow, however, the catalog gate remains authoritative and an external provider is the correct production route until an available speech model is advertised.
There is a catch. A unified contract cannot erase provider-specific data residency, retention, vocabulary, diarization, or accuracy requirements. If a tenant contract names a cloud, mandates a deployment boundary, or requires a feature absent from the common contract, pin that tenant to the qualifying provider and preserve the exception in policy data. The abstraction should make a vendor change controlled; it should not make the vendor invisible to auditors.
Keep allocation rules boring. Attribute direct ASR usage to the tenant that owns the invoice, place shared extraction overhead in a separately defined pool, and version the allocation policy. When the provider statement arrives, reconcile totals by provider, currency, and billing period before closing tenant entries. A cent-level mismatch should become an explicit reconciliation exception, not an unexplained rounding adjustment.
Which managed and self-hosted speech options deserve evaluation?
No provider wins on every axis. Evaluate the exact US and EU regions you can contract for, retention controls, supported audio formats, vocabulary needs, operational ownership, and the identifiers available for billing reconciliation. Accuracy should be tested with representative supplier accents, product codes, tax phrases, and noisy mobile recordings; public benchmark scores do not substitute for that corpus.
| Option | Operational boundary | Tenant cost attribution | When it fits | When to choose something else |
|---|---|---|---|---|
| OpenAI speech-to-text | Managed API | Tag the application ledger with operation and provider request identifiers | Teams that want a managed integration and can accept the applicable data terms | Choose a region-specific cloud option when the tenant contract requires a particular deployment boundary |
| Amazon Transcribe | Managed AWS service | Reconcile application operation IDs with AWS account and billing records | Marketplaces already governing tenant workloads inside AWS | Stick with another provider when the required language, contract, or region is a better match elsewhere |
| Google Cloud Speech-to-Text | Managed Google Cloud service | Join internal operation IDs to project-level usage and billing exports | Teams operating their governed audio path in Google Cloud | Choose another route when procurement or data controls mandate a different cloud |
| Azure AI Speech | Managed Azure service | Join tenant operations to the selected Azure subscription and billing data | Organizations whose tenant controls and contracts are centered on Azure | Use a different option when the approved region or feature set does not meet the workload |
| Self-hosted Whisper | Runtime and data plane operated by your team | Allocate measured compute and storage through an internal policy | Workloads requiring infrastructure control and teams prepared to operate inference | Avoid it when the team cannot own capacity, upgrades, monitoring, and model-risk review |
Those rows are architecture starting points, not claims of regulatory certification. Current product documentation and signed contracts decide what is permitted. Your mileage may vary sharply with language mix and recording quality.
The comparison also reveals why “use a chat model as speech-to-text” is the wrong fallback. Anthropic Claude, Google Gemini, OpenRouter, and Together are real model options, but their presence in a model stack does not prove that the specific endpoint receiving invoice audio provides ASR. A chat model consumes its documented modalities; it does not turn an unavailable transcription capability into speech recognition. Use a documented speech product for transcription, then send the resulting text to a suitable model for structured invoice-field extraction. If moderation is required, make that a separate, explicit policy stage rather than assuming transcription implies content review.
Retries are not routing.
How can the route change without breaking audit continuity?
Begin with shadow accounting: send production traffic through the approved external ASR provider, but calculate tenant allocation entries without charging tenants until reconciliation agrees with the provider statement. Sample completed transcripts for field-level quality on purchase order, amount, tax, currency, and supplier identity. Audio quality can differ by tenant, so aggregate accuracy is not enough.
Then enforce three controls. First, admit work only after region and tenant policy select an allowed provider. Second, cap retries by error class: retry 429 according to Retry-After, retry bounded transport failures with exponential backoff, and do not retry capability-unavailable decisions. Third, accept a transcript with a compare-and-set on the operation ID so a late duplicate cannot overwrite the result already used for invoice posting.
Keep the migration switch in policy data, not scattered through handlers. A change in catalog availability can qualify a new route for testing, but it should not silently move regulated production traffic. Run the same representative corpus, complete the privacy and compliance review, compare reconciliation fields, and promote tenant cohorts deliberately.
Small steps win.
The final acceptance test is an audit question: given invoice 18472, can an operator identify the tenant, original audio digest, routing policy version, provider and model, every attempt, accepted transcript, extracted fields, human correction, and reconciled cost without searching several unrelated systems? If the answer is no, the integration is not finished, even when the transcript text looks perfect.
Top comments (0)