A model-availability check should be a short-lived capability decision, not a prerequisite that makes every transcription request wait on a provider catalog. In a fintech moderation pipeline, I would ask the selected adapter for a cached, tenant-scoped capability snapshot, attach the snapshot version to the work item, and still treat the transcription call as authoritative. That handles model withdrawal, rate limiting, and stale discovery data without pretending that a preflight check can provide exactly-once behavior.
The check is not a reservation.
That is the trade-off.
The practical rule is: preflight for routing and operator visibility; enforce availability at the call boundary; record both decisions in an audit trail. A negative result should defer the report or choose an approved fallback. A positive result should permit an attempt, never guarantee success.
What does availability mean before a transcription call?
“Available” is too vague for a payment or moderation backend. It can mean that a model identifier exists in a catalog, that the current account may use it, that the requested region supports it, that the input is within policy, or that the provider is accepting traffic right now. Those are different predicates with different freshness.
For audio attached to a moderation report, I use a capability record with explicit dimensions: provider adapter, model family, input format, region, authorization scope, and expiry time. The record also carries a monotonically increasing snapshot version. The worker stores that version beside the report ID, audio digest, and policy revision, so a later reviewer can see what the system believed before the call and what the provider actually returned.
A catalog lookup answers “is this configured?” A capability probe answers “did this route recently work?” Neither answers “will this particular request succeed now?” The check must therefore be advisory. The call path remains the source of truth, and the report transition is idempotent: the same report ID and audio digest cannot produce two accepted transcription results because a worker retried after losing its response.
How should the preflight decision fit the workflow?
Start with an internal interface, not a provider-shaped health endpoint. The adapter owns provider-specific model names, authentication, and response parsing; the moderation workflow owns policy, audit records, and retry semantics.
type Capability struct {
Adapter, Model, Region string
Snapshot uint64
CheckedAt, ExpiresAt time.Time
Allowed bool
Reason string
}
type Transcriber interface {
Capability(context.Context, CapabilityRequest) (Capability, error)
Transcribe(context.Context, TranscriptionRequest) (Transcription, error)
}
The worker reads a capability snapshot only when it is missing or expired. A fresh positive snapshot lets the job enter an attempting state. An expired or negative snapshot moves it to deferred, with a reason an operator can query. It does not silently substitute a different model: a fallback is a policy decision, and changing the model can change language coverage, confidence interpretation, retention controls, and the evidence shown to a reviewer.
At the call boundary, the worker passes a deterministic idempotency key derived from the report ID, audio hash, model, and policy revision, where the selected provider contract supports such a key. If it does not, the application records an intent before the call and deduplicates completed results on its own key. This is an exactly-once mindset implemented through durable state and reconciliation, not a claim that a distributed network can make the remote side exactly once.
func process(ctx context.Context, t Transcriber, repo Repository, job Job) error {
cap, err := repo.FreshCapability(ctx, job.Adapter, job.Model, job.Region)
if err != nil { return repo.Defer(ctx, job.ID, "capability lookup unavailable") }
if !cap.Allowed { return repo.Defer(ctx, job.ID, cap.Reason) }
key := strings.Join([]string{job.ReportID, job.AudioHash,
job.Model, job.PolicyRevision}, ":")
if done, err := repo.CompletedByKey(ctx, key); err != nil {
return err
} else if done { return nil }
if err := repo.RecordAttempt(ctx, key, cap.Snapshot); err != nil {
return err
}
result, err := t.Transcribe(ctx, TranscriptionRequest{
ReportID: job.ReportID, AudioHash: job.AudioHash,
Model: job.Model, Region: job.Region,
})
if err != nil { return classifyAndSchedule(ctx, repo, key, err) }
return repo.CommitTranscription(ctx, key, result)
}
The example intentionally does not expose a provider route. Adapters can map Capability to a model-list endpoint, a supported-model document, or a guarded probe, while the rest of the service remains unchanged. That is useful portability: a replacement changes one adapter and its evidence tests, not the ledger of moderation decisions.
Which failures should the check prevent, and which can it not?
A preflight check prevents predictable work. It can catch a misspelled model identifier, an unauthorized model, an unsupported region, or a local policy that forbids sending the audio. It can also keep a queue from filling with requests after a snapshot turns negative.
It cannot reserve capacity. Between the check and the request, credentials can expire, a regional route can become unavailable, or the account can be throttled. HTTP 429 indicates that a server received too many requests in a period and may provide retry guidance; it is a response to handle at the call boundary, not proof that the earlier model check was wrong.
Retry transient transport failures and rate limits with bounded exponential backoff and jitter, respecting any server-provided delay. Do not retry authorization failure as congestion. Do not retry malformed audio until validation changes the input. After the retry budget, leave a durable needs_review or deferred state. The human-review queue must not interpret “no transcript” as “no moderation issue.”
I also separate availability evidence from content handling. Audio may contain personal or payment-related information, and a capability probe should not upload it merely to test a model. The probe should use metadata or a provider-supported non-content operation when possible; if a real test request is required, data classification, retention, and audit rules must be explicit. OWASP’s Top 10 for LLM Applications is a useful checklist for risks around untrusted input, sensitive information, and excessive agency.
What should be measured for portability?
A provider-neutral dashboard needs more than success rate. Track capability freshness, negative reasons, time from preflight to call, calls rejected after a positive snapshot, rate-limit responses, retry exhaustion, queue age, and the percentage of reports routed to an approved fallback. Partition those measurements by adapter, model, region, audio class, and policy revision. Otherwise a global “transcription healthy” number can conceal that one regulated region is failing.
The decision table is deliberately small:
| Observation | Action | Why |
|---|---|---|
| Fresh, permitted snapshot | Attempt the call | The route is eligible, not guaranteed |
| Missing or expired snapshot | Refresh or defer | Avoid an unexamined route |
| Provider rate limit | Back off and retry | Capacity changed after preflight |
| Permanent policy failure | Escalate or reject | Repeating cannot change policy |
This table has a limit: it cannot encode a provider’s full operational state. Keep the adapter contract narrow and record exceptions as policy data rather than hiding them in retry code.
The main limitation is temporal: a snapshot can be correct when recorded and stale before the request is accepted. A longer TTL reduces discovery traffic but increases stale routing; a shorter TTL improves freshness but makes the catalog a more visible dependency. For a regulated review queue, I would accept occasional deferral rather than hide that uncertainty behind an automatic model change, because the latter changes the evidence path and complicates reconciliation.
Log identifiers, not raw audio. A useful event has a report ID, audio digest, adapter, model, region, capability snapshot, policy revision, attempt number, outcome class, and timestamps. Keep the schema stable across adapters so reconciliation can compare providers without translating an entire provider vocabulary into the core ledger. Retain raw payloads only under the applicable data-retention policy.
Test the awkward interval deliberately: return a positive capability result, revoke it before transcription, then return a rate limit; simulate a worker timeout after the remote side accepted the request; replay the same job; and deliver a late result after escalation. The expected invariant is not “the check always predicts success.” It is “each report has one auditable decision path, and a late or duplicated result cannot create a second accepted outcome.”
This is where portability becomes measurable. Run the same contract tests against every adapter: capability normalization, authorization classification, retry hints, idempotency behavior, transcript metadata, and error mapping. A new provider is acceptable only when it preserves those contracts or the policy records the difference. “It has a Whisper-compatible model” is not a sufficient integration specification.
A compact rollout for production
Ship the capability record and audit event first, with the transcription path still authoritative. Populate snapshots in shadow mode and compare predictions with actual call outcomes. Next, use negative snapshots only to defer new work, while allowing already-attempting jobs to finish under normal retry policy. Finally, enable approved fallback rules one policy revision at a time, with queue review for every change affecting evidence or regional handling.
Keep snapshot TTLs short enough to limit stale decisions, but long enough to avoid turning the provider catalog into a dependency on every job. The right value comes from observed change frequency, queue latency, and the cost of a false positive, not from a universal number.
The durable design is modest: capability discovery informs routing, the transcription response decides execution, and the audit trail reconciles the two. That separation gives a fintech moderation system a defensible answer when a reviewer asks why audio was deferred, why a fallback was selected, or why a retry did not create duplicate evidence.
Top comments (0)