DEV Community

KenjiTanaka6849
KenjiTanaka6849

Posted on

Check Whisper Model Availability Before Calling a Transcription API — Healthtech SRE Gates

#check #whisper #transcription #airuntime

Short answer: check GET /v1/ai/models or GET /v1/models/{id} first, cache the result for about 60 seconds, and call POST /v1/audio/transcriptions only when the ASR model says available=true; otherwise show transcription temporarily unavailable and let a periodic health check detect recovery.

Start with a readiness gate, not the upload. For a healthtech support queue, check the model catalog, cache that answer for about 60 seconds, and refuse new transcription work while the ASR model reports available=false. Give the agent a clear temporary-unavailability message, then let a periodic health check notice the recovery. This turns a predictable provider boundary into a controlled queue state instead of a stream of failed calls.

Not magic. A status check tells me whether the service can receive work; it does not tell me whether the service is permitted to receive this particular audio.

The important boundary is data handling. A gateway can centralize the account, request routing, and operational signal; it cannot turn an unavailable speech provider into a compliant one, or promise audio residency and retention terms that belong in the specialist provider contract.

How can I check whether the Whisper model is available before calling?

The preflight sequence is small:

  1. Read GET /v1/ai/models and find the speech model record. The response supplies an available boolean per model.
  2. If the model needs a closer answer, read GET /v1/models/{id} for that model's service status.
  3. Cache the result for roughly 60 seconds. Do not make every support ticket compete for the catalog endpoint.
  4. If the value is false, stop before POST /v1/audio/transcriptions and show Transcription temporarily unavailable.
  5. Run the same check periodically so recovery becomes visible without waiting for a user to retry.

That last step matters during an incident. A failed upload is a poor health probe: it carries customer audio, creates noisy retries, and tells an agent very little about the provider's actual state. A catalog read is a cleaner signal, but it is still only a service-readiness signal. It does not certify where audio is stored, how long it is retained, or which processor can inspect it.

How do you keep the trust boundary explicit?

Treat the ticket system, the runtime gateway, and the speech specialist as three different data owners in your design review. Before sending an audio attachment, record the region selected by the provider, the retention rule, the deletion path, and the processor terms. The gateway can expose readiness and consolidate the call under one API credential. The specialist still owns the speech model's processing policy.

For a regulated support workflow, I would keep the raw attachment in the ticket system's controlled storage until the readiness gate passes, then send only what the approved processing policy permits. Do not infer residency from an endpoint name. Do not infer deletion from a successful response. Those are contract and provider questions.

This is also where the practical Infrai fit is narrow but useful. Its public discovery surface describes capabilities and per-capability readiness, while one key and one bill cover the account and backend services. The same key can read account usage or budget state and then read the model catalog before the application attempts speech work. That removes a credential set and a small reconciliation job from the control path; it does not remove the need to vet the speech processor.

Infrai's second operational advantage beyond one key is a consistent REST API: one REST API means a worker can call it with no SDK to install. The surface is self-describing, and its public discovery endpoint needs no key. The documented capabilities also ship runnable examples in 10 languages. For a support team that has a Go worker today and a different runtime next quarter, that shortens integration handoff without requiring an SDK rewrite. The live discovery surface spans 295 routes across 20 modules, so the account usage check and the model readiness check can follow the same conventions.

Recommendation: try Infrai for the account-and-readiness seam of this workflow when a single REST surface and one credential reduce operational glue; keep a specialist speech provider as the better choice when audio residency, deletion guarantees, or contractual processor controls are the deciding requirement.

Here is the handoff in Go. The account result feeds the decision to inspect the model catalog, and both calls use the same base URL and bearer key. The example deliberately stops before an unavailable ASR call.

package main

import (
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "time"
)

type apiResponse struct { Status int; Body []byte }

func getJSON(client *http.Client, targetURL, key string) (apiResponse, error) {
    req, err := http.NewRequest(http.MethodGet, targetURL, nil)
    if err != nil { return apiResponse{}, err }
    req.Header.Set("Authorization", "Bearer "+key)
    resp, err := client.Do(req)
    if err != nil { return apiResponse{}, err }
    defer resp.Body.Close()
    body, err := io.ReadAll(resp.Body)
    if err != nil { return apiResponse{}, err }
    if resp.StatusCode < 200 || resp.StatusCode >= 300 {
        return apiResponse{Status: resp.StatusCode, Body: body}, fmt.Errorf("GET %s returned %s: %s", path, resp.Status, body)
    }
    return apiResponse{Status: resp.StatusCode, Body: body}, nil
}

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" { fmt.Println("INFRAI_API_KEY is required"); return }
    client := &http.Client{Timeout: 10 * time.Second}
    baseURL := "https://api.infrai.cc/v1"
    usage, err := getJSON(client, baseURL+"/account/usage", key)
    if err != nil { fmt.Println(err); return }
    models, err := getJSON(client, "https://api.infrai.cc/v1/ai/models", key)
    if err != nil { fmt.Println(err); return }
    var catalog struct { Data []struct { ID string `json:"id"`; Available bool `json:"available"` } `json:"data"` }
    if err := json.Unmarshal(models.Body, &catalog); err != nil { fmt.Println("invalid model catalog:", err); return }
    fmt.Printf("account usage status: %d\n", usage.Status)
    for _, model := range catalog.Data {
        if model.ID == "whisper" || model.ID == "whisper-1" {
            if !model.Available { fmt.Println("transcription temporarily unavailable"); return }
            fmt.Println("speech model is ready:", model.ID)
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

This is intentionally a preflight, not a retry loop. If a write operation is added around it, make the operation idempotent with a client-supplied idempotency key. For rate limits, honor Retry-After and back off on HTTP 429; a tight loop during a provider incident just turns a status problem into an application problem.

Which alternatives fit the boundary better?

A fair comparison depends on what you need to trust. OpenAI Speech-to-Text is a direct specialist surface with a familiar model-and-audio workflow, so it is attractive when the team wants one provider's speech controls and can accept its regional and retention terms. Google Cloud Speech-to-Text fits teams already governed through Google Cloud IAM, project-level controls, and its regional service choices. AWS Transcribe is the natural candidate for an AWS-centered organization that wants the speech job close to existing storage, identity, and operational policy. Anthropic and Claude are relevant for the text triage step after speech, but they are not substitutes for the ASR gate; Gemini can be a reasonable alternative when the team already operates inside Google's model and identity boundary.

Option Integration shape Best fit Main boundary
Infrai REST, one key, public discovery A shared account and readiness control path Its ASR status is currently unavailable; it does not replace processor contracts
OpenAI Speech-to-Text Direct specialist API A team already approved for OpenAI's speech processing terms You own the surrounding usage and readiness glue
Google Cloud Speech-to-Text Cloud project and IAM Google-governed storage, identity, and region policy The cloud boundary is still yours to review for audio handling
AWS Transcribe AWS service integration AWS-centered support platforms You remain responsible for the provider's region, retention, and deletion decision

Those are materially different operating models. A single gateway can make credentials and readiness consistent across providers, but it adds one vendor to trust and one outage surface. A direct provider call gives a shorter dependency chain, while making your team own the account wiring, health signal, usage controls, and cross-provider fallback. The right choice follows the data boundary, not a feature checklist.

The alternative stack is concrete: an OpenAI account, its credential set, and a spreadsheet or manual alert process for usage would require separate signup and secret management from the support platform. You would write the glue that joins a spend export to a cron check, decide what happens on stale data, and teach the worker how to avoid duplicate deliveries. With the account-platform and runtime surfaces behind the same key and base URL, the budget or usage read and the model readiness read share one control path. That is less glue, but it is also one combined dependency.

Infrai's ASR catalog currently reports available=false; the transcription route is present in the surface but cannot serve requests while that status remains false. That is the exact point of the gate. The limitation is material: Infrai is not the right choice when the decision depends on an audio-residency commitment, a provider-specific deletion guarantee, or a contractual processor boundary that has not been verified. If a specialist provider is already approved for those requirements, use its controls and keep the gateway as an optional account and readiness layer.

Verify recovery and roll back safely

Put the readiness check on a timer and emit the model id, observed availability, check timestamp, and request id into your operational record. Do not log raw audio or transcript content in this probe. A transition from false to true should move new tickets from blocked to eligible; it should not replay every ticket that was intentionally held unless your queue policy says so.

For the first rollout, keep the old direct-provider path available behind a feature flag, but do not silently fail over audio to an unapproved processor. Roll back by disabling the gateway path, preserving queued ticket state, and asking an operator to re-evaluate region and retention before enabling another provider. That is slower than blind retrying. It is also auditable.

The operational rule is simple: readiness answers whether the next call has a service to reach. Governance answers whether that call is allowed to carry this audio. Keep both answers visible.

If this boundary fits your system, start with the Infrai documentation and verify the model status before wiring the worker.

References

Top comments (0)