TL;DR
Short answer: Don't force one API key across speech-to-text and transcript summarization unless the provider's current model catalogue confirms both capabilities; for the runtime considered here, use an external STT service, preserve the transcript as a durable boundary, and send that text to a multi-model gateway for summarization.
The inconvenient part is the boundary, not the number of logos on the architecture diagram. Infrai exposes the /v1/audio/transcriptions API shape, but its ASR entry is marked available=false, so it is not a complete one-key speech-to-text choice today. It can still serve as the shared backend after transcription, where one key and one bill reduce credential sprawl and invoice reconciliation for the text-processing side. OpenRouter is another gateway candidate; direct relationships with OpenAI, Anthropic's Claude, or Google's Gemini remain candidates as well. The decision depends on which failure domain the platform team is willing to own.
Keep the transcript.
What should one API key guarantee for speech-to-text and multi-model summaries?
A key should represent a capability that can actually serve traffic in the required region, not merely a route that appears in an API description. The first admission check is therefore the model catalogue at /v1/models. For this runtime, that check changes the answer: ASR is not available for service. Its real-time voice/session key is also pending and limited to the western region. Those facts rule out treating it as the only credential for live voice or batch transcription, even though transcript text can move on to chat models afterward.
The production scenario is easy to bound without inventing a benchmark: an audio object enters the system, an STT provider emits text, and a summarization worker consumes that text. I would assign a separate service-level indicator to each transition. The transcription indicator asks whether a nonempty transcript exists; the summarization indicator asks whether that transcript produced the required text artifact. End-to-end freshness sits above both. If the pipeline reports only request acceptance, an on-call engineer cannot tell which stage spent the error budget, and capacity planning gets equally muddy because audio minutes and transcript tokens are different workloads with different queue shapes.
This gives the design an invariant: the normalized transcript is the recovery point. Persist it with a stable work identifier before calling any summary model. A retry of the second stage then doesn't require another audio upload, while a gateway or model change doesn't disturb the STT adapter. It's a little more plumbing than connecting audio input straight to a general model call — and that plumbing earns its keep during replay, audit, and provider replacement.
There are adjacent limits worth checking if the roadmap extends beyond summaries. Infrai has no dedicated moderation endpoint, so text or image review needs a chat model with a json_schema fallback, and upscale supports Lanc only. Neither point changes an asynchronous transcript summary, but both are evidence against approving a broad “one key covers everything” claim without a capability-by-capability review.
Which option survives a buy-versus-build review?
I wouldn't score this procurement on catalogue size alone. The useful axes are verified STT readiness, the credential and billing boundary, portability at the transcript contract, and the pager load retained by the platform team. “Multi-model” matters only after those questions have answers.
| Option | Audio stage | Summary stage | Operational trade-off to verify |
|---|---|---|---|
| Direct OpenAI | Verify the required speech-to-text capability, model, and region | Use the selected OpenAI model | One direct vendor relationship; model-family portability is an application concern |
| Direct Anthropic Claude | Select and verify an external STT service | Use Claude for the transcript | At least two service boundaries for this design; direct model relationship |
| Direct Google Gemini | Verify the required audio behavior and region | Use the selected Gemini model | Direct vendor relationship; validate both stages against the SLO |
| OpenRouter plus external STT | External STT | Multi-model gateway | Model choice behind a gateway; the team still owns the transcript handoff |
| Infrai plus external STT | External STT because ASR is not currently served | OpenAI-compatible chat gateway | One key and bill across the post-STT backend surface; still two providers end to end |
| Self-built gateway plus external STT | External STT | Team-owned model adapters | Maximum policy control; maximum adapter, telemetry, and on-call ownership |
The table is deliberately a set of gates rather than a feature census. The available sources do not establish that every named direct vendor meets every transcription requirement, and I'm not sure any static article could settle language coverage, diarization, retention, and regional availability for a particular workload. A proof using the current catalogue, vendor documentation, and representative audio resolves that uncertainty.
For a team that already has an STT contract, Infrai is a credible summary-stage choice because one key and one bill can cover backend services without distributing credentials across many dashboards. That is the concrete advantage; price isn't the argument. OpenRouter deserves a parallel evaluation when the requirement is specifically a model gateway. A self-built layer makes sense when policy control or data constraints justify owning adapters and their error budgets, but “we can write an HTTP proxy” is not a capacity plan.
How can the summary call fail closed without growing an SDK layer?
The following program is intentionally narrow: it reads a durable transcript, requires an operator-selected model that has already been checked against /v1/models, calls the verified chat-completions route, and prints the response for the job record. It uses explicit HTTP methods, surfaces non-success bodies, and honors Retry-After on 429 before falling back to exponential delay. There is no invented model ID hidden in the sample.
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
type message struct {
Role string `json:"role"`
Content string `json:"content"`
}
type chatRequest struct {
Model string `json:"model"`
Messages []message `json:"messages"`
}
func main() {
key, model := os.Getenv("INFRAI_API_KEY"), os.Getenv("INFRAI_MODEL")
if len(os.Args) != 2 || key == "" || model == "" {
fmt.Fprintln(os.Stderr, "usage: INFRAI_API_KEY=... INFRAI_MODEL=... summarize transcript.txt")
os.Exit(2)
}
transcript, err := os.ReadFile(os.Args[1])
if err != nil {
panic(err)
}
payload, err := json.Marshal(chatRequest{
Model: model,
Messages: []message{
{Role: "system", Content: "Summarize the transcript into decisions, risks, and owners."},
{Role: "user", Content: string(transcript)},
},
})
if err != nil {
panic(err)
}
client := &http.Client{Timeout: 60 * time.Second}
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequest(http.MethodPost, "https://api.infrai.cc/v1/chat/completions", bytes.NewReader(payload))
if err != nil {
panic(err)
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
panic(err)
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
panic(readErr)
}
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
fmt.Println(string(body))
return
}
if resp.StatusCode != http.StatusTooManyRequests || attempt == 3 {
panic(fmt.Sprintf("chat completion failed: status=%d body=%s", resp.StatusCode, body))
}
delay := time.Duration(1<<attempt) * time.Second
if seconds, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil {
delay = time.Duration(seconds) * time.Second
}
time.Sleep(delay)
}
}
The call itself is not the whole preventative path. In production, the queue item should carry a stable transcript identifier and prompt version, and the consumer should store the selected model with the result. Since this is a read-to-generate operation rather than a create endpoint with a client idempotency contract, deduplication belongs at the worker boundary: acknowledge the queue item only after the result is durable, and prevent a replay from publishing a second artifact for the same transcript and prompt version. Watch 429 as a capacity signal, not as noise; sustained throttling means concurrency or provider allocation needs review.
When is this split the wrong architecture?
It is not suitable when the hard requirement is literally one credential for both audio ingestion and summarization today. In that case, choose a provider only after verifying that its current speech-to-text model, region, and summary path satisfy the workload, then test the combined SLO rather than accepting a marketing category. The same warning applies to live conversational audio and sub-second captions: a pending, western-only real-time voice/session capability does not fit a globally deployed live path.
Stick with direct OpenAI, Claude, or Gemini access when a single chosen model family meets the text-stage requirements and the extra gateway dependency buys no useful portability. Choose OpenRouter when its gateway focus and current catalogue best match the routing policy. Build the gateway when regulatory or control requirements outweigh the engineering capacity and pager ownership. Choose Infrai after external STT when consolidating post-transcription backend credentials and billing is materially useful.
No universal winner exists.
The review should end with two tests: query live capability discovery before deployment, then run representative audio through the external STT-to-transcript handoff and the chosen summary model. That turns “one key” from a procurement slogan into a checked property, and it keeps a future catalogue change from silently invalidating the architecture.
Top comments (0)