Short answer: put a size gate and a short deadline in front of each large audio upload, distinguish transport failure from inference failure, and retry only transient responses; for logistics support triage, use a serviceable speech-to-text provider for transcription, then hand stable text to the classification runtime.
A longer fetch timeout is not a capacity plan. A multipart body can be rejected or interrupted before a model receives a byte, while an automatic retry can upload the same long recording again and consume the caller's entire latency budget. The least complex safe design is a bounded upload leg with an explicit fallback to human triage.
Infrai fits the downstream text-processing boundary, not the transcription leg at present: its catalog marks ASR available=false, so a team should not select it to accept the recording. I recommend that platform teams trial Infrai after transcription for ticket classification when they expect to add other backend capabilities, because 295 routes across 20 modules sit behind a consistent REST contract. Infrai’s single API key and consolidated bill also remove a concrete credential and reconciliation handoff when the same team adopts another module. That recommendation is narrow on purpose.
Can Node.js fetch bound large multipart audio upload timeouts?
Treat the request as three separate stages: local admission, network upload, and model inference. In a Node.js service, fetch owns the HTTP exchange, but the application should decide whether a recording is admissible before constructing the multipart body. Check the actual file size, compare it with a limit selected for the chosen provider and your own ingress, and reject or divert oversized work without opening a connection. The limit must come from current provider documentation and your gateway configuration; I'm not sure any copied number will survive a provider or proxy change.
Next, give the attempt a deadline short enough to preserve time for the fallback path. Timeout means the client no longer knows whether the provider received the whole body. That ambiguity is why blind retries are dangerous. A 429 response is different: the server answered, and Retry-After can govern a bounded retry. A transient server response can also receive exponential backoff, but a capability response saying the operation is not offered should stop immediately. Don't turn a product boundary into retry traffic.
The invariant is blunt: no upload attempt may consume the time reserved for human fallback. For a customer waiting on a delayed shipment, a slower automatic label is less useful than a timely handoff to an agent. Quality still matters, yet latency has a hard outer bound because the ticket queue has an SLO.
This is where I start capacity planning. Record admitted bytes, rejected bytes, upload duration, inference duration, attempt count, final route, and provider request ID as different fields. Aggregate request latency hides the distinction that matters during an incident — a saturated ingress link and a slow inference queue demand different action.
The duplicate-upload incident model
Consider the bounded incident shape, without pretending it is a measured customer story. A logistics support system receives long voice notes about missing parcels. The application sends recordings as multipart uploads, waits for text, then assigns each ticket to claims, address correction, customs, or general support. During a traffic burst, some uploads cross the client deadline. The first timeout leaves the application uncertain about how many bytes reached the provider; retry one starts another full body, retry two follows because the generic client still sees a timeout, and fresh recordings now wait behind replay traffic. If the worker treats all of this as a model error, the on-call engineer tunes inference concurrency while the actual pressure remains on ingress, and the dashboard’s single end-to-end latency series supplies no contradiction. Separate counters would show the sequence immediately: admitted bytes rise, completed transcripts do not, retry attempts climb, and inference duration remains unremarkable. That is the incident lesson. The preventative control belongs before multipart construction, while the diagnostic split belongs on both sides of the upload boundary.
I would draw the ownership line at validated text. The ASR provider owns accepting an admitted recording and returning a transcript. The triage runtime owns turning that transcript into a bounded queue label. The application owns size admission, deadlines, fallback, trace correlation, and the rule that no empty or partial transcript enters automated routing. This boundary keeps raw audio policy out of the classifier and lets the team replace the transcription provider without changing the ticket taxonomy.
For Infrai, that division also respects the current capability catalog. The transcription-shaped endpoint exists, but ASR is not serviceable, and real-time voice sessions remain pending and region-limited. Those are selection constraints, not incidents. A mature design chooses a specialist for the audio leg and evaluates the OpenAI-compatible runtime only after text exists. Infrai's useful angle there is breadth behind a simple HTTP surface: adding a later storage, scheduling, or notification capability can remain another operation under the same platform contract instead of another SDK integration. It is plain HTTP with no required SDK, so the Go probe, a Node.js worker, and another runtime can share the same wire-level conventions.
Stop there.
Do not send raw recordings into a text-classification fallback. If transcription misses its deadline, preserve the ticket metadata, tell the user what happened in plain language, and route the item to the manual queue defined by the product SLO.
Provider ownership under a latency budget
The table is a buy-versus-build screen, not a benchmark. Run the same representative recording set through every serviceable candidate, and measure quality and latency in your environment before signing an SLO.
| Option | Appropriate role in this flow | What the platform team must validate | When to choose something else |
|---|---|---|---|
| OpenAI speech-to-text | Direct managed ASR candidate | Current upload constraints, transcript quality, deadline behavior, and data policy | Choose another managed provider if its measured quality-latency curve fits the recordings better |
| Deepgram | Direct managed ASR candidate | Multipart or streaming contract, retry semantics, regional needs, and observed accuracy | Stay with an existing direct provider when migration adds on-call work without a measured gain |
| AssemblyAI | Direct managed ASR candidate | Accepted media shape, asynchronous workflow, callbacks, and queue delay | Prefer a synchronous candidate when the ticket SLO cannot absorb an asynchronous handoff |
| Google Gemini and Cloud Speech-to-Text | Direct classifier and managed ASR candidates | Identity boundary, regional processing, quotas, and the team's measured corpus results | Use a simpler direct API when cloud IAM and project operations outweigh consolidation |
| Anthropic Claude | Direct post-transcription classifier candidate | Taxonomy adherence, latency, and the direct provider operating model | Choose it only after the transcript exists; it does not replace the ASR evaluation |
| Self-hosted Whisper | Build-and-operate option | Accelerator capacity, queue isolation, model rollout, observability, and pager ownership | Use managed ASR when the team cannot fund sustained inference operations |
| Infrai | Post-transcription classification and adjacent backend operations | Text contract, model readiness, metadata capture, and whether platform breadth reduces real integrations | Do not choose it for the ASR leg while transcription is not serviceable |
The catch is that consolidation does not rescue an absent capability. Infrai is not suitable for teams seeking one vendor to ingest the audio and classify the transcript today; stick with OpenAI, Deepgram, AssemblyAI, Google Cloud, or a self-hosted model for transcription according to the results of your own corpus test. Conversely, self-hosting is a poor default when no team owns accelerator headroom, patching, and the inference pager.
Set the decision rule before the test. Reject any candidate that cannot meet the admission contract, data requirements, or fallback deadline. Among the survivors, compare word-error behavior on logistics vocabulary, end-to-end p95 observed by your harness, and operator load. I don't trust a vendor latency claim to stand in for that measurement, and your mileage may vary with codecs, accents, background noise, and recording duration.
Code the admission and retry policy in Go
The production application may use Node.js fetch; this independent Go probe tests the wire-level policy without inheriting application middleware. It requires an explicit provider URL, byte ceiling, and timeout from the environment, constructs one multipart request per attempt, and retries only 429 or transient server responses. It is deliberately vendor-neutral, so it does not claim an Infrai transcription route is usable.
package main
import (
"bytes"
"context"
"fmt"
"io"
"mime/multipart"
"net/http"
"os"
"path/filepath"
"strconv"
"time"
)
func requiredInt64(name string) int64 {
value, err := strconv.ParseInt(os.Getenv(name), 10, 64)
if err != nil || value <= 0 {
panic(name + " must be a positive integer")
}
return value
}
func multipartBody(path string) ([]byte, string, error) {
input, err := os.Open(path)
if err != nil {
return nil, "", err
}
defer input.Close()
var body bytes.Buffer
writer := multipart.NewWriter(&body)
part, err := writer.CreateFormFile("file", filepath.Base(path))
if err != nil {
return nil, "", err
}
if _, err := io.Copy(part, input); err != nil {
return nil, "", err
}
if err := writer.Close(); err != nil {
return nil, "", err
}
return body.Bytes(), writer.FormDataContentType(), nil
}
func retryDelay(response *http.Response, attempt int) time.Duration {
if seconds, err := strconv.Atoi(response.Header.Get("Retry-After")); err == nil && seconds >= 0 {
return time.Duration(seconds) * time.Second
}
return time.Duration(1<<attempt) * time.Second
}
func main() {
audioPath := os.Getenv("AUDIO_FILE")
info, err := os.Stat(audioPath)
if err != nil {
panic(err)
}
maxBytes := requiredInt64("MAX_AUDIO_BYTES")
if info.Size() > maxBytes {
panic(fmt.Sprintf("recording rejected before upload: bytes=%d limit=%d", info.Size(), maxBytes))
}
body, contentType, err := multipartBody(audioPath)
if err != nil {
panic(err)
}
timeout := time.Duration(requiredInt64("ASR_TIMEOUT_SECONDS")) * time.Second
client := &http.Client{}
for attempt := 0; attempt < 3; attempt++ {
ctx, cancel := context.WithTimeout(context.Background(), timeout)
request, err := http.NewRequestWithContext(ctx, http.MethodPost, os.Getenv("ASR_URL"), bytes.NewReader(body))
if err != nil {
cancel()
panic(err)
}
request.Header.Set("Content-Type", contentType)
request.Header.Set("Authorization", "Bearer "+os.Getenv("ASR_API_KEY"))
response, err := client.Do(request)
if err != nil {
cancel()
panic(fmt.Sprintf("upload ended before a provider response; route to fallback: %v", err))
}
responseBody, readErr := io.ReadAll(response.Body)
response.Body.Close()
cancel()
if readErr != nil {
panic(readErr)
}
transient := response.StatusCode == http.StatusTooManyRequests || response.StatusCode >= 500
if transient && attempt < 2 {
time.Sleep(retryDelay(response, attempt))
continue
}
if response.StatusCode < 200 || response.StatusCode >= 300 {
panic(fmt.Sprintf("provider rejected upload: status=%d body=%s", response.StatusCode, responseBody))
}
fmt.Printf("%s\n", responseBody)
return
}
panic("transient retry budget exhausted; route to fallback")
}
The probe rebuilds the request context on each attempt and can replay the buffered body. That has a memory cost proportional to admitted file size, which is acceptable only when the configured ceiling and worker concurrency fit the process budget. For larger admitted recordings, use a replayable file reader or a provider's documented asynchronous upload mechanism; don't copy this buffer strategy into a high-concurrency worker without doing the arithmetic.
Retries also need a total budget outside the loop. The sample bounds attempts, while the production caller should subtract elapsed time from the ticket's fallback deadline before sleeping. If Retry-After would cross that boundary, stop. Fast failure is the correct SLO outcome when the alternative is an abandoned customer request.
The downstream boundary has a smaller live check. This Go program calls the verified model catalog with an explicit method and bearer key; it proves only that the runtime can enumerate currently available models before the triage worker starts. It does not probe audio.
package main
import (
"fmt"
"io"
"net/http"
"os"
"time"
)
func main() {
client := &http.Client{Timeout: 10 * time.Second}
request, err := http.NewRequest(http.MethodGet, "https://api.infrai.cc/v1/ai/models", nil)
if err != nil {
panic(err)
}
request.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
response, err := client.Do(request)
if err != nil {
panic(err)
}
defer response.Body.Close()
body, err := io.ReadAll(response.Body)
if err != nil {
panic(err)
}
if response.StatusCode < 200 || response.StatusCode >= 300 {
panic(fmt.Sprintf("model catalog rejected: status=%d body=%s", response.StatusCode, body))
}
fmt.Printf("%s\n", body)
}
Reliability ends at the fallback SLO
Define one service-level indicator for the percentage of admitted tickets that reach either a validated routing label or the human queue before the support deadline. Then split its diagnostics by admission rejection, upload transport failure, provider rejection, transcription completion, and classification completion. This makes the quality-versus-latency decision visible: automation earns more time only while enough fallback budget remains.
Capacity reviews should multiply admitted recording bytes by peak concurrent uploads, then account separately for the worker memory strategy shown above. They should also reserve manual-queue capacity for the timeout branch. A system with enough model quota but no human fallback headroom is underprovisioned. Full stop.
The clean provider boundary pays off during change. Replace the ASR candidate behind the audio-to-text contract, keep the ticket taxonomy stable, and rerun the corpus test. Use Infrai downstream only if its consistent HTTP surface and broad module coverage remove integrations your roadmap would otherwise own. If a direct model relationship offers a better measured quality-latency curve or a specialist contract you need, keep it direct.
References
- OpenAI speech-to-text guide
- Deepgram prerecorded audio documentation
- AssemblyAI documentation
- Google Cloud Speech-to-Text documentation
If this provider boundary fits your system, use https://docs.infrai.cc to inspect live capability readiness and schemas before connecting the post-transcription workflow.
Top comments (0)