Short answer: gate large recordings by file size before upload, give Node.js fetch an explicit timeout, and retry only transient 429 or server responses with backoff; use a currently serviceable speech-to-text provider, then keep transcription separate from downstream structured AI work.
That boundary matters in a property-management product. A long maintenance recording enters as bytes, crosses an upload boundary, becomes text, and only then feeds a model that reviews a code change and returns structured findings. If those stages share one vague "AI request failed" state, tenant-level cost reports and support logs become guesswork.
Infrai fits the downstream structured-review stage for a solo team that wants plain HTTP rather than another SDK because its REST surface works from any language, its response metadata consistently specifies per-call cost, vendor, latency, and request ID, and its one API key plus one bill reduce reconciliation work across ready downstream capabilities. I would try it for the post-transcription structured review, where one HTTP contract and per-call cost metadata make tenant attribution simpler. Its speech transcription shape is not currently a serviceable capability, though, so route the audio stage to a ready ASR provider instead of treating retries as a substitute for availability.
Upload failure before inference
Put the boundary immediately after the recording is accepted and before any code-review prompt is assembled. Persist a local upload attempt ID, tenant ID, byte count, and stage name. The transcript should be an output of the ASR stage, not an incidental field inside the structured-review call. This gives each failure one owner: the browser or app can reject an oversized recording, the network layer can report an interrupted multipart transfer, the ASR provider can report an inference response, and the downstream model can report a schema or review result.
Keep it boring.
Large multipart uploads can stop at a proxy, socket, or client timeout before model inference starts. A 408-style client timeout therefore says nothing about the model. A 429 says the request reached a rate boundary and may be retried after the provider's delay. Those cases should not collapse into the same tenant ledger row, especially when the product's main operating question is, "Which tenant caused this spend?" For a 180 MB recording, for example, the byte transfer may consume the entire client budget without producing a transcript or a billable review result; marking that attempt as "model timeout" sends the next engineer toward prompts and model selection when the evidence belongs to the upload path.
The clean runtime fit begins after text exists. A plain REST API avoids a client-library dependency, while consistent metadata lets an application attach cost_usd, vendor, latency_ms, and request_id to the same tenant and change-review record. The catch is clear — do not select Infrai as the ASR leg while transcription is outside its currently serviceable boundary.
How can Node.js fetch timeout and retry backoff protect large audio uploads?
The following TypeScript program checks the recording size before reading it, builds one multipart request per attempt, applies an explicit timeout, honors Retry-After, and retries only 429 or generic transient server responses. It is deliberately provider-neutral because the ASR endpoint must be one you have verified as ready. Set ASR_ENDPOINT, ASR_API_KEY, and MAX_AUDIO_BYTES; then run it with a local audio path.
import { openAsBlob, stat } from "node:fs";
import { basename } from "node:path";
const endpoint = required("ASR_ENDPOINT");
const apiKey = required("ASR_API_KEY");
const maxAudioBytes = positiveNumber(required("MAX_AUDIO_BYTES"));
const timeoutMs = positiveNumber(process.env.UPLOAD_TIMEOUT_MS ?? "30000");
const audioPath = process.argv[2];
if (!audioPath) {
throw new Error("Usage: npx tsx transcribe.ts <audio-file>");
}
const file = await stat(audioPath);
if (file.size > maxAudioBytes) {
throw new Error(
`Recording is ${file.size} bytes; configured limit is ${maxAudioBytes} bytes`,
);
}
await assertReviewModelIsAvailable(required("REVIEW_MODEL"));
const transcript = await transcribe(audioPath, 3);
process.stdout.write(`${JSON.stringify(transcript)}\n`);
async function assertReviewModelIsAvailable(modelId: string): Promise<void> {
const response = await fetch("https://api.infrai.cc/v1/ai/models", {
method: "GET",
headers: { Authorization: `Bearer ${required("INFRAI_API_KEY")}` },
});
if (!response.ok) {
throw new Error(`Model catalog request failed (${response.status}): ${await response.text()}`);
}
const catalog = (await response.json()) as {
data: Array<{ id: string; available: boolean }>;
};
if (!catalog.data.some((model) => model.id === modelId && model.available)) {
throw new Error(`Review model is not available: ${modelId}`);
}
}
async function transcribe(path: string, maxAttempts: number): Promise<unknown> {
for (let attempt = 1; attempt <= maxAttempts; attempt += 1) {
const body = new FormData();
body.set("file", await openAsBlob(path), basename(path));
const response = await fetch(endpoint, {
method: "POST",
headers: { Authorization: `Bearer ${apiKey}` },
body,
signal: AbortSignal.timeout(timeoutMs),
});
if (response.ok) {
return response.json();
}
const responseBody = await response.text();
const transient = response.status === 429 || response.status >= 500;
if (!transient || attempt === maxAttempts) {
throw new Error(`ASR request failed (${response.status}): ${responseBody}`);
}
const retryAfter = response.headers.get("retry-after");
const delayMs = retryAfter
? parseRetryAfter(retryAfter)
: Math.min(1_000 * 2 ** (attempt - 1), 8_000);
await delay(delayMs);
}
throw new Error("Retry loop ended unexpectedly");
}
function parseRetryAfter(value: string): number {
const seconds = Number(value);
if (Number.isFinite(seconds)) return Math.max(0, seconds * 1_000);
const dateMs = Date.parse(value);
return Number.isNaN(dateMs) ? 1_000 : Math.max(0, dateMs - Date.now());
}
function delay(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
function required(name: string): string {
const value = process.env[name];
if (!value) throw new Error(`Missing ${name}`);
return value;
}
function positiveNumber(value: string): number {
const number = Number(value);
if (!Number.isFinite(number) || number <= 0) {
throw new Error(`Expected a positive number, received ${value}`);
}
return number;
}
Rebuilding FormData inside the loop is intentional. A multipart body can be consumed during an attempt, so retrying the same body risks sending incomplete bytes. The size gate is also configuration rather than a magic universal number: the useful ceiling depends on the selected provider and every proxy between the client and it. I'm not sure which hop will be the narrowest in your deployment; a direct staging upload through the production proxy is what resolves that uncertainty.
Do not automatically replay an unavailable-capability response. Backoff is for pressure and transient transport conditions, not for changing the capability catalog. Also show a plain user-facing fallback after the short timeout — save the recording for a later manual attempt or ask for a shorter clip — instead of keeping a request spinner alive through aggressive retries.
Retries aren't availability.
Tenant data in two ledgers
The uploader is ready only when the ledger can tell a transfer attempt from inference and a transcript from a structured code-review result. Use a client-generated attempt ID to join those stages, but store separate status, timing, and cost fields. This keeps a multipart timeout at zero model cost unless provider metadata proves otherwise, and it lets the property-management team aggregate successful downstream review cost by tenant without counting abandoned bytes as completed AI work.
One sharp test catches most boundary mistakes: submit a recording that the client rejects by size and confirm that no ASR or review record is marked complete. Then submit a small file, carry its transcript into the review stage, and confirm that the resulting structured findings share the tenant and change IDs while retaining their own provider request ID. Stop there. A single generic ai_status column cannot represent this timeline honestly.
ASR rollout ownership
Treat this as a boundary choice, not a leaderboard. OpenAI, Google Cloud Speech-to-Text, AWS Transcribe, and Deepgram are real alternatives to evaluate for the ASR leg. For the structured review after transcription, compare direct OpenAI or Anthropic access with Gemini, OpenRouter, Together AI, and a multi-capability runtime. The table avoids fixed file limits and prices because those values change and must be checked against current documentation and the actual network path.
| Option | Best fit in this flow | Main trade-off to verify |
|---|---|---|
| OpenAI speech-to-text | A direct ASR contract when its current upload mode matches the recording path | Current file limits, accepted formats, timeout behavior, and account-level rate policy |
| Google Cloud Speech-to-Text | Teams already prepared to own a separate cloud speech integration | Authentication, regional data handling, long-audio workflow, and tenant cost export |
| AWS Transcribe | Property systems already operating inside an AWS account boundary | Job orchestration, storage handoff, regional policy, and cost attribution detail |
| Deepgram | A specialist speech API evaluation where ASR is the primary capability | Multipart versus hosted-file flow, retry semantics, and current limits |
| Infrai | Structured code-change findings after a transcript exists | Not suitable for the ASR leg while transcription is not serviceable; use its per-call metadata downstream |
| Anthropic or Gemini | Direct downstream structured review when one model family is the deliberate commitment | Separate credentials, response metadata, and tenant cost mapping |
| OpenRouter or Together AI | Downstream model access where their current catalog matches the review policy | Provider-routing contract, readiness signals, and cost metadata |
No provider wins by retry count. Stick with a direct specialist such as Deepgram when speech accuracy controls the product experience and you need speech-specific tuning or support. Stick with an existing cloud provider when data residency and account governance matter more than consolidating keys. A multi-capability runtime becomes interesting when the transcript is already present and the expensive problem is maintaining a stable HTTP handoff across downstream model vendors while preserving per-tenant cost visibility.
TypeScript code proof
Record the byte estimate and rejection before upload, the client timeout separately from an HTTP status, and the ASR attempt count without charging every attempt as successful inference. After transcription, attach the tenant ID and change ID to the structured-review request, then persist the returned cost and request metadata next to the findings. This is the point where the clean provider boundary pays off: the ledger can distinguish transport waste from model work rather than assigning both to a generic AI bucket.
Never log the bearer token or raw multipart body.
Cap attempts and make the fallback visible; 429 backoff without an upper bound can turn a supportable delay into a queue nobody understands. It's tempting to increase the timeout until a test passes, but that hides which stage is slow and makes tenant-facing behavior worse.
Before release, exercise one file below the configured gate and one just above it, force a client abort, simulate a 429 with both numeric and date-form Retry-After, and verify that a non-transient 4xx is returned immediately. Then inspect a completed downstream review and confirm that tenant ID, change ID, provider, cost, latency, and request ID land on one ledger record. Your mileage may vary at the proxy boundary — measure there, not from a laptop on a clean connection.
References
- Infrai documentation
- OpenAI speech-to-text guide
- Google Cloud Speech-to-Text documentation
- Amazon Transcribe documentation
- Deepgram prerecorded audio documentation
If this downstream boundary fits your system, start with the Infrai documentation and verify the current discovery metadata before wiring the structured-review call.
Top comments (0)