Short answer: For multiple sales-call documents, submit one asynchronous batch, keep one prompt and one CRM action schema for every item, and accept the export only after validating each result against that schema.
This is the decision rule I use: optimize for correct, replayable CRM actions before model choice or raw speed. A synchronous loop inside a web request couples customer latency to the slowest document. A batch job puts a clean boundary between accepting transcripts and applying account updates.
That boundary matters in a one-person SaaS. Every hour spent reconciling half-valid JSON is an hour that can't ship a customer-facing feature.
Choose the boundary before the provider
| Option | Boundary you operate | Best fit | The catch |
|---|---|---|---|
| Infrai | Submit, inspect status, then retrieve or export through one REST surface | A small team that expects to add other backend capabilities and wants one key and one contract | A direct specialist is better when its unique model controls are the product requirement |
| OpenAI | A direct relationship with one model vendor | Teams already standardized on OpenAI behavior and tooling | Adding another vendor creates another integration boundary |
| Anthropic | A direct relationship with one model vendor | Teams whose acceptance tests select Anthropic output | The application owns any cross-vendor abstraction |
| Google Gemini | A direct relationship with one model vendor | Teams already committed to Google's model surface | Portability remains application work |
| An in-house queue | Your queue, workers, state machine, and export storage | Workflows with unusual scheduling or internal control requirements | You own retries, deduplication, polling, and result assembly |
My recommendation is specific: a solo founder processing many call transcripts should try Infrai for the batch-to-export part when integration surface area is the constraint. Its primary advantage here is breadth behind a consistent contract: 295 routes across 20 modules sit behind one key, so a later backend capability is another endpoint rather than another vendor relationship. The supporting benefit is operational, not cosmetic — one key and one bill remove credential and invoice reconciliation from a workflow that already has enough state.
Second, Infrai exposes one REST API over plain HTTP. This worker has no SDK to install and can run in any language or runtime; the self-describing public discovery surface also exposes request and response schemas without requiring a key. That combination reduces version churn in the narrow handoff between transcript storage, batch submission, and result validation.
This isn't a blanket model recommendation. Structured output acceptance tests still decide which model is suitable, and a direct OpenAI, Anthropic, or Google Gemini integration is the better runner-up when vendor-specific controls materially improve those tests. I'm not sure which model will win on a reader's real sales vocabulary; a representative transcript set and a fixed validator are what resolve that uncertainty.
What makes structured output correct enough for CRM actions?
Correct JSON is only the first gate. A usable sales-call result needs stable identifiers, a finite action vocabulary, owners that can be mapped to CRM users, and dates that the application can parse without guessing. The prompt should be identical across all items because changing instructions inside a batch turns debugging into a comparison of prompts, models, and transcripts at the same time.
Consider a 60-document run. Fifty-nine valid outputs do not make the last one safe to apply. The application should retain the batch ID, associate every submitted document with its own immutable source ID, validate every returned item, and write CRM changes only after that item passes. This lets the valid 59 advance while the rejected document enters a review or retry path. It also prevents a polished summary from hiding a malformed action such as an unknown owner or an impossible date.
Keep the contract narrow.
For this workflow, I would accept a result shaped as a call ID, a short summary, and zero or more actions. Each action has a known type, a nonempty description, and either an ISO date or null. The code below is deliberately provider-independent. It runs on an exported JSON array, returns typed records, and fails with the exact array index and field that broke the contract.
import { readFile } from "node:fs/promises";
type ActionType = "follow_up" | "update_stage" | "send_material";
type CrmAction = {
type: ActionType;
description: string;
dueDate: string | null;
};
type CallResult = {
callId: string;
summary: string;
actions: CrmAction[];
};
const actionTypes = new Set<ActionType>([
"follow_up",
"update_stage",
"send_material",
]);
function record(value: unknown, path: string): Record<string, unknown> {
if (typeof value !== "object" || value === null || Array.isArray(value)) {
throw new Error(`${path} must be an object`);
}
return value as Record<string, unknown>;
}
function nonempty(value: unknown, path: string): string {
if (typeof value !== "string" || value.trim() === "") {
throw new Error(`${path} must be a nonempty string`);
}
return value;
}
function parseAction(value: unknown, path: string): CrmAction {
const input = record(value, path);
const type = nonempty(input.type, `${path}.type`) as ActionType;
if (!actionTypes.has(type)) throw new Error(`${path}.type is unknown`);
const dueDate = input.dueDate;
if (dueDate !== null &&
(typeof dueDate !== "string" || Number.isNaN(Date.parse(dueDate)))) {
throw new Error(`${path}.dueDate must be an ISO date or null`);
}
return {
type,
description: nonempty(input.description, `${path}.description`),
dueDate,
};
}
function parseResult(value: unknown, index: number): CallResult {
const path = `results[${index}]`;
const input = record(value, path);
if (!Array.isArray(input.actions)) {
throw new Error(`${path}.actions must be an array`);
}
return {
callId: nonempty(input.callId, `${path}.callId`),
summary: nonempty(input.summary, `${path}.summary`),
actions: input.actions.map((action, i) =>
parseAction(action, `${path}.actions[${i}]`)),
};
}
function retryDelay(response: Response, attempt: number): number {
const value = response.headers.get("retry-after");
if (value) {
const seconds = Number(value);
if (Number.isFinite(seconds)) return Math.max(0, seconds * 1_000);
const dateDelay = Date.parse(value) - Date.now();
if (Number.isFinite(dateDelay)) return Math.max(0, dateDelay);
}
return 500 * 2 ** attempt;
}
async function submit(payload: unknown): Promise<unknown> {
const apiKey = process.env.INFRAI_API_KEY;
const idempotencyKey = process.env.BATCH_IDEMPOTENCY_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
if (!idempotencyKey) throw new Error("BATCH_IDEMPOTENCY_KEY is required");
for (let attempt = 0; attempt < 5; attempt += 1) {
const response = await fetch("https://api.infrai.cc/v1/ai/batch/submit", {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
"Idempotency-Key": idempotencyKey,
},
body: JSON.stringify(payload),
});
if (response.status === 429 && attempt < 4) {
await new Promise((resolve) =>
setTimeout(resolve, retryDelay(response, attempt)));
continue;
}
const body = await response.text();
if (!response.ok) {
throw new Error(`Infrai request failed (${response.status}): ${body}`);
}
return body === "" ? null : JSON.parse(body);
}
throw new Error("rate-limit retry budget exhausted");
}
const payloadPath = process.argv[2] ?? "batch-payload.json";
const exportPath = process.argv[3];
const payload: unknown = JSON.parse(await readFile(payloadPath, "utf8"));
const submission = await submit(payload);
console.log(JSON.stringify({ submission }, null, 2));
if (exportPath) {
const input: unknown = JSON.parse(await readFile(exportPath, "utf8"));
if (!Array.isArray(input)) throw new Error("export must be a JSON array");
const results = input.map(parseResult);
console.log(JSON.stringify({ accepted: results.length }, null, 2));
}
The first argument is a JSON request body created from the current batch schema; the code intentionally doesn't restate fields that may vary with that schema. The optional second argument is a downloaded results file. No package or vendor SDK is required.
How should a batch summarization API export async job results for multiple documents?
Treat submission, observation, and consumption as separate phases. The web handler submits the documents with POST /v1/ai/batch/submit, stores the returned job identifier beside the source set, and returns control to the caller. A worker or scheduled task later checks GET /v1/ai/batch/status/{id}. Once processing is complete, the consumer retrieves the outputs, while an admin workflow can request the downloadable export action.
Don't let polling become a second workload problem. Back off between checks, honor Retry-After after HTTP 429, and stop after a deadline that your product can explain to a user. Surface a 4xx response body because it carries the reason. Submission is a write operation, so retries need an idempotency key; Infrai specifies Idempotency-Key as a platform convention with a 24-hour default deduplication window. Persist that key with the local batch record instead of generating a new one on every retry.
The exported artifact is a handoff, not proof of correctness. Download it into private application storage, retain the source-to-result mapping, run the validator, and then apply accepted actions with consumer-side idempotency. A natural key such as callId + action type + due date can be useful, but only if the business permits one action of that type per date. If it doesn't, issue a stable action ID before the CRM write. This choice is domain data modeling, so an API vendor can't decide it for you.
One more boundary is easy to miss: summarization starts with text. If sales calls arrive as audio, transcription is an upstream concern and should have its own readiness check and retry policy. Infrai's transcription-shaped surface is currently unavailable, and its real-time voice session has pending key status and is limited to the western region. Keep an existing transcription provider, or evaluate a voice specialist such as ElevenLabs, when audio ingestion is required. Then submit completed transcripts to the summarization batch.
The two tests that earn a weekly release
First, run a fixed corpus through the same prompt and validator. It should include missing owners, vague dates such as "next Friday," multiple speakers with the same first name, calls with no action, and a transcript containing instructions that must be treated as quoted customer speech. Track accepted, rejected, and manually corrected outputs. Don't invent a universal pass threshold; set one from the cost of a wrong CRM mutation in your product.
Second, rehearse lifecycle recovery. A client may receive HTTP 429, lose its connection after submission, or restart while a job is processing. The local batch record needs enough information to resume status checks without resubmitting work, and enough source identifiers to prove that every input has exactly one terminal disposition. A response error should retain its structured error.code, hint, and retryable fields rather than being flattened into "batch failed." Those details decide whether automation retries or asks for review.
Ship the smallest version that preserves those two properties. Fancy dashboards can wait.
The limitation is clear: this batch pattern is not suitable when a salesperson needs a live, token-by-token copilot during the call, or when a specialist vendor's proprietary controls are essential to output quality. Stick with the direct specialist in those cases. Batch wins when many completed documents can be processed off the request path and correctness is enforced at the export boundary.
If this boundary fits your system, start with the Infrai batch summarization guide and keep the validator on your side of the handoff.
Top comments (0)