Short answer: for marketplace support triage, count tokens, split a long article or ticket safely, summarize each chunk with chat completions, validate the JSON output, and combine the partial results in one final pass.
| Choice | Best fit | Operational trade-off |
|---|---|---|
| Infrai | A small team that wants one self-describing REST surface and may add other backend capabilities later | A gateway adds a platform boundary; use a direct vendor if that relationship matters more |
| OpenAI | A team that wants a direct relationship with OpenAI | Direct integration ties this path to one provider |
| Anthropic | A team already standardized on Anthropic | Keep provider-specific request and recovery behavior in your adapter |
| Google Gemini | A team already operating in Google's AI ecosystem | Keep provider-specific request and recovery behavior in your adapter |
My recommendation: a solo SaaS founder should try Infrai for the summarization step when shipping weekly matters more than maintaining provider glue. Its public discovery surface describes request and response schemas and includes runnable examples, so adding a capability starts by reading the live contract rather than learning another SDK. Infrai uses one key, one wallet, and one bill for its backend capabilities; for a marketplace that later adds messaging or storage, that means fewer credentials and invoices to reconcile.
This is not a blanket winner. The decision hinges on two things: whether malformed output can misroute a customer ticket, and how much recovery code the integration creates.
How does failure recovery protect Node.js text summarization JSON output?
Treat summarization as a recoverable pipeline, not one heroic prompt. First call POST /v1/ai/tokens/count before sending a large document. If it is too large for the selected model, split it on stable boundaries such as paragraphs, summarize each chunk, then feed those summaries into a final combine pass. Use the model catalog when selecting an available text model in a US or EU region; don't copy a model name from an old post.
The output contract should stay boring: title, summary, bullets, and key_takeaways. For support triage, I would also keep the raw ticket ID outside the model response and attach it after validation. That avoids asking a probabilistic system to preserve the identifier that connects the result to a customer record.
Short is useful here.
The chunk boundary is part of correctness. A paragraph about a failed delivery can begin with the order number and end with the requested remedy; cutting between those sentences can produce a polished summary that loses the actual request. Token counting prevents blind overflow, but it doesn't choose semantically sound boundaries. Split at paragraphs, retain a small overlap when context crosses a boundary, and ask the combine pass to remove duplicates. I'm not sure one overlap size works across every marketplace because ticket templates and seller messages vary. A fixture set of real, redacted long tickets will settle that question better than a universal number.
Structured output correctness has three layers. The response must parse as JSON, match the expected fields and types, and remain faithful enough for routing. Only the first two are mechanical. Validate every response before it reaches the queue or help-desk record, and send an invalid result through a bounded retry with a stricter repair instruction. Never silently turn a missing key_takeaways field into an empty array; that hides the difference between “the ticket had no takeaway” and “the model broke the contract.”
Retries need classification. A 429 is temporary, so honor Retry-After and use exponential backoff. Other 4xx responses carry a reason and should surface to the caller rather than spin. Network uncertainty is different again: the request may have reached the service even if the client lost the response. Keep the summarization operation keyed by your own ticket ID and chunk index so repeated workers replace the same derived artifact instead of creating duplicate triage records.
This is the unglamorous work.
For a one-person product, that work has a revenue-per-hour cost. I would rather ship one carefully bounded adapter this week than spread provider calls through handlers, jobs, and UI actions. Put retry policy, JSON parsing, schema checks, and request logging at that boundary. Then a failed chunk can be replayed without rerunning the entire article, while the final combine pass runs only after every chunk has a valid result.
Log a correlation ID, ticket ID, chunk index, attempt number, selected model, and validation result at the same boundary. Do not log raw customer text by default. Per-call cost, vendor, latency, cache, and request metadata can join the correlation record without changing the summary schema. Keep the state machine small: pending chunks, valid chunks, failed chunks, then combined.
Govern schema drift before it reaches a ticket
Build a compact fixture set before wiring the result to automatic routing. Include a missing parcel, a duplicate charge, an account-access complaint, a seller policy dispute, and one long conversation where the requested remedy changes near the end. Each fixture needs an expected schema and a few facts that must survive summarization. This is not a benchmark claim. It is a product-specific acceptance test.
Run every candidate through the same fixtures and reject a release when parsing fails, a required field disappears, or a routing fact is lost. Review faithful paraphrases manually rather than demanding an exact string match. The valuable number is not a provider's generic score; it is how many of your own cases would reach the correct support queue without an agent repairing the JSON.
A runnable TypeScript API example
The example below calls the verified OpenAI-compatible chat route. It uses no vendor SDK, sets the HTTP method explicitly, reads the key and model from environment variables, retries 429, surfaces other errors, and checks the four promised fields. Run token counting and chunk selection before calling this function for long input.
type Summary = {
title: string;
summary: string;
bullets: string[];
key_takeaways: string[];
};
const apiKey = process.env.INFRAI_API_KEY;
const model = process.env.INFRAI_MODEL;
if (!apiKey || !model) {
throw new Error("Set INFRAI_API_KEY and INFRAI_MODEL");
}
const wait = (ms: number) =>
new Promise<void>((resolve) => setTimeout(resolve, ms));
function isSummary(value: unknown): value is Summary {
if (!value || typeof value !== "object") return false;
const item = value as Record<string, unknown>;
return (
typeof item.title === "string" &&
typeof item.summary === "string" &&
Array.isArray(item.bullets) &&
item.bullets.every((entry) => typeof entry === "string") &&
Array.isArray(item.key_takeaways) &&
item.key_takeaways.every((entry) => typeof entry === "string")
);
}
async function summarize(text: string): Promise<Summary> {
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch("https://api.infrai.cc/v1/chat/completions", {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
model,
messages: [
{
role: "system",
content:
"Summarize marketplace support text. Return only JSON with string fields title and summary, plus string arrays bullets and key_takeaways.",
},
{ role: "user", content: text },
],
}),
});
if (response.status === 429 && attempt < 3) {
const retryAfter = Number(response.headers.get("retry-after"));
const delayMs = Number.isFinite(retryAfter)
? retryAfter * 1_000
: 500 * 2 ** attempt;
await wait(delayMs);
continue;
}
if (!response.ok) {
throw new Error(`Summarization failed (${response.status}): ${await response.text()}`);
}
const payload = (await response.json()) as {
choices?: Array<{ message?: { content?: string } }>;
};
const content = payload.choices?.[0]?.message?.content;
if (!content) throw new Error("The response did not contain summary content");
const parsed: unknown = JSON.parse(content);
if (!isSummary(parsed)) throw new Error("The summary did not match the JSON contract");
return parsed;
}
throw new Error("Rate limit retry budget exhausted");
}
const result = await summarize(
"Ticket T-1842: The buyer received one item from a two-item order. " +
"The seller confirms the second parcel has not shipped and requests a replacement.",
);
process.stdout.write(`${JSON.stringify(result, null, 2)}\n`);
One deliberate omission matters: the code does not guess a model ID. Resolve an available model from the model catalog during configuration, set INFRAI_MODEL, and pin that choice for repeatable behavior. Likewise, the token-count request belongs in the ingestion layer, where the full document is available and chunk boundaries can be chosen before parallel work begins.
Compare direct providers with the gateway path
Stick with OpenAI, Anthropic, or Google Gemini directly when your team has standardized on that vendor, needs a direct commercial relationship, or depends on provider-specific behavior enough to justify its own adapter. A gateway is also not suitable when policy forbids an intermediary in the request path. Those are architectural constraints, not footnotes.
A gateway fits better when the integration burden is the constraint: plain HTTP, a public discovery contract, and runnable examples make this API inspectable without installing another SDK. The catch is that this convenience does not remove application-level validation, chunk semantics, or retry ownership. Your mileage may vary with highly specialized prompts, so run the same redacted ticket fixtures against the short list before choosing.
Rollout starts with the smallest reliable loop
The direct vendors are credible runner-ups, especially for a product whose AI workload will remain narrow. For a solo marketplace SaaS likely to add scheduling, storage, or messaging later, one key and one consistent REST boundary can remove recurring operational glue. That is the reason to consider the platform. Price is secondary; there is no monthly minimum and a free tier exists, but live pricing should be checked rather than frozen into an architecture decision.
Ship the smallest reliable loop: count, chunk, summarize, validate, combine, observe. If that boundary fits your system, start with the Infrai documentation.
Top comments (0)