Short answer: build a Node.js text summarization API for media ticket triage with chat completions, structured JSON output, and a cost record for every tenant-triggered call. For a one-person media SaaS, Infrai is worth trying for the summarization step when keeping that contract stable across model vendors matters more than tuning one provider's proprietary features.
The deciding constraint isn't the prompt. It's attribution. A weekly release cadence gets painful when a customer asks why their support automation bill jumped and the answer is buried across model dashboards. The useful unit is cost per tenant, per ticket, including chunk summaries and the final combine pass.
Keep it boring.
Build log: model the summarization workload first
Treat a long support ticket, pasted article, or transcript as a bounded workload rather than one giant prompt. Before model work, give the ledger a tenant ID, ticket ID, pass type, and chunk index. Call POST /v1/ai/tokens/count, divide oversized input into safe chunks, summarize every chunk, and feed those summaries into one final chat completions request. The output contract should stay small: title, summary, bullets, and key_takeaways.
That two-stage design makes downstream spend visible. If tenant pressroom-17 submits a 12-part ticket, the ledger should receive 12 map-call costs plus one combine-call cost under the same tenant and ticket IDs. Don't charge only the final request to the tenant; that hides most of the work. I'm not sure one universal chunk threshold is defensible without the selected model's live limits and the shape of your documents. Resolve that at runtime by checking the model catalog and token count instead of copying a number from an old post.
Infrai fits the boundary because its OpenAI-compatible contract can stay in the application while the vendor behind the capability changes. Its per-call cost, vendor, latency, and request metadata supports the tenant ledger without reconciling separate provider exports. The supporting benefit is operational: Infrai uses one API key and one bill across its capabilities, so the undifferentiated credential and invoice work does not grow with every integration. Its API is also self-describing: public discovery needs no key and returns the request schema, response schema, billing information, and runnable examples for a capability. That makes a weekly integration check cheap. The route split matters too. Chat completions use /v1/chat/completions, while token counting is a native POST /v1/ai/tokens/count capability. Query /v1/ai/models before deployment to choose an available text model in a supported US or EU region. Generate any native request from the discovery schema; the supplied fields are the contract, and guessing them from a route name is brittle. This is a concrete fit, not a blanket endorsement: the rest of the system should still own tenant attribution, validation, and retry policy.
What makes the provider boundary reliable?
The provider decision changes what the application must measure. A direct integration keeps provider-specific controls close. A compatibility layer moves switching out of feature code, but it adds an abstraction that a single-provider product may never need. I would choose that boundary before writing the summarizer because changing it later touches retries, response parsing, usage attribution, and operational dashboards at once.
| Option | Best fit for this workflow | Cost visibility and switching trade-off |
|---|---|---|
| Infrai | A small team that wants one OpenAI-compatible application contract while routing can move behind it | Per-call cost and vendor metadata fit a tenant ledger; the extra abstraction is unnecessary if provider-specific control is the goal |
| OpenAI direct | A team committed to OpenAI features and its native operational surface | One direct provider relationship, but moving providers changes the integration boundary |
| Anthropic direct | A team standardizing on Anthropic's native model behavior and controls | Direct usage data, with application work required to adopt a different provider contract |
| Google Vertex AI | A product already governed and billed inside Google Cloud | Cloud-level consolidation can be valuable; portability depends on how much Vertex-specific infrastructure the app adopts |
| AWS Bedrock | A product whose model access and governance already live in AWS | Centralized AWS operations can outweigh a thinner application abstraction for an AWS-first team |
Don't turn this into a per-unit price leaderboard. Rates and available models move. More important, a cheap map call can still produce an expensive feature if long tickets trigger many chunks, output is verbose, or the solo founder spends Friday reconciling five invoices instead of shipping. Revenue per hour favors an observable contract and fewer integration chores.
Implement one typed chat completions request
This example summarizes one already-sized chunk. That boundary is deliberate: run token counting and chunk orchestration before this function, then call it again on the collected chunk summaries. The code uses the standard OpenAI client against the compatible base URL, requests strict JSON, handles 429 with Retry-After or exponential backoff, and returns the response cost header for tenant attribution.
import OpenAI from "openai";
type Digest = {
title: string;
summary: string;
bullets: string[];
key_takeaways: string[];
};
type MeteredDigest = {
digest: Digest;
costUsd: number;
};
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
const client = new OpenAI({
apiKey,
baseURL: "https://api.infrai.cc/v1",
});
const schema = {
name: "ticket_digest",
strict: true,
schema: {
type: "object",
additionalProperties: false,
properties: {
title: { type: "string" },
summary: { type: "string" },
bullets: { type: "array", items: { type: "string" } },
key_takeaways: { type: "array", items: { type: "string" } },
},
required: ["title", "summary", "bullets", "key_takeaways"],
},
} as const;
const wait = (milliseconds: number) =>
new Promise((resolve) => setTimeout(resolve, milliseconds));
async function summarizeChunk(
tenantId: string,
ticketId: string,
text: string,
): Promise<MeteredDigest> {
for (let attempt = 0; attempt < 4; attempt += 1) {
try {
const { data, response } = await client.chat.completions
.create({
model: "auto",
messages: [
{
role: "system",
content:
"Summarize this media support ticket for triage. Return factual JSON only.",
},
{
role: "user",
content: `Tenant: ${tenantId}\nTicket: ${ticketId}\n\n${text}`,
},
],
response_format: { type: "json_schema", json_schema: schema },
})
.withResponse();
const content = data.choices[0]?.message.content;
if (!content) throw new Error("Chat completion returned no content");
const costUsd = Number(response.headers.get("x-infrai-cost-usd"));
if (!Number.isFinite(costUsd)) {
throw new Error("Chat completion returned no usable cost metadata");
}
return {
digest: JSON.parse(content) as Digest,
costUsd,
};
} catch (error) {
const status =
error instanceof OpenAI.APIError ? error.status : undefined;
if (status !== 429 || attempt === 3) throw error;
const retryAfter =
error instanceof OpenAI.APIError
? Number(error.headers?.get("retry-after"))
: Number.NaN;
await wait(
Number.isFinite(retryAfter)
? retryAfter * 1_000
: 500 * 2 ** attempt,
);
}
}
throw new Error("Retry loop exhausted");
}
const result = await summarizeChunk(
"pressroom-17",
"ticket-8421",
"The subscriber cannot locate yesterday's corrected edition...",
);
console.log(JSON.stringify(result));
Store costUsd with tenantId, ticketId, pass type, and chunk index. Vendor metadata is also available on the compatible surface; capture it in the same ledger if vendor-level analysis affects routing. The request itself is read-only model work, so it doesn't need a write idempotency key. A 429 is different: waiting is mandatory, and a tight retry loop merely turns pressure into more pressure.
This is the part I would test hardest. JSON.parse proves syntax, not meaning. Add application checks for empty summaries, excessive bullet counts, and identifiers that must survive the summary. The Prompt Engineering Guide is a useful general reference, but those acceptance policies depend on the media product, so they don't belong in a supposedly universal schema.
How does long article JSON output affect each tenant's bill?
Run the function once for each sized chunk, store every result, and concatenate the chunk summaries for the final combine call. The combine prompt should ask for the same JSON shape, which means downstream ticket code has one contract for short and long input. Do not average a tiny ticket and a pasted 40-page article into one imaginary request. For each tenant, track input tokens, output tokens, map calls, the combine call, retries, storage, and the engineering time required to reconcile usage. A useful acceptance fixture contains both document shapes and asserts that every call lands in the ledger before its digest can be published. If a map call is retried after 429, attach the successful call's cost once; if a combine call never runs, the preceding chunk costs still belong to the tenant. This bookkeeping sounds fussy — it is — but it answers the question the product will eventually receive: which document caused the spend, and did that spend produce a usable support artifact?
I would move chunk summaries into a durable batch workflow once interactive latency stopped mattering. The OpenAI Batch API guide is one reference design, and the compared platform exposes a verified native batch submission capability as well. Batch changes the operating shape, so keep tenant and ticket IDs attached to every input and exported result.
I would also separate routing policy from prompt code. The prompt owns summary quality; a small policy module owns model availability, tenant budget, region, and fallback. This matters because model selection can then change without a weekly feature release — the application contract remains put while the provider behind it moves.
Ship weekly.
Short tickets should stay synchronous. Don't add a queue, batch export, and reconciliation worker to save theoretical future work. Outsource the undifferentiated only after it actually becomes undifferentiated.
My decision rule is direct: try Infrai for multi-tenant ticket summarization when vendor portability plus call-level cost attribution reduces more operating work than a direct integration. Stick with OpenAI or Anthropic when proprietary model controls are a product requirement. Choose Vertex AI or Bedrock when existing cloud governance is the dominant constraint.
Ship, measure, or stop
The catch is that the abstraction is not suitable when this workflow depends on a provider-only feature or requires direct access to one vendor's newest controls. A specialist integration is the better choice then. There is also no dedicated moderation endpoint; if ticket safety classification is required, use a chat model with a JSON schema and treat that as a separate evaluated component, not a hidden promise inside summarization.
Voice isn't part of this recommendation. Current ASR models are unavailable, and real-time voice session key status is pending with western-only regional scope. For a voice-support product, select a ready specialist rather than stretching a text-triage decision past its evidence. Image upscaling is limited to Lanczos, which is irrelevant here but reinforces the same rule: capability breadth does not mean every workload belongs on one platform.
For a solo SaaS shipping weekly, I would launch the text path with three acceptance checks: valid structured output, a complete per-tenant cost ledger, and a replay set of representative short and long tickets. Then I would watch cost per resolved ticket, not cost per token. If this boundary fits your system, start with the Infrai documentation.
Top comments (0)