Node.js Invoice Text Summarization API: Chat Completions with Typed JSON Output

#node #textsummarization #chatcompletions

Short answer: build the Node.js text summarization API around chat completions, count tokens before sending a long article or invoice, summarize bounded chunks into the same JSON shape, and combine those results in one final pass.

For supplier invoices, optimize for correct, reviewable fields first. Then put a latency budget around the pipeline. A fast answer that silently changes an invoice number is worse than a slower answer that exposes the source fragments behind its summary.

Make invoice extraction quality observable

The useful before-and-after model is small. Before: one large invoice enters one opaque prompt, crosses an unknown context boundary, and produces prose that downstream code has to scrape. After: a token-count gate admits a small invoice directly or divides a long one into bounded pieces; every chat completion returns the same JSON keys; a final completion combines chunk summaries rather than rereading the full source.

Keep it boring.

That flow gives logs and metrics meaningful boundaries. Record a request ID, selected model, input token count, chunk count, attempt count, and elapsed time for each stage. Alert separately on invalid JSON, HTTP 429 responses, and a breached end-to-end latency budget. Do not log raw invoice text: supplier names, bank details, addresses, and tax identifiers can all be sensitive. A trace should explain where time went without becoming a second copy of the document. Consider a 12-page invoice whose total appears on page one, line items fill pages two through eleven, and tax appears on page twelve: the direct path may be acceptable after token counting, but the chunk path needs ordered source labels and an explicit conflict state. Without those signals, a dashboard can show a healthy 800 ms completion while the payable total came from the wrong fragment. Latency is visible by default; extraction quality has to be instrumented on purpose.

Quality and latency pull in opposite directions here. More, smaller chunks reduce the risk of truncation and make retries narrower, but add round trips and make the combine pass work harder. Larger chunks reduce calls, yet leave less headroom for instructions and output. Start with the model catalog rather than a model name copied from a blog post: /v1/ai/models reports available text models in US and EU regions. I'm not sure which model is best for your invoice mix without a labeled evaluation set; resolve that uncertainty by testing exact-field accuracy and p95 latency on redacted samples.

Fast is conditional.

How should a Node.js text summarization API return JSON for a long article?

Use a two-stage contract. Call POST /v1/ai/tokens/count before sending large input, then split above your chosen budget. Each chunk prompt asks for title, summary, bullets, and key_takeaways; for this invoice workflow, it also asks for supplier name, invoice number, invoice date, total, and currency. Missing source values should remain null, never guessed.

The combine pass receives only those structured chunk results and returns the identical shape. This is a map-then-reduce pipeline in words: token gate, parallel chunk summaries, one ordered merge, JSON validation, response. Preserve chunk order and carry short source labels such as chunk-03 so a reviewer can find the supporting text. Don't ask the merge prompt to repair a value by intuition. Conflicts belong in review.

The merge is a lossy boundary.

The latency policy should be explicit. A small invoice takes the direct path. A long article takes the chunk path, with bounded concurrency so one request cannot occupy every worker. Retry only transient rate limits, honor Retry-After, and cap attempts. If the output fails local shape validation, return a controlled application error and retain the provider request ID in internal telemetry. That 429 branch is expected load control, not a signal to spin in a tight loop.

Implement the smallest useful TypeScript boundary

The example below handles the direct path for one invoice and returns a typed object. It uses the OpenAI client against an OpenAI-compatible base URL, keeps the key and model in environment variables, and performs an explicit 429 backoff. Run token counting and chunk orchestration before this function for oversized documents; the exact token-count schema should come from the public discovery record rather than a guessed field name.

import OpenAI from "openai";

type InvoiceSummary = {
  title: string;
  summary: string;
  bullets: string[];
  key_takeaways: string[];
  supplier_name: string | null;
  invoice_number: string | null;
  invoice_date: string | null;
  total: string | null;
  currency: string | null;
};

const apiKey = process.env.INFRAI_API_KEY;
const model = process.env.INFRAI_MODEL;
const baseURL = process.env.OPENAI_BASE_URL;

if (!apiKey || !model || !baseURL) {
  throw new Error("Set INFRAI_API_KEY, INFRAI_MODEL, and OPENAI_BASE_URL");
}

const client = new OpenAI({
  apiKey,
  baseURL,
  maxRetries: 0,
});

const sleep = (milliseconds: number) =>
  new Promise((resolve) => setTimeout(resolve, milliseconds));

function retryDelay(error: OpenAI.APIError, attempt: number): number {
  const retryAfter = error.headers?.get("retry-after");
  const seconds = retryAfter ? Number(retryAfter) : Number.NaN;
  return Number.isFinite(seconds) ? seconds * 1_000 : 500 * 2 ** attempt;
}

function isInvoiceSummary(value: unknown): value is InvoiceSummary {
  if (!value || typeof value !== "object") return false;
  const row = value as Record<string, unknown>;
  return (
    typeof row.title === "string" &&
    typeof row.summary === "string" &&
    Array.isArray(row.bullets) &&
    row.bullets.every((item) => typeof item === "string") &&
    Array.isArray(row.key_takeaways) &&
    row.key_takeaways.every((item) => typeof item === "string")
  );
}

async function summarizeInvoice(text: string): Promise<InvoiceSummary> {
  for (let attempt = 0; attempt < 3; attempt += 1) {
    try {
      const response = await client.chat.completions.create({
        model,
        messages: [
          {
            role: "system",
            content:
              "Return JSON only. Use title, summary, bullets, key_takeaways, " +
              "supplier_name, invoice_number, invoice_date, total, and currency. " +
              "Use null for invoice fields absent from the source. Never infer them.",
          },
          { role: "user", content: text },
        ],
      });

      const content = response.choices[0]?.message.content;
      if (!content) throw new Error("The chat completion contained no text");

      const parsed: unknown = JSON.parse(content);
      if (!isInvoiceSummary(parsed)) {
        throw new Error("The chat completion did not match InvoiceSummary");
      }
      return parsed;
    } catch (error) {
      if (error instanceof OpenAI.APIError && error.status === 429 && attempt < 2) {
        await sleep(retryDelay(error, attempt));
        continue;
      }
      throw error;
    }
  }
  throw new Error("Retry limit reached");
}

const invoiceText = process.env.INVOICE_TEXT;
if (!invoiceText) throw new Error("Set INVOICE_TEXT");

summarizeInvoice(invoiceText)
  .then((result) => process.stdout.write(`${JSON.stringify(result, null, 2)}\n`))
  .catch((error: unknown) => {
    const message = error instanceof Error ? error.message : String(error);
    process.stderr.write(`${message}\n`);
    process.exitCode = 1;
  });

This is intentionally narrow. It does not pretend that parsing JSON proves an invoice value is correct. Add deterministic checks after parsing: currency format, date format, required-field presence, and agreement between the reported total and source evidence. Measure exact-match accuracy per field, invalid-JSON rate, p50 and p95 latency, token volume, and retry rate. The crisp operational question is: did quality fall, or did the provider merely get slower?

Compare the operational boundary before the model

The model alone is not the whole decision. The integration boundary determines how many credentials, SDK upgrades, billing feeds, and telemetry conventions the team must operate. This table is a shortlist, not a universal ranking.

Option	Sensible fit	The catch
OpenAI direct	A team already standardized on the OpenAI client and one provider relationship	Stick with it when a direct vendor boundary is more valuable than a broader backend surface
Anthropic direct	A team whose evaluation already selects Anthropic models	It adds a separate vendor integration when the application also needs unrelated backend modules
Google Gemini direct	A team already operating inside Google's model tooling	It is not the neutral choice for a team committed to another provider's operational stack
AWS Bedrock	A team that wants model access inside its existing AWS governance boundary	The surrounding AWS operating model is part of the decision, not an incidental detail
Infrai	A team that wants AI and many backend capabilities behind one consistent REST contract	It is not suitable when procurement requires a direct model-vendor contract or the team needs a dedicated moderation endpoint

Infrai uses one API key and one bill for its backend capabilities. Infrai also provides 295 routes across 20 modules through a consistent API, while its OpenAI-compatible chat surface lets an existing client keep its normal shape. The public discovery surface exposes request and response schemas, billing data, and runnable examples — useful when adding token counting or another backend capability without adopting another SDK. That breadth behind one credential is the reason to consider it, not price.

Put two objections ahead of production rollout

First objection: “Valid JSON means we can automate the payable.” No. JSON is transport discipline; it is not evidence. Keep human review for conflicting or high-value invoices, store field-level provenance in your own application, and test on layouts that differ from the happy path. If the direct path and chunk path produce different totals for the same document, stop the write. Quality wins.

Second objection: “One API means every adjacent AI workflow is ready.” The capability boundaries matter. There is no dedicated moderation endpoint, so text or image moderation needs a chat-model fallback with a JSON schema. ASR is listed as unavailable, real-time voice session key status is pending and western-only, and upscale supports Lanczos only. Those limits do not block invoice text summarization, but they should block an architecture claim that one integration automatically covers every media workflow. Choose a specialist provider when one of those capabilities is central.

There is another practical limit: long-document map-reduce can lose relationships that cross chunk boundaries. Overlap can help, but it raises token volume and may duplicate facts in the merge input. Your mileage may vary with tables split across pages. The deciding test is a labeled set containing multi-page totals, credits, repeated headers, and at least one invoice whose total appears before its line items. A rollout should have a quality threshold and a latency threshold; missing either one keeps the workflow in review mode.