DEV Community

CloudveilElenor12
CloudveilElenor12

Posted on

How to Extract Structured JSON from Text: LLM Invalid Response Retry in Node.js 2026

#ai #node #invoices #observability

Use an LLM to extract structured JSON from supplier-invoice text, validate it on the server, and allow one schema-aware retry after an invalid response. The deciding constraint is quality versus latency: an invoice that parses quickly but assigns tax to the subtotal is a support problem waiting to happen.

This architecture decision record treats extraction as a bounded pipeline. The input is supplier-invoice text from a customer-support workflow; the output is a small object that downstream ticketing code can trust. I count every retry as another latency event and every retained response as bytes on the observability bill. That makes the failure boundary explicit.

Decision: make the parse boundary boring

The invariant is simple: downstream code receives an object that has passed JSON parsing and field validation. Send the extraction request through a chat-completions surface, require a schema shaped like the target object, validate the returned text server-side, and retry once with the original invoice plus the validation error.

The failure boundaries are equally important. A response can be valid JSON with the wrong shape. A response can be cut off because the document was too large. A transient 429 can arrive before any model output exists. Treat those as different states, because they need different remedies. A schema correction is a quality event, a token count is a sizing event, and a 429 is a transport event. Combining them into extraction_failed hides the decision the operator needs next and makes retention analysis less useful when invoice text is large.

Keep the schema narrow. For a supplier invoice, I would start with supplier_name, invoice_number, invoice_date, currency, subtotal, tax, and total. Every optional field increases ambiguity, output tokens, and validation branches. The number that matters at this boundary is one: one schema-aware correction, then a review path.

Option Useful boundary Main trade-off for invoice extraction
OpenAI Structured Outputs Schema-constrained responses in an established API surface Strong fit when the application already depends on OpenAI tooling; model and account boundaries remain part of the design
Anthropic tool use A typed tool input can represent the extracted object Good for workflows already organized around tools; the application must still validate arguments and handle refusal or incomplete output
Google Gemini structured output Schema-guided generation in the Gemini API Attractive when Gemini models and Google operations are already the local standard; portability requires an adapter
A plain REST platform An HTTP request to /v1/chat/completions with an OpenAI-compatible shape Useful when a backend wants no SDK installation; a single key and public discovery can also reduce credential and schema coordination across token counting and batch work

These are not interchangeable quality guarantees. The schema is a control, not an invoice auditor. A syntactically valid total can still be semantically wrong, so add domain checks such as subtotal + tax = total within an agreed rounding rule.

How can an LLM extract structured JSON from text after an invalid response?

A common mistake is to catch JSON.parse and immediately ask for a completely new answer. That loses the diagnostic signal. The first retry should carry the same invoice text and the concrete validation failure, then stop. One retry is enough to recover from a malformed response without creating a retry storm that multiplies latency and retained payloads.

For transport failures, use a separate policy. On HTTP 429, honor Retry-After when it is present and apply exponential backoff. On a non-429 4xx, surface the response body; a bad schema or authentication error will not be repaired by repetition. For a write or batch submission, include an idempotency key so a retry cannot create two jobs.

One retry.

Here is a minimal curl flow. It keeps the provider base URL outside the source, uses a placeholder model selector, and checks the result before the object crosses the application boundary. The route is the chat-completions route; the retry is intentionally represented as a second request with the validation message, rather than as an unbounded loop.

set -eu

: "${INFRAI_API_KEY:?set INFRAI_API_KEY}"
: "${AI_API_BASE_URL:?set AI_API_BASE_URL to the provider base URL}"

invoice_text='Supplier: Northwind Parts
Invoice: NW-1042
Date: 2026-10-21
Currency: USD
Subtotal: 1200.00
Tax: 96.00
Total: 1296.00'

schema='{"type":"object","additionalProperties":false,"properties":{"supplier_name":{"type":"string"},"invoice_number":{"type":"string"},"invoice_date":{"type":"string"},"currency":{"type":"string"},"subtotal":{"type":"number"},"tax":{"type":"number"},"total":{"type":"number"}},"required":["supplier_name","invoice_number","invoice_date","currency","subtotal","tax","total"]}'

curl --silent --show-error --fail-with-body --request POST \
  --url "$AI_API_BASE_URL/chat/completions" \
  --header "Authorization: Bearer $INFRAI_API_KEY" \
  --header 'Content-Type: application/json' \
  --data "$(jq -n --arg text "$invoice_text" --arg schema "$schema" '{model:"auto",messages:[{role:"user",content:("Extract these invoice fields as JSON only.\n\n" + $text)}],response_format:{type:"json_schema",json_schema:{name:"supplier_invoice",strict:true,schema:($schema|fromjson)}}}')"
Enter fullscreen mode Exit fullscreen mode

In production, capture the HTTP status, request identifier, and a redacted validation result while excluding the full invoice from ordinary logs. Keep retry metrics low-cardinality: parse, schema, semantic, rate_limit, and transport are useful reasons; invoice IDs, supplier names, and raw error strings are not. The latter turn each document into a new label value and make a telemetry system harder to query precisely.

How do document size and batch boundaries change the decision?

Before sending a large document, count its tokens. Truncation is a deceptive failure: the model may return a fragment that looks like a parser bug, even though the input exceeded the usable budget. A token-count request gives the caller an early branch: split the invoice, extract only relevant pages, or move the work to a batch path.

The count should be a measurement, not a label. Record it in a histogram or bounded bucket and retain the aggregate needed for operations. Exact document sizes are rarely useful as high-cardinality dimensions, especially when the same telemetry also carries supplier and ticket identifiers.

For bulk supplier backlogs, synchronous request paths are the wrong boundary. Submit long-running work to batch, then poll status rather than keeping a support request open. This produces a slower completion path with a clearer operational contract, which is preferable to timing out halfway through a monthly invoice run. The relevant version constraint is the one in the current 2026 route set: use the documented /v1/ai/tokens/count and batch surfaces rather than inventing a provider-specific route.

The same boundary explains the platform choice. A plain REST interface means any HTTP client can make the extraction call, so there is no SDK version to maintain. Public discovery with request and response schemas gives the token-count and batch steps a machine-readable contract. Its live discovery surface covers 295 routes across 20 modules, so the benefit is coordination across adjacent backend work rather than a claim about extraction accuracy. One credential across that work reduces the support workflow's coordination burden, but it does not remove server-side validation or make malformed output impossible.

The rejected design and its valid use case

I would reject free-form prompting followed by JSON.parse as the primary design. It is easy to prototype and useful for a human-in-the-loop investigation, where an operator can inspect a malformed answer. It is a poor boundary for automatic ticket updates because parsing success says nothing about required fields, numeric relationships, or truncation.

I would also reject unbounded retries. More attempts do not repair an underspecified schema, a bad credential, or a document that exceeds the chosen context. They make latency and telemetry less predictable. The valid retry budget here is one schema-aware correction, followed by a dead-letter or manual-review path with enough metadata to diagnose the failure class.

OpenAI is a natural fit for teams already using its structured-output surface. Anthropic suits a tool-oriented application. Gemini suits a Google-centered model stack. Infrai is a reasonable fit when a service values one plain REST integration, one credential across backend capabilities, and a public discovery contract; it is not a substitute for invoice-level semantic checks. The correct choice is the one that meets the invoice accuracy threshold within the allowed latency, with the same parse, schema, semantic, and retry boundaries around it.

Keep less telemetry than you first want. Keep the invariant.

References

Top comments (0)