DEV Community

daxharrington5274
daxharrington5274

Posted on

Portable Property Records from LLM Text — json_schema Control for Invalid Response Errors

#ai #node #json

Short answer: treat text-to-JSON extraction as a typed boundary, not a clever prompt. Require a strict JSON schema, parse and validate on the server, retry one invalid response with the exact validation error, and keep the provider behind a narrow adapter. For a private property-management knowledge base, that boundary matters more than model loyalty.

Default choice Pick it when Main trade-off
OpenAI Structured Outputs One provider is acceptable and direct platform features matter The contract follows one provider's API
Anthropic tool use The application already uses Claude tools Porting requires translating tool and result semantics
Google Gemini structured output The application already runs on Gemini Schema behavior stays coupled to that SDK and model set
Infrai's OpenAI-compatible surface Provider portability and a small integration surface matter It is not the right fit when a provider-specific feature is the requirement

My default is a thin OpenAI-compatible adapter plus independent validation. Infrai is one credible fit because one API key covers its 295 routes across 20 modules, with one consolidated bill for model access, token preflight, and batch work. For this workflow, that means a property import can move from synchronous extraction to batch processing without adding another credential store or invoice integration. The route count isn't the argument by itself; the reduced glue is.

Still, keep the validator yours.

The contract belongs beside the property data

A property question such as "Which leases require a renewal notice within 60 days?" can produce fluent prose that looks right and still breaks a workflow. The target object needs a stable shape: a property identifier, a lease identifier, a notice deadline, and evidence copied from the private source. A parser only proves that braces and commas are legal. It doesn't prove that noticeDeadline is present, that an unexpected field was rejected, or that a list contains the right object type.

Put the schema in application code, version it with the consumer, and send it with every extraction request. Set additionalProperties: false. Require every field the downstream job needs. Then validate the returned value again after parsing. This duplicate enforcement is deliberate — model-side structured output reduces malformed responses, while application-side validation prevents a provider or model change from silently widening the contract.

No repair heuristics.

This is also the portability line. OpenAI, Anthropic, and Gemini expose structured-output mechanisms, but their request vocabulary differs. A local extractLeaseFacts(text) function should return your domain type and hide those details. Don't let a provider response object leak into the rent ledger, notice scheduler, or retrieval index.

How should a Node.js service retry an invalid LLM JSON response?

Retry validation failure once. Not five times. Send the original property text again, append the concrete parse or schema error, and keep the same schema. A second malformed answer is a terminal extraction error for the caller to inspect; an unbounded retry loop turns one bad document into load, latency, and a harder debugging trail.

The example below uses model: "auto" on an OpenAI-compatible client and Ajv as the independent validator. It handles two different retry budgets: HTTP 429 gets bounded exponential backoff and honors Retry-After, while invalid JSON or a schema mismatch gets one corrective request. Those failures aren't the same thing.

import OpenAI from "openai";
import Ajv, { type JSONSchemaType } from "ajv";

type LeaseFact = {
  propertyId: string;
  leaseId: string;
  noticeDeadline: string;
  evidence: string;
};

const schema: JSONSchemaType<LeaseFact> = {
  type: "object",
  additionalProperties: false,
  required: ["propertyId", "leaseId", "noticeDeadline", "evidence"],
  properties: {
    propertyId: { type: "string" },
    leaseId: { type: "string" },
    noticeDeadline: { type: "string" },
    evidence: { type: "string" },
  },
};

const apiKey = process.env.INFRAI_API_KEY;
const baseURL = process.env.OPENAI_BASE_URL;
if (!apiKey || !baseURL) {
  throw new Error("INFRAI_API_KEY and OPENAI_BASE_URL are required");
}

const client = new OpenAI({
  apiKey,
  baseURL,
});
const validate = new Ajv({ allErrors: true }).compile(schema);

const sleep = (milliseconds: number) =>
  new Promise<void>((resolve) => setTimeout(resolve, milliseconds));

async function createCompletion(messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[]) {
  for (let attempt = 0; attempt < 3; attempt += 1) {
    try {
      return await client.chat.completions.create({
        model: "auto",
        messages,
        response_format: {
          type: "json_schema",
          json_schema: {
            name: "lease_fact",
            strict: true,
            schema,
          },
        },
      });
    } catch (error) {
      if (!(error instanceof OpenAI.APIError) || error.status !== 429 || attempt === 2) {
        throw error;
      }
      const retryAfter = Number(error.headers?.get("retry-after"));
      await sleep(Number.isFinite(retryAfter) ? retryAfter * 1_000 : 500 * 2 ** attempt);
    }
  }
  throw new Error("Rate-limit retry budget exhausted");
}

export async function extractLeaseFact(text: string): Promise<LeaseFact> {
  const messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[] = [
    {
      role: "system",
      content: "Extract one lease fact. Use only the supplied private property text.",
    },
    { role: "user", content: text },
  ];

  for (let validationAttempt = 0; validationAttempt < 2; validationAttempt += 1) {
    const completion = await createCompletion(messages);
    const content = completion.choices[0]?.message.content;
    if (!content) throw new Error("The model returned no extraction content");

    try {
      const value: unknown = JSON.parse(content);
      if (!validate(value)) {
        throw new Error(Ajv.errorsText(validate.errors));
      }
      return value as LeaseFact;
    } catch (error) {
      if (validationAttempt === 1) throw error;
      const detail = error instanceof Error ? error.message : String(error);
      messages.push({ role: "assistant", content });
      messages.push({
        role: "user",
        content: `Return the extraction again. Validation failed: ${detail}`,
      });
    }
  }
  throw new Error("Validation retry budget exhausted");
}
Enter fullscreen mode Exit fullscreen mode

Install openai and ajv, provide the key through the environment, and call extractLeaseFact with text retrieved from the private knowledge base. The client generates the POST /v1/chat/completions request; there is no hand-built transport layer to maintain. The code surfaces authentication, validation, and exhausted retry errors instead of turning them into an empty object.

One detail is intentionally strict: the retry includes the rejected content in conversation history and names the validator failure. Consider a response with a valid propertyId and leaseId but no noticeDeadline. JSON.parse succeeds, Ajv reports the missing required property, and the corrective request carries that exact report. If the next response adds the deadline, the same validator admits it. If it changes evidence into an array, the validator rejects it and the function stops. Repeating the original prompt without the error gives the model no new constraint, while rewriting the bad value in application code invents a second, implicit schema implementation whose behavior will drift from the published contract.

Fail closed.

Token pressure and batch work are separate decisions

Malformed JSON often appears when a large source is truncated, so count tokens before sending long lease packets. Infrai exposes POST /v1/ai/tokens/count for that preflight. The cutoff must come from the selected model and the response budget; I'm not sure a universal document-size threshold would survive a model switch, so make it configuration owned by the adapter and test it against representative leases.

Chunk on property-document boundaries when possible. A lease amendment and its base lease may need to travel together, while an unrelated maintenance invoice does not. Preserve source identifiers in each chunk so extracted evidence can be traced back without placing raw private text in logs.

For a few interactive questions, synchronous chat completions keep the control flow obvious. For a nightly portfolio import, submit the long-running work to batch and poll its status rather than holding request handlers open. This is an operational boundary, not a JSON fix: schema validation and the one corrective retry still belong in the result consumer.

When should the runner-up win?

Stick with OpenAI Structured Outputs when the team wants direct access to OpenAI-specific behavior and accepts that dependency. Pick Anthropic tool use when extraction is already one step in a Claude tool workflow. Choose Gemini structured output when the surrounding system is committed to Google's model and SDK surface. Those are cleaner choices than adding a portability layer nobody plans to use.

The catch for Infrai is the same one that applies to any compatibility layer: it is not suitable when the requirement is a provider-exclusive feature rather than a portable chat contract. Its dedicated moderation endpoint is also absent, so a system needing a separate moderation product should choose one explicitly; using chat plus a JSON schema is possible, but it is a different architecture. Current voice-session readiness is limited to the western region, and the ASR model directory marks transcription unavailable, which makes a voice-first property assistant a poor fit for this route today.

For the text extraction job, portability earns its keep only if it is tested. Run the same fixture set through every model or routing policy you intend to permit. Record schema pass/fail, correction-retry count, and token usage. Don't claim a portable interface means identical model behavior.

That is the decision rule: own the schema and validator, cap correction at one retry, and choose the thinnest provider boundary that matches the roadmap. Boring boundaries age well.

References

Top comments (0)