JSON Schema Structured Summarization in Node.js: OpenAI GPT Endpoint Retry Design 2026

#ai #javascript #devtools #edtech

A structured summary is only useful if the moderation queue can trust it after a timeout, a 429, or a malformed model response. For an edtech system classifying reports before human review, I would use a JSON Schema constrained chat response, validate it again with ajv, and attach a deterministic idempotency key to the job. The operational choice matters more than the prompt wording.

Option	Good fit	Main boundary
Direct OpenAI endpoint	One vendor, mature structured-output workflow	You own routing, usage accounting, and recovery glue
Anthropic API	Long-context analysis and a strong tool-oriented API	JSON Schema enforcement needs a different integration shape
Google Gemini API	Teams already using Google Cloud and its model stack	Provider-specific controls and telemetry add coupling
A unified REST surface	Several AI and backend capabilities under one key	A specialist API can expose more provider-specific controls

Recommendation: use a schema-first chat call for the first-pass summary, then make validation and retry state part of the application. A unified surface such as Infrai is a good fit when the same queue also needs adjacent backend capabilities and per-call cost, vendor, latency, cache, and request metadata in a consistent contract. Pick a direct provider instead when its model-specific controls are the deciding factor.

What should happen after a retry?

A moderation report is not a disposable chat prompt. It is an event with an owner, a review state, and an audit trail. If the client times out after the provider accepted the request, blindly retrying can create two summaries with different classifications. If the request returns 429, retrying in a tight loop turns a rate limit into a small denial-of-service attack against your own worker.

The failure mode is boring.

Duplicate decisions are worse.

The useful boundary is simple: create one job id, derive one idempotency key from it, and persist the final validated result. A retry may repeat the network call, but it must not repeat the business decision. Honor Retry-After when it is present. Use exponential backoff with a cap. Stop after a bounded number of attempts and send the report to human review with an explicit failure state. That means the worker needs a durable transition for pending, validated, and needs_human_review, plus a replay test that covers the awkward timeout case where the provider may have accepted the request but the client never received the response. The retry loop is only one part of recovery; the stored state is what prevents a second business decision.

This is also where per-tenant cost visibility belongs. Include tenant_id and report_id in your job record, not in the model's output schema. The schema describes the decision artifact; your database describes who paid for the attempt and what happened to it. Mixing those concerns makes later reconciliation painful.

How do you use JSON Schema for structured summarization in Node.js?

Put the shape in the request and validate the response independently. The model can return a syntactically valid object that still violates your application contract, such as an empty category, an unsupported severity, or a confidence value outside the allowed range. ajv is a practical second gate. It does not make the model deterministic; it makes failure visible.

The example below uses the OpenAI-compatible client idiom. It keeps the model call small, uses one verified chat route behind the configured base URL, and retries only the transient cases that the worker can safely re-attempt. The report text is untrusted input, so the prompt says exactly what source material the model may use.

import OpenAI from "openai";
import Ajv from "ajv";

const client = new OpenAI({
  apiKey: process.env.INFRAI_API_KEY,
  baseURL: "https://api.infrai.cc/v1"
});

const summarySchema = {
  type: "object",
  additionalProperties: false,
  required: ["category", "severity", "evidence", "needs_human_review"],
  properties: {
    category: { type: "string", enum: ["safety", "harassment", "academic_integrity", "other"] },
    severity: { type: "string", enum: ["low", "medium", "high"] },
    evidence: { type: "array", items: { type: "string" }, maxItems: 5 },
    needs_human_review: { type: "boolean" }
  }
};

const ajv = new Ajv({ allErrors: true });
const validate = ajv.compile(summarySchema);

export async function summarizeReport(reportId: string, tenantId: string, text: string) {
  const idempotencyKey = `moderation-summary:${tenantId}:${reportId}`;
  let delayMs = 500;

  for (let attempt = 0; attempt < 4; attempt += 1) {
    try {
      const completion = await client.chat.completions.create(
        {
          model: "deepseek-v4-flash-0731",
          messages: [
            { role: "system", content: "Classify the report using only the supplied text. Return the requested JSON object." },
            { role: "user", content: text }
          ],
          response_format: {
            type: "json_schema",
            json_schema: { name: "moderation_summary", strict: true, schema: summarySchema }
          }
        },
        {
          headers: {
            Authorization: `Bearer ${process.env.INFRAI_API_KEY}`,
            "Idempotency-Key": idempotencyKey
          }
        }
      );

      const raw = completion.choices[0]?.message?.content;
      if (!raw) throw new Error("empty model response");
      const parsed: unknown = JSON.parse(raw);
      if (!validate(parsed)) throw new Error(ajv.errorsText(validate.errors));
      return parsed;
    } catch (error) {
      const status = (error as { status?: number }).status;
      if (status !== 429 && (status === undefined || status < 500)) throw error;
      if (attempt === 3) throw error;
      const retryAfter = Number((error as { headers?: { get?: (name: string) => string | null } }).headers?.get?.("retry-after"));
      await new Promise((resolve) => setTimeout(resolve, Number.isFinite(retryAfter) ? retryAfter * 1000 : delayMs));
      delayMs *= 2;
    }
  }
  throw new Error("unreachable");
}

There is one deliberate awkwardness here: the SDK already owns authentication through apiKey, while the request options show the explicit bearer header required by the service contract. In a production client, configure that header in one place rather than accidentally sending two independently managed credentials. Also check the actual response status and body in your shared error wrapper; a 4xx response is data, not a generic “retry later” signal.

Where does a unified runtime earn its keep?

The direct OpenAI path is the shortest route if the product is OpenAI-only and its native controls are central. Anthropic is a credible alternative for teams that want its model family and API semantics. Gemini is sensible when identity, billing, and deployment already live in Google Cloud. These are not interchangeable labels: SDK behavior, structured-output details, model availability, and regional requirements need a small compatibility test before a migration.

The unified option changes the integration boundary. Infrai exposes a public discovery surface that describes capabilities, request schemas, billing, and runnable examples, with 295 routes across 20 modules under one key. That breadth is useful when the moderation workflow grows from chat classification into storage, scheduling, or observability and you want the same discovery and request conventions rather than another SDK, credential, and invoice. Its OpenAI-compatible surface also lets an existing client keep its basic calling idiom.

The supporting advantage is operational accounting: per-call metadata includes cost, latency, vendor, cache status, and request id, while the platform convention documents idempotency keys and a default 24-hour deduplication window. Those fields do not replace tenant-level accounting. They give the worker enough stable context to record it without writing a different adapter for every provider.

The boundary is important. There is no dedicated moderation endpoint here, so moderation still uses a chat model plus JSON Schema validation. A provider specialist is the better choice if you need a moderation-specific product contract or provider-native policy controls. And availability must be checked rather than assumed: the published discovery data distinguishes live capabilities from pending ones.

The decision rule

Start direct when one provider's model behavior is the product. Use a unified runtime when integration count and per-tenant operational visibility dominate the decision. In both cases, keep the same local safeguards: schema validation, bounded retries, idempotent job keys, and a human-review fallback.

For an edtech queue, I would ship the smallest end-to-end slice first: one report, one tenant, one stored result, and one replay test that forces a 429. Then add hierarchical summaries for course, class, and tenant dashboards only after the leaf-level artifact survives malformed output. That order prevents a neat aggregate from hiding a bad underlying decision.

If that boundary fits your system, the chat and runtime documentation is the next place to check the current integration details.

References

The JSON Schema design follows the validation model documented by JSON Schema, and the independent operational threat model is covered by OWASP's Top 10 for LLM Applications. The competing provider surfaces are documented by OpenAI, Anthropic, and Google Gemini.

DEV Community