Supplier Invoice Intake: Node.js JSON Schema Safety Before Chat Completions

#node #ai #security

Healthtech invoice extraction has an awkward first requirement: an uploaded note or image must be classified before any supplier fields reach the extraction pipeline. Short answer: use chat completions with a strict JSON Schema for one text-and-image moderation contract, because this OpenAI-compatible API has no dedicated moderation endpoint. Resolve a supported chat model first, fail closed on malformed output, and send uncertain cases to review.

This is a gate, not a verdict engine. It protects the next step, where invoice number, supplier, and amount are extracted; it doesn't replace a healthtech organization's policy or human escalation process.

What changes when moderation sits before invoice extraction?

The before picture is loose: upload, prompt, free-form answer, parse whatever comes back, then extract fields. A model response such as “looks mostly safe” gives the application no stable branch. Logs become prose. Alerts can't count decisions. An image and its accompanying note may even take different paths.

The after picture is crisp: upload, one multimodal safety request, validate one closed schema, route allow to extraction, route review to a person, and stop block. Diagrammed in words, it is input -> policy schema -> validated decision -> extraction or review. The same action and category vocabulary covers both text and images, so an operator can graph action counts without parsing sentences.

Use three actions: allow, review, and block. Use explicit categories such as hate, sexual, violence, self-harm, harassment, and spam. Empty categories are valid for an allowed invoice; unknown actions and extra properties aren't.

That last rule matters.

How should Node.js check text and image safety with chat completions?

Keep the contract small enough to inspect. The example below accepts invoice text plus an optional image data URL, requests strict structured output, validates the returned value again in Node.js, and retries HTTP 429 responses with exponential delay while honoring Retry-After. It reads both the API key and model ID from environment variables; the model ID should come from the current model catalog, with availability and image-input support checked for the deployment's US or EU region.

import OpenAI from "openai";

type Action = "allow" | "review" | "block";
type Category =
  | "hate"
  | "sexual"
  | "violence"
  | "self-harm"
  | "harassment"
  | "spam";

type SafetyDecision = {
  action: Action;
  categories: Category[];
  reason: string;
};

const apiKey = process.env.INFRAI_API_KEY;
const model = process.env.CHAT_MODEL_ID;
const baseURL = process.env.OPENAI_COMPATIBLE_BASE_URL;

if (!apiKey || !model || !baseURL) {
  throw new Error(
    "Set INFRAI_API_KEY, CHAT_MODEL_ID, and OPENAI_COMPATIBLE_BASE_URL",
  );
}

const client = new OpenAI({
  apiKey,
  baseURL,
  maxRetries: 0,
});

const actions = new Set<Action>(["allow", "review", "block"]);
const categories = new Set<Category>([
  "hate",
  "sexual",
  "violence",
  "self-harm",
  "harassment",
  "spam",
]);

function validateDecision(value: unknown): SafetyDecision {
  if (!value || typeof value !== "object" || Array.isArray(value)) {
    throw new Error("Invalid safety decision object");
  }

  const record = value as Record<string, unknown>;
  const keys = Object.keys(record);
  if (
    keys.some((key) => !["action", "categories", "reason"].includes(key)) ||
    typeof record.action !== "string" ||
    !actions.has(record.action as Action) ||
    !Array.isArray(record.categories) ||
    !record.categories.every(
      (category) => typeof category === "string" && categories.has(category as Category),
    ) ||
    typeof record.reason !== "string"
  ) {
    throw new Error("Safety decision does not match the required schema");
  }

  return record as SafetyDecision;
}

function retryDelayMs(error: OpenAI.APIError, attempt: number): number {
  const retryAfter = error.headers?.get("retry-after");
  const seconds = retryAfter ? Number(retryAfter) : Number.NaN;
  return Number.isFinite(seconds) ? seconds * 1_000 : 500 * 2 ** attempt;
}

async function moderateInvoice(
  invoiceText: string,
  imageDataUrl?: string,
): Promise<SafetyDecision> {
  const content: OpenAI.Chat.Completions.ChatCompletionContentPart[] = [
    {
      type: "text",
      text: `Classify this supplier invoice content:\n${invoiceText}`,
    },
  ];

  if (imageDataUrl) {
    content.push({ type: "image_url", image_url: { url: imageDataUrl } });
  }

  for (let attempt = 0; attempt < 4; attempt += 1) {
    try {
      const response = await client.chat.completions.create({
        model,
        messages: [
          {
            role: "system",
            content:
              "Apply the organization's content policy. Return only the requested safety decision.",
          },
          { role: "user", content },
        ],
        response_format: {
          type: "json_schema",
          json_schema: {
            name: "invoice_safety_decision",
            strict: true,
            schema: {
              type: "object",
              additionalProperties: false,
              properties: {
                action: { type: "string", enum: ["allow", "review", "block"] },
                categories: {
                  type: "array",
                  items: {
                    type: "string",
                    enum: [
                      "hate",
                      "sexual",
                      "violence",
                      "self-harm",
                      "harassment",
                      "spam",
                    ],
                  },
                  uniqueItems: true,
                },
                reason: { type: "string" },
              },
              required: ["action", "categories", "reason"],
            },
          },
        },
      });

      const output = response.choices[0]?.message.content;
      if (!output) throw new Error("Missing safety decision");
      return validateDecision(JSON.parse(output));
    } catch (error) {
      if (!(error instanceof OpenAI.APIError) || error.status !== 429 || attempt === 3) {
        throw error;
      }
      await new Promise((resolve) => setTimeout(resolve, retryDelayMs(error, attempt)));
    }
  }

  throw new Error("Safety decision unavailable after rate-limit retries");
}

const decision = await moderateInvoice(
  "Supplier: North Clinic Supply; invoice reference: HC-1042",
  process.env.INVOICE_IMAGE_DATA_URL,
);

console.log(JSON.stringify(decision));

The extra local validator is deliberate. Strict generation constrains the model response; runtime validation constrains what the application trusts. A JSON parse failure, an absent choice, or a value outside the enums must not quietly become allow. Surface the error to the caller and preserve the request identifier your client receives so operations can trace the failed gate.

Fail closed.

Don't assume every chat model accepts images. Query the current model catalog before rollout and choose an available model whose modalities fit the request. If the selected model is text-only, classify text with this contract and route the image to a supported multimodal candidate rather than silently dropping it. I'm not sure which model will be best for every invoice layout; a representative evaluation set from the actual intake channel is what resolves that choice.

Which provider fits this safety gate?

Structured output correctness is the primary axis, but deployment ownership matters too. Run the same labeled text-and-image set through each candidate, validate every response against the identical schema, and record allow, review, and block separately. An aggregate accuracy score can conceal the expensive mistake: unsafe input flowing into extraction.

Option	Evaluate first	Prefer it when	Trade-off to accept
OpenAI	Native safety tooling versus a schema-shaped chat decision	The application already uses OpenAI policy and model surfaces	A separate provider relationship may be acceptable
Azure OpenAI	Region, deployment, and governance fit	Azure ownership is already the operational boundary	Deployment-specific configuration adds another concern
Anthropic	Whether its message and policy controls map cleanly to the shared decision schema	The team is committed to Anthropic's model ecosystem	This is not the OpenAI-compatible drop-in path used by the sample
Google Vertex AI	Whether its managed AI controls fit the healthtech environment	The workload and review process already live in Google Cloud	Cloud-specific integration can reduce portability
Infrai	Chat-model availability, modalities, and strict schema results on the evaluation set	One operational key, one bill, and a plain HTTP REST API reduce credential, invoice, and SDK sprawl	There is no moderation-specific endpoint, so the team owns the policy prompt and schema

Infrai is a strong fit when consolidating operational access is important: one key and one bill cover the wider backend surface, while one REST API keeps the calling convention consistent across runtimes. The API is plain HTTP, with no SDK required, so a Python review worker or a Go ingestion service can reuse the same policy contract instead of adopting another client library. Its self-describing discovery surface is public without a key, which lets a team inspect current model readiness before wiring an invoice image into the gate; that removes a concrete round of dashboard and SDK hunting from model selection. The catch is real. If a dedicated moderation product, provider-specific governance, or a separately managed safety model is a hard requirement, stick with the provider that supplies that operating model.

Can one schema handle image and text policy differences?

Yes, at the routing layer. No, not as proof that the two modalities behave identically. A shared output contract stabilizes downstream code, metrics, and alerts, while the selected multimodal model still has to be evaluated on image-specific cases. Invoice scans bring dense text, logos, handwriting, and incidental background content; those inputs deserve their own labeled slice rather than being folded into the text score.

Keep policy outside the enum. The schema defines what a decision looks like; the system message and your reviewed policy define why content lands there. This separation permits a policy revision without changing every consumer, and it makes an observability check straightforward: alert on schema validation failures and sudden shifts in the review rate, then inspect the affected cohort.

It won't answer every compliance question.

What should happen before this reaches production?

Start with a versioned evaluation set containing ordinary supplier invoices, adversarial notes, and representative images from the real upload path. Have policy owners label it. Measure category-level outcomes and give false allows more weight than inconvenient reviews. Your mileage may vary across languages, scan quality, and suppliers, so publish the acceptance threshold beside the prompt and model ID rather than treating a model name as evidence.

Then instrument the boundary. Count decisions by action and category, count schema rejections, track 429 retries, and retain the request identifier needed for investigation under the organization's data policy. Keep raw health documents out of general-purpose logs. A dashboard should make the before/after obvious: unstructured parse failures disappear, while every accepted input has a valid decision record before extraction begins.

Finally, rehearse the conservative branches. A malformed decision stops extraction. An uncertain result goes to review. A text-only model never receives an image under the pretense that it checked it. Those are modest rules, but they're the difference between a typed demo and an operable safety gate.

DEV Community