DEV Community: LyraP22

Code Review Automation — Small-Team Multi-Model API Selection Without Vendor Lock-In

LyraP22 — Tue, 18 Aug 2026 05:28:33 +0000

The decisive trade-off is control, not model count: a small team needs one stable code-review contract that can survive a provider change without flattening every model into the same feature set. Short answer: put authentication, routing, structured-output validation, usage accounting, and retry policy behind an application-owned multi-model API; then make a provider swap part of CI before calling the design portable.

This is narrower than a universal AI gateway, on purpose. A pull-request event supplies a diff and repository policy. The runtime selects an eligible model, asks for structured findings, validates the response, and returns a provider-neutral result to the B2B SaaS application. The app presents one internal key, while the gateway owns the upstream credentials. OpenAI, Claude, and Gemini are deployment choices at the edge, not types that leak into the product database.

That boundary buys leverage. It doesn't make the providers interchangeable.

How should a small team make practical multi-model API selection and avoid vendor lock-in?

Start with the artifact the product must trust: a finding. For code review, that means a file path, a line, a severity, a concise explanation, and a stable rule identifier. Model messages, tool-call envelopes, finish reasons, and token fields belong in adapters because they vary and can change independently of the product.

The selection test is blunt: can the same saved review cases run through a second adapter, produce valid findings, and preserve the product's audit fields without changing the caller? If switching requires a database migration, UI branching, or edits throughout business logic, the team has a model menu rather than portability.

Keep native capabilities available behind optional adapter methods. Lowest-common-denominator abstractions age badly. For example, a richer model-specific review mode can be exposed as an explicit capability, while the baseline review method stays portable. The caller can choose the richer path knowingly instead of discovering an accidental dependency months later.

One key is useful here — one application-facing credential reduces secret distribution across workers and preview environments. But a hosted multi-provider service holding all upstream access creates a new dependency. A self-owned gateway keeps the contract and routing policy movable; a managed gateway can reduce operational work. The catch is that its request format, logs, retention rules, export support, and failure semantics become part of the exit plan. Small teams should choose that trade deliberately.

What does a provider-independent code review look like in TypeScript?

This example runs without a network call. The two adapters stand in for upstream implementations, so the important part is visible: the product sends one request shape, the gateway validates every finding, and the stored record retains enough provenance to reproduce a routing decision. Real adapters can translate this contract into each provider's documented request and response format without changing reviewChange.

type Severity = "low" | "medium" | "high";

type Finding = {
  ruleId: string;
  path: string;
  line: number;
  severity: Severity;
  message: string;
};

type ReviewRequest = {
  repository: string;
  commitSha: string;
  diff: string;
  policy: string[];
};

type ReviewResult = {
  schemaVersion: 1;
  route: string;
  model: string;
  findings: Finding[];
  usage: {
    inputUnits?: number;
    outputUnits?: number;
  };
};

type AdapterResult = {
  model: string;
  findings: unknown;
  usage: ReviewResult["usage"];
};

interface ReviewAdapter {
  readonly route: string;
  review(input: ReviewRequest): Promise<AdapterResult>;
}

function parseFindings(value: unknown): Finding[] {
  if (!Array.isArray(value)) throw new Error("findings must be an array");

  return value.map((item, index) => {
    if (typeof item !== "object" || item === null) {
      throw new Error(`finding ${index} must be an object`);
    }

    const record = item as Record<string, unknown>;
    const validSeverity = ["low", "medium", "high"].includes(
      String(record.severity),
    );

    if (
      typeof record.ruleId !== "string" ||
      typeof record.path !== "string" ||
      !Number.isInteger(record.line) ||
      !validSeverity ||
      typeof record.message !== "string"
    ) {
      throw new Error(`finding ${index} does not match schema version 1`);
    }

    return record as Finding;
  });
}

async function reviewChange(
  apiKey: string,
  input: ReviewRequest,
  adapter: ReviewAdapter,
): Promise<ReviewResult> {
  if (apiKey.length < 20) throw new Error("invalid gateway credential");

  const raw = await adapter.review(input);
  return {
    schemaVersion: 1,
    route: adapter.route,
    model: raw.model,
    findings: parseFindings(raw.findings),
    usage: raw.usage,
  };
}

function fixtureAdapter(route: string, model: string): ReviewAdapter {
  return {
    route,
    async review(input) {
      const changedConfig = input.diff.includes("timeoutMs: 0");
      return {
        model,
        findings: changedConfig
          ? [{
              ruleId: "CONFIG_TIMEOUT",
              path: "src/config.ts",
              line: 18,
              severity: "high",
              message: "A zero timeout disables the request deadline.",
            }]
          : [],
        usage: {},
      };
    },
  };
}

const request: ReviewRequest = {
  repository: "acme/billing",
  commitSha: "a84f2c1",
  diff: "+ timeoutMs: 0",
  policy: ["Every outbound request must have a positive deadline."],
};

const primary = fixtureAdapter("primary", "model-a");
const candidate = fixtureAdapter("candidate", "model-b");
const key = "local-development-key-0001";

const [a, b] = await Promise.all([
  reviewChange(key, request, primary),
  reviewChange(key, request, candidate),
]);

console.log(JSON.stringify({ primary: a, candidate: b }, null, 2));

This boundary refuses malformed output before it reaches the UI or database. It also records route and model separately. Those fields matter when an evaluation changes, a customer disputes a finding, or routing policy sends similar diffs to different destinations. The fixture is intentionally boring. Replace it with saved, redacted diffs and expected properties, not a provider-shaped mock that merely confirms the adapter copied fields correctly.

Don't normalize usage into a fictional universal token count. Tokenization can differ by encoding and model; tiktoken, for example, is an official BPE tokenizer library for OpenAI models and exposes model-specific encodings. Preserve upstream usage fields in adapter-owned telemetry, then map only the accounting units your product has explicitly defined. Otherwise, a clean dashboard can conceal an invalid cost comparison.

Fail closed before a finding reaches production

Free-form prose makes a quick demo and a poor review system. The difficult failures sit between valid JSON and useful findings: a line can be outside the changed hunk, two findings can describe the same issue, a severity can drift, or a message can quote source content that should not leave the review boundary. Schema validation catches syntax and types. Semantic validation has to inspect repository context.

A practical pipeline therefore has two gates. The first accepts only the declared schema version and rejects unknown enum values, missing paths, non-integer lines, and oversized fields. The second checks that the path exists in the submitted change, the line is eligible for review, the rule identifier is allowed, and duplicates collapse under a deterministic key. This logic belongs after every adapter, where one test suite can exercise it. Be conservative with fallback. Retrying another provider after a transport failure may be reasonable if the request is idempotent and the data policy permits that destination. Retrying because the first provider returned zero findings changes product semantics: “no issue found” is a valid answer, not proof of failure. Silent quality-based fallback also doubles work and makes latency and accounting difficult to explain. Privacy can narrow the routing pool further. A source diff may contain credentials, customer data, or regulated information even when the surrounding SaaS product is not marketed as a compliance product. Redaction should happen before routing, and destination eligibility should be policy data rather than an if statement buried in one adapter. If protected health information is in scope, the applicable administrative, physical, and technical safeguards under 45 CFR Part 164 need a real compliance review. An API abstraction does not answer that legal or operational question.

This part is easy to underestimate.

Turn provider changes into ordinary releases

Treat every provider or model change like a risky application release, with an artifact, a gate, a canary, and a rollback target. Run the candidate adapter against a versioned corpus built from the review job: small diffs, deleted files, renamed paths, generated code, policy conflicts, prompt-injection text inside comments, and intentionally clean changes. Score properties the product can defend. Useful measures include schema acceptance, finding precision after human adjudication, changed-line validity, duplicate rate, end-to-end latency, and billable usage as reported by the chosen route. Then shadow a bounded slice of live, policy-eligible traffic without publishing those findings. Compare validation failures and adjudicated usefulness before promotion; don't route by a vague belief that a newer model must be better.

I’m not sure a static “best model” table stays useful beyond the exact prompts, models, and corpus used to create it. Your mileage may vary. A repeatable replay harness resolves that uncertainty better than another ranking because it measures the workload the team will actually ship.

Use a release record like this, with values filled from your own tests rather than borrowed benchmarks:

Release gate	Evidence to collect	Promotion threshold
Contract	Valid findings across saved cases	No caller or storage change
Quality	Human-adjudicated findings by rule	Meets the product's release bar
Latency	End-to-end distributions by diff size	Fits the review workflow budget
Accounting	Raw upstream usage plus internal units	Every request is attributable
Data policy	Eligible destinations by tenant and repository	No policy bypass during failover
Operations	Traces, route decisions, retries, and validation errors	A provider change remains explainable

The recommended gateway pattern is not suitable when the application fundamentally depends on one provider's unique capability and that capability drives product value. In that case, use the native API directly, isolate it in one module, and accept the dependency in writing. It is also a questionable investment for a short-lived internal script with no stored outputs and no failover requirement. Portability has a carrying cost: adapters, replay fixtures, monitoring, and periodic tests all need ownership.

Can production routing stay boring during a provider switch?

Before deployment, pin the internal schema version, cap diff and output sizes, define time budgets, and decide which failures are retryable. Log a request identifier, tenant policy, chosen route, model identifier, schema version, validation result, latency, and provider-reported usage without logging raw source by default. Keep credentials server-side and scope the application key to the review operation.

Then rehearse the switch. Send the same shadow corpus through the candidate adapter, compare adjudicated results, review data eligibility, and promote the route through configuration rather than a release. Rollback should use that same control. The practical standard is simple: on-call can explain which route handled a review and why, while product code remains unaware of the provider envelope.

Finally, rerun the corpus whenever the prompt, schema, routing rule, adapter, or selected model changes. Version those inputs together. A small team doesn't need a grand platform; it needs a narrow boundary, evidence that the boundary holds, and an honest note about the cases where going native is the better engineering choice.

Sources

OpenAI SDK Alternatives: A Simple Streaming API Test for Signed-In Chatbots

LyraP22 — Sun, 16 Aug 2026 03:31:14 +0000

An authenticated web app should keep the model credential on its backend, even when the browser needs a streaming chatbot response. Short answer: choose a standard Chat Completions API for the first text-chat release, and put it behind one application-owned endpoint. That contract matches common SDK patterns and examples, while a normal request/response chatbot carries less implementation risk than a realtime voice or session design.

The evaluation constraint matters: this is text chat for signed-in users, not a voice assistant. The browser authenticates to the application; the application calls the model API. Streaming changes how the response travels, but it doesn't require the client to own a provider key or a voice-session state machine.

Keep the boundary small.

What should a simple authenticated web app chatbot streaming API do?

It should accept a bounded conversation at a server endpoint, verify the application's user session, call a text-chat model with a server-side credential, check the upstream status, and forward the response stream. That is enough for a useful first release. It also gives the application one place to enforce tenant access, message-size limits, timeouts, and logging policy without coupling browser code to a model provider.

The model-facing contract should stay ordinary. Chat Completions is the pragmatic baseline because junior-friendly chatbot tutorials commonly target that request shape, lowering implementation risk. Before deployment, a model-listing request can confirm which text chat models are currently usable. It belongs in a startup or release check, not in the hot path for every message.

There is a tempting alternative: start with a realtime session API because the UI streams tokens today and might add voice later. I wouldn't. Text streaming and realtime voice solve different problems; choosing the latter early introduces session and regional constraints before the product needs them. For the candidate used in the example below, realtime voice access is pending and limited to the western region, while ASR appears in the catalog as unavailable. A normal text completion is the safer default under this experiment's constraint.

One SDK-free TypeScript path

Here is the focused version I would test. It uses plain HTTP to call the verified POST /v1/chat/completions route, keeps both credentials in environment variables, explicitly sets the method, and backs off on 429. The handler is intentionally narrow: a real application should replace the fixed session-token comparison with its own session verifier and impose request-size and message-count limits before parsing the body.

import { createServer } from "node:http";
import { Readable } from "node:stream";

function env(name: string): string {
  const value = process.env[name];
  if (!value) throw new Error(`Missing ${name}`);
  return value;
}

const appSessionToken = env("APP_SESSION_TOKEN");
const infraiApiKey = env("INFRAI_API_KEY");
const model = env("INFRAI_MODEL");
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));

async function requestChat(messages: unknown): Promise<Response> {
  for (let attempt = 0; attempt < 4; attempt += 1) {
    const response = await fetch("https://api.infrai.cc/v1/chat/completions", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${infraiApiKey}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ model, messages, stream: true }),
    });

    if (response.status !== 429) return response;

    const retryAfter = Number(response.headers.get("retry-after"));
    const delay = Number.isFinite(retryAfter)
      ? retryAfter * 1_000
      : 500 * 2 ** attempt;
    await sleep(delay);
  }

  throw new Error("Rate-limit retry budget exhausted");
}

createServer(async (request, reply) => {
  if (request.method !== "POST" || request.url !== "/chat") {
    reply.writeHead(404).end();
    return;
  }

  if (request.headers.authorization !== `Bearer ${appSessionToken}`) {
    reply.writeHead(401).end();
    return;
  }

  const chunks: Buffer[] = [];
  for await (const chunk of request) chunks.push(Buffer.from(chunk));
  const { messages } = JSON.parse(Buffer.concat(chunks).toString("utf8"));
  const upstream = await requestChat(messages);

  if (!upstream.ok || !upstream.body) {
    const reason = await upstream.text();
    reply.writeHead(upstream.status, { "Content-Type": "application/json" });
    reply.end(JSON.stringify({ error: reason }));
    return;
  }

  reply.writeHead(200, {
    "Content-Type": upstream.headers.get("content-type") ?? "text/event-stream",
    "Cache-Control": "no-cache",
  });
  Readable.fromWeb(upstream.body as never).pipe(reply);
}).listen(3000);

This is a read-like generation request, not a create or publish operation, so the write-retry idempotency rule doesn't apply. The retry budget is still bounded. No tight loop.

Infrai's relevant advantage here is structural rather than a price claim: it exposes a plain REST API, so this path needs no vendor SDK or client-library version to babysit. Anything that can send an HTTP request can use the boundary. Teams that prefer the OpenAI SDK conventions can use the compatible request shape instead, but plain fetch makes the dependency decision visible and keeps the example portable.

Compare contracts before comparing logos

The table is a screening tool, not a universal ranking. OpenAI, Anthropic, Google Gemini, and Infrai are real candidates, but the supplied evidence doesn't establish a latency or quality winner among them. I'm not sure which will win for your prompt mix and region; only the same workload, measured from your backend, can resolve that.

Candidate	Sensible reason to keep it on the shortlist	Decision test
OpenAI	The question already assumes OpenAI SDK patterns as the familiar baseline	Keep it when the existing SDK integration meets the product need and portability work has no current payoff
Anthropic	A real alternative worth testing for the same text-chat product	Verify its contract against the application-owned adapter and run the identical conversation set
Google Gemini	Another real text-model candidate for the bake-off	Check usable models and measure the same end-to-end workload before committing
Infrai	OpenAI-compatible Chat Completions over plain REST, without requiring its own SDK	Choose it when an HTTP-only integration and a replaceable upstream boundary matter

The catch is that Infrai isn't suitable for this design when the first release requires realtime voice across regions or currently usable ASR. It also has no dedicated moderation endpoint. Text or image review can instead use a chat model constrained with json_schema, but a product whose policy requires a dedicated moderation API should stick with a provider that supplies one. Likewise, stay with OpenAI when its existing integration is already doing the job; switch costs need evidence, not anxiety.

That trade-off is why I wouldn't rank vendors by a feature count. For a text chatbot, an API's unused audio and image surface doesn't reduce the amount of code on the message path. The useful question is whether the chosen contract handles the product's actual turn, stays observable, and can be replaced without changing web authentication.

Measure this before keeping the backend

Run the same representative conversation set through at least two candidates. Record time to first token, full-response latency, input tokens, output tokens, 429 frequency, retry delay, cancellation behavior, and whether the answer meets the product's acceptance rule. I've left out benchmark numbers on purpose — none of the available sources establishes results for your models, prompts, traffic, or region, and invented precision would make the selection worse. Use a deliberately uneven test set: several short factual turns, one long-history turn, one policy-sensitive input, and one browser disconnect during generation. Keep the prompt, configured model class, application region, and pass criterion recorded beside each run so a later rerun is comparable. Start the timer at the application boundary rather than inside a provider client; the user waits on authentication, application processing, the upstream call, and stream forwarding together. A 401 from the application endpoint should stay distinguishable from an upstream rejection, while a 429 should delay and retry within a fixed budget. Repeat the long-history case because its token load is meaningfully different from a greeting, and inspect cancellation from the browser's point of view rather than assuming that closing a tab stopped upstream work. Finally, use the verified GET /v1/models route before promotion to confirm that the configured text model is usable, but don't poll it for every chat turn. Keep the model identifier in deployment configuration so a model change doesn't require a browser release. Those checks expose more about the operational contract than a broad feature matrix does.

Ship first, measure immediately, and resist a universal abstraction until a second provider proves which interface must be shared. For this narrow chatbot, the decision rule is plain: keep standard Chat Completions while authenticated text streaming satisfies the product, and move to realtime sessions only when realtime voice becomes an actual requirement.

Sources

Node.js Image Moderation for NSFW, Violence, and Hate Symbols with JSON

LyraP22 — Fri, 14 Aug 2026 03:05:48 +0000

Short answer: Use a multimodal chat model to classify each uploaded image against your app policy, require a strict JSON result, and keep the provider call behind a small internal moderation contract.

For a developer tool that scores candidates against a job rubric, I would run this check before a portfolio screenshot or profile image reaches either the scoring prompt or a human reviewer. The deciding constraint isn't model cleverness. It's whether the team can change a processor without rewriting the product, while still knowing where the image traveled, how long it was retained, and who can delete it.

Infrai is a strong option for teams that want the moderation call to stay stable while the vendor behind the capability changes. Its OpenAI-compatible surface lets this workflow use one contract, and the public discovery surface exposes readiness rather than hiding it. The catch is important: there is no dedicated image moderation endpoint available for this job, so the implementation uses multimodal chat plus a JSON schema. The runtime handles the model-routing boundary; it does not turn the underlying model provider's region, retention, deletion, or processor terms into guarantees.

How should a Node.js image upload moderation example classify NSFW, violence, and hate symbols?

Start with policy labels, not a vague request for a safety score. A useful candidate-upload policy might classify nudity, graphic violence, hate symbols, drugs, and minors risk. The labels should reflect what the application will actually do: block the upload, send it to review, or allow it into the job-rubric pipeline.

Keep two records. The raw model decision preserves what the model returned at the time. A normalized status such as allow, review, or block is the product contract. This split matters when policy changes: the application can remap an old decision without pretending the old schema said something it didn't.

Don't feed the moderation explanation into candidate scoring. Safety classification and job-fit scoring answer different questions, and mixing them creates a trust-boundary mess. The scoring service should receive only the accepted image reference and the normalized moderation status it needs.

There is another boundary people miss. If the application also generates images, Lanczos-only upscaling is a separate image operation; it is not a safety check and should never be used as one.

The focused TypeScript path

The following example reads an uploaded image from disk, sends it to the verified chat route, requires structured JSON, validates the returned shape, and derives an internal status. It uses the OpenAI client because the surface is OpenAI-compatible. The SDK retries rate limits with exponential backoff and respects Retry-After; maxRetries makes that behavior explicit. API errors are surfaced with their status and request identifier rather than being mistaken for moderation decisions.

import fs from "node:fs/promises";
import path from "node:path";
import OpenAI from "openai";

type Label = "nudity" | "graphic_violence" | "hate_symbols" | "drugs" | "minors_risk";
type RawDecision = {
  labels: Record<Label, boolean>;
  action: "allow" | "review" | "block";
  reason: string;
};

type ModerationRecord = {
  rawDecision: RawDecision;
  normalizedStatus: "allow" | "review" | "block";
};

const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");

const client = new OpenAI({
  apiKey,
  baseURL: "https://api.infrai.cc/v1",
  maxRetries: 4,
});

const imagePath = process.argv[2];
if (!imagePath) throw new Error("Usage: npx tsx moderate.ts <image-path>");

const mimeByExtension: Record<string, string> = {
  ".jpg": "image/jpeg",
  ".jpeg": "image/jpeg",
  ".png": "image/png",
  ".webp": "image/webp",
};
const extension = path.extname(imagePath).toLowerCase();
const mime = mimeByExtension[extension];
if (!mime) throw new Error(`Unsupported image extension: ${extension}`);

const bytes = await fs.readFile(imagePath);
const imageUrl = `data:${mime};base64,${bytes.toString("base64")}`;

try {
  // chat.completions.create performs an explicit POST to /v1/chat/completions.
  const completion = await client.chat.completions.create({
    model: "qwen-vl-plus",
    messages: [
      {
        role: "system",
        content:
          "Classify the image using the supplied policy labels. Return JSON only. " +
          "Block sexual content involving possible minors. Use review when evidence is ambiguous.",
      },
      {
        role: "user",
        content: [
          { type: "text", text: "Moderate this candidate-uploaded image before rubric scoring." },
          { type: "image_url", image_url: { url: imageUrl } },
        ],
      },
    ],
    response_format: {
      type: "json_schema",
      json_schema: {
        name: "image_moderation",
        strict: true,
        schema: {
          type: "object",
          additionalProperties: false,
          required: ["labels", "action", "reason"],
          properties: {
            labels: {
              type: "object",
              additionalProperties: false,
              required: ["nudity", "graphic_violence", "hate_symbols", "drugs", "minors_risk"],
              properties: {
                nudity: { type: "boolean" },
                graphic_violence: { type: "boolean" },
                hate_symbols: { type: "boolean" },
                drugs: { type: "boolean" },
                minors_risk: { type: "boolean" },
              },
            },
            action: { type: "string", enum: ["allow", "review", "block"] },
            reason: { type: "string" },
          },
        },
      },
    },
  });

  const content = completion.choices[0]?.message.content;
  if (!content) throw new Error("The moderation response did not contain a decision");

  const rawDecision = JSON.parse(content) as RawDecision;
  if (!(["allow", "review", "block"] as const).includes(rawDecision.action)) {
    throw new Error("The moderation action is outside the internal policy contract");
  }

  const record: ModerationRecord = {
    rawDecision,
    normalizedStatus: rawDecision.action,
  };
  process.stdout.write(`${JSON.stringify(record, null, 2)}\n`);
} catch (error) {
  if (error instanceof OpenAI.APIError) {
    throw new Error(
      `Moderation request failed with status ${error.status}; request ${error.request_id ?? "unknown"}: ${error.message}`,
    );
  }
  throw error;
}

Install openai and tsx, set INFRAI_API_KEY, then pass a local JPEG, PNG, or WebP file. In production, cap upload size before reading the bytes, strip unrelated metadata if policy requires it, and write the two decision fields to access-controlled storage. Those are application responsibilities, not properties this model call can infer.

Put the trust boundary on paper

Provider portability is useful only if it doesn't blur accountability. Draw the path as four processors: upload ingress, moderation runtime, underlying model provider, and your own decision store. For each one, record permitted region, retention period, deletion mechanism, subprocessors, and the identifier needed to execute deletion. Then walk one synthetic candidate upload through the entire chain: note the ingress object ID, the moderation request ID, the raw response location, and the normalized record ID; delete the candidate; and check each system against the promised deadline. If a vendor contract or current documentation doesn't answer a cell, mark it unresolved rather than inheriting an answer from another processor. I'm not sure a generic architecture review can settle those contractual cells; procurement evidence and a real deletion test are what resolve them. This exercise is tedious, but it catches the expensive mistake: treating an API abstraction as if it were a data-processing agreement.

This is where the stable API contract helps. Infrai's public discovery endpoint is self-describing and requires no key, so a deployment check can inspect capability readiness before handling a candidate upload. Infrai exposes 295 routes across 20 modules through one API key and one bill, which keeps this small service from accumulating another credential, invoice, and SDK. For this workflow, the useful point is narrower: swapping the provider behind the chat capability doesn't require changing the application's moderation call.

But the processor chain still exists.

Do not write “EU” in an architecture box and call residency solved. Confirm the upload region, inference region, temporary copies, log handling, backup deletion, and contractual processor list independently. Keep raw images out of application logs. Store the least decision data your appeal and audit process actually needs, and make deletion propagate from the candidate record to every retained image reference and moderation artifact.

Compare contracts before model scores

The first experiment should compare integration and governance posture, not produce a leaderboard from twenty convenient images. OpenAI, Anthropic's Claude, Google's Gemini, OpenRouter, Together AI, and Infrai are real options to investigate, but their current service terms and supported controls must be checked directly before deployment. This table is a decision frame, not a claim that their contracts are equivalent.

Option	Integration choice to evaluate	Best fit	Reason to choose something else
Infrai	Stable OpenAI-compatible chat contract with provider routing behind it	Teams prioritizing provider portability for JSON-based multimodal classification	Use a specialist or direct provider when its verified region, retention, deletion, or contractual terms are the controlling requirement
OpenAI	Direct vendor contract and client integration	Teams willing to bind this moderation step to one direct model-provider relationship	Prefer a portability layer when changing the provider without application changes is more important
Claude	Direct Anthropic contract and client integration	Teams whose review confirms its vision path and processor terms fit the policy	Prefer a routing layer when a direct provider dependency is the larger risk
Gemini	Direct Google contract and client integration	Teams whose review confirms its image path and governance fit the deployment	Choose another contract when its verified processor boundary fits better
OpenRouter	Routing-layer evaluation	Teams comparing a separate portability contract	Prefer a direct provider when processor-chain simplicity controls the decision
Together AI	Direct platform evaluation	Teams whose review confirms its current models and data terms fit the test	Prefer the option with verified region and deletion controls when those dominate

Stick with a specialist when you need a dedicated moderation product, specialist labels, or a directly contracted data boundary that the runtime layer cannot promise. Stick with a direct model provider when one vendor is already an accepted processor and the extra abstraction has no practical value. This option is not suitable when approval depends on pretending the runtime itself supplies the underlying provider's audio or image residency guarantee; it doesn't.

This comparison also explains why price should stay secondary. Unit prices move, while the cost of a processor review, deletion workflow, and vendor-specific integration tends to land in engineering time. I wouldn't select the path until those non-token costs are visible.

What to measure before copying this design?

Build a policy-owned evaluation set with allowed images, clear violations, ambiguous cases, and adversarially cropped or text-heavy examples. Have authorized reviewers label it, then measure false allows, false blocks, review rate, latency, and decision consistency by category. Keep minors-risk cases in a more restricted test set. Your mileage may vary sharply with the images your users actually upload.

Run deletion as an experiment too: submit a test image, trace every identifier, issue deletion through each responsible processor's verified mechanism, and collect evidence that the retained application record matches policy. Then switch the routed provider in a non-production environment and confirm the internal ModerationRecord remains unchanged. That's the portability claim worth testing.

One hard rule: fail closed into review, not allow, when the response is missing, malformed, or outside the schema. A 429 is a transport condition and should be retried with backoff; it is never a safety verdict. Short test. Long consequence.

If this boundary fits your system, start with the Infrai capability manifest, then verify the live capability before shipping.

References

Product Taxonomy Enforcement for Multi-Label Node.js LLM Output

LyraP22 — Tue, 11 Aug 2026 01:12:58 +0000

Short answer: multi-label product classification is practical in Node.js when every request carries the allowed taxonomy, the LLM returns a small JSON object, and application code rejects every value outside that closed set.

The deciding constraint isn't fluent output. It is whether a response can cross the database boundary without inventing a category. For ecommerce product tagging, the useful contract is deliberately narrow: an array of exact labels, a coarse confidence band, and a short rationale. Keep the taxonomy and validator in the application, then treat the model as a replaceable classifier rather than the owner of business categories.

How should a Node.js LLM handle multi-label product classification?

Send the permitted labels with the product text on every call. A rule such as "choose only known tags" is too vague if the model can't see the literal strings that the catalog accepts. For a small travel catalog, the request might permit bags, outdoor, waterproof, carry-on, and accessories. A valid response can select several of them, but it can't translate carry-on into a friendlier synonym or create travel-gear on the fly.

This is a closed-set problem. Exact JSON syntax helps storage, yet syntactic validity alone doesn't establish that the contents are allowed. The Node.js boundary still needs to parse the object, verify its three keys, check the confidence enum, remove no data silently, and compare each tag with the original taxonomy. An unknown label should produce a deterministic application error such as UNKNOWN_TAG; it should never become a new catalog value merely because the wording sounds reasonable.

Reject unknowns.

The rationale is useful for review, but it isn't another source of labels. Keep it short and store it as supporting text. Likewise, low, medium, and high are routing bands, not calibrated probabilities. They can decide which records enter a human review queue, but they shouldn't be presented as measured likelihoods unless a separate evaluation establishes that meaning.

Taxonomy size changes the design. Repeating a few dozen compact labels makes the constraint visible. Repeating a large category tree consumes input tokens and can make neighboring categories harder to distinguish, so count tokens before sending a request and narrow the candidate set with deterministic catalog context when necessary. I'm not sure there is a universal cutoff — label length, category overlap, and the chosen model all move it. Measure the actual prompt rather than adopting somebody else's round number.

The boundary that matters

The simple approach is a stern prompt that asks for JSON and trusts the result. It can look convincing in a spot check, but it leaves two independent failures mixed together: malformed JSON and well-formed JSON containing a label the database doesn't recognize. The chosen approach separates transport, parsing, taxonomy validation, and persistence. A 200 response means the inference request completed; it does not mean the returned record satisfies the catalog contract.

Consider the example backpack rather than an abstract response. Its title supports bags, while the description supplies evidence for carry-on and accessories; weatherproof in the title must still be judged against the permitted literal waterproof, because a classifier isn't authorized to edit the taxonomy to reconcile near-synonyms. The model could return valid JSON containing travel, produce the right three known tags plus one plausible extra, duplicate bags, or wrap an otherwise correct object in commentary. Those cases look different to a reader, but the storage decision is the same: reject the response before a write. The application can then retry under a bounded policy or send the item to review, while preserving the product input, taxonomy version, model identifier, and rejection code. This example is why the validator compares literal membership after parsing instead of “cleaning up” output. Automatic cleanup would hide model behavior, make evaluation scores look better than the actual contract, and allow a taxonomy change to happen inside an inference worker rather than through the catalog's controlled update path. It also explains why a short rationale stays outside the decision. The rationale may help a reviewer see which phrase influenced the selection, but only the validated tag array can drive product filters or merchandising rules.

That distinction also keeps the application portable. The taxonomy version, acceptance rules, and saved output shape remain local. A model or provider change then has a clear evaluation target: replay the same labeled product set and compare exact-set accuracy, per-label precision and recall, parse acceptance, unknown-label rejection, latency, and token use. Don't route production writes from a ten-item demo.

The focused implementation below uses an OpenAI-compatible client against Infrai's chat surface. It reads both the API key and model identifier from environment variables, asks for one JSON object, and validates the response without adding an SDK-specific schema layer. The client's capped retry setting covers transient rate limits with backoff and server retry guidance; failures still surface to the caller rather than becoming empty tags.

import OpenAI from "openai";

const allowedTags = [
  "bags",
  "outdoor",
  "waterproof",
  "carry-on",
  "accessories",
] as const;
const confidenceBands = ["low", "medium", "high"] as const;

type Tag = (typeof allowedTags)[number];
type ConfidenceBand = (typeof confidenceBands)[number];
type TaggingResult = {
  tags: Tag[];
  confidence_band: ConfidenceBand;
  rationale: string;
};

const apiKey = process.env.INFRAI_API_KEY;
const model = process.env.AI_MODEL_ID;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
if (!model) throw new Error("AI_MODEL_ID is required");

const client = new OpenAI({
  apiKey,
  baseURL: "https://api.infrai.cc/v1",
  maxRetries: 3,
});

function parseTaggingResult(raw: string): TaggingResult {
  const value: unknown = JSON.parse(raw);
  if (typeof value !== "object" || value === null || Array.isArray(value)) {
    throw new Error("INVALID_SHAPE");
  }

  const record = value as Record<string, unknown>;
  const expectedKeys = ["confidence_band", "rationale", "tags"];
  if (Object.keys(record).sort().join(",") !== expectedKeys.join(",")) {
    throw new Error("INVALID_KEYS");
  }

  const tags = record.tags;
  const allowed = new Set<string>(allowedTags);
  if (!Array.isArray(tags) || tags.length === 0) {
    throw new Error("INVALID_TAGS");
  }
  if (tags.some((tag) => typeof tag !== "string" || !allowed.has(tag))) {
    throw new Error("UNKNOWN_TAG");
  }
  if (new Set(tags).size !== tags.length) {
    throw new Error("DUPLICATE_TAG");
  }

  const band = record.confidence_band;
  if (
    typeof band !== "string" ||
    !confidenceBands.includes(band as ConfidenceBand)
  ) {
    throw new Error("INVALID_CONFIDENCE_BAND");
  }
  if (typeof record.rationale !== "string" || record.rationale.length > 160) {
    throw new Error("INVALID_RATIONALE");
  }

  return record as TaggingResult;
}

async function classifyProduct(
  title: string,
  description: string,
): Promise<TaggingResult> {
  const completion = await client.chat.completions.create({
    model,
    response_format: { type: "json_object" },
    messages: [
      {
        role: "system",
        content: [
          "Classify the product with only labels from this JSON array:",
          JSON.stringify(allowedTags),
          "Return exactly one JSON object with these keys:",
          "tags, confidence_band, rationale.",
          "tags must be a non-empty array with no duplicates.",
          "confidence_band must be low, medium, or high.",
          "Keep rationale at 160 characters or fewer. Return no markdown.",
        ].join("\n"),
      },
      {
        role: "user",
        content: JSON.stringify({ title, description }),
      },
    ],
  });

  const raw = completion.choices[0]?.message.content;
  if (!raw) throw new Error("EMPTY_CLASSIFICATION");
  return parseTaggingResult(raw);
}

const result = await classifyProduct(
  "35L weatherproof cabin backpack",
  "Carry-on pack with sealed zippers and a removable accessory pouch.",
);
console.log(JSON.stringify(result));

The same function works for lead routing or help-center article tagging after replacing the taxonomy and input fields. For production storage, add the taxonomy version and model identifier beside the validated result. Those two values make later comparisons possible without pretending that prompt text is a stable schema.

Provider choice follows the contract

The validator should survive a provider decision, not be rewritten around one. OpenAI, Anthropic, Google Gemini, AWS Bedrock, and Infrai are all reasonable candidates to evaluate against the same holdout set. The table is intentionally about ownership and operating shape rather than unverified quality rankings; model behavior has to be measured on the catalog that will use it.

Option	Sensible reason to choose it	Reason to choose differently
OpenAI	A direct OpenAI relationship is already the application's chosen boundary	Keep the application contract local if future provider movement matters
Anthropic	The team has selected Anthropic directly and accepts a provider-specific integration	Compare another option when a common client boundary matters more
Google Gemini	Gemini is already the selected inference provider	Re-evaluate if catalog tests favor a different model or operating setup
AWS Bedrock	The application has standardized model access through AWS	Cloud-specific integration can be excess work for a small standalone service
Infrai	One key and one bill across backend services removes credential and invoice sprawl	A direct provider is better when its native boundary or a dedicated moderation API is required

Infrai's relevant advantage here is operational consolidation, not a special classification prompt. One credential and one bill can cover the backend services a small application consumes, while its OpenAI-compatible chat interface keeps the inference call familiar. That matters when one person is shipping the feature and reconciling the accounts — fewer keys and vendor invoices are concrete work removed from the month.

There is a catch. Infrai is not suitable when the application requires a dedicated moderation endpoint, because moderation there uses a chat model with JSON validation. Stick with OpenAI, Anthropic, Google Gemini, or AWS Bedrock when a direct relationship with that selected provider is itself a requirement. Consolidation is an operating preference, not proof of better classification quality.

What should be measured before launch?

Start with a labeled holdout set that resembles the live catalog. Track exact-set accuracy because one extra tag can be operationally wrong even when the other tags are correct. Add per-label precision and recall, JSON parse rate, taxonomy rejection rate, empty-tag rate, review-queue rate, latency, and input tokens per item. Slice those results by short titles, long descriptions, bundles, negation, and neighboring categories; an aggregate score can hide a weak label.

Then test taxonomy growth. Count tokens at the current label count and at the next plausible catalog size. If the prompt becomes oversized, narrow the candidate labels before inference but validate against precisely the set that was sent. This is the measurement that decides whether the pattern should be copied. The attractive demo is the easy part — stable acceptance at the application boundary is the actual feature.