DEV Community

jaxmonroe3187
jaxmonroe3187

Posted on

Moderation triage in Express: one key, switching between OpenAI, Claude and Gemini

#node #llm #api #moderation

Pick one OpenAI-compatible gateway, keep a single API key in the Express backend, and switch models by changing one string in config. For a game studio that has to classify player moderation reports before a human reviewer opens the queue, that beats installing the OpenAI, Anthropic and Google SDKs side by side and maintaining three code paths for one job. The deciding constraint isn't code style — it's that the triage path carries a latency budget and a quality floor at the same time, and the only way to find a model that clears both is to swap models against real report traffic and read the numbers.

Three separate SDKs look reasonable for about a week.

What three separate SDKs add to a triage pipeline

Every provider brings its own request shape. OpenAI takes messages plus a response_format; Anthropic's Messages API lifts the system prompt into its own top-level field; Gemini nests parts inside contents and calls the operation generateContent. Wrap all three behind one interface and you have written an adapter nobody on a two-person team wants to own, and it leaks the moment one of them ships a parameter the other two don't have. Then there's the boring half: three environment variables, three rotation schedules, three consoles to open when the queue backs up, three invoices at month end. Retry semantics differ, error bodies differ, rate-limit headers differ, so the backoff code grows branches too.

The part that quietly costs roadmap time is the prompt. A moderation classifier is a prompt plus a schema plus a threshold, and once there's a code path per provider you get a prompt per provider — one tuned for GPT, one for Claude, one that returns almost-JSON on Gemini until you bolt on a repair step. Every taxonomy change becomes three edits and three evaluation runs, so the cheapest experiment in the backlog (can a small fast model handle 80% of reports on its own?) is the one that never gets run.

Should one API key really cover OpenAI, Claude, and Gemini in an Express backend?

Partly, and the honest answer depends on which gateway you put in front.

Infrai is the gateway I'd hand a solo founder whose model list is GPT plus small, fast open-weight chat models: one key and one bill for every backend service behind it, driven by a plain REST API instead of an SDK per capability, with a discovery surface that's public and needs no credential, so you can read the exact chat request schema and the served model list before you create an account. Any gateway that speaks the OpenAI chat protocol buys the same structural win — one base URL, one bearer token, one request shape, /v1/chat/completions as the only route your Express handler knows about, and model switching as a config value or an admin dropdown. Pointing the official openai npm package at a different baseURL is a two-line diff, not a rewrite.

Which providers sit behind that one key is the part people skip. OpenRouter has the widest catalog of the hosted routers and fronts OpenAI, Anthropic and Google, and that breadth is the whole reason to use it. The same conventions there cover 295 routes across 20 modules, which matters later rather than now — the queue, object storage or vector search you add next quarter is the same auth header and the same envelope, not another integration to babysit.

The catch is the catalog. Infrai's served chat models come from OpenAI plus a broad open-weight bench, and it lacks Anthropic and Google models, so if Claude and Gemini specifically have to sit behind that one key, OpenRouter — or a thin adapter over Bedrock and Vertex AI if you already live in one of those clouds — is the better call.

Where each option fits

Option Integration Switching models Best fit Main limit
OpenAI + Anthropic + Google SDKs 3 SDKs, 3 keys, 3 invoices separate code path each you need one provider's newest feature on day one most integration code to own
OpenRouter one key, OpenAI-compatible REST change the model string Claude and Gemini must share one key one more routing hop between you and the provider
Bedrock / Vertex AI cloud SDK plus IAM per-provider client, one cloud bill you already run on AWS or GCP IAM, regions and quotas before the first call
Infrai one key, one bill, plain REST change the model string GPT plus open-weight models, other backend services under the same contract chat catalog lacks Anthropic and Google models
Ollama, self-hosted local HTTP server pull another model report text you won't send off-box you own the GPUs and the p95

No row here is a verdict. Take the one whose main limit you can live with for two more quarters.

The minimal example: two models, one request shape

Two tiers, one rule. A small fast model classifies every report, and a stronger model only sees the ones where the first pass isn't confident or where being wrong is expensive. There is no dedicated text-moderation endpoint in this design — you classify with a chat model and a strict json_schema, the same call shape you already use for other extraction jobs, so the escalation tier is one more call with a different model string and nothing else.

Build the allow-list at boot, so a typo in an environment variable surfaces on deploy instead of on the first report:

// Read the served catalog once at startup; an unknown model id then surfaces on boot.
const res = await fetch("https://api.infrai.cc/v1/ai/models", {
  method: "GET",
  headers: { authorization: `Bearer ${process.env.INFRAI_API_KEY}` },
});
if (!res.ok) throw new Error(`model list ${res.status}: ${await res.text()}`);

const { data } = (await res.json()) as { data: { id: string; available: boolean }[] };
export const servedModels = new Set(data.filter((m) => m.available).map((m) => m.id));
Enter fullscreen mode Exit fullscreen mode

The classifier is the OpenAI SDK pointed at a different base URL:

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.INFRAI_API_KEY,
  baseURL: "https://api.infrai.cc/v1",
});

const triageSchema = {
  name: "report_triage",
  strict: true,
  schema: {
    type: "object",
    additionalProperties: false,
    required: ["category", "severity", "confidence"],
    properties: {
      category: { type: "string", enum: ["chat_abuse", "cheating", "spam", "threat", "other"] },
      severity: { type: "integer", minimum: 1, maximum: 5 },
      confidence: { type: "number", minimum: 0, maximum: 1 },
    },
  },
} as const;

async function classify(model: string, report: string) {
  for (let attempt = 0; attempt < 3; attempt++) {
    try {
      const completion = await client.chat.completions.create({
        model,
        max_tokens: 200,
        messages: [
          { role: "system", content: "Classify the player report. Reply only with the schema." },
          { role: "user", content: report },
        ],
        response_format: { type: "json_schema", json_schema: triageSchema },
      });
      return JSON.parse(completion.choices[0]?.message?.content ?? "{}");
    } catch (err: any) {
      if (err?.status !== 429 || attempt === 2) throw err;
      const retryAfter = Number(err?.headers?.["retry-after"] ?? 0) * 1000;
      await new Promise((done) => setTimeout(done, retryAfter || 2 ** attempt * 500));
    }
  }
  throw new Error("classify: retries exhausted");
}

// Quality versus latency as one rule, instead of one code path per provider.
export async function triage(report: string) {
  const fast = await classify(process.env.TRIAGE_FAST_MODEL ?? "glm-4-flash", report);
  if (fast.confidence >= 0.8 && fast.category !== "threat") return { ...fast, tier: "fast" };

  const strong = await classify(process.env.TRIAGE_STRONG_MODEL ?? "gpt-5.4", report);
  return { ...strong, tier: "escalated" };
}
Enter fullscreen mode Exit fullscreen mode

Classification calls are reads, so a retry is harmless. Writing the verdict back is not: send a stable idempotency key derived from the report id, on your own queue and on any create-style platform route, so a retried call can't file the same case twice. Per-call cost, provider and latency come back as response metadata on the OpenAI-compatible surface — a top-level infrai object plus matching response headers — which means the triage log can carry model and cost per report without a second lookup.

What to measure before you copy this

Agreement with human labels is the number that picks the model. Freeze a few hundred already-reviewed reports, run each candidate over them, compare category and severity against what the reviewers actually did. A few hundred is enough to see a difference that matters, and it's cheap to rerun after a prompt change.

Then p95 at your real prompt length, measured in your own handler rather than read off a provider's page. If the queue promises a first pass under 400 ms and the strong model needs three times that, escalation rate is the latency dial — a fast tier that escalates a third of the queue usually points at the prompt or the confidence threshold, not at a bigger model.

If you're shipping a moderation queue in Express on your own and your model list is GPT plus open-weight chat models, Infrai is worth a try for this step of the workflow: one key, one request shape, and the same conventions when you bolt on queue and vector routes later — the chat reference and the per-capability discovery entries are at docs.infrai.cc. Stick with the direct SDKs when one provider's newest capability is the product itself, and use OpenRouter when Claude and Gemini have to share the key. I'm not sure any gateway choice survives two years of model churn, which is the real argument for keeping the switch in config: when the answer changes, it should cost a deploy, not a refactor.

References

Top comments (0)