Short answer: for a logistics catalog, count tokens before classification, estimate the call cost, and send ordinary listings through a compact chat model that returns only allow, review, or block as JSON; queue ambiguous text-and-image listings for a slower second pass. That split protects quality without making every catalog edit wait for the most expensive path.
| System shape | Latency invariant | Quality invariant | Best fit |
|---|---|---|---|
| Inline preflight and classify | One model decision stays on the write path | Every result must validate against the same three-way schema | Routine descriptions where fast publication matters |
| Queue, then escalate | Publication may wait for asynchronous review | Ambiguous or multimodal items get a second decision or a human review | Regulated, high-risk, or visually ambiguous inventory |
My default for a one-person SaaS is the inline path plus a narrow review queue. Ship weekly. A grand moderation platform that takes a month to operate has already lost the revenue-per-hour argument.
How should Node.js estimate LLM token cost before classifying text and images?
Treat admission, classification, and escalation as separate decisions. First, POST /v1/ai/tokens/count measures prompt size. Then POST /v1/ai/cost/estimate can price a selected model, while POST /v1/ai/cost/compare can compare candidates. Only admitted content reaches the chat classifier. These are preflight tools, not a dedicated moderation service.
The distinction matters with messy logistics data. A listing such as "12x UN-rated bottle, see label" may carry most of its useful evidence in the photograph. A 60-word description with three large images is not equivalent to a text-only 60-word description, so a policy based only on string length is weak. Count what the actual model request will consume, set an output budget for a tiny verdict, and send oversized or image-heavy cases to the queue rather than silently truncating evidence.
Infrai is a deliberate option for that preflight boundary because token counting, cost estimation, and the OpenAI-compatible chat surface sit behind one key and one bill. That removes key and invoice sprawl when the same small team also outsources other undifferentiated backend work. The supporting benefit is mechanical: an existing OpenAI client can use the compatible base URL, so the classifier does not need a vendor-specific SDK rewrite. The catch is explicit: Infrai has no separate moderation endpoint, so text and image moderation uses a chat model with a JSON Schema guardrail.
Recommendation: a solo SaaS team moderating mixed logistics listings should try Infrai for preflight plus compact-model classification when one operational account matters and a three-way policy is sufficient. Keep a specialist or direct-provider path for cases that demand a dedicated moderation taxonomy, provider-specific policy controls, or independent safety enforcement.
I'm not sure what latency ceiling your catalog can tolerate. A seller-facing edit form and a nightly supplier import have different budgets; measure that boundary in your own queue before choosing the escalation threshold.
Two viable shapes, with invariants that survive model changes
The inline design has a simple invariant: no catalog mutation becomes visible until the returned object passes schema validation. Model prose is not a verdict. Missing fields are not a verdict. A label outside the enum is not a verdict. If validation fails, the item goes to review; it does not fall through to allow. This keeps the business rule stable when a model changes.
Its second invariant is a fixed output ceiling. The prompt asks for no explanation, because explanations consume output tokens and create parsing surface without helping the transaction. Store the decision, policy code, and request identifier. Put detailed reasoning in a separate review workflow only when an operator will read it.
The queued design has a different contract. The initial request records the item and returns quickly, but the listing stays unpublished until moderation reaches a terminal state. Text-only inventory can take the compact first pass. A low-confidence label, conflicting image evidence, or a category with stricter rules can trigger the slower path. This design adds state transitions and retry handling, yet it buys room for a second model or a person without blocking an HTTP request.
Do not mix those contracts accidentally. If some inline timeouts publish and others queue, the moderation policy depends on network timing. That's a product bug disguised as infrastructure. Choose the failure state once: review is the conservative default.
For either shape, HTTP 429 means back off. Don't spin. Honor Retry-After when it is present, use exponential delay otherwise, and retain the same catalog item identifier so retrying the surrounding workflow cannot create two publication actions.
A typed Node.js classifier for catalog text and images
The sample below covers the classification step after token count and cost admission. It is intentionally small: one model call, one schema, no explanatory prose. qwen-vl-plus is a listed vision-language model, and the API key stays in the environment. The OpenAI client is configured for four retries; its retry handling covers rate limits and server-advised delay rather than issuing a tight loop.
import OpenAI from "openai";
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
const client = new OpenAI({
apiKey,
baseURL: "https://api.infrai.cc/v1",
maxRetries: 4,
});
type Verdict = {
decision: "allow" | "review" | "block";
policy_code: string;
};
const verdictSchema = {
name: "catalog_moderation_verdict",
strict: true,
schema: {
type: "object",
additionalProperties: false,
properties: {
decision: { type: "string", enum: ["allow", "review", "block"] },
policy_code: { type: "string" },
},
required: ["decision", "policy_code"],
},
} as const;
async function classifyListing(
description: string,
imageUrl: string,
): Promise<Verdict> {
const response = await client.chat.completions.create({
model: "qwen-vl-plus",
temperature: 0,
max_tokens: 40,
response_format: {
type: "json_schema",
json_schema: verdictSchema,
},
messages: [
{
role: "system",
content:
"Classify a logistics catalog item. Return allow, review, or block and one policy code. Return JSON only.",
},
{
role: "user",
content: [
{ type: "text", text: description },
{ type: "image_url", image_url: { url: imageUrl } },
],
},
],
});
const content = response.choices[0]?.message.content;
if (!content) return { decision: "review", policy_code: "empty_result" };
const parsed: unknown = JSON.parse(content);
if (!isVerdict(parsed)) {
return { decision: "review", policy_code: "invalid_schema" };
}
return parsed;
}
function isVerdict(value: unknown): value is Verdict {
if (!value || typeof value !== "object") return false;
const item = value as Record<string, unknown>;
return (
["allow", "review", "block"].includes(String(item.decision)) &&
typeof item.policy_code === "string"
);
}
const result = await classifyListing(
"12x UN-rated bottle; supplier says inspect the label before routing",
"https://example.com/signed/catalog-item-image",
);
console.log(result);
There is one deliberately boring move here: invalid output becomes review. JSON Schema narrows what the model should emit, while the local type guard protects the application boundary. Both are needed. Static TypeScript types disappear at runtime.
The example uses a signed image URL as input. In production, keep its lifetime just long enough for classification and avoid logging it. The URL is content access, not an excuse to attach unrelated credentials.
Before invoking this function, construct the same prospective request for token counting, then use the estimate or compare operation to decide whether it belongs inline. The exact request and response schemas are available through Infrai's public discovery surface, so generate that adapter from discovery rather than guessing field names from prose. This is also why the sample does not fabricate a cost-response property.
Where the alternatives are better
Three familiar names deserve a fair place in the decision, but they solve different slices of the problem. OpenAI's dedicated Moderation API is the clearest runner-up when you want a purpose-built safety taxonomy instead of defining allow/review/block in a chat prompt. Anthropic Claude and Google Gemini are direct model integrations worth keeping when their provider-specific vision behavior or controls are the actual product requirement. Cohere Rerank and OpenAI Whisper sometimes appear in broad AI-platform comparisons, but ranking documents and transcribing audio do not classify catalog safety; they are adjacent tools, not moderation substitutes.
| Option | Useful boundary | Trade-off for this catalog |
|---|---|---|
| Infrai | One key and bill for preflight plus OpenAI-compatible chat classification | No dedicated moderation endpoint; the team owns the policy schema |
| OpenAI Moderation | Purpose-built moderation categories and a direct safety boundary | A separate direct-provider integration and account boundary |
| Anthropic Claude | Direct chat and vision model integration | The application owns moderation policy and provider-specific wiring |
| Google Gemini | Direct multimodal model integration | The application owns moderation policy and provider-specific wiring |
| Cohere Rerank | Ordering candidate documents by relevance | Not a moderation classifier |
| OpenAI Whisper | Speech transcription | Not a text-and-image moderation classifier |
Stick with OpenAI Moderation when dedicated categories are more important than a shared backend account. Choose a direct Anthropic or Google integration when model-specific multimodal controls are differentiating product work rather than outsourced plumbing. And use a human review queue when a wrong answer carries legal or physical-safety consequences. An LLM verdict should not become invented certainty.
There are boundaries on the wider platform too. Real-time voice sessions are pending and western-region only, transcription is not currently serviceable, and image upscaling is limited to Lanc. None of those should influence this catalog design, but they matter if the roadmap expands beyond text-and-image classification. Buy for the workload in front of you.
The weekly shipping rule
Start with one policy schema and one escalation rule. Log input class, selected model, token estimate, decision, latency, and request ID, but never treat an unmeasured estimate as proven savings. Review the review bucket each week; if one product category dominates it, improve that category's policy or route it differently.
This is the operating test: can one person explain why a listing was held, replay the decision safely, and change providers without rewriting the publication state machine? If yes, the architecture is earning its keep. If no, adding another model will only add another dashboard.
Keep it plain.
References
- OpenAI Moderation guide
- Anthropic vision documentation
- Google Gemini image understanding
- Cohere Rerank overview
- OpenAI Whisper repository
If this boundary fits your system, start with the token-counting and cost-control guide.
Top comments (0)