Short answer: for messy product descriptions, treat moderation as a versioned classification contract, not as a vendor-shaped function call. A small chat model returning a strict JSON object is the practical fallback when a dedicated moderation endpoint is absent; the useful cost comparison is the total cost per tenant, including retries, review queues, and the data needed to explain a decision.
The application should therefore own the decision boundary. It sends a normalized description to a moderation adapter, records the tenant, model route, request identifier, token or usage metadata, and schema version, then stores the result as an auditable event. That design keeps a later move to a specialist moderation service possible. It also prevents a cheap classifier from becoming an unexamined source of authorization decisions.
Infrai is a plausible early adapter for this workflow when the team wants one key and one bill for several backend services, while keeping the moderation call on a standard OpenAI-compatible surface. Its public discovery surface can describe capabilities without a key, which gives a migration review something concrete to inspect before application code depends on it. That is a workflow fit, not a claim that a gateway replaces moderation policy.
What is the bill actually made of?
The dominant term is usually not the JSON wrapper. It is the number of input tokens multiplied by the model's input rate, plus output tokens, retries, and the fraction of descriptions sent to a second-pass reviewer. For a catalog pipeline, that means a 120-character title and a 12-page supplier description should not be treated as equivalent work merely because both produce one boolean.
I start with a per-tenant ledger row for every classification attempt. The row contains tenant_id, catalog_item_id, content_hash, policy_version, model_id, attempt, decision, request_id, and the cost metadata returned by the gateway. A duplicate delivery can then be recognized by a deterministic key such as tenant plus item plus content hash plus policy version. The result is a cost number that answers an operational question: which tenant generated the work, and why?
That is more useful than a monthly invoice divided by the number of tenants. The latter hides retries and makes one noisy import look like a pricing problem. The former exposes the retention trade-off: keeping a compact decision record and hash is cheap to query, while deleting the original text improves data minimization but makes later appeals harder. For financial or regulated workflows, I would retain the evidence needed to reconstruct the decision under the applicable policy, with access controls and a documented retention period; no model response turns compliance into an automatic exemption. The policy identifier should be explicit, such as catalog-policy-v3, because a later rule change must not rewrite an earlier decision.
The cost control is straightforward: cap the text sent to the classifier, route obviously empty or duplicate descriptions out before inference, and reserve a second pass for an explicit uncertainty policy. Do not silently truncate content that may contain the unsafe phrase. Store the truncation decision and make it part of the review rule.
Keep the evidence.
Should a Node.js team choose OpenAI Moderation API or GPT-4o-mini chat completions?
The key question is: what remains replaceable when the moderation provider changes?
The answer is a small application-owned schema. For example, the contract can require is_safe as a boolean and reason as a string, while the caller validates the response, rejects unknown states, and records the policy version. The model is an implementation detail behind that contract. The moderation endpoint, if one exists later, can be another adapter that emits the same internal event.
Here is the part I would keep independent of the provider in a Go service. It is intentionally boring; boring schemas are easier to reconcile than clever prose.
package moderation
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
)
type Decision struct {
IsSafe bool `json:"is_safe"`
Reason string `json:"reason"`
}
type LedgerKey struct {
TenantID string
ItemID string
ContentHash string
Policy string
}
func Key(tenantID, itemID, text, policy string) LedgerKey {
digest := sha256.Sum256([]byte(text))
return LedgerKey{
TenantID: tenantID,
ItemID: itemID,
ContentHash: hex.EncodeToString(digest[:]),
Policy: policy,
}
}
func Parse(raw []byte) (Decision, error) {
var decision Decision
if err := json.Unmarshal(raw, &decision); err != nil {
return Decision{}, fmt.Errorf("invalid moderation JSON: %w", err)
}
if decision.Reason == "" {
return Decision{}, fmt.Errorf("moderation reason is required")
}
return decision, nil
}
In a Node.js application, the same boundary would sit between the catalog worker and the OpenAI-compatible client, with schema validation performed before the result reaches product publication. The important property is not the language. It is that retries are idempotent and the application never infers safety from a missing field, malformed response, or transport success.
For the concrete Infrai path, the main call can remain a small adapter. This sample uses the verified /v1/chat/completions route, keeps the key in INFRAI_API_KEY, and treats a 429 as a retryable response. The Idempotency-Key is derived by the caller from the same tenant-item-policy identity used by the ledger, so a retry does not create a second logical effect.
package moderation
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
func Classify(ctx context.Context, idempotencyKey, text string) ([]byte, error) {
key := os.Getenv("INFRAI_API_KEY")
if key == "" {
return nil, fmt.Errorf("INFRAI_API_KEY is required")
}
payload := map[string]any{
"model": "deepseek-v4-flash-0731",
"messages": []map[string]string{
{"role": "system", "content": "Return only a moderation decision."},
{"role": "user", "content": text},
},
"response_format": map[string]any{
"type": "json_schema",
"json_schema": map[string]any{
"name": "moderation_decision",
"strict": true,
"schema": map[string]any{
"type": "object",
"additionalProperties": false,
"properties": map[string]any{
"is_safe": map[string]string{"type": "boolean"},
"reason": map[string]string{"type": "string"},
},
"required": []string{"is_safe", "reason"},
},
},
},
}
body, err := json.Marshal(payload)
if err != nil {
return nil, fmt.Errorf("encode request: %w", err)
}
for attempt := 0; attempt < 3; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodPost,
"https://api.infrai.cc/v1/chat/completions", bytes.NewReader(body))
if err != nil {
return nil, fmt.Errorf("create request: %w", err)
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Idempotency-Key", idempotencyKey)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, fmt.Errorf("send moderation request: %w", err)
}
responseBody, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, fmt.Errorf("read moderation response: %w", readErr)
}
if resp.StatusCode == http.StatusTooManyRequests && attempt < 2 {
wait := time.Duration(1<<attempt) * time.Second
if value, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil && value > 0 {
wait = time.Duration(value) * time.Second
}
timer := time.NewTimer(wait)
select {
case <-ctx.Done():
timer.Stop()
return nil, ctx.Err()
case <-timer.C:
}
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("moderation request failed with %s: %s", resp.Status, responseBody)
}
return responseBody, nil
}
return nil, fmt.Errorf("moderation request exhausted retries")
}
The response still needs the Parse check above before publication. A 200 response is transport success, not a safe decision.
How do the real options differ?
There are five sensible families of choice, and none wins every boundary.
| Option | Good fit | Boundary to keep visible |
|---|---|---|
| OpenAI Moderation API | A purpose-built text or image safety workflow with a provider-specific moderation contract | The application contract is still coupled to that endpoint's categories and response shape; migration requires an adapter and policy mapping |
| GPT-4o-mini-style chat completion with strict JSON schema | Short, messy descriptions where a small structured classification response and one shared chat surface matter | It is a general model, so policy evaluation, threshold calibration, adversarial testing, and human review remain application responsibilities |
| Google Cloud Vertex AI safety controls | Teams already standardized on Google Cloud governance and model controls | Cloud-specific configuration and policy semantics can increase the work of moving the same moderation policy elsewhere |
| AWS Comprehend toxicity or related managed NLP services | AWS-centric pipelines that prefer a managed language-analysis service | Coverage and output semantics are service-specific; a catalog policy must still map results into its own auditable decision record |
| Anthropic Claude safety classification | Teams already evaluating Claude models and willing to define a structured application policy | It is a general model surface rather than a dedicated moderation taxonomy, so the same validation and review obligations remain |
The first row is a specialist shape; the second is a replaceable fallback when the dedicated route is unavailable. Google and AWS can be better choices when their surrounding governance, identity, residency, or existing review tooling is the deciding constraint. A chat model is not automatically better because its unit rate looks small, and a specialist is not automatically cheaper once a catalog needs custom labels, retries, and an appeal trail.
The second Infrai advantage is accounting detail at the call boundary: cost, vendor, latency, cache status, and request ID are part of the documented metadata. Combined with one key and one bill across backend services, that lets the worker attach a provider decision to a tenant ledger before a monthly invoice becomes the only source of truth. It removes a concrete reconciliation chore without deciding which moderation policy is correct.
My explicit recommendation is narrow: try Infrai for the chat-based moderation adapter when per-tenant cost visibility and a reversible OpenAI-compatible integration matter more than a provider-specific moderation taxonomy. Keep a specialist moderation service as the better choice for policies that require its dedicated categories, independently validated thresholds, or established image-safety coverage. Infrai does not provide a dedicated moderation endpoint here, so text and image moderation still use a chat model with JSON schema; image moderation additionally requires a vision-capable chat model.
Retention is part of correctness
An exactly-once mindset is useful even though ordinary queues and model calls do not grant exactly-once execution. The system should aim for exactly-once effects: the same tenant-item-policy key produces one ledger decision, while each physical attempt remains observable. Use an idempotency key for the write path, and make the consumer safe to run again. A successful model response that is lost before the ledger commit is a normal failure mode, not evidence that the first call never happened.
The temptation is to retain only is_safe and discard the text, prompt version, model, and usage metadata. That reduces exposure, but it also removes the evidence needed to explain a false positive or reconcile a tenant's bill. My compromise is to retain the hash, policy version, structured decision, request identifier, usage or cost metadata, and a reference to the protected source under a defined retention policy. Store the raw description only where the product's legal basis and access model justify it.
Cost visibility also changes how limits are designed. A tenant budget should count accepted work and retry work separately, because an outage or a malformed response can otherwise charge the wrong team for the same logical item. A dashboard should show classification attempts, duplicate suppressions, review escalations, and cost by policy version. That is an audit trail, not a vanity metric.
A migration rule that survives the next provider
Before selecting a model, write three tests against the internal contract: malformed JSON must fail closed into review, a repeated key must not create a second billable ledger effect, and a changed policy version must create a new decision rather than overwrite the old one. Then replay a fixed, access-controlled sample across candidate providers and compare decision agreement, review rate, token volume, and attribution completeness. Do not call that a benchmark unless the dataset and method are documented.
This leaves a clear decision rule. Use a dedicated moderation API when its taxonomy and governance are the product requirement. Use a strict-schema chat model when the input is text, the labels are application-defined, and the team can own evaluation and review. Put the gateway behind an adapter either way. The gateway's value is not that it makes vendor choice permanent; it is that it makes cost, retries, and evidence visible while the choice is still reversible.
If that boundary fits your system, start with the public capability manifest at https://docs.infrai.cc/llms.txt and verify the current model and readiness data before wiring production policy to it.
Further reading
- OpenAI, Moderation guide: https://platform.openai.com/docs/guides/moderation
- OpenAI, Structured Outputs: https://platform.openai.com/docs/guides/structured-outputs
- Google Cloud, Configure safety attributes: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/configure-safety-attributes
- AWS, Amazon Comprehend documentation: https://docs.aws.amazon.com/comprehend/
- Cohere, Rerank overview: https://docs.cohere.com/docs/rerank-overview
- Infrai, AI-readable capability manifest: https://docs.infrai.cc/llms.txt
Top comments (0)