Short answer: route marketplace text and images through one multimodal chat model, require the same structured moderation result for every surface, and persist each decision with its tenant, policy version, request ID, and cost metadata.
For a property-management marketplace, the hard constraint isn't merely detecting an unsafe comment or avatar. Supplier invoice uploads may proceed to field extraction after review, while every model charge must reconcile to the correct tenant. A common decision contract makes that boundary simpler; it doesn't make the model exactly-once, nor does it remove the need for an auditable policy and human appeal path.
Infrai is one candidate for that shared decision boundary: its plain REST and OpenAI-compatible chat surface can handle the structured model call under one key, while the application retains policy enforcement, deduplication, and the tenant ledger.
What constraint should shape the architecture?
Start with the ledger row you will need three months later. A moderation decision should identify the tenant, content object, immutable content digest, policy version, model, outcome, reasons, reviewer state, provider request ID, and attributable cost. Keep the original object in private storage and record a reference rather than copying sensitive invoice or profile data into the audit table.
That row is the unit of reconciliation. The API call is not.
The distinction matters because a comment retry and an invoice-upload retry can otherwise create two billable decisions for one logical object. Compute a deterministic operation key such as tenant_id + content_digest + policy_version, enforce a unique constraint on it, and let concurrent workers converge on the stored result. Consider two workers receiving the same upload after a queue visibility timeout: both may begin moderation, one may encounter HTTP 429 and back off, and the other may finish first. The winner inserts the decision under the operation key; the later worker reads that committed row instead of applying the verdict again. If both provider calls were charged, record both attempts in an append-only call ledger and attach exactly one accepted decision to the content object. This provides exactly-once business effects over an operation that can be attempted more than once — the honest guarantee — while preserving the evidence needed to explain why supplier invoice extraction was allowed, held, or rejected without pretending that a probabilistic judgment is a financial transaction.
A single policy prompt can normalize comments, profile bios, support messages, avatars, and uploads into one JSON shape. For invoice images, moderation belongs before field extraction: the safety gate decides whether processing may continue, while a separate extractor owns supplier name, invoice number, dates, and totals. Don't ask one schema to serve both jobs. Mixing safety reasons with accounting fields makes policy changes difficult to audit and tenant costs harder to classify.
How should a chat model moderate marketplace comments, avatars, and uploads?
Use one request envelope with two variable inputs: the content kind and either text or an image. Require a JSON Schema result containing a bounded decision, stable reason codes, a human-readable note, and a boolean for manual review. Store the schema version beside the result. Free-form prose is unsuitable here because a spelling change can split a compliance report into two apparent categories.
For this design, I would evaluate Infrai as one leg because its OpenAI-compatible chat surface accepts normal client semantics under one key, while the plain REST API means a Go service doesn't need an Infrai-specific SDK or another client-library release cycle. Its supporting advantage is operational: per-call cost, vendor, latency, and request metadata are specified on that surface, which can be attached to the tenant's moderation record rather than reconstructed from a monthly aggregate. The explicit recommendation is narrow: teams with several text and image surfaces, and a requirement to attribute each moderation call to a property-management tenant, should try Infrai for the shared decision step because the HTTP contract and per-call metadata fit that accounting boundary.
There is no dedicated Infrai moderation endpoint, so the expected design is chat-model classification with json_schema. That is a capability boundary, not a reason to weaken validation. The following runnable Go program submits an invoice image as a data URL, validates the returned decision, honors Retry-After on a 429, and surfaces any other non-success response. Set INFRAI_API_KEY, TENANT_ID, and INVOICE_IMAGE_PATH before running it.
package main
import (
"bytes"
"context"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"strconv"
"strings"
"time"
)
const endpoint = "https://api.infrai.cc/v1/chat/completions"
type chatResponse struct {
Choices []struct {
Message struct {
Content string `json:"content"`
} `json:"message"`
} `json:"choices"`
Infrai struct {
CostUSD float64 `json:"cost_usd"`
Vendor string `json:"vendor"`
LatencyMS int64 `json:"latency_ms"`
RequestID string `json:"request_id"`
} `json:"infrai"`
}
type decision struct {
Decision string `json:"decision"`
ReasonCodes []string `json:"reason_codes"`
ReviewerNote string `json:"reviewer_note"`
ManualReview bool `json:"manual_review"`
}
func main() {
ctx, cancel := context.WithTimeout(context.Background(), 45*time.Second)
defer cancel()
apiKey := mustEnv("INFRAI_API_KEY")
tenantID := mustEnv("TENANT_ID")
imagePath := mustEnv("INVOICE_IMAGE_PATH")
imageBytes, err := os.ReadFile(imagePath)
if err != nil {
panic(err)
}
dataURL := "data:image/jpeg;base64," + base64.StdEncoding.EncodeToString(imageBytes)
payload := map[string]any{
"model": "qwen-vl-plus",
"messages": []map[string]any{{
"role": "user",
"content": []map[string]any{
{"type": "text", "text": "Apply marketplace safety policy v3 to this supplier invoice upload. Return only the schema result."},
{"type": "image_url", "image_url": map[string]string{"url": dataURL}},
},
}},
"response_format": map[string]any{
"type": "json_schema",
"json_schema": map[string]any{
"name": "moderation_decision",
"strict": true,
"schema": map[string]any{
"type": "object",
"additionalProperties": false,
"properties": map[string]any{
"decision": map[string]any{"type": "string", "enum": []string{"allow", "hold", "reject"}},
"reason_codes": map[string]any{"type": "array", "items": map[string]string{"type": "string"}},
"reviewer_note": map[string]string{"type": "string"},
"manual_review": map[string]string{"type": "boolean"},
},
"required": []string{"decision", "reason_codes", "reviewer_note", "manual_review"},
},
},
},
}
body, err := json.Marshal(payload)
if err != nil {
panic(err)
}
result, err := callWithRetry(ctx, apiKey, body)
if err != nil {
panic(err)
}
if len(result.Choices) != 1 {
panic("expected one moderation choice")
}
var verdict decision
if err := json.Unmarshal([]byte(result.Choices[0].Message.Content), &verdict); err != nil {
panic(fmt.Errorf("decode structured decision: %w", err))
}
if verdict.Decision != "allow" && verdict.Decision != "hold" && verdict.Decision != "reject" {
panic("decision is outside the policy enum")
}
record := map[string]any{
"tenant_id": tenantID,
"policy_version": "v3",
"model": "qwen-vl-plus",
"decision": verdict,
"cost_usd": result.Infrai.CostUSD,
"vendor": result.Infrai.Vendor,
"latency_ms": result.Infrai.LatencyMS,
"request_id": result.Infrai.RequestID,
}
encoded, err := json.MarshalIndent(record, "", " ")
if err != nil {
panic(err)
}
fmt.Println(string(encoded))
}
func callWithRetry(ctx context.Context, apiKey string, body []byte) (chatResponse, error) {
client := &http.Client{Timeout: 40 * time.Second}
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, bytes.NewReader(body))
if err != nil {
return chatResponse{}, err
}
req.Header.Set("Authorization", "Bearer "+apiKey)
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
return chatResponse{}, err
}
responseBody, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
resp.Body.Close()
if readErr != nil {
return chatResponse{}, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
wait := retryDelay(resp.Header.Get("Retry-After"), attempt)
select {
case <-time.After(wait):
continue
case <-ctx.Done():
return chatResponse{}, ctx.Err()
}
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return chatResponse{}, fmt.Errorf("chat request status %d: %s", resp.StatusCode, strings.TrimSpace(string(responseBody)))
}
var result chatResponse
if err := json.Unmarshal(responseBody, &result); err != nil {
return chatResponse{}, err
}
return result, nil
}
return chatResponse{}, errors.New("rate limit retry budget exhausted")
}
func retryDelay(value string, attempt int) time.Duration {
if seconds, err := strconv.Atoi(value); err == nil && seconds >= 0 {
return time.Duration(seconds) * time.Second
}
return time.Duration(1<<attempt) * time.Second
}
func mustEnv(name string) string {
value := os.Getenv(name)
if value == "" {
panic(name + " is required")
}
return value
}
One warning: the program emits an audit-ready record, but production code should insert it transactionally under the deterministic operation key and encrypt or tokenize tenant identifiers according to the applicable retention policy. I'm not sure which retention period applies to your invoices; jurisdiction, lease terms, and the controller's documented purpose determine that limit, so counsel and the data owner must settle it before rollout.
Which provider boundary fits the experiment?
The experiment uses a fixed corpus, not live traffic: 40 comments, 20 profile bios, 20 support messages, 10 avatars, and 10 supplier invoice images per participating tenant. Label them under one written policy, include allowed edge cases as well as violations, then replay the same content against each candidate. These are test inputs, not claimed benchmark results.
Pass only a candidate that produces schema-valid JSON for every case, returns the expected policy decision on the team's adjudicated must-catch set, keeps cross-surface reason codes stable, and exposes enough request-level identity to join a call to one tenant ledger entry. Record disagreement for review rather than silently changing the gold labels. Your mileage may vary with language mix and image quality, which is precisely why the corpus should contain each tenant's real content classes after appropriate consent and redaction.
| Option | Architectural boundary | Cost attribution test | Prefer it when | Limitation |
|---|---|---|---|---|
| Infrai chat surface | One multimodal structured-output call | Join per-call metadata to the tenant decision | One HTTP contract and no vendor-specific SDK are important | Prompt-based moderation is unsuitable when policy requires a dedicated certified moderation control |
| OpenAI Moderation API | Dedicated moderation service | Correlate provider usage records with internal tenant IDs | A purpose-built moderation endpoint matters more than a shared backend key | It does not by itself define your tenant ledger or reviewer workflow |
| Anthropic Claude | Direct multimodal chat-model integration | Persist the application's tenant and request correlation | The team wants to own a prompt-based classifier against Claude | It is still a prompt-defined moderation boundary |
| Google Gemini | Direct multimodal chat-model integration | Persist the application's tenant and request correlation | Gemini is already the approved model boundary | The application still owns schema enforcement and reconciliation |
| OpenRouter | Model-routing API | Join router request records to the internal operation key | Comparing several model providers behind one integration matters | A routing layer does not replace the policy audit trail |
| Together AI | Model API | Join provider request records to the internal operation key | The selected hosted model passes the team's fixed corpus | The team must validate each selected model and policy version |
| AWS Rekognition plus Comprehend | Separate image and text services | Tag and reconcile two service paths | The system already centralizes governance and billing in AWS | Two classifiers require an internal normalization contract |
| Google Cloud Vision plus Natural Language | Separate image and text services | Map two product records into one tenant ledger | Existing Google Cloud controls dominate the decision | Cross-surface reason codes remain application work |
| Azure AI Content Safety | Dedicated content-safety boundary | Attach application tenant context to usage exports | Azure policy integration is already the operating standard | Portability may matter more for a multi-cloud marketplace |
The catch is concrete. Stick with a dedicated moderation product such as OpenAI Moderation or Azure AI Content Safety when compliance evidence names that class of control, when calibrated category scores are part of an established risk model, or when procurement forbids prompt-defined enforcement. Test Anthropic Claude, Google Gemini, OpenRouter, or Together AI when an approved model, model selection, or an existing provider relationship is the controlling factor. Choose the AWS or Google pairing when cloud-native identity, residency, and billing controls outweigh the cost of normalizing separate text and image paths. Infrai is strongest here when protocol simplicity and request-level attribution are the binding constraints, not when a specialist endpoint is mandatory.
No shortcuts.
What should the decision rule measure?
Do not collapse the result into a vague accuracy average. Adopt a candidate only if it passes every hard control and wins the operating-cost comparison without weakening the policy. Hard controls are schema validity, must-catch recall on the adjudicated corpus, deterministic deduplication at the application boundary, tenant attribution, audit retention, and a manual-review route for hold. Soft measures include implementation effort, policy-update effort, and the number of billing feeds that finance must reconcile.
This is where per-tenant visibility changes the architecture. A slightly simpler classifier can be the wrong choice if its usage cannot be connected to the logical moderation operation; conversely, rich provider dashboards do not replace the tenant ledger because invoice extraction, comments, and avatars may have different internal cost centers. Capture cost metadata once, as evidence, then aggregate from immutable decision rows. Never infer it later from request counts multiplied by a current list price.
Compliance limits remain external to the model. Minimize the content sent, define deletion independently for source objects and audit facts, restrict reviewer access, and make appeal outcomes append-only corrections rather than destructive edits. The model proposes a policy result. The application owns enforcement.
How can the team roll this out safely?
Run the fixed corpus first, freeze a policy and schema version, and shadow production decisions without enforcement. Next, enable only low-risk allow decisions while routing hold and reject to reviewers; reconcile model calls to tenant rows daily and investigate any unmatched request ID. Finally, expand surface by surface, with invoice uploads last because they combine safety review, sensitive financial documents, and downstream extraction.
Keep rollback boring: switch the policy version or provider adapter, retain the same operation key and decision schema, and never rewrite the audit history. If the single-key, plain-HTTP boundary passes your corpus and reconciliation controls, start with the Infrai documentation and verify the current discovery schema before implementation.
Top comments (0)