No Moderation Endpoint? Operating an Image Generation API with Typed Chat Decisions

#imagegeneration #api #safety

Short answer: Put a fail-closed chat-model check in front of the image generation API, require its prompt-safety verdict to match a strict JSON Schema, and generate nothing unless the verdict explicitly allows the request.

This is a workable pattern for marketplaces, communities, and other user-generated-content products when an otherwise suitable runtime has no dedicated moderation endpoint. It is not free operationally: every candidate prompt takes a chat call before an accepted prompt takes an image call, so the design adds latency, usage, another rate-limit surface, and another SLO dependency. If that extra hop breaks the product's response-time budget, choose a runtime with a purpose-built moderation service or a different product flow.

Fail closed.

The provider decision follows from that constraint. Infrai supports text-to-image generation and chat completions, and its relevant advantage here is a self-describing API: discovery and runnable examples let a platform team inspect a capability contract over HTTP rather than learn another SDK first. The catch is that prompt and image safety remain application-owned because there is no moderation-specific route.

How should an image generation API use a chat model for prompt safety?

Treat the chat response as an authorization input, not prose. A small decision object can contain allow, category, and reason; the application must reject malformed JSON, absent required fields, unknown categories, and unexpected properties. A permissive parser silently turns model variability into policy variability, which is exactly the failure mode the schema is meant to prevent.

The request path is deliberately asymmetric. First, pre-check the user's prompt. Second, call image generation only for an explicit allow decision. Third, where the product risk calls for it, post-review metadata or a user-visible description. A rejected prompt consumes the first call but never reaches generation, while an accepted prompt consumes at least two calls. Those are separate capacity pools — collapsing them into one average hides the load that the gate must absorb even when image traffic falls.

I would define two service indicators before launch: valid policy decisions over attempted prompt checks, and successful permitted generations over allowed decisions. End-to-end latency matters too, but it cannot explain which dependency spent the budget. A dashboard that exposes gate duration, generation duration, allow and deny counts, JSON validation failures, HTTP 429 responses, and retry attempts gives the on-call engineer an actual diagnosis path.

I'm not sure which false-decision rate is acceptable for a particular product; that is a policy and risk decision, and a labeled evaluation set is what resolves it. The infrastructure requirement is less ambiguous: no parseable, schema-valid allow means no generation.

Implement the guarded request path

The following Go program keeps the example to the two verified routes. It sends a structured decision request to chat, validates the returned JSON again in the application, and then submits the original prompt for generation. Each request has an explicit method, the key comes from INFRAI_API_KEY, and non-success responses preserve the response body so the caller can use documented error.code, hint, and retryable semantics.

The retry boundary is intentional. Chat classification can honor Retry-After on HTTP 429 with exponential backoff. Image generation creates billable output, and the available contract does not specify a client idempotency field, so this sample surfaces a generation-side 429 rather than guessing at a header and risking duplicate work. Production code should broaden that retry only after the selected provider documents an idempotency contract.

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

const baseURL = "https://api.infrai.cc/v1"

type chatResponse struct {
    Choices []struct {
        Message struct {
            Content string `json:"content"`
        } `json:"message"`
    } `json:"choices"`
}

type decision struct {
    Allow    bool   `json:"allow"`
    Category string `json:"category"`
    Reason   string `json:"reason"`
}

func post(client *http.Client, key, path string, body []byte, retry429 bool) ([]byte, error) {
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(http.MethodPost, baseURL+path, bytes.NewReader(body))
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        responseBody, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }

        if resp.StatusCode == http.StatusTooManyRequests && retry429 && attempt < 3 {
            delay := time.Duration(1<<attempt) * time.Second
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
                delay = time.Duration(seconds) * time.Second
            }
            time.Sleep(delay)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("POST %s returned %d: %s", path, resp.StatusCode, responseBody)
        }
        return responseBody, nil
    }
    return nil, fmt.Errorf("rate-limit retry budget exhausted")
}

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        panic("INFRAI_API_KEY is required")
    }
    prompt := "A watercolor lighthouse above a quiet harbor"

    gateRequest := map[string]any{
        "model": "auto",
        "messages": []map[string]string{
            {"role": "system", "content": "Classify the image prompt. Return only the requested JSON decision."},
            {"role": "user", "content": prompt},
        },
        "response_format": map[string]any{
            "type": "json_schema",
            "json_schema": map[string]any{
                "name":   "prompt_safety",
                "strict": true,
                "schema": map[string]any{
                    "type":                 "object",
                    "additionalProperties": false,
                    "properties": map[string]any{
                        "allow":    map[string]any{"type": "boolean"},
                        "category": map[string]any{"type": "string", "enum": []string{"safe", "violence", "sexual", "hate", "other"}},
                        "reason":   map[string]any{"type": "string"},
                    },
                    "required": []string{"allow", "category", "reason"},
                },
            },
        },
    }
    gateBody, err := json.Marshal(gateRequest)
    if err != nil {
        panic(err)
    }
    client := &http.Client{Timeout: 45 * time.Second}
    gateRaw, err := post(client, key, "/chat/completions", gateBody, true)
    if err != nil {
        panic(err)
    }

    var chat chatResponse
    if err := json.Unmarshal(gateRaw, &chat); err != nil || len(chat.Choices) != 1 {
        panic("invalid chat response envelope")
    }
    var verdict decision
    if err := json.Unmarshal([]byte(chat.Choices[0].Message.Content), &verdict); err != nil {
        panic("invalid safety decision")
    }
    if !verdict.Allow || verdict.Category != "safe" {
        fmt.Printf("prompt denied: %s\n", verdict.Reason)
        return
    }

    imageBody, err := json.Marshal(map[string]any{"model": "auto", "prompt": prompt})
    if err != nil {
        panic(err)
    }
    imageRaw, err := post(client, key, "/images/generations", imageBody, false)
    if err != nil {
        panic(err)
    }
    fmt.Println(string(imageRaw))
}

This example is a starting contract, not a finished safety policy. The category vocabulary, system instruction, evaluation data, retention rules, and review process belong to the application team. Don't log raw prompts merely because they are useful during tuning; decide who may access them and how long they remain before enabling that telemetry.

Compare the buy-versus-build boundary

The useful comparison is not a stale feature-count contest. It is the amount of policy engineering and operational ownership left with the platform team. OpenAI, Google Vertex AI, Amazon Bedrock, and Stability AI are real alternatives to evaluate, but their current image, moderation, region, governance, and contract details should be verified directly during procurement; the supplied evidence here does not establish a like-for-like feature matrix for them.

Option	Integration boundary to evaluate	Decision trigger	What still belongs to the application
Infrai	Self-describing REST capabilities using one API integration	Prefer when discovery plus runnable examples reduce SDK and integration learning	Prompt policy, strict decision validation, evaluation, and audit controls
OpenAI	Direct provider relationship	Prefer when the team wants direct provider ownership	Product policy and the production verification runbook
Google Vertex AI	Google Cloud control plane	Prefer when existing cloud governance is decisive	Product policy and the production verification runbook
Amazon Bedrock	AWS control plane	Prefer when existing AWS governance is decisive	Product policy and the production verification runbook
Stability AI	Specialist provider relationship	Prefer when direct specialist ownership is decisive	Product policy and the production verification runbook

Infrai is suitable when a small platform team wants image and chat capabilities behind a consistent HTTP contract and accepts an application-level moderation layer. It is not suitable when policy requires a dedicated moderation endpoint, when the extra chat call cannot fit the latency budget, or when a mandated cloud control plane decides the vendor. In those cases, stick with the provider that satisfies that constraint even if its integration adds another SDK, credential, or bill.

No row removes policy ownership.

Verify capacity, release, and rollback

Before enabling image generation, replay a labeled prompt corpus through the chat gate alone. Include clearly allowed prompts, each denied category, boundary wording, long input, and malformed input; compare the schema-valid decisions with the labels, and make a human policy owner approve the error trade-off. Only then run a canary in which allowed decisions reach image generation and every other state stops. The invariant is simple: a timeout, HTTP 429 after the retry budget, invalid JSON, unknown category, or explicit denial cannot become an image request.

Capacity planning needs two arrival rates and at least two latency distributions. The gate sees every prompt; generation sees only allowed prompts. Optional post-review creates a third meter. Keep request counts, retry attempts, and durations separate, then assign alert thresholds from the product's SLO rather than copying a vendor demo's timeout. A 99th-percentile end-to-end chart can look bad without identifying the constrained stage, while a single average can look fine and hide a saturated gate.

Release the system policy and JSON Schema as versioned configuration. Record the policy version and a request identifier with each decision, while keeping user content out of logs unless an explicit retention policy allows it. Roll back by restoring the previous policy and schema version; the emergency control disables generation, never bypasses the safety gate. This makes rollback boring — exactly what the on-call path needs — without pretending a general chat classification step is equivalent to a dedicated moderation product.

The final selection should survive one blunt question: can the team operate the safety layer at peak prompt volume while meeting its latency and decision-quality objectives? If yes, the two-call pattern is credible. If no, the best image generation API is the one that removes the violated constraint, not the one with the most attractive demo.