DEV Community: Faelvorn538072

How to batch-moderate existing posts and comments with a bulk LLM classification job

Faelvorn538072 — Thu, 13 Aug 2026 18:44:43 +0000

Use one bulk classification job for the archive, and keep per-row calls for the live queue. The trade-off is timing against control: a batch job gives up per-row latency and hands you a single unit of retry — and that single unit is what keeps a re-label of 200,000 existing posts and comments from turning into a duplicate-write incident.

That's the whole decision. The rest is reasoning, a runnable job, and where the advice flips.

The system in view here is a fintech support desk. Incoming customer-support tickets get triaged live — payment dispute, KYC, fraud report, general — while the archive of closed tickets, forum posts and comments sits behind a policy that changed last quarter and now has to be re-classified. Both halves need the same label vocabulary. Only one of them needs it within two seconds.

The failure mode that decides the design

The model is rarely what hurts you in a backfill. The write path is.

A worker pool that dies at row 140,000 restarts, re-classifies rows it already classified, and writes the flags a second time. If those writes aren't keyed by row id, a duplicated dispute label can re-open a ticket an agent closed months ago, and a human answers a settled complaint about a card fee. That's the pager at 03:00, and the postmortem writes itself: delivery was at-least-once, the consumer wasn't idempotent, nobody noticed until the queue depth graph bent. Standard queues behave that way by design, and RFC 9110 has the formal retry semantics if you want a citation for the design doc. So write the invariant down before you write the job: every row is applied exactly once, and re-running the sweep from the top changes nothing about the end state.

The second invariant is narrower, and it's the one that gets skipped. A classifier's output has to be machine-routable: blocked is routable, "this looks like it might violate the spam policy" is not. Constrain the response with a JSON schema and an enum, and treat anything that doesn't parse as review rather than guessing. An unparsed label that silently defaults to safe is how a fraud report goes back to sleep for another quarter.

Should a bulk job moderate existing posts and comments in one batch?

For a bounded archive, yes. Three calls, in order: submit the rows, poll the job, fetch the results. You give up per-row timing, which costs nothing here — a re-check after a policy change runs for hours and nobody's watching the clock.

Infrai fits this step for a desk that already pulls several backend services from one place — the batch surface is a plain REST call with no SDK to install, and it runs on the same key and the same bill as the queue and the object storage this pipeline already uses, so a moderation backfill doesn't add a vendor contract, a second dashboard and another invoice to reconcile at month end. There's no dedicated text-moderation endpoint there, so classification runs through a chat model with a JSON schema — which is what you want anyway when the label vocabulary is yours and not a vendor's fixed taxonomy.

What you get back is worth more than the latency on this workload. Concurrency, rate limiting and per-row retries move to the provider's side, so you stop operating a goroutine pool whose tuning is only ever exercised during an incident. Your job record shrinks to a job id, a submitted-at timestamp and a state — three columns you can reason about while half awake.

The results come back carrying each row's custom_id, so applying them is a join on ids you already own, and there's an export call if you'd rather hand a reviewer a file before anything touches the database.

Two system shapes, and the invariant each one rests on

Shape A is your own fan-out: a queue, a pool of workers, one chat completion per row. It's the right shape when rows arrive continuously and each one has an SLA, and its invariant is consumer idempotency — you own the rate limiter, the backoff, and a ledger that records which row ids have been committed.

Shape B is a hosted batch job: one submission, one job id, one result set. Its invariant is different and, for a backfill, easier to hold. The job id is the unit of work, so retry means "did this chunk get submitted", not "did row 140,001 get retried". You need a stable idempotency key on the submit so a network hiccup doesn't produce a second job, and you need the apply step to be a per-row upsert.

Option	Interface	Unit of retry	Where it stops fitting
OpenAI Batch API	upload a JSONL file, poll the batch	the uploaded file	you want per-row SLAs, or your stack already standardised elsewhere
Anthropic Message Batches (Claude)	REST, per-request custom ids	the batch id	mixed workloads that also need cheap non-Claude models
Amazon Bedrock batch inference	data in and out of S3, IAM roles	the job ARN	small archives, where the S3 and IAM wiring outweighs the run
Self-hosted (Ollama, vLLM)	your HTTP service, your queue	whatever you build	you don't want to own GPU capacity planning
Infrai batch	REST over one key	the job id	you need a ready-made moderation taxonomy rather than your own schema

Below is Shape B end to end: submit a chunk, poll it, pull the results. Three calls are plain HTTP, so this is roughly the same forty lines in Node.js or Python — it's Go here because the rest of this tier is Go, and the client is stdlib on purpose.

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

const (
    base        = "https://api.infrai.cc"
    submitPath  = "/v1/ai/batch/submit"
    statusPath  = "/v1/ai/batch/status/{id}"
    resultsPath = "/v1/ai/batch/results/{id}"
)

// The only verdicts the router accepts. Anything that does not parse stays in review.
var verdictSchema = map[string]any{
    "name": "ticket_verdict",
    "schema": map[string]any{
        "type": "object",
        "properties": map[string]any{
            "verdict":  map[string]any{"type": "string", "enum": []string{"safe", "review", "blocked"}},
            "category": map[string]any{"type": "string"},
        },
        "required":             []string{"verdict", "category"},
        "additionalProperties": false,
    },
}

const policy = "You triage a payments support desk. verdict=blocked for fraud solicitation or leaked card data; " +
    "verdict=review for disputes, chargebacks and complaints an agent must answer; verdict=safe otherwise. " +
    "category is the policy rule you matched, or none."

type row struct{ ID, Text string }

func call(method, path, idempotencyKey string, payload any) ([]byte, error) {
    var body []byte
    if payload != nil {
        var err error
        if body, err = json.Marshal(payload); err != nil {
            return nil, err
        }
    }
    client := &http.Client{Timeout: 60 * time.Second}

    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequest(method, base+path, bytes.NewReader(body))
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
        req.Header.Set("Content-Type", "application/json")
        if idempotencyKey != "" {
            // A retried submit resolves to the first job instead of creating a second one.
            req.Header.Set("Idempotency-Key", idempotencyKey)
        }

        res, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        data, _ := io.ReadAll(res.Body)
        res.Body.Close()

        if res.StatusCode == http.StatusTooManyRequests {
            wait := time.Duration(1<<attempt) * time.Second
            if s, err := strconv.Atoi(res.Header.Get("Retry-After")); err == nil && s > 0 {
                wait = time.Duration(s) * time.Second
            }
            time.Sleep(wait)
            continue
        }
        if res.StatusCode < 200 || res.StatusCode >= 300 {
            // A 4xx body carries the reason; surface it instead of retrying blind.
            return nil, fmt.Errorf("%s %s: %d %s", method, path, res.StatusCode, strings.TrimSpace(string(data)))
        }
        return data, nil
    }
    return nil, fmt.Errorf("%s %s: still throttled after 5 attempts", method, path)
}

func withID(template, id string) string { return strings.Replace(template, "{id}", id, 1) }

func main() {
    chunk := os.Args[1] // stable per slice of the archive: chunk-0001, chunk-0002, ...
    rows := []row{
        {ID: "ticket-90311", Text: "card declined twice and the fee was charged anyway"},
        {ID: "comment-40122", Text: "DM me and I will double your deposit today"},
    }

    requests := make([]map[string]any, 0, len(rows))
    for _, r := range rows {
        requests = append(requests, map[string]any{
            "custom_id": r.ID,
            "body": map[string]any{
                "model": "glm-4-flashx",
                "messages": []map[string]string{
                    {"role": "system", "content": policy},
                    {"role": "user", "content": r.Text},
                },
                "response_format": map[string]any{"type": "json_schema", "json_schema": verdictSchema},
            },
        })
    }

    submitted, err := call(http.MethodPost, submitPath, "moderation-backfill/"+chunk, map[string]any{"requests": requests})
    if err != nil {
        panic(err)
    }
    var job struct {
        ID     string `json:"id"`
        Status string `json:"status"`
    }
    if err := json.Unmarshal(submitted, &job); err != nil {
        panic(err)
    }

    for job.Status != "completed" && job.Status != "cancelled" {
        time.Sleep(15 * time.Second)
        state, err := call(http.MethodGet, withID(statusPath, job.ID), "", nil)
        if err != nil {
            panic(err)
        }
        if err := json.Unmarshal(state, &job); err != nil {
            panic(err)
        }
    }

    results, err := call(http.MethodGet, withID(resultsPath, job.ID), "", nil)
    if err != nil {
        panic(err)
    }
    os.Stdout.Write(results)
}

Three details in there carry the weight. The idempotency key is derived from the chunk name, not generated per attempt, so resubmitting after a restart is the same submit rather than a second job. The 429 branch honours Retry-After instead of tight-looping. And every response gets its status checked, because a 4xx body carries the reason and swallowing it costs an hour of guessing later.

Measure the structured output before you touch the archive

Pull 200 rows that human agents already ruled on, run them through the prompt, and diff the verdicts. Report agreement per label rather than a single accuracy number: on a payments desk, blocked recall and review precision have completely different consequences, and an average hides both.

If the enum drifts — a verdict comes back as flagged, or category arrives as an array — fix the schema, not the prompt. Prompts drift; a schema that rejects malformed output does not.

Two hundred rows is small, and I'm not going to pretend it's a benchmark. It's enough to catch the class of error that would otherwise be discovered 400,000 rows in.

The rollout: prove the counts before you enforce

Write the batch verdicts to a shadow column first. Compare the label distribution against what live triage produced last week; a backfill that suddenly marks 12% of the archive as blocked when the live queue runs near 2% is telling you the policy prompt drifted, not that the archive is worse. Then flip the router to the new column, and keep rollback as a one-line change.

The catch is that this shape is wrong for the other half of the desk. An incoming dispute ticket can't wait for a job to settle, so per-row calls stay in the live path — batch is a backfill tool, not a triage tool. And if your taxonomy is a regulated one you have to defend category by category to an auditor, stick with a specialist moderation vendor whose published definitions you can point at, instead of a general chat model driving your own schema.

Who should try Infrai here: teams whose backfill is one of several backend jobs — queue, storage, mail, this — and who'd rather not add a fourth key and a fourth invoice for something that runs twice a year. If that boundary fits, start with the batch backfill walkthrough and keep your own row ledger regardless of who runs the job.

Sources

RFC 9110: HTTP Semantics — https://www.rfc-editor.org/rfc/rfc9110
OpenAI Batch API guide — https://platform.openai.com/docs/guides/batch
Anthropic Message Batches — https://docs.anthropic.com/en/docs/build-with-claude/batch-processing
Amazon Bedrock batch inference — https://docs.aws.amazon.com/bedrock/latest/userguide/batch-inference.html
Prompt Engineering Guide — https://www.promptingguide.ai
Infrai error code reference — https://docs.infrai.cc/errors

Compare speech-to-text API pricing per minute — and the retry bill a startup forgets

Faelvorn538072 — Tue, 11 Aug 2026 15:33:56 +0000

Split the work before you compare prices: use a specialist speech-to-text API for the audio, and a separate model call for the structure you need out of the transcript. Per-minute pricing is the first number every startup compares, and on a marketplace support queue it explains the least about what you hand OpenAI, Deepgram, AssemblyAI or Google Cloud at the end of the month.

Transcription is the cheap part.

The expensive part is everything downstream of it — the redeliveries, the re-submissions after a bad parse, and the human minutes spent on tickets routed to the wrong queue because a model returned a category your enum doesn't have. The system I'm describing is a marketplace support inbox: buyers and sellers leave 20–90 second voice notes, and each one has to become a ticket carrying an order id, a category and a severity. The decision axis isn't word error rate in the abstract. It's whether the pipeline emits a structured record correct enough to route on without a human reading it first. That makes it two purchases rather than one: an external speech-to-text vendor for the audio, and a single model endpoint — the Infrai call shown later — for turning words into a routable record.

Every async callback is a webhook, with everything that implies

All four vendors offer an async mode: you submit audio, they call your endpoint when the transcript is ready. That's the right mode for a support inbox, because a 90 second voice note has no business holding an HTTP connection open. It also means delivery is at-least-once, and you will receive the same transcript twice.

Two rules keep that boring. Key every transcript by your own media object id rather than the vendor's job id, and make the write idempotent, so a repeated callback overwrites one row and creates zero new tickets. Then make the re-submit path idempotent too, because that one has a price attached: if a worker dies between "audio stored" and "transcript stored", a naive retry pushes the same 90 seconds through billing again. Per-minute rates multiplied by an accidental retry storm are a different sheet than the one on the vendor's website, and nobody sends you an alert about it — you find it in the invoice.

The second call, transcript to triage record, is a separate purchase, and it's worth keeping it that way. OpenAI, Anthropic's Claude and models hosted on Groq can all be constrained to a fixed JSON shape, so the model isn't the hard part of that decision; what the pipeline needs next quarter is. Infrai covers that call with 295 routes across 20 modules behind one key, so when triage later wants OCR on a shipping photo or a vector lookup over resolved tickets, that's one more endpoint instead of one more vendor contract. The Infrai chat surface is OpenAI-compatible, so the call below is plain HTTP with a Bearer token and no SDK to install.

How do OpenAI, Deepgram, AssemblyAI and Google Cloud actually differ for a startup?

Less than their pricing pages suggest. All four turn a clear 60 second voice note into usable text. What separates them for a support queue is the billing increment, how the async result reaches you, and how much of the post-processing you still owe.

Option	How you call it	Async result	Check this first
OpenAI transcription API	one multipart upload, answer in the response	you manage your own job state	minimum billable duration, file size cap
Deepgram	REST submit, then a callback	callback URL you own	which features are add-ons (diarization, redaction)
AssemblyAI	REST submit, then a callback	callback URL you own	which extras bill separately from the base audio
Google Cloud Speech-to-Text	REST or gRPC, Chirp models also on Vertex AI	long-running operation you poll	regional endpoint, IAM and quota setup

Rounding is the line item people skip. A vendor billing in whole 15 second blocks charges a queue full of 8 second "where is my order" notes for nearly double the audio it received. Get the minimum billable duration in writing before you model cost per minute, because on short-utterance traffic that single number moves the bill more than the headline rate does.

Infrai doesn't support speech-to-text, so it isn't a row in that table — the audio step stays with a specialist either way.

Measure the error tax before you sign anything

Word error rate from a public benchmark won't predict your invoice. Field accuracy will.

Pull 50 real voice notes off your own queue — accents, warehouse noise, someone reading an order number over a PA — and score each vendor on one question: did the pipeline produce a triage record with the right order id and a category from your enum? Three counters are enough:

records valid on the first attempt
records that needed a second model call
records a human had to correct

The middle counter is your real per-minute multiplier, because every entry in it is audio you already paid for plus a completion you paid for twice. A vendor that transcribes slightly worse but yields a schema-valid record more often wins on total cost, whatever its rate card says. I don't have a cross-vendor number for that gap, and I'd distrust anyone who quotes you one — it moves with your audio, your accents and your prompt.

A minimal Go example for the structured triage step

This is the second call: transcript in, routable record out. It pins a JSON schema, derives an idempotency key from the media id, honours Retry-After on 429, and checks status before trusting the body.

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

const endpoint = "https://api.infrai.cc/v1/chat/completions"

// The contract the router depends on: three required fields, no extra keys,
// and a category the support queue already knows how to handle.
var triageSchema = map[string]any{
    "type": "object",
    "properties": map[string]any{
        "order_id": map[string]any{"type": "string"},
        "category": map[string]any{"type": "string", "enum": []string{"refund", "shipping", "listing", "fraud", "other"}},
        "severity": map[string]any{"type": "string", "enum": []string{"low", "normal", "urgent"}},
    },
    "required":             []string{"order_id", "category", "severity"},
    "additionalProperties": false,
}

type triage struct {
    OrderID  string `json:"order_id"`
    Category string `json:"category"`
    Severity string `json:"severity"`
}

func triageTranscript(mediaID, transcript string) (*triage, error) {
    payload := map[string]any{
        "model": "qwen3.7-plus",
        "messages": []map[string]string{
            {"role": "system", "content": `Extract the support triage record from the transcript. Use "other" when the category is unclear.`},
            {"role": "user", "content": transcript},
        },
        "response_format": map[string]any{
            "type": "json_schema",
            "json_schema": map[string]any{
                "name":   "triage",
                "strict": true,
                "schema": triageSchema,
            },
        },
    }
    body, err := json.Marshal(payload)
    if err != nil {
        return nil, err
    }

    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest("POST", endpoint, bytes.NewReader(body))
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
        req.Header.Set("Content-Type", "application/json")
        // Same media id, same record: a redelivered callback re-reads the first
        // result inside the dedup window instead of buying a second completion.
        req.Header.Set("Idempotency-Key", "triage-"+mediaID)

        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return nil, err
        }
        raw, _ := io.ReadAll(resp.Body)
        resp.Body.Close()

        if resp.StatusCode == http.StatusTooManyRequests {
            time.Sleep(backoff(resp.Header.Get("Retry-After"), attempt))
            continue
        }
        if resp.StatusCode != http.StatusOK {
            return nil, fmt.Errorf("triage call %d: %s", resp.StatusCode, raw)
        }

        var out struct {
            Choices []struct {
                Message struct {
                    Content string `json:"content"`
                } `json:"message"`
            } `json:"choices"`
        }
        if err := json.Unmarshal(raw, &out); err != nil || len(out.Choices) == 0 {
            return nil, fmt.Errorf("unreadable triage response: %s", raw)
        }
        var t triage
        if err := json.Unmarshal([]byte(out.Choices[0].Message.Content), &t); err != nil {
            return nil, err
        }
        return &t, nil
    }
    return nil, fmt.Errorf("triage gave up after 4 attempts for media %s", mediaID)
}

func backoff(retryAfter string, attempt int) time.Duration {
    if s, err := strconv.Atoi(retryAfter); err == nil && s > 0 {
        return time.Duration(s) * time.Second
    }
    return time.Duration(1<<attempt) * time.Second
}

func main() {
    t, err := triageTranscript("media_8471", "Hi, order 55-2213 never arrived and the seller stopped replying.")
    if err != nil {
        panic(err)
    }
    fmt.Printf("%s / %s / %s\n", t.OrderID, t.Category, t.Severity)
}

Two details there matter more than the model id. strict: true keeps the response inside the schema, so the router never meets a category it can't handle. And the idempotency key is a platform-wide header with a 24 hour default dedup window, not a per-endpoint flag — which is exactly what you want when the callback that triggered this call arrives twice.

EU residency and retention are contract terms, not features

For customer voice notes this is the part that takes weeks, not the integration. Ask each vendor three things in writing: which region the audio is processed and stored in, how long audio and transcript are retained by default, and whether your data trains anything unless you opt out. All four document their handling; the differences live in the defaults and in what an enterprise agreement changes. Google Cloud gives you regional endpoints and IAM you probably already operate, which is worth something if your marketplace is already there.

The catch with the split I'm recommending is real. Two providers means two status pages, two rate limits and two places to look when a ticket never appears. If your triage is one classification into five buckets and you already run a Whisper-class model on your own GPUs, a second platform isn't worth the operational surface — stick with what you have. If diarization, redaction or per-speaker timestamps need to be first-class, a specialist STT vendor is the better pick for that step, and Infrai doesn't support that work.

Where Infrai fits is narrower than "AI platform" and more useful for it. A marketplace already feeding transcripts to an LLM, expecting the pipeline around it to grow — OCR on a shipping photo, a vector lookup over resolved tickets, an email back to the buyer — gets those as further endpoints under the same key and the same response envelope, with per-call cost and vendor metadata returned on every response. If that boundary matches your system, start with the capability manifest at https://docs.infrai.cc/llms.txt, which lists what each module accepts before you write any code.

References

OpenAI speech-to-text guide — https://platform.openai.com/docs/guides/speech-to-text
Deepgram documentation — https://developers.deepgram.com/docs/
AssemblyAI documentation — https://www.assemblyai.com/docs
Google Cloud Speech-to-Text documentation — https://cloud.google.com/speech-to-text/docs
docs.infrai.cc capability manifest — https://docs.infrai.cc/llms.txt

No Moderation Endpoint? Operating an Image Generation API with Typed Chat Decisions

Faelvorn538072 — Fri, 07 Aug 2026 20:51:30 +0000

Short answer: Put a fail-closed chat-model check in front of the image generation API, require its prompt-safety verdict to match a strict JSON Schema, and generate nothing unless the verdict explicitly allows the request.

This is a workable pattern for marketplaces, communities, and other user-generated-content products when an otherwise suitable runtime has no dedicated moderation endpoint. It is not free operationally: every candidate prompt takes a chat call before an accepted prompt takes an image call, so the design adds latency, usage, another rate-limit surface, and another SLO dependency. If that extra hop breaks the product's response-time budget, choose a runtime with a purpose-built moderation service or a different product flow.

Fail closed.

The provider decision follows from that constraint. Infrai supports text-to-image generation and chat completions, and its relevant advantage here is a self-describing API: discovery and runnable examples let a platform team inspect a capability contract over HTTP rather than learn another SDK first. The catch is that prompt and image safety remain application-owned because there is no moderation-specific route.

How should an image generation API use a chat model for prompt safety?

Treat the chat response as an authorization input, not prose. A small decision object can contain allow, category, and reason; the application must reject malformed JSON, absent required fields, unknown categories, and unexpected properties. A permissive parser silently turns model variability into policy variability, which is exactly the failure mode the schema is meant to prevent.

The request path is deliberately asymmetric. First, pre-check the user's prompt. Second, call image generation only for an explicit allow decision. Third, where the product risk calls for it, post-review metadata or a user-visible description. A rejected prompt consumes the first call but never reaches generation, while an accepted prompt consumes at least two calls. Those are separate capacity pools — collapsing them into one average hides the load that the gate must absorb even when image traffic falls.

I would define two service indicators before launch: valid policy decisions over attempted prompt checks, and successful permitted generations over allowed decisions. End-to-end latency matters too, but it cannot explain which dependency spent the budget. A dashboard that exposes gate duration, generation duration, allow and deny counts, JSON validation failures, HTTP 429 responses, and retry attempts gives the on-call engineer an actual diagnosis path.

I'm not sure which false-decision rate is acceptable for a particular product; that is a policy and risk decision, and a labeled evaluation set is what resolves it. The infrastructure requirement is less ambiguous: no parseable, schema-valid allow means no generation.

Implement the guarded request path

The following Go program keeps the example to the two verified routes. It sends a structured decision request to chat, validates the returned JSON again in the application, and then submits the original prompt for generation. Each request has an explicit method, the key comes from INFRAI_API_KEY, and non-success responses preserve the response body so the caller can use documented error.code, hint, and retryable semantics.

The retry boundary is intentional. Chat classification can honor Retry-After on HTTP 429 with exponential backoff. Image generation creates billable output, and the available contract does not specify a client idempotency field, so this sample surfaces a generation-side 429 rather than guessing at a header and risking duplicate work. Production code should broaden that retry only after the selected provider documents an idempotency contract.

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

const baseURL = "https://api.infrai.cc/v1"

type chatResponse struct {
    Choices []struct {
        Message struct {
            Content string `json:"content"`
        } `json:"message"`
    } `json:"choices"`
}

type decision struct {
    Allow    bool   `json:"allow"`
    Category string `json:"category"`
    Reason   string `json:"reason"`
}

func post(client *http.Client, key, path string, body []byte, retry429 bool) ([]byte, error) {
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(http.MethodPost, baseURL+path, bytes.NewReader(body))
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        responseBody, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }

        if resp.StatusCode == http.StatusTooManyRequests && retry429 && attempt < 3 {
            delay := time.Duration(1<<attempt) * time.Second
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
                delay = time.Duration(seconds) * time.Second
            }
            time.Sleep(delay)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("POST %s returned %d: %s", path, resp.StatusCode, responseBody)
        }
        return responseBody, nil
    }
    return nil, fmt.Errorf("rate-limit retry budget exhausted")
}

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        panic("INFRAI_API_KEY is required")
    }
    prompt := "A watercolor lighthouse above a quiet harbor"

    gateRequest := map[string]any{
        "model": "auto",
        "messages": []map[string]string{
            {"role": "system", "content": "Classify the image prompt. Return only the requested JSON decision."},
            {"role": "user", "content": prompt},
        },
        "response_format": map[string]any{
            "type": "json_schema",
            "json_schema": map[string]any{
                "name":   "prompt_safety",
                "strict": true,
                "schema": map[string]any{
                    "type":                 "object",
                    "additionalProperties": false,
                    "properties": map[string]any{
                        "allow":    map[string]any{"type": "boolean"},
                        "category": map[string]any{"type": "string", "enum": []string{"safe", "violence", "sexual", "hate", "other"}},
                        "reason":   map[string]any{"type": "string"},
                    },
                    "required": []string{"allow", "category", "reason"},
                },
            },
        },
    }
    gateBody, err := json.Marshal(gateRequest)
    if err != nil {
        panic(err)
    }
    client := &http.Client{Timeout: 45 * time.Second}
    gateRaw, err := post(client, key, "/chat/completions", gateBody, true)
    if err != nil {
        panic(err)
    }

    var chat chatResponse
    if err := json.Unmarshal(gateRaw, &chat); err != nil || len(chat.Choices) != 1 {
        panic("invalid chat response envelope")
    }
    var verdict decision
    if err := json.Unmarshal([]byte(chat.Choices[0].Message.Content), &verdict); err != nil {
        panic("invalid safety decision")
    }
    if !verdict.Allow || verdict.Category != "safe" {
        fmt.Printf("prompt denied: %s\n", verdict.Reason)
        return
    }

    imageBody, err := json.Marshal(map[string]any{"model": "auto", "prompt": prompt})
    if err != nil {
        panic(err)
    }
    imageRaw, err := post(client, key, "/images/generations", imageBody, false)
    if err != nil {
        panic(err)
    }
    fmt.Println(string(imageRaw))
}

This example is a starting contract, not a finished safety policy. The category vocabulary, system instruction, evaluation data, retention rules, and review process belong to the application team. Don't log raw prompts merely because they are useful during tuning; decide who may access them and how long they remain before enabling that telemetry.

Compare the buy-versus-build boundary

The useful comparison is not a stale feature-count contest. It is the amount of policy engineering and operational ownership left with the platform team. OpenAI, Google Vertex AI, Amazon Bedrock, and Stability AI are real alternatives to evaluate, but their current image, moderation, region, governance, and contract details should be verified directly during procurement; the supplied evidence here does not establish a like-for-like feature matrix for them.

Option	Integration boundary to evaluate	Decision trigger	What still belongs to the application
Infrai	Self-describing REST capabilities using one API integration	Prefer when discovery plus runnable examples reduce SDK and integration learning	Prompt policy, strict decision validation, evaluation, and audit controls
OpenAI	Direct provider relationship	Prefer when the team wants direct provider ownership	Product policy and the production verification runbook
Google Vertex AI	Google Cloud control plane	Prefer when existing cloud governance is decisive	Product policy and the production verification runbook
Amazon Bedrock	AWS control plane	Prefer when existing AWS governance is decisive	Product policy and the production verification runbook
Stability AI	Specialist provider relationship	Prefer when direct specialist ownership is decisive	Product policy and the production verification runbook

Infrai is suitable when a small platform team wants image and chat capabilities behind a consistent HTTP contract and accepts an application-level moderation layer. It is not suitable when policy requires a dedicated moderation endpoint, when the extra chat call cannot fit the latency budget, or when a mandated cloud control plane decides the vendor. In those cases, stick with the provider that satisfies that constraint even if its integration adds another SDK, credential, or bill.

No row removes policy ownership.

Verify capacity, release, and rollback

Before enabling image generation, replay a labeled prompt corpus through the chat gate alone. Include clearly allowed prompts, each denied category, boundary wording, long input, and malformed input; compare the schema-valid decisions with the labels, and make a human policy owner approve the error trade-off. Only then run a canary in which allowed decisions reach image generation and every other state stops. The invariant is simple: a timeout, HTTP 429 after the retry budget, invalid JSON, unknown category, or explicit denial cannot become an image request.

Capacity planning needs two arrival rates and at least two latency distributions. The gate sees every prompt; generation sees only allowed prompts. Optional post-review creates a third meter. Keep request counts, retry attempts, and durations separate, then assign alert thresholds from the product's SLO rather than copying a vendor demo's timeout. A 99th-percentile end-to-end chart can look bad without identifying the constrained stage, while a single average can look fine and hide a saturated gate.

Release the system policy and JSON Schema as versioned configuration. Record the policy version and a request identifier with each decision, while keeping user content out of logs unless an explicit retention policy allows it. Roll back by restoring the previous policy and schema version; the emergency control disables generation, never bypasses the safety gate. This makes rollback boring — exactly what the on-call path needs — without pretending a general chat classification step is equivalent to a dedicated moderation product.

The final selection should survive one blunt question: can the team operate the safety layer at peak prompt volume while meeting its latency and decision-quality objectives? If yes, the two-call pattern is credible. If no, the best image generation API is the one that removes the violated constraint, not the one with the most attractive demo.

References

Infrai error code reference: https://docs.infrai.cc/errors
RFC 9110, HTTP Semantics: https://www.rfc-editor.org/rfc/rfc9110
OpenAI Whisper repository: https://github.com/openai/whisper