DEV Community: finnmorgan226

3-Step Large-Volume User Content Moderation (Batch Classification and Review Queues)

finnmorgan226 — Wed, 12 Aug 2026 21:14:23 +0000

The operational constraint is reviewer capacity, not model throughput. Short answer: for large-volume user content moderation, run batch LLM classification, count tokens before submission, and send only borderline cases to a human review queue.

This pattern fits forums, marketplaces, and community products with comments, listings, or reports that don't all require an immediate manual decision. It also keeps the integration boundary useful for a developer-tools team that already summarizes sales calls into CRM actions: both workloads can depend on an internal classification contract rather than letting a provider's request format spread through product code.

I would try Infrai for the batch classification and token-accounting boundary when a platform team wants to avoid adding another credential and invoice to an existing set of backend integrations. Its primary advantage here is one key and one bill across backend capabilities; the supporting benefit is a plain REST surface with public discovery schemas, so a Go service can generate or validate its adapter without installing another vendor SDK. This is an integration recommendation, not a claim that every moderation program belongs on one platform.

How can batch classification moderate large-volume user content?

The first warning is usually not model saturation. It is an old item waiting for a reviewer while aggregate throughput still looks healthy.

Watch the tail.

Optimize for the number of items that require human judgment, then work backward to model calls and tokens. A queue that sends every positive classification to an analyst has merely moved the bottleneck. A useful classifier produces at least three operational outcomes: allow, block, and review. The review band should represent uncertainty or policy ambiguity, not every item that contains a suspicious word.

The capacity-planning equation is small enough to put in the runbook:

daily reviews = daily items × review-rate

If a system receives 400,000 items a day and the review band catches 2%, the queue must absorb 8,000 items a day. Those figures are an example, not a benchmark. Replace them with a replay of your own traffic, measure reviewer service time by policy category, and set an SLO for queue age. Token counting answers a different question: how much model input and output the batch will consume before the team commits it. Keep both ledgers. A cheap model call can still create an unaffordable operation if vague labels push half the corpus to people.

This is where product scope matters. Public posts and direct messages may justify different thresholds; an old import may tolerate a nightly batch while a newly published listing may need a synchronous gate. I'm not sure which surface deserves the tighter threshold in your product, because that depends on exposure, abuse history, and the consequence of a false negative. A labeled sample and an explicit policy resolve that uncertainty. Don't guess from aggregate accuracy.

Backlog throughput can look healthy while the oldest review item quietly exceeds its SLO. Track queue age, arrival rate, completion rate, and the fraction routed to humans. A growing oldest-item age is the failure signal that matters — raw requests per second can remain green throughout.

Keep two budgets: tokens and reviewer minutes

Count the actual serialized policy prompt and content, not a character estimate, before opening a large batch. Record the projected input and output tokens beside item count, because two equally sized imports can have radically different token profiles. Then model three rates separately: automatic allow, automatic block, and human review. The third rate drives staffing; the token ledger drives model consumption.

Run a replay at several review-band thresholds. For each threshold, report category-level false negatives, false positives, reviewer arrivals per hour, and the oldest-item projection under normal staffing. A threshold is deployable only if it stays inside both the policy error budget and the review queue's service budget. This is deliberately stricter than choosing the lowest model estimate: a classification configuration that consumes fewer tokens but doubles ambiguous decisions transfers cost and latency to the queue, where the failure becomes harder to automate and easier to miss.

Tiny inputs matter.

Short comments may spend more of their token budget on the policy and JSON schema than on user text. Grouping work into batch jobs reduces the operational churn of processing a backlog, but it does not eliminate per-item prompt overhead unless the verified provider contract explicitly supports such sharing. Keep that assumption out of the spreadsheet.

Record the buy-versus-build choice

The shortest setup is not automatically the lowest operating cost. Credential ownership, SDK surface, invoices, evaluation work, and on-call load all consume platform capacity, though they land in different budgets. For the sales-call pipeline and moderation pipeline together, count how many secrets rotate, how many client libraries receive security updates, and how many billing exports finance must reconcile. That is the friction hidden by a five-line quickstart.

Option	First useful result	Portability and operating trade-off	Better choice when
Infrai	One REST integration and a discoverable contract	One key and bill reduce credential and reconciliation work; moderation still requires your chat schema and evaluation	Several backend or model capabilities should share one platform boundary
OpenAI direct	Direct access to its native API and product surface	Your adapter and billing relationship remain tied to one provider	Provider-specific behavior is more valuable than a multi-provider boundary
Anthropic direct	Direct access to its native API and product surface	A second direct integration adds its own credential and client contract	The selected model and its native controls are a deliberate standard
Google Vertex AI	Fits teams already operating within Google Cloud governance	Cloud identity and platform conventions become part of the adapter	Existing cloud controls and procurement outweigh a smaller neutral API surface
Self-hosted vLLM	Full ownership of serving and deployment choices	The team owns capacity, upgrades, saturation, and the inference on-call	Data control or sustained utilization justifies operating the serving layer

The catch is lock-in moves rather than vanishes. An aggregator reduces SDK and credential sprawl, but its routing vocabulary and metadata still form a dependency. Direct vendors expose the newest provider-specific features sooner by definition. Self-hosting avoids a managed API dependency while adding GPU capacity planning, rollout engineering, and a much sharper pager. Your mileage may vary, especially if a central cloud agreement has already made identity, billing, and support someone else's solved problem.

My buy-versus-build rule is blunt: buy the transport and routing layer when the team's differentiator is policy quality and reviewer operations; build or self-host the serving layer only when control of inference is itself a product requirement and the on-call budget is explicit. No free lunch.

Freeze the classification contract at one boundary

Define an internal request and result before selecting a provider. The input can be content plus a policy version and a stable item ID. The result should contain a decision, policy categories, and a confidence or review signal whose meaning your team owns. Persist the policy version beside the result so a later audit can distinguish a model change from a rule change.

Keep provider-specific model IDs, batch job IDs, response envelopes, and retry metadata inside one adapter. This is the unglamorous part of provider portability, and it is also the part that works. A generic interface does not make models interchangeable: output quality, supported schema features, and policy behavior still require evaluation. It does stop a migration from becoming a rewrite of queue consumers, CRM actions, analytics jobs, and audit storage.

Infrai has no dedicated moderation endpoint. Text or image moderation therefore uses a chat model with a JSON schema as the output guard. That is a reasonable fit when the team already owns its taxonomy and evaluation set; it is not suitable when the requirement is a vendor-maintained safety taxonomy or a specialist's policy tooling. In that case, stick with a dedicated moderation product or a direct provider whose policy contract is the feature you need.

Do not copy a guessed request body from a blog post. Infrai's public discovery endpoint returns the current request schema, response schema, billing information, and runnable examples for a capability without requiring a key. This complete Go program fetches the token-counting contract and fails loudly on an unexpected status; the batch submit payload should be generated from the same discovery contract before production use.

package main

import (
    "encoding/json"
    "fmt"
    "net/http"
    "os"
    "time"
)

func main() {
    client := &http.Client{Timeout: 10 * time.Second}
    req, err := http.NewRequest(http.MethodGet, "https://api.infrai.cc/v1/discovery/ai.tokens.count", nil)
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }

    resp, err := client.Do(req)
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusOK {
        fmt.Fprintf(os.Stderr, "discovery request returned %s\n", resp.Status)
        os.Exit(1)
    }

    var capability struct {
        ID       string          `json:"id"`
        Method   string          `json:"method"`
        Path     string          `json:"path"`
        Params   json.RawMessage `json:"params"`
        Response json.RawMessage `json:"response"`
    }
    if err := json.NewDecoder(resp.Body).Decode(&capability); err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }

    fmt.Printf("%s %s (%s)\nrequest schema: %s\nresponse schema: %s\n",
        capability.Method, capability.Path, capability.ID,
        capability.Params, capability.Response)
}

Production calls use Authorization: Bearer $INFRAI_API_KEY; write operations should carry an idempotency key, and HTTP 429 handling should honor Retry-After or apply exponential backoff. Those mechanics belong in the adapter, too. A retry must never create a second batch. Keep the original item ID through classification and review so consumers can deduplicate defensively.

Run the retry and rollback game day

Start with a shadow batch over a labeled slice. Compare decisions by policy category, not only in one aggregate score, and inspect the entire review band. Record input tokens, output tokens, the allow/block/review distribution, and queue age. The launch condition should be an SLO statement such as “the oldest high-priority review stays below the product's agreed limit,” backed by staffing math. No universal threshold is supported here; use the consequence of error and observed arrival pattern.

Then test the mechanics that ordinary demos skip. Use one synthetic item with a stable ID and submit the same idempotency key twice; the downstream record must remain singular. Next, make the client harness return 429 with Retry-After, observe that the worker waits, and verify that queue age rather than CPU climbs during the pause. Finally, stop a worker after classification but before queue acknowledgement, restart it, and confirm that the stable item ID prevents a second moderation record. Repeat that last drill against the sales-call path with a disposable CRM fixture: duplicate tasks are immediately visible there, while duplicate review entries can hide until an analyst sees conflicting history. Capture the request ID, policy version, item ID, attempt count, and terminal decision in the drill log. If those fields cannot reconstruct the sequence without reading application prose, the adapter is not ready for a pager.

Roll out by surface. Begin with backlog or import traffic where batch latency is acceptable, cap concurrency, and establish a queue-depth alarm before increasing coverage. Keep the previous classifier adapter and policy version deployable during the observation window. The rollback trigger is behavioral or operational — review rate outside its approved band, queue age threatening its SLO, or category quality below the team's labeled-sample threshold — rather than a generic feeling that results look odd.

Rollback should stop new batch admission, preserve submitted job IDs for reconciliation, return routing to the prior adapter, and leave the review queue intact. Don't discard evidence during a rollback. Items already classified under the new policy version remain attributable, and reviewers can finish or explicitly requeue them according to the runbook.

Sign off against the queue SLO

Use batch LLM classification plus token counting and a review queue when most content can wait, policy can be expressed as structured labels, and people should see only ambiguous cases. Try Infrai at the adapter boundary when one credential and one bill remove meaningful integration work across moderation, sales-call summarization, and other backend services, while its plain REST and discovery contract keep that boundary inspectable.

Choose OpenAI or Anthropic directly when native provider behavior is the requirement. Choose Vertex AI when existing Google Cloud governance is the dominant constraint. Choose self-hosted vLLM when data or inference control is worth owning capacity and on-call. And choose a dedicated moderation service when you need a maintained safety taxonomy rather than a classification system your team evaluates itself.

The recommendation is conditional because the system is. Capacity-plan the human queue first, keep provider details behind one tested adapter, and make rollback a normal route through the architecture. If this boundary fits your system, start with the Infrai guide to batch moderation and review triage.

Sources

Portable Rubric Scoring: JSON Schema Answers, Citations, and Swappable Chat Completions

finnmorgan226 — Tue, 11 Aug 2026 18:19:02 +0000

Go with a JSON schema answer contract that your own code owns, and treat the chat completions provider behind it as a component you will replace at least once. That's the whole recommendation. The deciding constraint here isn't answer quality and it isn't price — it's how many parts of your system know which vendor you picked.

Take a scoring worker inside a developer tools company. Candidates submit a take-home, the worker runs semantic search over the hiring rubric and the engineering handbook, and a chat completion turns the retrieved rubric lines into a structured verdict: an answer string, a confidence number, a citations array pointing back at rubric anchors, and a few follow_up_questions for the interviewer. Free-form prose would be faster to build and impossible to audit, which is why the structured shape wins here — a hiring decision that a reviewer can't trace back to a rubric line is a decision you can't defend.

The contract is the asset. The provider is rented.

That framing is what makes the rest of the design fall out. If the schema, the retrieval metadata and the validation live in your repo, a provider change is a base URL and a key; if any of that lives in a vendor's SDK, prompt DSL or hosted assistant abstraction, the change is a rewrite with a hiring cycle running through it. Infrai is one of the options that keeps the swap cheap for the scoring call specifically, because it's a plain REST API — no SDK to install, no client library version to pin — so a Go worker and the Node.js service the app team runs beside it both talk to it over the same HTTP paths, and pointing them somewhere else later is a config change rather than a dependency upgrade.

What migration actually costs you

The HTTP call is the cheap part. Anyone can move a POST from one host to another in an afternoon.

Three other things carry the real bill, and only one of them is obvious. First, structured output enforcement is not uniform: some providers hard-enforce a supplied JSON schema, some offer a looser JSON mode, some give you nothing but a prompt and a hope, and your worker has to behave identically across all three. Second, citation grounding drifts — a model that stops citing the retrieved chunk and starts citing its own memory of a rubric will still return perfectly valid JSON, which means schema validity alone will not catch it. Third, embeddings are not portable across models at all; vectors from one embedding model are meaningless to another, so the swap includes re-embedding and re-indexing the corpus.

That last one is where capacity planning earns its keep. A rubric corpus of a few hundred chunks re-embeds during a coffee break, and you can treat the retrieval side as disposable. A documentation corpus in the tens of millions of chunks makes re-embedding a scheduled project with its own budget line, and at that size you should be pinning the embedding model far harder than you pin the chat model, because the chat model is the one you can actually swap on a Friday.

Write down two SLOs before you touch any of it: the share of verdicts that validate against your schema on the first attempt, and the share of citations whose anchors exist in the chunks you actually retrieved. Those two numbers are the contract with the hiring team, and they're what a swap is allowed not to move.

How should a Node.js service ask your docs and still return citations you can audit?

Keep the schema in your repository as data, send it with every request, and validate the response locally even when the provider promises strict enforcement. The verdict shape for the rubric case is small enough to read in one screen:

{
  "type": "object",
  "required": ["answer", "confidence", "citations", "follow_up_questions"],
  "properties": {
    "answer": { "type": "string" },
    "confidence": { "type": "number" },
    "citations": {
      "type": "array",
      "items": {
        "type": "object",
        "required": ["doc_id", "section", "anchor"],
        "properties": {
          "doc_id": { "type": "string" },
          "section": { "type": "string" },
          "anchor": { "type": "string" }
        }
      }
    },
    "follow_up_questions": { "type": "array", "items": { "type": "string" } }
  }
}

Every citation field maps to metadata you already had before the model ran: the document ID of the rubric revision, the section heading, the URL anchor your reviewer will click. So the check after the call is a set membership test, not a judgement call — if a returned anchor isn't in the set you retrieved, you reject the verdict and re-ask with a tighter evidence block. Embeddings decide which rubric lines get in front of the model; the chat completion only decides what to say about the evidence it was handed. Keeping those two jobs separate is what lets you swap either one without touching the other.

None of that is language-specific, which is rather the point: the Node.js service the app team owns and the Go worker underneath it validate against the same schema file, reject the same unanchored citations, and neither one imports a vendor package to do it.

One more habit that survives migrations: key your verdict cache on a hash of the rubric revision, the criterion and the submission, so a retried batch reuses stored verdicts instead of paying for them twice.

Wiring the retrieval and the model call in Go

Roughly seventy lines, no framework, no client library. The base URL and key arrive as configuration because that's the entire portability story in practice.

package main

import (
    "bytes"
    "encoding/json"
    "errors"
    "fmt"
    "io"
    "log"
    "math"
    "net/http"
    "os"
    "strconv"
    "time"
)

var (
    baseURL = envOr("SCORER_BASE_URL", "https://api.infrai.cc/v1")
    apiKey  = os.Getenv("SCORER_API_KEY") // export SCORER_API_KEY="$INFRAI_API_KEY"
)

type Chunk struct{ DocID, Section, Anchor, Text string }

type Citation struct {
    DocID   string `json:"doc_id"`
    Section string `json:"section"`
    Anchor  string `json:"anchor"`
}

type Verdict struct {
    Answer     string     `json:"answer"`
    Confidence float64    `json:"confidence"`
    Citations  []Citation `json:"citations"`
    FollowUps  []string   `json:"follow_up_questions"`
}

func main() {
    rubric := []Chunk{
        {"rubric-2026-eng", "Debugging depth", "#debugging-depth",
            "A strong candidate reproduces the reported behaviour before editing code and names the hypothesis under test."},
        {"rubric-2026-eng", "Operational judgement", "#operational-judgement",
            "A strong candidate describes rollback and monitoring for the change, not the change alone."},
    }
    submission := "I added a retry loop, shipped it behind a flag, and watched the error rate for an hour before calling it done."

    texts := []string{submission}
    for _, c := range rubric {
        texts = append(texts, c.Text)
    }
    vecs, err := embed(texts)
    if err != nil {
        log.Fatal(err)
    }

    best, top := 0, -1.0
    for i := range rubric {
        if s := cosine(vecs[0], vecs[i+1]); s > top {
            best, top = i, s
        }
    }

    v, err := score(rubric[best], submission)
    if err != nil {
        log.Fatal(err)
    }
    for _, c := range v.Citations {
        if c.Anchor != rubric[best].Anchor {
            log.Fatalf("citation outside retrieved evidence: %+v", c)
        }
    }
    fmt.Printf("criterion=%s similarity=%.3f confidence=%.2f\n%s\n",
        rubric[best].Section, top, v.Confidence, v.Answer)
}

func embed(input []string) ([][]float64, error) {
    var out struct {
        Data []struct {
            Embedding []float64 `json:"embedding"`
        } `json:"data"`
    }
    err := call("POST", baseURL+"/embeddings", map[string]any{
        "model": "text-embedding-v4",
        "input": input,
    }, &out)
    if err != nil {
        return nil, err
    }
    vecs := make([][]float64, len(out.Data))
    for i, d := range out.Data {
        vecs[i] = d.Embedding
    }
    return vecs, nil
}

func score(c Chunk, submission string) (Verdict, error) {
    schema := map[string]any{
        "type":     "object",
        "required": []string{"answer", "confidence", "citations", "follow_up_questions"},
        "properties": map[string]any{
            "answer":     map[string]any{"type": "string"},
            "confidence": map[string]any{"type": "number"},
            "citations": map[string]any{
                "type": "array",
                "items": map[string]any{
                    "type":     "object",
                    "required": []string{"doc_id", "section", "anchor"},
                    "properties": map[string]any{
                        "doc_id":  map[string]any{"type": "string"},
                        "section": map[string]any{"type": "string"},
                        "anchor":  map[string]any{"type": "string"},
                    },
                },
            },
            "follow_up_questions": map[string]any{
                "type":  "array",
                "items": map[string]any{"type": "string"},
            },
        },
    }
    evidence := fmt.Sprintf("doc_id=%s section=%s anchor=%s\n%s",
        c.DocID, c.Section, c.Anchor, c.Text)

    var out struct {
        Choices []struct {
            Message struct {
                Content string `json:"content"`
            } `json:"message"`
        } `json:"choices"`
    }
    err := call("POST", baseURL+"/chat/completions", map[string]any{
        "model":       "qwen3.7-plus",
        "temperature": 0,
        "messages": []map[string]string{
            {"role": "system", "content": "Score the submission against the rubric evidence supplied. Cite only that evidence."},
            {"role": "user", "content": "Rubric evidence:\n" + evidence + "\n\nSubmission:\n" + submission},
        },
        "response_format": map[string]any{
            "type": "json_schema",
            "json_schema": map[string]any{
                "name":   "rubric_verdict",
                "strict": true,
                "schema": schema,
            },
        },
    }, &out)
    if err != nil {
        return Verdict{}, err
    }
    if len(out.Choices) == 0 {
        return Verdict{}, errors.New("no choices in response")
    }
    var v Verdict
    return v, json.Unmarshal([]byte(out.Choices[0].Message.Content), &v)
}

func call(method, url string, payload, out any) error {
    body, err := json.Marshal(payload)
    if err != nil {
        return err
    }
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(method, url, bytes.NewReader(body))
        if err != nil {
            return err
        }
        req.Header.Set("Authorization", "Bearer "+apiKey)
        req.Header.Set("Content-Type", "application/json")

        res, err := http.DefaultClient.Do(req)
        if err != nil {
            return err
        }
        raw, _ := io.ReadAll(res.Body)
        res.Body.Close()

        if res.StatusCode == http.StatusTooManyRequests {
            time.Sleep(backoff(res.Header.Get("Retry-After"), attempt))
            continue
        }
        if res.StatusCode >= 400 {
            return fmt.Errorf("%s %s: status %d: %s", method, url, res.StatusCode, string(raw))
        }
        return json.Unmarshal(raw, out)
    }
    return errors.New("rate limited on 4 consecutive attempts")
}

func backoff(retryAfter string, attempt int) time.Duration {
    if secs, err := strconv.Atoi(retryAfter); err == nil && secs > 0 {
        return time.Duration(secs) * time.Second
    }
    return time.Duration(math.Pow(2, float64(attempt))) * time.Second
}

func cosine(a, b []float64) float64 {
    var dot, na, nb float64
    for i := range a {
        dot += a[i] * b[i]
        na += a[i] * a[i]
        nb += b[i] * b[i]
    }
    return dot / (math.Sqrt(na) * math.Sqrt(nb))
}

func envOr(k, fallback string) string {
    if v := os.Getenv(k); v != "" {
        return v
    }
    return fallback
}

The interesting line is the one that rejects a citation whose anchor wasn't retrieved. Everything above it is plumbing you can move; that check is the part that makes the output auditable no matter who serves the model.

Swapping providers is then a shell exercise, which is exactly the property you were buying:

SCORER_BASE_URL=https://api.infrai.cc/v1 SCORER_API_KEY="$INFRAI_API_KEY" go run ./cmd/scorer
SCORER_BASE_URL=https://api.openai.com/v1 SCORER_API_KEY="$OPENAI_API_KEY" go run ./cmd/scorer

Buy, build, or rent: how the options actually differ

Option	How you call it	Cost of leaving	Fits when	Main limit
Ollama on your own boxes	Local HTTP, your own index	Low code cost, high ops cost	Submissions can't leave your network	You own capacity, upgrades and the on-call page
OpenAI direct	Official SDKs or `/v1/chat/completions`	Low if you stayed on plain HTTP	You want the strictest structured output support	Single vendor for both retrieval and generation
Azure OpenAI or Bedrock	Cloud-specific auth and resource naming	High — auth, naming and quotas are not portable	Procurement or residency rules decide for you	Regional model availability lags
OpenRouter	One key across many upstream models	Low	You're still choosing a model	Behaviour varies with whichever upstream serves you
Infrai	Same OpenAI-compatible paths, different base URL	Low	You want one contract over several backend jobs	Small vendor; check the readiness of any capability you depend on

Two rows deserve a note rather than a cell. Bedrock and Azure OpenAI are the right answer when a compliance program, not an engineer, is making the choice — the auth and resource model is genuinely harder to leave, and that's the price of the paperwork they solve. Infrai earns its row for a different reason: the routing across upstream vendors sits behind one consistent interface, so moving the scorer from one underlying model vendor to another is a field in the request body instead of a second integration, and the same key covers the vector and batch endpoints the pipeline needs around the scoring call.

If you're a small platform team that already owns retrieval and wants the model provider to be a config value, Infrai is worth a trial run for the scoring call itself, on the strength of that plain-HTTP surface and the single contract around it.

Test the swap on a schedule, and rehearse the rollback

Start with a smoke test that says nothing about quality and everything about whether the contract still holds:

curl -s -X POST https://api.infrai.cc/v1/chat/completions \
  -H "Authorization: Bearer $INFRAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen3.7-plus","messages":[{"role":"user","content":"reply with the word ok"}]}'

Then the real check. Keep fifty submissions that humans have already scored, with the rubric revision pinned. Run them against the candidate provider on a schedule, not on the day you're forced to move, and compare three numbers against the incumbent: first-attempt schema validity, citation anchor match rate, and agreement with the human score inside one band. A provider that holds the first two and loses the third is a prompt problem. One that loses the first two is a contract problem, and no prompt will fix it.

Rollback should be boring: previous base URL, previous key, previous model id, rerun the golden set, done. Keep both configs live for one hiring cycle so the rollback path is exercised rather than theoretical.

The catch, and I'd rather say it plainly: none of this makes a small provider equivalent to a hyperscaler for every job. If your legal team needs a named data-residency commitment or a specific contractual instrument, stick with Azure OpenAI or Bedrock and accept the higher exit cost. If you need a dedicated content moderation endpoint, Infrai doesn't offer one — you fold that check into a chat call with its own JSON schema, which is fine for a rubric pipeline and probably not fine for a public user-generated content firehose. And if retrieval quality is your bottleneck rather than portability, spend the week on chunking and a reranker instead; Cohere's rerank documentation is a better use of your afternoon than any provider comparison, including this one. If the boundary I've described fits your system, the retry and error semantics at https://docs.infrai.cc/errors are the next thing to read, because that's what your worker's backoff code has to agree with.

References

Cohere Rerank documentation — https://docs.cohere.com/docs/rerank-overview
OpenAI structured outputs guide — https://platform.openai.com/docs/guides/structured-outputs
Ollama API reference — https://github.com/ollama/ollama/blob/main/docs/api.md
Amazon Bedrock user guide — https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html
Prompt Engineering Guide — https://www.promptingguide.ai
Infrai error code reference — https://docs.infrai.cc/errors

A Production Runbook for Marketing Image APIs: Posters, Social Ads, and Style Drift

finnmorgan226 — Fri, 07 Aug 2026 22:41:50 +0000

Short answer: choose a text-to-image API only after it passes a repeatable test built from your own marketing briefs, target poster and social-ad dimensions, style rules, upscale path, and campaign traffic shape. The useful output is not the prettiest sample. It is a verified, publishable asset delivered inside an end-to-end SLO, with a rollback target already proved against the same corpus.

Don't let a gallery make this decision.

How should a marketing app test text-to-image API resolution and style control?

Start with a frozen evaluation corpus, not an open-ended prompt contest. Each case should describe the intended subject, composition, palette, negative space, placement, crop, and final pixel dimensions. Include the awkward work that exposes weak controls: a product beside reflective packaging, a subject that must remain clear after a vertical crop, a poster with a quiet region reserved for copy, and a campaign family that needs to look related across several aspect ratios. Strip customer data from this corpus. If an approved workflow could contain regulated information, classification and access controls belong before evaluation, not after a provider has been selected; for US healthcare data, the security and privacy requirements in 45 CFR Part 164 are a concrete review boundary, not a marketing checkbox.

Generate several candidates per case because one lucky output says little about repeatability. Preserve the request envelope, model or capability identifier when exposed, timestamps, original bytes, transformed bytes, reviewer result, and rejection reason. Blind the review so provider labels cannot influence it. A reviewer should grade separate properties: adherence to the brief, composition, visual defects, brand-style consistency, crop survival, and editability. "High quality" is otherwise too vague to operate. It can mean attractive at first glance, faithful to a product reference, clean at final size, or fast enough for an approval session; those are different indicators and they fail for different reasons.

Resolution needs the same discipline. Requested dimensions are an input. Decoded dimensions are evidence. Usable resolution is a reviewer judgment made after the exact resize, crop, and compression chain that production will apply. An upscale step belongs in its own test cell, paired with the non-upscaled output, because enlargement can change edges and texture without fixing composition or brief adherence. Render the final ad with deterministic typography and brand assets outside the generated image unless the experiment is specifically measuring embedded text. This keeps spelling, legal copy, hierarchy, and logo placement under application control.

Use explicit denominators. Brief acceptance rate is accepted generations divided by completed generations. Workflow completion rate is verified publishable assets divided by submitted jobs. Deadline attainment is publishable assets ready before the internal cutoff divided by submitted jobs. Track p50, p95, and p99 time to a verified asset, but set the objective on the percentile that reflects the real approval workflow. A median hides the long queue that wakes the on-call engineer during a campaign launch.

I would also require a minimum sample count per creative class before treating a difference as meaningful. I'm not sure a universal sample size exists here; variance depends on prompt breadth, reviewer agreement, and how close candidates are to the acceptance boundary. Record those three inputs and expand the sample until the decision stops moving materially. Your mileage may vary. The uncertainty is useful because it tells the team what another evaluation run could resolve.

Make "ready" an observable state

An API response is one step in a media pipeline. The application still has to decode the object, verify its type and dimensions, apply deterministic transforms, store it, attach policy and provenance metadata, and make the resulting asset readable by the publisher. Define ready at that boundary. A 200 from generation does not prove that the campaign can retrieve the final object, and a successful upscale job does not prove that the result survived the placement crop.

Keep the provider boundary narrow — boring is good here — and translate business intent through an adapter. The internal request should carry the brief, target geometry, style reference identifier, and an idempotency key owned by the application. Vendor-specific options stay inside the adapter rather than leaking into every workflow. If routing decisions are produced by a model through structured tool calls, validate the chosen function and every argument in application code before dispatch; the function-calling guide in Further reading documents that separation between model output and application-side execution.

The following Go boundary validates bytes before a workflow may move to ready. It deliberately avoids any commercial route or SDK type.

package creative

import (
    "bytes"
    "context"
    "crypto/sha256"
    "encoding/hex"
    "errors"
    "fmt"
    "image"
    _ "image/jpeg"
    _ "image/png"
    "io"
)

const maxImageBytes = 40 << 20

type Request struct {
    Brief          string
    Width          int
    Height         int
    StyleRef       string
    IdempotencyKey string
}

type Generator interface {
    Generate(context.Context, Request) (io.ReadCloser, error)
}

type Asset struct {
    Bytes  []byte
    Format string
    Width  int
    Height int
    SHA256 string
}

func GenerateAndVerify(ctx context.Context, g Generator, req Request) (Asset, error) {
    if req.Brief == "" || req.Width <= 0 || req.Height <= 0 || req.IdempotencyKey == "" {
        return Asset{}, errors.New("incomplete generation request")
    }

    body, err := g.Generate(ctx, req)
    if err != nil {
        return Asset{}, fmt.Errorf("generate image: %w", err)
    }
    defer body.Close()

    data, err := io.ReadAll(io.LimitReader(body, maxImageBytes+1))
    if err != nil {
        return Asset{}, fmt.Errorf("read image: %w", err)
    }
    if len(data) > maxImageBytes {
        return Asset{}, errors.New("image exceeds size limit")
    }

    cfg, format, err := image.DecodeConfig(bytes.NewReader(data))
    if err != nil {
        return Asset{}, fmt.Errorf("decode image: %w", err)
    }
    if format != "jpeg" && format != "png" {
        return Asset{}, fmt.Errorf("unsupported image format %q", format)
    }
    if cfg.Width != req.Width || cfg.Height != req.Height {
        return Asset{}, fmt.Errorf(
            "dimensions %dx%d; want %dx%d",
            cfg.Width, cfg.Height, req.Width, req.Height,
        )
    }

    sum := sha256.Sum256(data)
    return Asset{
        Bytes: data, Format: format, Width: cfg.Width, Height: cfg.Height,
        SHA256: hex.EncodeToString(sum[:]),
    }, nil
}

The byte limit is application policy, so set it from measured assets and memory budgets rather than copying 40 MiB blindly. Production code should then write the verified bytes to durable storage, read the object back through the same identity used by the publishing stage, and atomically record the checksum, dimensions, transform version, and storage location. Only that record may advance the job. Keep generated artwork and overlaid copy as separate lineage inputs; when legal text changes, the system can render a new final asset without asking a probabilistic generator to recreate the scene.

Retries need a state machine, not a loop around Generate. Distinguish a request that is safe to retry from a transform or publication step whose side effect may already exist. Use the application idempotency key across attempts, bound the attempt count, obey dependency retry guidance, and add jitter where appropriate. Reconciliation should scan for contradictions such as a job marked ready with no readable object, or a stored object whose checksum differs from the publication record. Those are user-visible workflow failures even if every individual request appeared healthy.

Small point, big consequence.

Capacity-plan the operating model

Run the quality corpus through a load profile that resembles campaign work: bounded bursts, concurrent variants, human review pauses, rejected candidates, and upscale tasks. Average requests per second won't reveal a queue that grows faster than workers can drain it. Measure queue age, active jobs, attempt count, dependency latency, bytes held in memory, transform CPU, storage-write duration, acceptance rate, and end-to-end completion time. Then raise offered load until the deadline SLO breaks, identify the actual bottleneck, and establish headroom from that observed limit. Do this independently for generation and upscale; they may have different concurrency and resource constraints.

The operating model determines which team owns that headroom. There is no default winner.

Model	Platform team owns	Suitable when	The catch is
Managed API	Evaluation, adapter, queue, asset verification, policy integration, and dependency budgets	Demand varies and the team wants to avoid operating model serving	External quotas, policy changes, and capability lifecycle remain dependencies
Self-hosted inference	All application controls plus accelerators, serving, scaling, artifacts, safety operations, and upgrades	Utilization is sustained and the organization can staff inference operations	Spare capacity, rollouts, and incident response become on-call work
Split path	Two adapters, routing policy, cross-path evaluation, and reconciliation	A measured continuity objective justifies independent paths	Test combinations and operational surface grow quickly

This is a buy-versus-build review with an SLO attached. A managed path is not suitable when its data boundary, deployment isolation, required controls, or dependency objective cannot satisfy policy. Self-hosting is not suitable when accelerator capacity and model-serving incidents have no durable owner. Stick with one narrow managed adapter when traffic is uneven and dependency risk fits the error budget; consider self-hosting only when measured utilization, control requirements, staffing, and upgrade ownership make the extra system rational. A split design earns its keep only if the continuity objective is valuable enough to fund two integrations and continuous equivalence testing.

Cost belongs in the same worksheet, but inference charges alone are a misleading numerator. Include rejected generations, upscale work, storage, egress, accelerator idle capacity, reviewer time, evaluation runs, integration maintenance, and on-call ownership. Use your observed acceptance rate and traffic distribution in the model. Don't publish a universal cost ranking from a synthetic prompt set; commercial terms move, while labor and rejection costs vary by workflow.

Before approval, write the decision as assumptions rather than adjectives: expected submissions per campaign, burst concurrency, final dimensions, acceptance target, deadline percentile, maximum queue age, data classification, recovery objective, owner, and review date. That record makes later reconsideration possible without reopening a debate about which demo looked better.

Verify releases and rehearse rollback

Treat a change to the model, provider adapter, prompt template, style reference, safety policy, crop, encoder, or upscale stage as a release. Run the frozen corpus first, compare outputs through blinded review, and allow a bounded canary only after the offline gate passes. The canary dashboard must terminate at the verified publication asset. Generation latency is diagnostic; time to readable, correctly sized, approved creative is the service indicator.

Predeclare stop conditions. Roll back when the canary spends the error budget for time to verified asset, acceptance drops below the approved baseline, queue age breaches its limit, or reconciliation finds an invalid ready state. Route new jobs to the last accepted configuration through versioned configuration, while in-flight work follows a documented finish-or-expire rule. Never make rollback depend on editing code during the incident.

Run the rollback before launch. Verify that the prior configuration still has capacity, that its credentials and model reference remain valid, that old and new request envelopes can be read, and that replay cannot publish twice. Preserve requests, checksums, transform versions, reviewer outcomes, and state transitions long enough to separate creative variance from adapter, crop, storage, or publishing changes during review. If the alternate path cannot meet the same final-asset gate, it is not a rollback target; it is another experiment.

The final selection should remain conditional. Choose the API and operating model that pass this workload's quality, resolution, style-control, upscale, data, capacity, and recovery gates, then rerun the decision when the briefs, placements, traffic shape, policy boundary, or staffing model changes.

Feature flags for Node.js and Next.js: percentage rollout and basic targeting

finnmorgan226 — Wed, 05 Aug 2026 12:05:19 +0000

Bottom line: for a kill switch, a percentage rollout, and basic targeting inside a Node.js or Next.js app, a cheap flags API over plain HTTP does the job, and LaunchDarkly is an expensive way to buy those three things. The moment you need an audit trail on every flag change, per-variant evaluation metrics, or dependencies between flags, the alternative stops being an alternative and you go back to the incumbent.

I run a platform team. Buy-vs-build is most of my job, and I've been wrong about it in both directions.

The month our flag telemetry outran the flags themselves

Two quarters ago we put a new checkout behind a flag and rolled it to 10% of traffic. Fine so far. Then somebody on my team wanted to answer the obvious question — which cohort actually saw the new flow — and tagged every custom metric with the flag key plus the served variant, which promptly multiplied against user tier, region, and payment method. Active series went from roughly 400 to a little over 52,000. Our Datadog custom-metrics line came in at $2,340 that month against the $180 I'd planned for, and I heard about it from finance rather than from any dashboard I own. I ended up ripping the tags back out the same week, which cost us the cohort answer entirely.

Flag evaluation is cheap. Observing flag evaluation is not.

That's the invariant, and it's the part nobody puts in the comparison table. Sentry's flag integration attaches the last handful of evaluations to an error event instead of emitting one event per render, and the OpenTelemetry semantic conventions for feature flags push in the same direction — hang the evaluation off a signal you were already paying to collect. PostHog makes the opposite trade deliberately, bundling flags with product analytics, and that's a perfectly good trade if you actually wanted the analytics. I hadn't decided to buy analytics. I was just paying for them.

Is a cheap LaunchDarkly alternative enough for percentage rollout and basic targeting?

For hiding unfinished work, canarying a release, and killing a bad deploy: yes, comfortably. Those three jobs need a boolean, a percentage, and a sticky bucketing unit so one user doesn't flip between variants on every request. Everything past that is where the money goes, and most teams evaluating a switch are paying for the "past that" without using it.

Option	How you call it	What you get	Where it stops
LaunchDarkly	SDK per runtime, streaming updates	audit log, experiments, flag dependencies, approvals	operational weight and cost for three toggles
Unleash (self-hosted)	SDK plus a service you run	full control, open source, no per-seat maths	you own the uptime, upgrades and on-call rotation
PostHog	SDK, flags wired into product analytics	flags and experiment readouts in one place	analytics ingest you may not want to fund
Flagsmith	SDK or REST, hosted or self-hosted	reasonable middle ground on both axes	smaller ecosystem, fewer runtime integrations
Infrai	plain REST, one key	flags next to logs, metrics and error capture	no change audit, no evaluation stats, clients poll

Infrai landed on my shortlist for a boring reason: the same key already covered the logs, metrics and error capture I'd otherwise buy separately, so adding a flag was one more endpoint against a contract my services had already wired up — not one more vendor, one more SDK, one more on-call runbook, one more invoice to reconcile. Across 295 routes and 20 modules the conventions hold still, which is worth more to me than any single module being best in class. Flags are a small corner of that surface, and I'd rather they were a small corner of something than a whole vendor relationship.

The rollout call, and why it carries an idempotency key

Our app is Next.js on Node 22.14; the thing that actually moves the percentage is a small Go binary that runs in the deploy pipeline, because I want the rollout change to live in the same commit as the deploy that needs it. POST /v1/flags/rollout/{key} is the write and GET /v1/flags/is_enabled/{key} is the read, which a Next.js route handler or server component can call directly over plain HTTP.

One detail matters more than the endpoints themselves. The write carries a client-supplied idempotency key, so a retry after a network blip re-applies the same 10% instead of stacking a second rollout on top of the first. Deploy pipelines retry — mine has retried a step at exactly the wrong moment more than once, usually at 17:40 on a Friday — and a rollout call without an idempotency key is a rollout call that can double-apply. Ask that question of whichever platform you land on, because at-least-once delivery is the working assumption everywhere else in your stack and flags are not special.

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

const base = "https://api.infrai.cc/v1"

// call issues one authenticated request and backs off on 429 instead of
// hammering the endpoint. idem is the client-supplied key that keeps a
// retried write from applying twice.
func call(method, path string, payload []byte, idem string) ([]byte, error) {
    for attempt := 0; ; attempt++ {
        req, err := http.NewRequest(method, base+path, bytes.NewReader(payload))
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
        req.Header.Set("Content-Type", "application/json")
        if idem != "" {
            req.Header.Set("Idempotency-Key", idem)
        }

        res, err := http.DefaultClient.Do(req)
        if err != nil {
            return nil, err
        }
        body, _ := io.ReadAll(res.Body)
        res.Body.Close()

        if res.StatusCode == http.StatusTooManyRequests && attempt < 3 {
            wait := time.Duration(1<<attempt) * time.Second
            if v, err := strconv.Atoi(res.Header.Get("Retry-After")); err == nil && v > 0 {
                wait = time.Duration(v) * time.Second
            }
            time.Sleep(wait)
            continue
        }
        if res.StatusCode < 200 || res.StatusCode > 299 {
            return nil, fmt.Errorf("%s %s -> %d: %s", method, path, res.StatusCode, body)
        }
        return body, nil
    }
}

func main() {
    plan, err := json.Marshal(map[string]any{
        "percentage": 10,
    })
    if err != nil {
        panic(err)
    }

    // One idempotency key per rollout step, reused on every retry, so a blip
    // during deploy cannot push checkout-v2 to 20 percent.
    if _, err := call("POST", "/flags/rollout/checkout-v2", plan, "deploy-4711-checkout-v2-10"); err != nil {
        panic(err)
    }

    state, err := call("GET", "/flags/is_enabled/checkout-v2", nil, "")
    if err != nil {
        panic(err)
    }
    fmt.Printf("checkout-v2: %s\n", state)
}

Before you trust any canary, find out what the percentage is bucketed on. Per-session or per-device bucketing looks equivalent to per-user on a dashboard, and then one person gets the old checkout on their phone and the new one on their laptop, which generates a support ticket you'll spend an afternoon on.

Where the cheap option stops being the right one

The catch is change management. There's no per-change audit trail, so "who moved checkout to 50% at 23:40 and why" has to be answered by your deploy log instead of by the flag service. That's survivable when flag changes ship through CI, and it doesn't support the case where a compliance auditor wants the flag system itself to be the record. If you're in a regulated shop, stick with LaunchDarkly and stop reading comparison posts like this one.

Two more edges, both about latency budgets. Clients poll rather than stream, so a kill switch reaches every open browser tab one poll interval later — we poll every 30 seconds and keep every money-touching check server-side, which means the browser copy is a rendering hint and never an authorisation decision. If your SLO says a bad flag must be dark everywhere within five seconds, streaming updates are worth paying for and this whole category is the wrong shape. And there are no per-variant evaluation stats, so measuring whether variant B converted better is a job for your analytics stack, not the flag API.

I'm not sure how much of the incumbent's price tag is the streaming transport and how much is the compliance surface — as far as I can tell it's mostly the latter, so teams buying it for percentage rollouts are funding an audit capability they never open. Your mileage may vary. Mine went the other way twice.

Deleting a flag is permanent, with no recycle bin, so we prefix each key with its ticket id and leave it toggled off for a sprint before anyone removes it. Cheap discipline. It has saved me twice.

References

Infrai capability sheet (llms.txt): https://docs.infrai.cc/llms.txt
LaunchDarkly documentation: https://docs.launchdarkly.com/
Unleash documentation: https://docs.getunleash.io/
OpenFeature specification: https://openfeature.dev/specification/
PostHog feature flags: https://posthog.com/docs/feature-flags
OpenTelemetry logs signal concepts: https://opentelemetry.io/docs/concepts/signals/logs/
Datadog custom metrics billing: https://docs.datadoghq.com/account_management/billing/custom_metrics/

A Simple Feature Flag Kill Switch API for Node.js Production Incident Rollback

finnmorgan226 — Sun, 02 Aug 2026 22:28:00 +0000

If you just want the recommendation: Short answer: use a dedicated feature flag as a production kill switch when rollback must be immediate, but treat flag evaluation, incident automation, ownership, and evidence as separate design problems. For a Node.js service, check that flag immediately before the risky integration or background job, then take a deliberately boring safe path when it is off. A flag is a circuit breaker operated by people or by automation you own; it is not an incident-management system.

I would make this a buy decision before a build decision. The control plane looks small until the platform team owns propagation, authentication, stale reads, auditability, and the 03:00 escalation path. My SLO question is blunt: can the application reach a known-safe state inside the rollback budget even if the deploy pipeline is busy? If yes, a kill switch earns its keep.

How should a Node.js production incident use a feature flag kill switch for rollback?

Put the check at the last responsible moment. If payments.partner_v2.kill protects a partner call, evaluate it just before that call, not once during process startup and not five stack frames above the behavior it governs. When the switch is active, return a cached result, enqueue work for later, or disable the optional path; pick one safe behavior in advance and test it. The naming should identify the capability, owner, and destructive meaning without forcing an on-call engineer to remember whether true means safe or risky.

This is the invariant I want: one control-plane change must stop new exposure without requiring a new artifact, while the data plane retains an explicit fallback. In Node.js, I usually wrap the remote evaluation behind a tiny application interface and keep provider-specific HTTP outside business logic. The wrapper also owns a short timeout and a written fail-open or fail-closed policy. A recommendation engine can fail open; an unbounded export job may need to fail closed. There is no universal default, and I'm not sure teams appreciate that distinction until an incident forces it.

Fast is useful. Predictable wins.

Test the fallback.

The flag should be dedicated to emergency control rather than shared with a percentage rollout. Rollout intent changes gradually; a kill switch has a binary operational meaning and a named responder. Write the runbook so the operator knows the safe value, the expected user effect, the verification signal, and the person authorized to restore service. If the provider only supports client polling, include the polling interval in the rollback objective: a 60-second refresh cannot honestly support a 20-second mitigation target. Your mileage may vary with process count and cache behavior, so test propagation under realistic fleet conditions rather than reading a marketing adjective as an SLO.

The config footgun that changed my rollback checklist

I learned this during one bounded incident involving an internal flag gateway, not an application defect. A worker fleet had FLAG_REGION=us-east1, while the gateway expected us-east-1; the auth proxy answered 401, our wrapper interpreted the evaluation as unavailable, and 26 workers kept using their locally cached value for 37 minutes. The dashboard showed successful worker requests because the cached decision let processing continue, while the gateway panel showed rejected evaluations, so neither view alone explained why the switch appeared to move but the risky traffic did not. I checked token age, deployment order, and proxy policy before comparing the rendered environment against the gateway's region list character by character. The header was structurally correct. The credential was current. The missing hyphen sat in deployment configuration, far away from the code whose behavior we were trying to stop, and rolling the corrected environment finally made the fleet converge on the new decision. I wrote the exact invalid region and the observed cache state into the incident timeline, then added configuration validation at startup and a separate alert for evaluation-unavailable state. I'm still annoyed by how ordinary it was — one missing hyphen defeated an otherwise sensible rollback plan.

The lesson was not “cache nothing.” A remote control plane will sometimes be unreachable, and forcing every request through it can turn a mitigation tool into a dependency amplifier. The lesson was to make failure policy observable and intentional. We now separate four states in the adapter: enabled, disabled, evaluation unavailable, and configuration invalid. The application maps those states to a safe behavior; the monitoring layer counts them; the runbook says which one an operator should expect after a toggle. This is capacity planning in miniature, because the polling interval, fleet size, and synchronized refresh pattern determine control-plane load.

I also require a rollback drill before a risky launch. An engineer flips the switch in a non-production environment, confirms the risky call count reaches zero, confirms the fallback remains inside its latency objective, and restores the original value. For production, the responder records the incident timestamp and the flag action in the incident timeline. That evidence matters when the flag system has no native change history.

Keep the logs narrow. Flag key, evaluation state, request ID, and a non-sensitive reason are usually enough; don't put tokens or customer payloads into an emergency breadcrumb. OWASP's logging guidance is the useful baseline here, particularly because incident pressure makes over-logging feel temporarily reasonable. It rarely feels reasonable during the later security review.

Choosing the control plane without pretending every option is equivalent

I score flag systems against the operating model, not the demo. A platform team that needs delegated approvals and detailed change evidence has a different requirement from a small service team that wants a plain remote switch. These are real alternatives, and the shortlist should survive a buy-versus-build review.

Option	Operating model I would evaluate	Good shortlist condition	The catch to test
LaunchDarkly	Dedicated managed feature-management platform	The organization wants a specialized flag product and will standardize around it	Validate cost, SDK lifecycle, governance, and lock-in against fleet size
Unleash	Feature-management platform with a self-hosting path	Data location or control-plane ownership makes self-hosting important	The platform team accepts upgrades, storage, capacity, and on-call ownership
ConfigCat	Managed feature-flag service with client integration	Teams prefer a focused hosted product and its integration model fits their applications	Verify required audit, dependency, and incident-routing behavior before rollout
Infrai	Plain REST API within a broader backend API	A small team wants any language to call one HTTP interface without installing a flag SDK	No native flag alert routing, change audit history, evaluation statistics, or dependency graph
In-house service	Code, storage, propagation, and operations owned internally	Regulation or unusual semantics make vendor products unsuitable	The apparent small build creates a permanent reliability and security service

Infrai is interesting here for one concrete reason: it is a plain REST API, so there is no client library version to babysit and anything capable of an authenticated HTTP request can use it. That is useful in a mixed fleet where Node.js workers, Go control jobs, and shell-based incident tooling need the same narrow control surface. Its public discovery surface is self-describing, and the wider platform exposes 295 routes across 20 modules under one key, but breadth should not distract from the flag-specific limits.

I keep the incident-detection shortlist separate because a flag control plane does not replace observability. Datadog is a managed candidate when a team wants logs and operational monitoring under one commercial service; Grafana is the candidate I examine when dashboards and an existing metrics stack are central to the operating model; Sentry belongs in the review when application errors are the primary signal; Better Stack is another hosted option to assess for monitoring and response workflows. Those products do not become kill switches merely because they can tell an engineer that production is unhealthy. I compare their alert delivery, retention, ingestion model, access controls, and on-call fit in a second buy-versus-build table, then connect the chosen signal to the flag runbook or to automation we explicitly own. This separation prevents a dangerous procurement shortcut: buying a strong detector and assuming rollback now exists, or buying a convenient flag API and assuming detection came with it. The Datadog pricing model, for example, distinguishes log ingestion from indexing, which is exactly the sort of capacity-sensitive detail I forecast before sending verbose flag evaluations into a logging platform.

The catch is substantial. Infrai flags have no built-in notification routing, change audit history, evaluation statistics, parent-child dependencies, or recycle bin, and clients can only poll. It is not suitable when native incident alerts, approval evidence, or complex flag relationships are hard requirements; keep a dedicated platform such as LaunchDarkly, Unleash, or ConfigCat on the shortlist in those cases. Likewise, choose Unleash's self-hosting path over a managed control plane when owning the data plane is a requirement and your team has budgeted the operational load.

A minimal poller for the preventative path

The following Go program makes one explicit read against the verified kill-switch route, authenticates from an environment variable, honors Retry-After on 429, applies exponential backoff otherwise, and surfaces non-success bodies. It deliberately prints the response rather than guessing a response field that the API schema should define. In an application adapter, generate the typed response model from the public discovery schema, then map that model to the Node.js interface described earlier.

package main

import (
    "context"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

const endpoint = "https://api.infrai.cc/v1/flags/is_enabled/payments.partner_v2.kill"

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        fmt.Fprintln(os.Stderr, "INFRAI_API_KEY is required")
        os.Exit(2)
    }

    ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
    defer cancel()

    body, err := getWithBackoff(ctx, http.DefaultClient, key)
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    fmt.Println(string(body))
}

func getWithBackoff(ctx context.Context, client *http.Client, key string) ([]byte, error) {
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            return body, nil
        }
        if resp.StatusCode != http.StatusTooManyRequests {
            return nil, fmt.Errorf("flag query failed: status=%d body=%s", resp.StatusCode, body)
        }

        delay := time.Duration(1<<attempt) * time.Second
        if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
            delay = time.Duration(seconds) * time.Second
        }
        select {
        case <-time.After(delay):
        case <-ctx.Done():
            return nil, ctx.Err()
        }
    }
    return nil, fmt.Errorf("flag query remained rate limited after retries")
}

This sample is a diagnostic building block, not the request-path architecture. I would run polling in one adapter per process, add jitter, cache the last valid evaluation for a bounded period, and expose adapter states to the application's metrics system. Node.js business code should consume a local decision and never know the URL or bearer token. For a write or toggle, use the documented write route and an idempotency strategy from its discovery schema; don't improvise a path or retry a mutation blindly.

Where the kill-switch pattern stops helping

A feature flag cannot detect the incident it is meant to mitigate. Infrai has no native alerting or notification routing tied to flags, so automated rollback requires a poller or an incident workflow you build. Its observability surface also does not provide distributed trace queries or span trees, source-map decoding, crash symbolication, Session Replay, synthetic checks, or heartbeat monitoring. I would pair silent-job detection with a Healthchecks-style tool and keep trace analysis in the tracing system already accountable for that SLO.

That boundary changes the recommendation. If the requirement says “page the on-call engineer when error rate crosses a threshold, show the exact flag change, and automatically restore after approval,” buy a dedicated incident and flag workflow that natively supplies those controls. If the requirement says “give this service a small, language-neutral emergency switch and our existing incident automation will operate it,” a polled REST flag can be a clean fit. Manual control is acceptable only when the response objective and staffing model make it acceptable.

Recovery deserves the same design effort as shutdown. Define the evidence required to turn the risky path back on, restore it gradually if the underlying integration can be overloaded, and watch saturation plus user-visible errors during the recovery window. A kill switch without a restoration policy tends to become permanent configuration, after which nobody is sure whether the old path is safe to delete.

Ownership is the control.

My final gate is an ownership line in the service catalog: one team owns the flag, one runbook defines its safe value, and one test proves the fallback. No dependency graph or audit feature can manufacture that discipline. Products can preserve evidence and reduce toil, but the platform roadmap still has to fund the operational behavior around the switch.

References

Private AI images in Node.js: object storage and temporary signed download links

finnmorgan226 — Thu, 30 Jul 2026 21:35:41 +0000

Use object storage with short-lived presigned GET links when a Node.js app has to store AI-generated images and hand each user a temporary download link for private files, and reach for signed CDN cookies only once a single session pulls hundreds of images at a time. Public buckets, base64 blobs in Postgres, and proxy routes that stream bytes back through your own API all look simpler on day one, and all three turn into the thing you get paged about later.

I've been paged about all three.

My job is the platform roadmap, so I weigh managed storage against a self-hosted MinIO cluster on three axes that actually show up in a budget: egress, on-call hours, and how hard it would be to leave in eighteen months. The signing pattern below is the one I've carried through two of those migrations, and it survives because it barely touches the storage vendor.

How do I create temporary signed download links for private AI-generated images in Node.js?

The shape is always the same. Your Node service writes the object with a private ACL, records where it went, and — at the moment a user clicks download — asks the storage API for a URL that carries its own signature and dies on a timer. The browser then fetches that URL directly. Your API never sees the image bytes, which is the whole point: a 6 MB PNG streamed back through an Express route burns a request slot, a socket, and a slice of your p99 for nothing.

Sign at click time, not at write time.

Links you mint during generation sit in your database going stale, and every stale link is a support ticket. I keep the TTL at five minutes for a single download and fifteen for a gallery page, because the browser only needs the link long enough to start the transfer — an in-flight GET isn't cut off when the signature expires, though I've only checked that against S3 and R2, so your mileage may vary elsewhere.

Here's the signer I run as a sidecar. Our product code is Node; this piece is Go 1.21 because it's the one path where I want a single static binary and no dependency tree to audit.

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "net/url"
    "os"
    "strconv"
    "time"
)

const base = "https://api.infrai.cc/v1"

// The native envelope: ok / data / error / metadata.
type envelope struct {
    OK   bool `json:"ok"`
    Data struct {
        URL       string `json:"url"`
        Method    string `json:"method"`
        ExpiresAt string `json:"expires_at"`
    } `json:"data"`
    Error json.RawMessage `json:"error"`
}

func backoff(attempt int) time.Duration {
    return time.Duration(250*(1<<attempt)) * time.Millisecond
}

// presignGet returns a short-lived download URL for one private object.
// The URL comes back already signed, so it is fetched WITHOUT the platform key.
func presignGet(client *http.Client, bucket, key string, ttl time.Duration) (string, string, error) {
    payload, err := json.Marshal(map[string]any{
        "op":              "get",
        "expires_seconds": int(ttl.Seconds()),
    })
    if err != nil {
        return "", "", err
    }
    endpoint := fmt.Sprintf("%s/storage/object/presign/%s/%s",
        base, url.PathEscape(bucket), url.PathEscape(key))

    var lastErr error
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest("POST", endpoint, bytes.NewReader(payload))
        if err != nil {
            return "", "", err
        }
        req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
        req.Header.Set("Content-Type", "application/json")
        // Same key on every retry of the same request, so a retry never issues a second link.
        req.Header.Set("Idempotency-Key",
            "presign:"+bucket+":"+key+":"+strconv.Itoa(int(ttl.Seconds())))

        resp, err := client.Do(req)
        if err != nil {
            lastErr = err
            time.Sleep(backoff(attempt))
            continue
        }
        raw, _ := io.ReadAll(resp.Body)
        resp.Body.Close()

        if resp.StatusCode == http.StatusTooManyRequests {
            wait := backoff(attempt)
            if secs, convErr := strconv.Atoi(resp.Header.Get("Retry-After")); convErr == nil {
                wait = time.Duration(secs) * time.Second
            }
            lastErr = fmt.Errorf("rate limited: %s", raw)
            time.Sleep(wait)
            continue
        }
        if resp.StatusCode != http.StatusOK {
            return "", "", fmt.Errorf("presign %s: status %d: %s", key, resp.StatusCode, raw)
        }

        var env envelope
        if jsonErr := json.Unmarshal(raw, &env); jsonErr != nil {
            return "", "", jsonErr
        }
        return env.Data.URL, env.Data.ExpiresAt, nil
    }
    return "", "", fmt.Errorf("presign %s: giving up after 4 attempts: %w", key, lastErr)
}

func main() {
    client := &http.Client{Timeout: 10 * time.Second}
    link, expiresAt, err := presignGet(client, "renders-prod", "u/8412/job-7f3a.png", 5*time.Minute)
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    // Hand this straight to the browser. No Authorization header travels with it.
    fmt.Printf("%s (valid until %s)\n", link, expiresAt)
}

export INFRAI_API_KEY=ifr_your_key_here
go run ./cmd/imagelinks

That's it. The returned URL goes to the browser as-is, and the one rule people trip over is that you must not attach your platform key to it — the signature in the query string is the credential, and stacking a bearer token on top of it only muddies the request.

The record you keep matters more than the bucket you pick

Object stores are not databases, and the ones I've run don't let you search metadata server-side — list operations filter by prefix and nothing else. So the row you write when a render finishes is the only index you will ever have. Mine carries user id, job id, bucket, object key, mime type, byte size, and the model that produced it, and that last column has paid for itself twice: once when a vendor retired a model and we had to re-render roughly 40,000 images, and once when finance wanted per-model storage attribution and I could answer with a single query instead of a week of crawling. Key layout matters as much as the columns do. I use u/{user_id}/{job_id}.png, so a prefix list returns one user's whole library, and a lifecycle rule can expire a whole tier of accounts without touching the database. If you need the same bytes under a second prefix — a share copy, a thumbnail tree — use the store's server-side copy call instead of pulling the object down and pushing it back up through your app server, because that round trip costs you egress, ingress, and a worker slot to move bytes you already own.

Verify before you offer the button.

A head request — GET /v1/storage/object/head/{bucket}/{key} on an API-first store, headObject on any S3-compatible client — tells you the object exists and how large it is. That's enough to render an accurate download control with a real file size on it, rather than a link that 404s in the user's face because a cleanup job got there first.

Buy vs build, priced in on-call hours

The honest comparison isn't feature grids, it's who carries the pager. Every option below issues a temporary signed link in roughly the same number of lines; what separates them is what else you inherit when you adopt them, and how expensive the exit is once you have a few hundred million objects laid out in someone's key convention.

Option	How you issue a temporary link	What it costs you operationally	Where it stops fitting
Amazon S3	`getSignedUrl` in the AWS SDK	IAM review, egress modelling, CloudFront in front	Small teams drown in the IAM surface
Cloudflare R2	S3-compatible presign	Very little; no egress charge is the draw	Fewer regions, thinner lifecycle tooling
MinIO, self-hosted	S3-compatible presign	Disks, upgrades, quorum, your pager	Under ~50 TB it rarely pays for itself
Supabase Storage	`createSignedUrl`	Bundled with the rest of that platform	You're adopting a platform, not a bucket
Cloudinary	Signed delivery URLs	Transformations come included	Priced for media pipelines, not cold archives
Infrai	One presign call under the same key	One key and one bill across modules	Backend list is r2, s3, oss, cos — no gcs, no b2

Infrai is the one on that list I had to go read about rather than recognise, and the thing that made it worth a row is that its storage module behaves like the S3 contract everyone already knows: the ACL enum is private or signed-only, presign takes op and expires_seconds, and lifecycle expiry is expressed in whole days (the storage reference spells the enum out). Its discovery endpoint is public and needs no key, which is the part a platform team should care about, because I can diff the route and field surface on a schedule and find out about changes before an engineer does. For a team that's already buying queues and email from the same vendor, folding image storage into the same key removes an entire billing relationship. For a team whose only backend dependency is a bucket, that consolidation argument doesn't apply and S3 or R2 is the shorter path.

The bill I got wrong by 9x

Last quarter I modelled our image storage at about 240 USD a month: twelve terabytes at rest, plus what I assumed was a modest egress tail. The invoice landed at 2,140. It took me most of a day in the access logs to find the cause, and it was embarrassing. Our gallery component preloaded full-resolution originals for every tile on the page, twenty tiles per scroll, and each preload was a fresh signed GET straight against the bucket. Thumbnails existed. Nobody had wired them into the grid. So a casual scroll through one user's history dragged roughly 60 MB of egress that no human ever looked at, and because links were minted per render we also carried a 40x multiplier on presign calls that nobody had capacity-planned for. Two repairs, both boring: generate a 320px derivative at write time and point the grid at that, then cache the derivative's signed link for its full TTL instead of re-signing on every render. The line went back under 300 USD the following cycle. I'm still not sure why our synthetic checks never caught the egress curve — as far as I can tell the probe only ever loaded a single image detail page, which is the one route that behaved correctly all along.

The lesson I'd hand to anyone designing this: capacity-plan the link mint rate, not just the bytes.

Where each of these stops being the right answer

Presigned links are the wrong tool for anything you want indexed. There's no way to give a search engine a URL that expires in five minutes, so a public marketing gallery belongs behind a CDN with a genuinely public origin — and several API-first stores don't support public-read ACLs at all, which makes that decision for you before you get to argue about it.

Versioning is the gap I'd check before committing anything irreversible. If a re-render can overwrite a live object key and you need the previous copy back, you want a store with object versioning switched on; some of the newer API-first options lack it, and "we'll simply never reuse a key" is a policy that survives right up until the first backfill script. Same story for immutability: if your compliance people say WORM, stick with a vendor that publishes an object-lock guarantee, and don't let anyone talk you into emulating retention in application code.

Cross-region copies are a manual job on most of the lightweight options. The catch is that disaster recovery becomes something you own and schedule rather than a checkbox you tick.

Browser-to-bucket uploads deserve one last look before you commit, because CORS configuration is not uniformly self-service across these platforms, and an upload flow that assumes you can set your own origin rules can stall on exactly that. Routing the upload through your own service sidesteps the question for the price of one extra hop, which for AI-generated images is usually fine — you're already holding the bytes when the render completes.

Pick the store, then design the record. The link itself is the easy part.