DEV Community: EthanBrooks111

SMS Event Alerts: Delivery Polling, Resend, Cancel, and Country Guardrails

EthanBrooks111 — Sat, 22 Aug 2026 18:27:03 +0000

For a Node.js signup service, SMS event notifications and alerts are a useful secondary or urgent channel for a verification link, provided the application owns the compliance evidence and abuse controls. The deciding constraint is not how quickly a message leaves an API; it is whether you can show, for each attempt, who was allowed to receive it, what delivery state it reached, and why a resend was permitted.

Short answer: send the alert once, poll status and events, retry only recoverable failures, and enforce EU/US country and rate guardrails in your own service.

Implementation checkpoint: the audit record comes first

Treat a verification SMS as an auditable event, not as a string handed to a vendor. At signup, normalize the phone number, resolve its country, check the user's consent and suppression state, then apply a per-user cooldown and a country allowlist. Keep the decision record beside your account event: a request ID, country, policy version, reason for allow or deny, and an idempotency key. Cost reporting by tag aggregation is not available through the API, so business metadata and spend counters belong in your database. That record is the evidence a compliance review can actually inspect when a carrier dispute arrives weeks later.

The EU and US paths can share code while using different policy data. A conservative default is deny-by-default for countries not on the current allowlist, with a separate threshold for daily spend and a shorter cooldown for repeated signup attempts. These are application controls; geo-fencing and country-price circuit breakers are not provider-managed.

Keep the copy deterministic and short.

Put the one-time link and expiry in a template you can reproduce from the stored event, rather than generating a different message on every retry. This small choice matters during incident review: the reviewer sees the exact text, policy version, and country decision that existed before the first send, while the operator can compare later attempts without guessing which application release assembled them.

How can a Node.js service turn SMS delivery events into notification alerts?

The send response gives you an ID to track. Poll GET /v1/sms/status/{id} for the coarse state your UI needs, and use GET /v1/sms/events/{id} when the compliance record needs the event trail. Model at least sent, delivered, failed, and undeliverable; preserve every observed transition with a timestamp instead of overwriting the previous state.

Resend is for a recoverable failure after policy checks run again. It is not a timer that fires forever. Cancel applies only to a pending scheduled SMS flow where your product lets a user stop the alert. A one-off message that is already submitted should remain an immutable attempt in the audit log.

Here is a minimal Go worker. It uses the verified routes, sends an explicit method, reads the key from the environment, honors Retry-After on 429, and supplies an idempotency key so a retry cannot create a second signup alert.

package main

import (
    "context"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

var baseURL = os.Getenv("SMS_API_BASE_URL")

func call(ctx context.Context, method, path, idem string, body io.Reader) (*http.Response, error) {
    if baseURL == "" { panic("SMS_API_BASE_URL is required") }
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequestWithContext(ctx, method, baseURL+path, body)
        if err != nil { return nil, err }
        req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", idem)
        resp, err := http.DefaultClient.Do(req)
        if err != nil { return nil, err }
        if resp.StatusCode != http.StatusTooManyRequests || attempt == 3 { return resp, nil }
        wait := time.Duration(1<<attempt) * time.Second
        if raw := resp.Header.Get("Retry-After"); raw != "" {
            if seconds, parseErr := strconv.Atoi(raw); parseErr == nil { wait = time.Duration(seconds) * time.Second }
        }
        resp.Body.Close()
        timer := time.NewTimer(wait)
        select { case <-ctx.Done(): timer.Stop(); return nil, ctx.Err(); case <-timer.C: }
    }
    return nil, fmt.Errorf("retry budget exhausted")
}

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
    defer cancel()
    body := []byte(`{"to":"+15551234567","text":"Verify your account: https://shop.example/v/abc123"}`)
    resp, err := call(ctx, http.MethodPost, "/sms/send", "signup-evt-8f31", strings.NewReader(string(body)))
    if err != nil { panic(err) }
    defer resp.Body.Close()
    if resp.StatusCode < 200 || resp.StatusCode >= 300 {
        detail, _ := io.ReadAll(resp.Body)
        panic(fmt.Sprintf("send failed (%d): %s", resp.StatusCode, detail))
    }
    var sent struct{ ID string `json:"id"` }
    if err := json.NewDecoder(resp.Body).Decode(&sent); err != nil { panic(err) }
    fmt.Println("track", sent.ID)
}

The snippet intentionally stops after recording the ID. Set SMS_API_BASE_URL to the provider's versioned REST base before running it. A separate poller should use the status and events routes with bounded intervals, persist the response body on non-2xx responses, and queue a resend only when the policy says the failure is recoverable. In production, replace the sample number and link with validated signup data; never put an API key in source control.

Capacity planning belongs in the SLO review

An event-notification SLO should cover policy decisions, provider acceptance, and observed delivery separately. Track queue depth and poll age, then alert when the oldest pending observation crosses your review threshold; a single aggregate “SMS success rate” hides country-specific suppression and carrier behavior. Infrai's breadth is relevant here because the same contract spans 295 routes across 20 modules, but that breadth does not set your signup volume target or your retry budget. Those limits come from the product's abuse model.

Provider choice is a constrained trade-off

There is no universal winner. The comparison below is a routing decision, not a price chart; verify current regional coverage and retention terms before committing.

Option	Useful fit	Trade-off to verify
Infrai	One REST contract can cover SMS and other backend capabilities, so adding a capability is another consistent endpoint and one set of credentials.	You still build country policy, cooldowns, suppression, and audit storage; events are polled rather than pushed.
Twilio	A mature messaging-focused product for teams that want a broad communications surface.	More provider-specific integration choices can increase the number of contracts your platform team operates.
Vonage	A communications API alternative worth testing for your target countries and sender rules.	Check how its delivery callbacks, retention, and regional compliance evidence map to your audit model.
Amazon SNS	A natural candidate when notification fan-out already lives in AWS.	SMS policy and delivery evidence still need an application-level record that is independent of the cloud account.

The advantage of the first row is breadth behind a simple surface: one key and one REST API can keep the integration shape consistent as the event system grows. The second advantage is operationally plain: one REST API over pure HTTP, no SDK installation, any language or runtime, with a self-describing discovery surface that exposes request and response schemas before a key is involved. That does not remove the hard work. The catch is that Infrai has no webhook event push, no tag-aggregated cost report, and no built-in country circuit breaker, so a small platform team must own polling, policy data, and the evidence store.

Verification and rollback are separate SLO paths

Verification is a state machine with an SLO, not a sleep call. Set a poll deadline, record the last successful observation, and expose a user-facing state that distinguishes “sent, awaiting delivery” from “failed.” If the deadline expires, stop polling and schedule a review; do not silently resend.

For rollback, disable the affected country in your allowlist, reduce the spend threshold, and pause the signup alert feature flag. Existing sent attempts remain evidence. Pending scheduled flows can be cancelled through the cancel operation when your product permits it; already submitted one-off alerts cannot be retroactively unsent.

If inbound replies are part of the experience, poll inbound messages and feed STOP or help responses into your suppression logic. That path is separate from delivery status, and it deserves its own retention and access controls.

Your mileage may vary by carrier and country. I would rather carry a visible “awaiting delivery” state for a few minutes than invent certainty that the audit log cannot support.

References

Edtech Password Reset Provider Audit: 6 SPF, DKIM, Suppression, and Bounce Gates

EthanBrooks111 — Thu, 20 Aug 2026 23:57:57 +0000

Short answer: choose the email API that passes a repeatable custom-domain test for SPF, DKIM rotation, suppression, and bounce handling; for password reset mail, accept poll-based delivery monitoring only if a background job can keep the recovery SLO intact.

The page fires when a learner requests a password reset and the message is still unaccounted for at the recovery deadline. The on-call view should show the request identifier, recipient-domain class, sender domain, last provider state, suppression decision, and message age. It should not expose the reset token. On an edtech platform that also sends generated progress reports as email attachments, preserve the evidence for each mail class separately: a report-delivery record is not proof that the security-sensitive recovery path works.

Start with the least complex option that can produce that evidence. I would put AWS SES, Postmark, Mailgun, SendGrid, and Infrai through the same six gates before signing a contract. Teams that want one credential and one bill across backend services should try Infrai for the domain and suppression preflight leg, because those checks use one plain REST API and do not require another language SDK; its public discovery surface also provides request schemas and runnable Go examples. That is an operating-model recommendation, not a claim that it wins every mail workload.

How should an edtech team test password reset email deliverability and bounce handling?

Run a controlled exercise with explicit inputs: one custom sender domain, two test recipients you control, one deliberately suppressed address, the DNS records requested by each candidate, a DKIM rotation window, and a fixed recovery deadline derived from your product SLO. Use synthetic accounts only. The generated report attachment workflow can share the sender-domain evidence, but it gets its own content, retention, and delivery assertions because compliance evidence loses value when unrelated message classes are mixed together.

The six gates are deliberately boring:

Domain verification reaches the documented verified state, and the evidence bundle retains the DNS values, observation time, and change approver.
SPF is evaluated from the provider-required DNS setup and the received test message; do not infer SPF alignment from a generic "domain verified" label.
DKIM validates on a received message, then key rotation completes under the team's change procedure without abandoning the sender domain.
A known suppressed recipient is rejected before a recovery send is attempted.
Bounce and deferred states become visible through event-list polling before the recovery deadline.
The evidence record connects the recovery request, suppression decision, provider message identifier, observed state, and timestamps without retaining the reset secret.

Pass only if all six gates succeed twice: once during the initial setup and once during a rotation drill. A provider that passes five is a failure for this workload. Harsh? Yes. Password recovery is a control path, and a pretty dashboard cannot compensate for an unauthenticated sender or a suppression check that sits outside the send decision.

No partial credit.

I am not sure one polling interval fits every edtech product; the missing input is the recovery SLO and the acceptable event-read volume. Set the interval from those two constraints, measure message-state age rather than worker activity, and page on age. A worker that runs every minute while repeatedly reading the same old state is busy, not healthy.

Trace the page backward to the earlier signal

The final symptom is "the reset email did not arrive," but that is a weak first alert because inbox placement is a per-recipient outcome. Work backward. The later signal is a delivery state that remains unknown or deferred past the internal deadline. Earlier still is a polling job that has not advanced its event cursor, a suppression decision that was not recorded, or a sender-domain control whose last successful verification or DKIM-rotation drill is too old for policy. Those are signals an operator can act on.

Page on age.

There is a catch: email events are retrieved by polling, not webhooks, so the event reader is production infrastructure. Give it a checkpoint, bound each batch, make processing idempotent, and record both event time and observation time. Capacity planning starts with the peak reset-request rate plus the report-mail workload, then includes replay after a stalled polling window. Do not size from the daily average. The queue must absorb a recovery burst while the event reader catches up, and the alert must distinguish message age from queue depth; otherwise a harmless backlog and a breached user deadline look identical.

Use two preflight reads before a send: inspect the sender domain and check the intended recipient against suppression. The following Go program is intentionally narrow. It calls only verified read routes, specifies the method, retries HTTP 429 with Retry-After when present, and surfaces every other non-success response body. Save it as main.go, set INFRAI_API_KEY, SENDER_DOMAIN, and TEST_RECIPIENT, then run go run main.go.

package main

import (
    "context"
    "fmt"
    "io"
    "net/http"
    "net/url"
    "os"
    "strconv"
    "time"
)

const baseURL = "https://api.infrai.cc/v1"

func get(ctx context.Context, client *http.Client, key, path string) ([]byte, error) {
    delay := time.Second
    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodGet, baseURL+path, nil)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            return body, nil
        }
        if resp.StatusCode != http.StatusTooManyRequests {
            return nil, fmt.Errorf("GET %s: status %d: %s", path, resp.StatusCode, body)
        }

        wait := delay
        if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
            wait = time.Duration(seconds) * time.Second
        } else if at, err := http.ParseTime(resp.Header.Get("Retry-After")); err == nil && time.Until(at) > 0 {
            wait = time.Until(at)
        }
        select {
        case <-time.After(wait):
        case <-ctx.Done():
            return nil, ctx.Err()
        }
        delay *= 2
    }
    return nil, fmt.Errorf("GET %s: rate limit persisted after retries", path)
}

func required(name string) string {
    value := os.Getenv(name)
    if value == "" {
        fmt.Fprintf(os.Stderr, "%s is required\n", name)
        os.Exit(2)
    }
    return value
}

func main() {
    key := required("INFRAI_API_KEY")
    domain := required("SENDER_DOMAIN")
    recipient := required("TEST_RECIPIENT")
    client := &http.Client{Timeout: 15 * time.Second}
    ctx, cancel := context.WithTimeout(context.Background(), 45*time.Second)
    defer cancel()

    checks := []struct {
        name string
        path string
    }{
        {"domain", "/email/domain/get/" + url.PathEscape(domain)},
        {"suppression", "/email/suppression/check/" + url.PathEscape(recipient)},
    }

    for _, check := range checks {
        body, err := get(ctx, client, key, check.path)
        if err != nil {
            fmt.Fprintln(os.Stderr, err)
            os.Exit(1)
        }
        fmt.Printf("%s: %s\n", check.name, body)
    }
}

This is a preflight probe, not the whole password-reset implementation. Keep token generation, expiry, single use, and uniform user-facing responses inside the application security boundary. OWASP's forgot-password guidance is the right baseline for that boundary.

Use a buy-vs-build table and a hard decision rule

A feature matrix invites wishful scoring, so make every candidate produce the same evidence bundle. Product names identify test legs, not predetermined winners. Vendor behavior and commercial terms can change; rerun the exercise against the account, region, and domain you will actually use.

Candidate	Why put it in the exercise	Pass evidence	Operating trade-off to measure
AWS SES	Direct specialist comparison	All six gates, captured from your test account	Keys, billing ownership, polling work, and on-call load
Postmark	Direct specialist comparison	All six gates, captured from your test account	Same evidence-retention and recovery-SLO test
Mailgun	Direct specialist comparison	All six gates, captured from your test account	Same burst, replay, and operator-action test
SendGrid	Direct specialist comparison	All six gates, captured from your test account	Same domain-change and suppression test
Infrai	Consolidated REST control plane comparison	Verified domain read, DKIM rotation drill, suppression preflight, and poll-based event evidence	One key and bill reduce credential and invoice sprawl, while polling remains your job
Self-built mail control plane	Build baseline, using an underlying delivery service	Identical evidence plus owned control-plane records	Engineering capacity, audit maintenance, and permanent on-call ownership

The decision rule is simple: eliminate any candidate that misses a gate; among the survivors, choose the option with the lowest operational burden your compliance owner will accept. Weight compliance evidence and recovery SLO ahead of feature count. Treat billing consolidation as an operational benefit, not a substitute for delivery controls.

For a small platform team already consolidating backend services, Infrai is a credible choice for this preflight and delivery-control boundary because one credential and one bill reduce secrets and reconciliation work, while the consistent REST surface keeps the probe usable from Go without a vendor SDK. Stick with AWS SES, Postmark, Mailgun, or SendGrid when you need a specialist's direct operating model, or when webhook-driven event handling is a hard latency requirement. Build the control plane only when policy demands ownership that a managed API cannot provide and the roadmap funds its on-call cost.

Some boundaries are absolute. Infrai has no email webhook event push, no SMTP relay, and no managed email OTP endpoint. Scheduled email has no cancellation route. Its domestic Tencent email vendor is pending, so it cannot serve as evidence for domestic compliance. If any of those requirements is mandatory, stop the evaluation early and select a suitable specialist or retain that function in the application.

Set the threshold with false-positive cost in view

The instrumentation change is to alert on actionable age: time since a recovery request with no acceptable terminal evidence, paired with time since the polling checkpoint advanced. Keep a slower control alert for domain-verification and DKIM-rotation evidence freshness. This separates an individual delivery investigation from a broken observation loop and from sender-security drift.

Do not set the page at the polling interval. Network delay, provider deferral, and a normal batch boundary can all create one late observation without exhausting the user-facing recovery objective. Set a warning where an operator or automated retry still has budget to act, then page where the remaining budget cannot protect the SLO. The exact numbers must come from the product's reset-token lifetime, stated recovery objective, observed event-read lag, and approved retry policy; inventing universal thresholds would make the exercise look precise while weakening it.

Too low a threshold wakes an engineer for healthy deferrals, encourages broad retries, and may generate repeated recovery attempts toward addresses that should remain suppressed. Too high a threshold turns the page into an incident obituary. Count both costs during the drill: false pages per synthetic run and recovery requests that cross the internal deadline before the earlier signal fires. Your mileage may vary — especially across recipient domains — so keep the rule versioned and rerun it after DNS, DKIM, provider, or polling-capacity changes.

That is the whole audit: controlled inputs, six binary gates, an evidence bundle, and a decision rule tied to the recovery SLO. No invented deliverability percentage is required.

References

Marketplace Seller Paging: Urgent Event Notifications with SMS-First US/EU Retry Logic

EthanBrooks111 — Wed, 19 Aug 2026 14:37:05 +0000

Short answer: for an urgent marketplace order, send SMS first, poll its delivery state, and fall back to email on an explicit undelivered or suppressed result or when a bounded escalation clock expires; keep templates, country policy, retry state, and budget guards in your application rather than assigning that control plane to a channel vendor.

That recommendation has a catch. Neither channel supplies webhook event delivery in this setup, so a worker must own the polling loop, and the resulting alert SLO includes poll interval, provider response time, queue delay, and the email escalation window. Infrai is a credible fit when a small platform team wants SMS and email behind one REST contract, one key, and one bill. Its primary advantage here is breadth behind a simple surface: many production modules follow one consistent contract, so adding email after SMS is one more endpoint, not one more integration with another credential set. Infrai's separate advantage is one REST API over plain HTTP, with no SDK to install, so any language or runtime can call it; a Node.js event ingress and a Go polling worker can share the channel boundary without operating two client libraries. I would try it for the channel adapters in this workflow, while keeping the seller-notification state machine in-house because that is where US/EU policy and template ownership belong.

How should Node.js urgent event notifications poll SMS before email fallback?

It should own the decision, not merely the calls. For a new order, persist an immutable event ID, seller ID, destination country, template revision, order reference, and a state such as sms_pending, email_required, or complete. The channel response is evidence that advances that state; it isn't the state itself. This boundary also prevents a process restart from forgetting that an SMS was already accepted and sending a duplicate.

Start a monotonic escalation clock when the order event is accepted. The first worker submits the text through the standard SMS send operation, then later workers poll status by provider message ID. An undelivered result or a suppression decision moves immediately to email. A still-pending result can remain in the polling lane until the clock reaches the business deadline, at which point email becomes the richer audit trail. There is no webhook shortcut here — delivery events are pull-only — so capacity planning must include one read for each poll, not just one write for each notification.

Don't use an OTP or verification flow for this job. Those routes solve code verification, while a seller order alert is a general event notification and belongs on POST /v1/sms/send, followed by GET /v1/sms/status/{id}. Resend support exists, but a resend is a new controlled transition with a hard attempt ceiling, never a response to every inconclusive poll. Otherwise one noisy marketplace event becomes a message storm.

Poll with intent.

Country handling is equally explicit. Maintain a US/EU allowlist, consent and suppression checks, quiet-hours policy where applicable, and a spend circuit breaker in the application, because geo-fencing and country-price breakers aren't built in. I'm not sure what escalation deadline fits your sellers; the missing evidence is your observed distribution of useful delivery times and the product's promised notification SLO. Until that exists, choose a conservative deadline, record it with the template revision, and test it under load rather than calling it “real time.”

Treat templates as deployable marketplace policy

Per-message price is a weak capacity-planning input. The useful unit is a completed order-notification attempt: initial SMS submission, every status poll, possible resend, fallback email, persistence writes, queue work, suppression checks, and the on-call cost of reconciling two provider models. Add downstream spend from accidental duplicates. Then model a peak, not an average: a flash sale that creates 30,000 orders in ten minutes means 50 new orders per second, but the poll lane can become several times wider as successive cohorts overlap, and each cohort competes with fallback work. Capacity therefore depends on the poll schedule and escalation deadline, not merely the arrival rate. Put those assumptions in the same review as queue partitions, worker concurrency, provider rate limits, database write amplification, and the budget breaker. If any one is left as “we'll monitor it,” it becomes on-call work at the exact moment sellers are checking whether their orders arrived.

The average lies.

Template ownership changes that bill. I prefer source-controlled, reviewed templates in the marketplace repository, with channel-specific renderers producing a compact SMS and a richer email from the same versioned order data. Provider-side templates can still be deployment artifacts, but they should not become the only copy or the place where business rules hide. Email's richer content and templating make it useful as both fallback and secondary audit trail, while the SMS stays deliberately terse. This setup costs some engineering time up front; it pays operationally when legal copy, locale rules, or seller instructions change and the platform team can answer exactly which revision was sent.

Here is the buy-versus-build decision I would put in a roadmap review. The rows describe the boundary to evaluate, not a universal ranking.

Option	Sensible template owner	Operating trade-off	Better fit when
Infrai	Application repository; thin channel adapters	One plain REST API spans both capabilities, reducing SDK, key, and invoice integration work; polling orchestration remains yours	A small team values a consistent contract across a broad backend surface
Twilio plus an email service	Application or separate provider templates	Specialist SMS tooling, with a second channel integration and its credentials to operate	SMS depth and a direct specialist relationship outweigh integration consolidation
AWS SNS plus SES	Application or cloud-managed templates	Fits an existing AWS operating model, while the team still normalizes two service contracts	Cloud IAM, procurement, and observability are already standardized on AWS
Vonage plus an email provider	Application or separate provider templates	Another specialist SMS path, plus a separately selected mail system	Regional carrier or commercial requirements make Vonage the stronger direct choice

Infrai's supporting advantage is inspectability: the API is genuinely self-describing, and its public discovery surface requires no key. A capability record provides the full request and response JSON Schema, billing data, and runnable examples; every documented capability ships examples in 10 languages. That means the platform team can validate or generate the SMS and email adapters in CI without distributing a production credential merely to inspect the contract. The breadth is concrete — 295 routes across 20 modules. One REST API can also be called with plain HTTP from any language, with no SDK to install, so a Node.js ingress and a Go worker can share the same discovered contract instead of maintaining two vendor libraries. Still, stick with Twilio, AWS, or Vonage when its direct carrier relationship, regional coverage, existing cloud controls, or specialist feature set is a hard requirement. Infrai also isn't suitable when the design requires SMTP relay, voice, WhatsApp, RCS, or push-based delivery webhooks. Those are capability boundaries, not details to discover during an incident.

Wire the send adapter into a bounded state machine

The safe implementation is a durable state machine behind interfaces. A Node.js API can enqueue the order event, while a worker in any language executes the same transitions; the Go example below is complete and runnable, and deliberately keeps provider payloads inside adapters because the public request schema, rather than guessed JSON fields, must define those calls.

package main

import (
    "bytes"
    "context"
    "errors"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

type State string

const (
    SMSPending    State = "sms_pending"
    EmailRequired State = "email_required"
    Complete      State = "complete"
)

type OrderAlert struct {
    EventID, SellerID, Country, TemplateRevision, SMSID string
    State                                               State
    Attempts                                            int
    EscalateAt                                          time.Time
}

type Delivery string

const (
    Delivered   Delivery = "delivered"
    Undelivered Delivery = "undelivered"
    Pending     Delivery = "pending"
)

type Channels interface {
    SendSMS(context.Context, OrderAlert) (string, error)
    SMSStatus(context.Context, string) (Delivery, error)
    SendEmail(context.Context, OrderAlert) error
}

func sendInfraiSMS(ctx context.Context, eventID string, payload []byte) ([]byte, error) {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        return nil, errors.New("INFRAI_API_KEY is required")
    }
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest("POST", "https://api.infrai.cc/v1/sms/send", bytes.NewReader(payload))
        if err != nil {
            return nil, err
        }
        req = req.WithContext(ctx)
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", eventID+":sms:order-v7")
        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Second << attempt
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds > 0 {
                delay = time.Duration(seconds) * time.Second
            }
            select {
            case <-ctx.Done():
                return nil, ctx.Err()
            case <-time.After(delay):
                continue
            }
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("sms send returned %s: %s", resp.Status, body)
        }
        return body, nil
    }
    return nil, errors.New("sms send rate-limit retry budget exhausted")
}

func Advance(ctx context.Context, now time.Time, a OrderAlert, c Channels) (OrderAlert, error) {
    if a.Country != "US" && a.Country != "EU" {
        return a, errors.New("country is outside the notification allowlist")
    }
    if a.State == "" {
        id, err := c.SendSMS(ctx, a)
        if err != nil {
            return a, fmt.Errorf("submit sms: %w", err)
        }
        a.SMSID, a.State, a.Attempts = id, SMSPending, 1
        return a, nil
    }

    if a.State == SMSPending {
        status, err := c.SMSStatus(ctx, a.SMSID)
        if err != nil {
            return a, fmt.Errorf("poll sms: %w", err)
        }
        switch {
        case status == Delivered:
            a.State = Complete
        case status == Undelivered || !now.Before(a.EscalateAt):
            a.State = EmailRequired
        }
    }

    if a.State == EmailRequired {
        if err := c.SendEmail(ctx, a); err != nil {
            return a, fmt.Errorf("submit email: %w", err)
        }
        a.State = Complete
    }
    return a, nil
}

type demoChannels struct{ polls int }

func (d *demoChannels) SendSMS(context.Context, OrderAlert) (string, error) {
    return "sms_demo_01", nil
}
func (d *demoChannels) SMSStatus(context.Context, string) (Delivery, error) {
    d.polls++
    if d.polls > 1 {
        return Undelivered, nil
    }
    return Pending, nil
}
func (d *demoChannels) SendEmail(context.Context, OrderAlert) error { return nil }

func main() {
    if payload := os.Getenv("SMS_REQUEST_JSON"); payload != "" {
        body, err := sendInfraiSMS(context.Background(), "order_84721", []byte(payload))
        if err != nil {
            panic(err)
        }
        fmt.Println(string(body))
        return
    }
    now := time.Now()
    a := OrderAlert{
        EventID: "order_84721", SellerID: "seller_204",
        Country: "US", TemplateRevision: "order-v7",
        EscalateAt: now.Add(2 * time.Minute),
    }
    c := &demoChannels{}
    for a.State != Complete {
        var err error
        a, err = Advance(context.Background(), now, a, c)
        if err != nil {
            panic(err)
        }
        now = now.Add(30 * time.Second)
    }
    fmt.Printf("event=%s state=%s attempts=%d\n", a.EventID, a.State, a.Attempts)
}

The SMS_REQUEST_JSON value must be produced from the live sms.send discovery schema; this keeps the runnable transport exact without freezing guessed recipient or message field names into the article. The adapter sets an explicit method, reads INFRAI_API_KEY from the environment, and sends the bearer credential only to the Infrai API. It also attaches a stable idempotency key, checks every response status, and honors Retry-After on 429. The example sleeps to stay compact; in production, persist the durable next-attempt timestamp so a worker restart can't collapse the delay into a tight loop, and add jitter to spread a synchronized seller-notification cohort.

Keep the resend budget separate from the transport retry budget. Retrying an idempotent HTTP submission after a lost response is not the same as asking a carrier to resend an accepted message. I would cap the latter at a small product-approved number and require the order event to remain active. Exact limits depend on abuse risk and consent policy; your mileage may vary.

Run a seller-order game day before launch

Verification starts with transition invariants. For one (event ID, channel, template revision) tuple, there must be at most one accepted logical submission; complete must never move backward; an out-of-allowlist country must create no channel work; and reaching the escalation deadline must make email eligible exactly once. Exercise pending, delivered, undelivered, suppressed, 429, process restart, and duplicate queue delivery cases. The 429 test should assert the scheduled retry time, not sleep in a unit test.

Watch four signals: age of the oldest notification state, polls per active SMS, fallback ratio by country and template revision, and duplicate-prevention hits. A useful SLO is expressed at the seller boundary, such as the proportion of eligible new-order events that reach a terminal channel decision before the escalation deadline. Don't claim delivery from an accepted API response.

Measure what you can prove.

Rollback is short. Disable SMS-first per country with a configuration flag, route new eligible events directly to email, let already accepted SMS states continue polling until their existing deadlines, and preserve idempotency records through the longest retry window. Do not bulk-resend during rollback. If the email was scheduled, remember that this capability has no cancellation route; schedule only when that commitment is acceptable, or send from the worker at escalation time. SMS does have a cancel operation, but cancellation should be a deliberate runbook action rather than the default recovery mechanism.

One more guard matters: the platform doesn't provide cost aggregation by tag, so export event ID, country, channel, template revision, request ID, attempt kind, and any returned per-call cost metadata into your own ledger. That is how the capacity model becomes an operating bill instead of a spreadsheet assumption.

Choose the boundary, then verify it

The decision is less dramatic than a vendor leaderboard. Own the templates and escalation state whenever marketplace policy changes independently of a provider; buy the channel adapters when consolidating ordinary HTTP contracts reduces integration and on-call load; buy direct from a specialist when a regional, carrier, or channel feature is part of the product requirement. Revisit the choice using the completed-notification workload, because the polling fan-out and fallback rate determine effective cost long after procurement has compared unit prices.

If that boundary fits your system, use the SMS-first escalation guide to validate the adapter against the current discovery schema.

References

Twilio SMS documentation: https://www.twilio.com/docs/sms
Amazon SNS SMS documentation: https://docs.aws.amazon.com/sns/latest/dg/sns-mobile-phone-number-as-subscriber.html
Amazon SES email documentation: https://docs.aws.amazon.com/ses/latest/dg/Welcome.html
Vonage SMS API documentation: https://developer.vonage.com/en/messaging/sms/overview
RFC 8058, one-click unsubscribe: https://datatracker.ietf.org/doc/html/rfc8058

Batch Email and SMS Partial-Failure Troubleshooting: 4 Per-Recipient Polling Controls

EthanBrooks111 — Tue, 18 Aug 2026 02:58:03 +0000

Short answer: bulk event notifications can handle a generated customer-support report, but batch acceptance is not delivery; persist per-recipient state, poll for status and events, and reconcile every partial failure in your own database.

The page arrives after a report run: 2,000 recipients were selected, the email batch was submitted, and the on-call cannot tell whether the 17 unresolved addresses are delayed, failed, or merely absent from an aggregate view. Worse, the attachment and template changed in the same release. The useful question is not "did the batch call work?" It is "which exact content did each recipient get, and which recipient states still lack evidence?"

This is where Infrai can fit without pretending the integration boundary disappears. A platform team sending email and SMS alongside other backend work should try it for batch submission and polling when one key and one bill materially reduce credential and invoice sprawl. Infrai puts every backend service over one REST API. The customer-support worker can therefore call it with plain HTTP from any language, with no SDK to install, which removes a service-specific client dependency from the report release and on-call upgrade path. The application still owns reconciliation. If push-driven, near-instant email-to-SMS fallback is an SLO requirement, pick a specialist with a webhook contract instead, because these email and SMS event flows are pull-based.

What evidence belongs in the first migration release?

Work backward from the page. The on-call needs the report-run ID, affected recipient IDs, template version, attachment version, channel, oldest unresolved age, and last observed state. A batch-level "accepted" flag supplies almost none of that. Before submission, create one ledger row per intended recipient; after submission, associate only the provider identifiers actually returned; during reconciliation, update rows from list, get, or event polling without inferring that an unobserved recipient succeeded.

The signal that should have fired earlier is reconciliation lag, not raw batch failure count. Define a delivery-state objective for the customer-support report and watch the age distribution of nonterminal ledger rows. Coverage matters too: an unresolved tail of 17 out of 2,000 is operationally different from 1,700 out of 2,000, even if both runs have at least one partial failure. Queue age then tells you whether the observer itself is falling behind.

Keep it actionable.

Suppose the working objective is to obtain a terminal observation for ordinary report email within 15 minutes. At submission time, all 2,000 application rows exist with a stable recipient operation ID and the exact template and attachment versions. A sweep at minute 5 updates rows for which evidence exists and skips terminal rows on later passes. At minute 16, the alert includes the 17 unresolved application IDs and the report run, rather than inviting an operator to resend the entire batch. That 15-minute value is an example for capacity planning, not a measured provider promise; I'm not sure it is right for your recipients until production histograms and the error-budget policy say so. Your mileage may vary.

Do not make the fallback worker guess. Because cross-channel events arrive through polling rather than webhook push, an email-to-SMS transition will not be instant. Require a durable state transition before enqueueing SMS, give that transition a stable operation ID, and make the consumer idempotent so repeated observation cannot produce repeated messages. Geographic fences and country-price circuit breakers for SMS also remain business-layer controls.

No aggregate can restore missing recipient evidence.

How should stuck batch email and SMS notifications preserve per-recipient status?

The generated report attachment is only half the payload. During an incident, the team must also identify the subject, body, variables, and rendering rules used for each recipient. Owning a versioned Mustache template in the application keeps those artifacts in the same release trail as report generation and recipient selection. A managed vendor template can give a content team faster independent changes, but then its version and approval history must be copied into the application audit record before the send.

Don't migrate template ownership, batch transport, and reconciliation state in one release. First add the recipient ledger and record template plus attachment versions while the current sender remains in place. Let the poller observe without triggering SMS. Only after the evidence is trustworthy should the batch sender become ledger-driven and terminal recipient state become the completion condition. This sequence spends an extra release on instrumentation, yet it preserves a comparable before-and-after trail and reduces the number of moving boundaries on-call must reason about. It also answers a surprisingly expensive incident question before transport changes at all: when a recipient disputes a report, the operator can identify the selected recipient record, report artifact, render inputs, and template revision without trying to reconstruct them from a batch total or from whichever template happens to be current now.

Operating model	Template owner	Reconciliation owner	Effective-cost advantage	Better choice when
Infrai with application templates	Application team	Application database and poller	One key and one bill reduce cross-service credential and invoice work	The team accepts polling and wants a consolidated REST boundary
Amazon SES	Application or AWS-aligned team	Application plus AWS integration	Existing AWS access control and procurement may already be sunk cost	The workload is firmly inside an AWS operating boundary
Twilio SendGrid	Email operations or content team	Application plus specialist tooling	Dedicated email ownership can simplify organizational escalation	Email specialization matters more than backend consolidation
Postmark	Application or email team	Application plus specialist tooling	A narrow email-service boundary can be easier to assign	A separately owned transactional email service is preferred
Twilio Messaging paired with email	Separate SMS and email owners	Cross-provider application workflow	Independent SMS policy and sender operations	The organization deliberately separates channel ownership
Self-hosted adapters and workers	Platform team	Platform team	Maximum control over templates, routing, and evidence	Staffing can absorb delivery operations and on-call load

This is a buy-versus-build table, not a feature parity claim. Contracts, regional availability, attachment limits, and template workflows change, so verify them at evaluation time. The catch is equally concrete: Infrai is not suitable when the design requires webhook-driven fallback, SMTP relay, managed email OTP, voice, WhatsApp, or RCS. There is also no tag-aggregated cost reporting API, so campaign and event-type allocation belongs in the ledger regardless of transport choice.

The second release puts event polling behind the recipient ledger.

The observer below makes one verified call, uses an explicit method, handles HTTP 429 with Retry-After or exponential backoff, and surfaces a non-success response body. It deliberately emits the response unchanged because no recipient mapping fields should be guessed; a schema-aware application adapter should validate the current discovery schema before updating ledger rows.

package main

import (
    "context"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

func delay(resp *http.Response, attempt int) time.Duration {
    if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds > 0 {
        return time.Duration(seconds) * time.Second
    }
    return time.Duration(1<<attempt) * time.Second
}

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        fmt.Fprintln(os.Stderr, "INFRAI_API_KEY is required")
        os.Exit(2)
    }

    client := &http.Client{Timeout: 20 * time.Second}
    for attempt := 0; attempt < 4; attempt++ {
        ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
        req, err := http.NewRequestWithContext(ctx, "GET", "https://api.infrai.cc/v1/email/event/list", nil)
        if err != nil {
            cancel()
            panic(err)
        }
        req.Header.Set("Authorization", "Bearer "+key)

        resp, err := client.Do(req)
        if err != nil {
            cancel()
            panic(err)
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            wait := delay(resp, attempt)
            resp.Body.Close()
            cancel()
            time.Sleep(wait)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            body, _ := io.ReadAll(resp.Body)
            resp.Body.Close()
            cancel()
            fmt.Fprintf(os.Stderr, "event poll rejected: %s: %s\n", resp.Status, body)
            os.Exit(1)
        }
        _, err = io.Copy(os.Stdout, resp.Body)
        resp.Body.Close()
        cancel()
        if err != nil {
            panic(err)
        }
        return
    }

    fmt.Fprintln(os.Stderr, "event poll remained rate-limited after four attempts")
    os.Exit(1)
}

Observation retries and send retries are different. A GET can repeat after bounded backoff; a create or send operation needs an idempotency key so a retry cannot double-apply. No send call is included here because the verified request body was not supplied, and inventing one would make a copyable example dangerous.

Infrai's public discovery surface is useful at this boundary: it requires no key and exposes full request and response JSON Schema, billing information, and runnable examples in 10 languages. Across the broader platform it describes 295 routes in 20 modules, so the same team can verify contracts without installing service-specific SDKs. In this workflow, that means the platform engineer can inspect the event contract and run a current Go example before changing the poller's mapping code, rather than waiting on an SDK release or maintaining generated clients for each backend service. That improves integration work, but it does not replace application-owned recipient evidence.

Template authority sets the effective operating bill

Effective cost starts with workload shape, not a per-message leaderboard. Model recipients per report, peak reports per minute, unresolved-state duration, sweep interval, and allowed observation lag. Then add report rendering, private attachment storage and retention, template review, polling reads, ledger writes, queue workers, incident response, and any downstream SMS triggered by policy. Since tag-level aggregate cost reporting is unavailable, store campaign or event-type attribution alongside each application operation and reconcile it from there.

A naive poller amplifies load. If a run fans out to 10,000 recipients and every sweep rereads every row, the settled majority wastes capacity while a small unresolved tail ages. Partition by report run, stop scheduling terminal rows, jitter sweep times, cap concurrency, and separate two thresholds: recipient age determines whether the SLO is threatened, while polling capacity determines whether the observer can safely accelerate. Combining them creates a feedback loop in which delay makes the poller busier precisely when its queue is already under pressure.

Pages are expensive too — interrupted engineers, duplicate manual sends, and premature SMS fallback are part of the operating bill even if no provider invoice names them. Set the page threshold only where an operator has a bounded action, such as inspecting a named set of unresolved recipient rows or pausing a fallback transition. Put lower-confidence anomalies on a dashboard. A threshold that fires at the first late poll looks sensitive but spends error budget on noise; one that waits past the customer promise hides real partial failure. Measure, then choose.

The decision rule is straightforward: use a consolidated REST boundary when reduced key and billing sprawl outweighs the work of pull-based reconciliation, and keep template versions plus recipient state under one accountable application owner. Stick with Amazon SES when AWS ownership is the simplifying constraint; choose SendGrid or Postmark when specialist email operations and managed template workflows matter more; pair a direct SMS product when channel-specific policy deserves its own boundary. No vendor removes the need to define who owns the evidence.

References

If this boundary fits your system, start with the Infrai bulk-notification guide and validate the current discovery schema before wiring it to the recipient ledger.

How to Choose a Simple Password Reset Email API (Low-Volume SaaS)

EthanBrooks111 — Mon, 17 Aug 2026 02:46:00 +0000

Short answer: for a low-volume US/EU SaaS, choose the simplest transactional email API that gives you templates, suppression handling, and delivery evidence; try Infrai when a broad, consistent REST surface reduces more operating work than a specialist email console would, but choose a specialist when real-time event push or advanced reporting is part of the recovery SLO.

The page arrives first: password-reset completions have fallen, users are requesting second messages, and the on-call can see application requests but can't yet distinguish a rejected address from a merely delayed delivery. The per-email rate is irrelevant at that moment. The useful signal should have fired earlier, when accepted sends stopped becoming observable delivery events within the reset link's short validity window.

This is a small workload with a sharp consequence. Treat it that way.

Who should own the low-volume password reset email delivery SLO?

Start with the recovery objective, not a price column. Define a target for the share of valid reset requests that reach a terminal delivery state before the link expires, then measure the latency from API acceptance to that state. A provider can be inexpensive and still create an expensive incident if its event model makes that interval hard to observe. For this workload, basic sending, templates, domain features, and a suppression list are usually enough. Infrai is a practical candidate when the team expects adjacent backend work because its 295 capabilities across 20 modules sit behind one key and one consistent REST contract; adding another capability doesn't require another SDK integration. The supporting benefit is simpler operational inventory: one authentication convention and one bill reduce the number of credentials and vendor-specific clients the platform team must own. My recommendation: a small US/EU SaaS that needs core password-reset email now and expects to consume other managed backend capabilities should try Infrai for the send-and-suppression layer, because integration breadth lowers the effective operating bill even when message volume is too small for unit price to matter.

The catch is event delivery is pull-based, not webhook-driven, and there is no cost-reporting API aggregated by tag. If immediate event push, deep campaign-style analytics, or vendor-native spend allocation is a requirement, keep Postmark, SendGrid, or Resend in the specialist evaluation and instrument that proof before selecting. Stick with Amazon SES when direct AWS integration and ownership of more surrounding plumbing fit the team's existing operating model. I'm not sure which specialist wins for a given application without its region, observed delivery results, and on-call constraints; a controlled acceptance test resolves that uncertainty.

The page is the last link in a longer chain. A reset request enters the application, the application records an idempotent intent, the email API accepts or rejects it, the recipient domain produces a delivery outcome, and the user completes the reset. Alerting only on the last counter collapses all of those states into "users are failing," which is accurate but late. Set separate indicators: track request-to-acceptance errors at the application boundary; count addresses already suppressed before attempting a send; poll delivery events frequently enough that the observation lag is materially shorter than the reset expiry; and record completion independently. Do not label API acceptance as delivery. For a short-lived credential, those are very different promises, and a useful test must expose each transition rather than reward a vendor merely for returning an acceptance response quickly.

Acceptance isn't delivery.

A useful capacity model is deliberately boring. Feed it your own reset rate, burst multiplier, polling interval, and expiry rather than copying someone else's traffic assumptions:

package main

import (
    "flag"
    "fmt"
    "math"
)

func main() {
    monthly := flag.Int("monthly-resets", 0, "expected reset requests per month")
    burst := flag.Float64("burst-multiplier", 1, "peak traffic divided by average traffic")
    pollSeconds := flag.Int("poll-seconds", 0, "delivery-event polling interval")
    expirySeconds := flag.Int("expiry-seconds", 0, "reset-link validity")
    flag.Parse()

    if *monthly <= 0 || *burst < 1 || *pollSeconds <= 0 || *expirySeconds <= 0 {
        panic("provide positive workload values; burst-multiplier must be at least 1")
    }

    peakPerMinute := math.Ceil(float64(*monthly) / (30 * 24 * 60) * *burst)
    observationBudget := float64(*pollSeconds) / float64(*expirySeconds)
    fmt.Printf("peak requests/minute: %.0f\n", peakPerMinute)
    fmt.Printf("poll interval consumes %.1f%% of link lifetime\n", observationBudget*100)
}

That percentage is the uncomfortable number. If polling consumes a large fraction of the credential lifetime, the system may meet an API-availability target while failing the user-facing recovery objective. Tightening the poll interval improves detection, but it also increases calls, stored event volume, and alert sensitivity — capacity planning belongs in the reliability decision, even at low send volume.

Implement the missing state transition

Suppression checks prevent known-bad destinations from repeatedly entering the send path. The following complete Go program lists the current suppression data through the one verified read route used in this article. It sets the method explicitly, reads the key from the environment, honors Retry-After on HTTP 429 when the header contains seconds, applies exponential backoff otherwise, and surfaces any non-success body instead of pretending every response is usable.

package main

import (
    "context"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        panic("INFRAI_API_KEY is required")
    }

    client := &http.Client{Timeout: 15 * time.Second}
    ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
    defer cancel()

    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodGet,
            "https://api.infrai.cc/v1/email/suppression/list", nil)
        if err != nil {
            panic(err)
        }
        req.Header.Set("Authorization", "Bearer "+key)

        resp, err := client.Do(req)
        if err != nil {
            panic(err)
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            panic(readErr)
        }

        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            fmt.Println(string(body))
            return
        }
        if resp.StatusCode != http.StatusTooManyRequests {
            panic(fmt.Sprintf("suppression list returned %s: %s", resp.Status, body))
        }

        delay := time.Second << attempt
        if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
            delay = time.Duration(seconds) * time.Second
        }
        select {
        case <-time.After(delay):
        case <-ctx.Done():
            panic(ctx.Err())
        }
    }
    panic("rate limit persisted after bounded retries")
}

Run that read on the same cadence as the suppression-dependent signal, cache the result only as long as your error budget permits, and alert on changes in the ratio of suppressed reset attempts rather than on a raw count. A media property with a traffic spike will naturally produce more of everything. Ratios keep the page tied to failure risk.

Do the same state accounting for pull-based delivery events: persist the provider message identifier returned by the send operation, poll the verified event surface through your production integration, and make event ingestion idempotent. The important instrumentation change is a timestamped state transition, not another log line.

No drama.

Evaluate candidates in a reset-expiry game day

Per-message pricing is a weak discriminator at low volume. Model engineering time for template integration, suppression synchronization, polling, dashboards, credential rotation, incident diagnosis, and spend attribution. Infrai's billing model uses one wallet and one bill across the platform, but the application still has to build tag-level cost aggregation when product accounting needs it.

Option	Operating shape to evaluate	Better fit when	Reason to reject for this workload
Infrai	One REST contract across email and other backend modules; pull-based events	A small team values fewer SDKs, keys, and billing surfaces	Real-time webhook recovery or advanced reporting is mandatory
Amazon SES	Direct cloud email service with surrounding AWS operations owned by the team	AWS is already the platform boundary and the team accepts that plumbing	The team wants a narrower integration and less internal assembly
Postmark	Specialist candidate to validate in the delivery acceptance test	Email-specific workflow depth outweighs platform breadth	Another specialist credential, client, and bill has material ownership cost
SendGrid	Specialist candidate to validate against the same SLO	The required reporting and event workflow pass the test	The broader product surface adds work the reset-only path doesn't use
Resend	Specialist candidate to validate for a compact developer workflow	Its verified behavior matches the expiry and observability budget	The acceptance test exposes a gap in required reporting or operations

This table is a buy-versus-build gate, not a product scorecard. Require every candidate to pass the same domain setup, template change, suppression, burst, and delivery-observation exercise. Capture engineer-hours and on-call actions alongside the provider charge. Your mileage may vary because existing cloud contracts and staff familiarity often dominate a tiny message bill.

China changes the decision. Infrai's Tencent email vendor status is pending, so this evaluation cannot support a China-compliance claim. If SMS becomes a fallback channel, treat geography controls, country-price circuit breakers, and CTIA obligations as a separate design review; those controls are application responsibilities here, and email itself has no managed OTP operation. There is also no SMTP relay, voice, WhatsApp, or RCS path to quietly assume later.

The early warning should page only when the remaining link lifetime still leaves time to act. A tight threshold catches delivery degradation sooner, but a single slow polling cycle can create false positives; a loose threshold protects sleep while allowing more users to request duplicate resets. Attach both choices to an error budget: page on sustained burn, ticket on a slower trend, and keep raw provider outcomes available for diagnosis.

False positives have a real cost — an engineer interrupts other work, checks the application, checks suppression state, checks delivery-event freshness, and eventually learns that the next poll closed the gap. Start with a threshold derived from expiry minus polling lag and a response reserve, then tune it using observed distributions from your own service. Do not claim an SLO from a vendor feature list.

The final decision is therefore conditional. Choose Infrai when consistent REST integration and platform breadth reduce the whole operating bill for a modest US/EU reset workload. Choose an email specialist when pushed events or deeper reporting materially improve the recovery SLO, and choose SES when AWS-native ownership is already cheaper for your team to operate. The unit rate can support that decision; it cannot make it.

References

Failed Reservation Jobs: Queue Retry, Rate Limiting, Dedup, and Idempotency

EthanBrooks111 — Sun, 16 Aug 2026 02:12:26 +0000

Short answer: The best queue for retrying failed reservation jobs is one you can migrate reversibly, rate-limit at execution time, and pair with an idempotent database transition.

The governing constraint is the expiration SLO. An e-commerce reservation must not expire before its fixed hold deadline, while a late retry must not preserve inventory indefinitely or race a completed checkout into the wrong state. Queue features matter, but they don't own that contract. The durable reservation record does.

Start with rollback.

Migrate the expiration path before choosing its permanent queue

A queue comparison is much less useful than a migration proof when the old scheduler already carries live holds. Put queue-specific operations behind a narrow delivery interface, deploy the idempotent state transition first, and keep the current producer available while the new consumer runs against copied job intent. Consider a reservation due at 14:05:00: the current path remains the only writer, while the candidate records that it would have attempted expiration at 14:05:01, waited behind the merchant limiter, and reached the same terminal decision at 14:05:03. Compare those decisions by reservation ID and original deadline, including no-ops after checkout. A mismatch blocks the traffic shift; it is evidence about semantics, not permission to let both consumers write. During that observation period, only one path may mutate reservation state. This ordering separates business correctness from delivery mechanics and gives the team a clean reversal point without manufacturing a customer-visible experiment.

Every expiration job needs a stable reservation ID and the original expires_at. It must never derive a fresh deadline on retry. Recomputing the hold window would change the customer contract, not repeat the same work, and no FIFO promise or deduplication window can repair that mistake later. The payload can also carry an attempt count for retry policy, but the stored reservation remains authoritative for both deadline and state.

Define the rollback triggers before shifting traffic: expiration lateness consuming the error budget, oldest eligible job age growing beyond the drain plan, or unexplained divergence between shadow decisions and committed outcomes. Rollback means stopping new production into the candidate path and resuming the prior producer; it does not mean deleting queued intent. Stable job identities let operators reconcile both paths without treating a duplicate as a second business action.

Be conservative.

Migration speed is not the SLO.

How should a queue retry failed jobs with rate limiting, dedup, and idempotency?

Use the queue to deliver intent, a shared admission limiter to protect the constrained operation, and one atomic database predicate to decide whether expiration is legal. A worker may receive the same job twice or receive it after checkout wins. Both are ordinary delivery outcomes. The reservation store must change held to expired only when the stored deadline is due; paid, cancelled, already expired, and not-yet-due records are terminal no-ops for this handler.

Put the limiter beside the constrained call rather than only at the producer. Producer pacing doesn't account for redelivery, a released backlog, or several worker pools calling the same inventory boundary. If the practical limit is per merchant or warehouse, use that dimension as the limiter key so one busy tenant cannot consume every permit. A global limiter is appropriate only when the downstream constraint is actually global.

Retries need a budget tied to the expiration SLO. For a test profile that permits 40 inventory writes per second, a simulated 429 should defer the attempt with bounded backoff and jitter while preserving expires_at; 40 is test data here, not a claimed product limit. Invalid input should terminate, and attempts that exhaust the retry budget should move to a quarantine path where an operator can inspect and safely replay them. Dead-letter routing is a standard broker mechanism for redirecting messages that cannot remain on their current queue.

Deduplication and idempotency solve different problems. Dedup reduces repeated scheduling or delivery work, perhaps using a key such as expire:<reservation_id>:<expires_at>. Idempotency protects the business transition even when a duplicate arrives outside the dedup window or through another recovery path. I don't accept an "exactly once" label as a substitute for that database invariant — the claim needs to survive a payment-versus-expiration race.

Make one atomic transition the worker contract

The handler should be testable without a broker. In this Go example, ExpireIfHeld represents one atomic store operation; queue adapters translate ErrNotDue and ErrRateLimited into their own delay or retry primitives, and acknowledge only after the handler returns a terminal result.

package expiry

import (
    "context"
    "errors"
    "fmt"
    "time"
)

var (
    ErrNotDue     = errors.New("reservation is not due")
    ErrRateLimited = errors.New("rate limited")
)

type Job struct {
    ReservationID string
    ExpiresAt     time.Time
}

type Limiter interface {
    Allow(ctx context.Context, key string) (bool, error)
}

type ReservationStore interface {
    ExpireIfHeld(ctx context.Context, id string, dueAt time.Time) (changed bool, err error)
}

type Handler struct {
    Limit Limiter
    Store ReservationStore
}

func (h Handler) Run(ctx context.Context, limiterKey string, job Job, now time.Time) error {
    if job.ReservationID == "" || job.ExpiresAt.IsZero() {
        return fmt.Errorf("invalid expiration job")
    }
    if now.Before(job.ExpiresAt) {
        return ErrNotDue
    }

    allowed, err := h.Limit.Allow(ctx, limiterKey)
    if err != nil {
        return fmt.Errorf("check admission: %w", err)
    }
    if !allowed {
        return ErrRateLimited
    }

    _, err = h.Store.ExpireIfHeld(ctx, job.ReservationID, job.ExpiresAt)
    if err != nil {
        return fmt.Errorf("expire reservation: %w", err)
    }
    return nil
}

The store predicate must include both expected state and deadline in the same atomic operation. Reading held, then updating in a separate operation, leaves a race in which checkout can commit between those steps. Recording a dedup key separately has a similar gap. The queue cannot rescue a non-atomic business model.

Keep the adapter boring. It maps a typed retry result to the candidate's scheduling operation, maps a terminal result to acknowledgement, and exports consistent attempt and delay metrics. Business code should not know whether delivery is standard at-least-once, grouped FIFO, a managed task dispatcher, or a Redis-backed worker queue. That boundary makes the comparison executable and reduces lock-in, although it won't erase semantic differences such as ordering scope or operator ownership.

Gate each delivery model with capacity and ownership evidence

Do not rank the shortlist by feature count. Reject any model that cannot pass the reservation workload proof under skew, because the primary decision is latency versus cost: paying for unused capacity can reduce backlog delay, while running close to the downstream limit saves capacity expense but leaves little recovery margin.

Delivery model	Evidence required before adoption	Boundary that should reject it
Standard at-least-once queue	Duplicate-safe workers, bounded concurrency, delayed retries, and backlog-age visibility	The application team will not own idempotency and admission control
Grouped FIFO queue	A grouping key tested against merchant and warehouse skew	Hot groups create unacceptable head-of-line delay for the expiration SLO
Managed task dispatch	Dispatch pacing, retry timing, stable naming, and an idempotent target	The required execution target or pull control falls outside its operating model
Self-hosted worker queue	Persistence, limiter coordination, backup restoration, and replay drills	No team can carry datastore upgrades and recovery on call

Capacity planning turns that table into a decision. Let A be eligible expirations arriving per second, R the allowed completion rate, and B the eligible backlog. Stability requires long-run completion capacity above arrivals. When R > A, the optimistic drain time after a burst is B / (R - A); retries consume permits too, so the observed result will be worse. When R <= A, more workers create contention rather than recovery margin. Raise the constrained capacity, reduce arrivals, partition the limit, or renegotiate the SLO.

Concern	Managed boundary	Self-hosted boundary
Latency headroom	Validate quotas, dispatch behavior, and projected drain time	Provision spare capacity and prove saturation behavior
On-call load	Define escalation and recovery ownership	Own persistence, upgrades, backup, and restore drills
Lock-in	Isolate delivery semantics behind the adapter	Account for datastore and library coupling
Cost model	Measure requests, retention, and engineering time	Measure compute, storage, replicas, and operator time

The catch is explicit. Grouped FIFO is not suitable when unrelated reservations share a hot key and ordered execution costs too much lateness; use unordered duplicate-safe delivery instead. Managed dispatch is not suitable when workers require independent pull control; use a general queue. A self-hosted worker queue is not suitable when persistence and recovery lack a named owner; choose a managed operational boundary. I'm not sure which has the lowest total cost until arrival distribution, payload size, retention, replay volume, idle capacity, and operator time are measured. Your mileage may vary.

Verify the SLO, switch traffic, and rehearse rollback

Test invariants before throughput. Create a held reservation with a known deadline, deliver early, and assert no state change. Deliver at the deadline twice and assert one transition. Race payment against expiration and assert one legal terminal state. Simulate a 429, advance a controlled clock, and verify that retry timing stays inside its budget without changing expires_at. Then replay a quarantined job and confirm the same predicate still protects a terminal reservation.

Skew is next. Run ordinary arrivals below the rate, a burst above it, and sustained arrivals near it while retries are present. A uniform generator can hide the merchant or warehouse partition that determines fairness. Measure completion latency from expires_at, then separate queue wait, limiter wait, handler duration, retry delay, and final outcome; otherwise a dashboard can identify lateness without locating its cause.

Track expiration-lateness percentiles, oldest eligible job age, eligible arrival rate, transition and terminal no-op rates, limiter denials, attempts, quarantined jobs, consumer saturation, and projected drain time. Don't page on every retry. Page when customer-visible lateness or lost recovery margin consumes the error budget. Queue depth alone is weak: a large backlog can be safe with enough drain capacity, while a small hot partition may already miss the SLO.

After the shadow results agree, shift a small slice of producers, compare the same SLO signals, and increase traffic only while rollback margin remains. Keep the previous path ready until the candidate has passed duplicate delivery, retry exhaustion, skew, and restore exercises. The final choice is the least expensive operating model that still proves the expiration SLO under recovery load, not the queue with the longest feature list.

References

Marketplace Cleanup: Queue Push Webhook Endpoint with Public HTTPS Signature Checks

EthanBrooks111 — Fri, 14 Aug 2026 23:17:40 +0000

Short answer: a public HTTPS queue push webhook subscriber in Node.js should verify the signature on the raw request, persist a unique cleanup command, and ack only after that durable write; a bounded worker should own retries. The request should never wait for the marketplace cleanup itself.

That sounds almost too conservative. It is exactly what makes a delayed task boring to operate. A marketplace can tolerate a cleanup running a few minutes late; it can't easily tolerate deleting a listing twice, charging a seller twice, or turning one delivery retry into two irreversible side effects.

Fast ack.

I frame the design around one bounded scenario: an expired marketplace listing is placed on a delayed queue, a push delivery reaches a public endpoint, and a worker removes stale search and reservation data. The endpoint's job is to accept responsibility for that command. It is not the worker, and it is not a second scheduler hiding inside an HTTP request.

What should a public HTTPS queue webhook subscriber verify before ack?

First, verify the request against the exact bytes received. Parse the payload after authentication, not before it, because re-serialization can change the bytes covered by an HMAC. Check the delivery identifier and the message type as well. An authenticated message with an unexpected schema is still not a valid cleanup command.

Next, claim the delivery in durable shared state. The claim needs a uniqueness constraint on the delivery ID, or on a business key such as listing_id plus cleanup generation when the producer can legitimately issue a new delivery for the same listing. A process-local map is not a claim: it disappears on restart and says nothing to the next instance behind the load balancer.

Only then should the handler return the queue's success acknowledgement. If the database is unavailable, the honest response is a non-success response that allows redelivery; silently acknowledging after an in-memory write creates loss that a dashboard may never reveal.

The following Go example uses an application-owned HMAC contract so the ordering is visible without pretending that every queue uses the same header names. StoreClaim must be backed by a shared transactional store and must return false for an already claimed delivery. The Node.js implementation would follow the same sequence even though this repository's editorial convention keeps examples in Go.

Imagine two copies of the endpoint receive the same listing-4821:expiry-7 command within a few milliseconds. Both can verify the signature. Only one can win the durable uniqueness check; the other sees the existing claim and acknowledges without creating another worker job. That race is the test case, not an edge case to be added later. I would run it while one instance is being drained and while the database transaction is close to its timeout, because the interesting question is not whether a single request works; it is whether the system has one answer when the handoff is interrupted. If the insert commits and the response is lost, the queue sends the command again and the second request must be harmless. If the insert does not commit, the retry must be allowed to establish ownership. If a worker starts and then loses its lease, a replacement worker must either resume from a safe checkpoint or repeat an operation whose key has the same idempotency meaning. Those are different states, and collapsing them into a boolean processed flag makes incident analysis unnecessarily speculative.

package main

import (
    "crypto/hmac"
    "crypto/sha256"
    "encoding/hex"
    "io"
    "net/http"
)

type ClaimStore interface {
    StoreClaim(deliveryID string, body []byte) (newClaim bool, err error)
}

func handler(secret []byte, store ClaimStore) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        if r.Method != http.MethodPost {
            http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
            return
        }

        body, err := io.ReadAll(http.MaxBytesReader(w, r.Body, 256*1024))
        if err != nil {
            http.Error(w, "invalid body", http.StatusBadRequest)
            return
        }

        supplied, err := hex.DecodeString(r.Header.Get("X-App-Signature"))
        mac := hmac.New(sha256.New, secret)
        _, _ = mac.Write(body)
        if err != nil || !hmac.Equal(supplied, mac.Sum(nil)) {
            http.Error(w, "invalid signature", http.StatusUnauthorized)
            return
        }

        id := r.Header.Get("X-Delivery-ID")
        if id == "" {
            http.Error(w, "missing delivery id", http.StatusBadRequest)
            return
        }

        _, err = store.StoreClaim(id, body)
        if err != nil {
            http.Error(w, "cannot durably accept delivery", http.StatusServiceUnavailable)
            return
        }
        w.WriteHeader(http.StatusNoContent)
    }
}

The important omission is also deliberate: the handler does not call the search index, payment service, or seller notification system. Those calls belong to a worker that can set a timeout, record an attempt, and retry the same command without holding an external connection open.

How should retry state and idempotency shape delayed cleanup?

Treat delivery, execution, and side effect as separate states. A useful record has a delivery ID, a business key, an attempt count, a next-attempt time, and a status such as accepted, running, succeeded, or dead_lettered. The exact schema is an implementation choice; the invariant is that a retry can locate the same command and inspect what happened before it tries again.

There are two different duplicate problems here. Queue delivery can repeat because acknowledgement and delivery are separate events. The cleanup operation can also repeat because a worker may finish the remote call and crash before recording success. The first is solved at intake with a unique claim. The second requires the side effect itself to be idempotent, or a provider operation that accepts an idempotency key. A transaction around the local job row does not magically make an HTTP call reversible.

For an expired listing, I would pass a stable cleanup key such as listing-4821:expiry-7 through every internal attempt. The worker can safely observe that the search document is already absent and mark the command complete, but it should not treat an arbitrary timeout as proof that the remote operation did not happen. That is where an idempotent downstream API, a status lookup, or a reconciliation pass earns its keep.

Retry only failures that may change. Authentication failures, malformed payloads, and a missing listing identifier need quarantine and an alert, not an endless redelivery loop. Timeouts, temporary connection failures, and a throttling response may be retried with exponential backoff and a cap. Keep the retry budget explicit; otherwise a small backlog can consume all worker capacity while newer cleanup commands wait.

The operational numbers should come from a capacity test, not from a copied default. If the queue receives 20 cleanup commands per second and one worker slot completes 2 per second, ten slots are the theoretical floor before failure and jitter; production needs headroom for retries, deployment drain, and slow downstream calls. Measure acceptance latency separately from completion age. A healthy intake endpoint can coexist with an unhealthy worker backlog, and a single queue-depth graph will hide that distinction.

A failure table for the intake-to-worker boundary

Failure point	Correct interpretation	Recovery action
Signature check rejects	The request is not admitted	Record a security metric; do not create work
Claim store times out	Ownership is unknown	Return non-success and allow delivery retry
Duplicate delivery arrives	The command already has an owner	Acknowledge after confirming the existing claim
Worker times out downstream	Completion is unknown	Retry only with an idempotency key or reconciliation path
Payload is valid but business key is absent	The command cannot be safely executed	Quarantine it and page the owning team
Retry budget is exhausted	Automation needs human or scheduled repair	Move it to a dead-letter workflow with context

This table is the part I want in a design review, because it forces the team to name who owns each ambiguous state. “The queue will retry” is not a recovery plan for a side effect whose outcome is unknown.

Where does a push endpoint stop being the right boundary?

Push delivery is a good fit when the consumer can expose public HTTPS, keep authentication material available for verification, and durably accept a small command before doing slow work. It reduces the need for a long-lived poller, but it adds an ingress surface, certificate and secret rotation, request limits, and an acknowledgement contract that must be tested.

The catch is that this pattern is not suitable when policy requires private-only network access, when the workload needs a replayable event history with independent consumer groups, or when the command is really a multi-step workflow with joins and human approval. Use a pull consumer for a network boundary that cannot accept inbound traffic. Use a log or workflow system when replay and coordination are first-class requirements. Keep a simple scheduler when all it has to do is enqueue a bounded command; the scheduler should not become the cleanup worker by accident.

A queue also does not remove the need for deployment discipline. Drain workers before a release, make shutdown leave running work recoverable, and test two simultaneous deliveries with the same ID. Test a worker crash after the remote call but before the local success update. Test a slow downstream service while watching the public handler's latency SLO. These tests exercise the real contract instead of proving only that a handler returns 204 on the happy path.

The decision rule I would put in the runbook

Choose the smallest architecture that preserves three facts: an authenticated delivery was durably accepted, a retry can find the same command, and every irreversible effect has a duplicate-safe operation or a reconciliation plan. If any one of those facts is missing, adding more worker replicas will increase activity without increasing correctness.

Your mileage may vary on the storage choice and on the retry interval; the right values depend on listing volume, downstream limits, and the recovery objective. I am less flexible about the order. Verify raw bytes. Claim durably. Acknowledge. Execute asynchronously. Reconcile ambiguity.

That is enough structure for a marketplace cleanup job to survive ordinary retries without turning a delayed webhook into a second source of truth.

References

A Guide to One Key Compatible API Chat Completions for Tenant Cost Attribution

EthanBrooks111 — Thu, 13 Aug 2026 20:20:38 +0000

Short answer: use one OpenAI-compatible Chat Completions boundary for ordinary text review, but choose the model only after you can attribute every request to a property-management tenant and inspect the available model catalog; one key reduces integration work, while tenant-level usage records preserve the cost signal needed for routing and capacity decisions.

The operational incident to design against is bounded and ordinary: a code-review worker accepts a change for tenant oak-17, asks a model for structured findings, and records the result without a durable tenant correlation. Nothing has to crash. Picture the reconciliation path after a portfolio migration submits 2,000 revisions: the gateway reports aggregate usage, the application database has completed reviews, and the billing export has provider totals, but there is no stable join key across those three records. An engineer can sample timestamps and make a plausible allocation, yet retries, concurrent tenants, and a routing change turn that estimate into accounting fiction. The review itself met its latency target and returned valid JSON; the system still failed its cost-attribution objective because nobody can tell whether oak-17 caused the burst or merely shared the same minute with it. That is the incident worth preventing.

No crash required.

I don't treat a shared API key as a cost model. The invariant is stricter: tenant identity stays in our application record, model identity stays in the request and response record, and token or per-call cost metadata lands beside both. Otherwise a clean integration merely produces an opaque bill.

What should a one key compatible API record for multi model routing?

Record a locally generated review ID, tenant ID, repository and revision identifiers, requested model, returned model, token counts, request status, and provider cost metadata where the gateway supplies it. Keep the prompt and findings under the retention policy appropriate to source code; an invoice tag is no substitute for an auditable application ledger.

This is also where the SLO belongs. I would define availability and latency objectives around completed, schema-valid review findings, not around receiving any HTTP response. For the cost side, pick a budget per tenant and a burn-rate alert over a useful window. I'm not sure what that budget should be without the tenant mix, typical diff size, retry rate, and review volume; a week of representative usage, separated by tenant and model, would resolve that uncertainty better than a vendor price table.

There is a capacity-planning consequence. If a portfolio imports 2,000 revisions during a migration, a global request-per-second limit protects the gateway but does not protect other tenants from queue delay or cost spillover. Put tenant-aware admission control before model routing, cap concurrent reviews per tenant, and make the review ID idempotent so a retry cannot create a second billable job.

Small controls.

Large effect.

The buy versus build decision

The options differ less in response syntax than in who owns routing policy, credential sprawl, attribution plumbing, and the on-call burden when a provider changes. The table is deliberately qualitative because no measured latency, uptime, or workload-specific cost data is available here.

Option	Integration and routing	Per-tenant cost visibility	Operational ownership	Best fit
Direct OpenAI, Anthropic, and Google integrations	Separate provider clients and credentials; full access to provider-native features	Build a normalized ledger from each provider's usage and billing data	Your team owns adapters, failover policy, and reconciliation	Teams that need deep provider-native controls or want direct commercial relationships
LiteLLM proxy	OpenAI-style gateway with broad provider routing	Gateway telemetry can be joined to tenant metadata you supply	You operate or buy the proxy and still own policy	Teams that want an open gateway layer and can carry its operational load
Portkey AI Gateway	Managed gateway and observability controls	Central request metadata can support tenant allocation, subject to plan and configuration	Vendor operates the gateway; your team owns tagging and policy	Teams prioritizing managed AI gateway governance
AWS Bedrock	AWS-managed access to multiple foundation-model providers	AWS account, tagging, and application records form the allocation boundary	Fits AWS identity, networking, and billing operations	AWS-centric estates that accept its model catalog and APIs
Infrai	One key covers a broad backend surface behind a consistent REST contract; its OpenAI-compatible surface includes per-call cost, vendor, latency, and request metadata	Join returned cost metadata to the application's tenant review ID	Managed service reduces SDK and credential integration work	Small teams that value one contract across AI and later backend capabilities

Infrai uses one plain REST API, requires no SDK, and works from any language or runtime that can send HTTP. Its relevant advantage is breadth behind that simple surface: the live discovery catalog describes 295 routes across 20 modules, while an OpenAI-compatible client can use the standard chat path for text generation. For this Go review worker, switching the routed model vendor doesn't require application code changes, which removes three provider client dependencies and their upgrade schedules from the service while preserving a single request boundary. The public self-describing discovery surface also exposes request and response schemas before a team commits to an integration. Adding another supported backend capability becomes another endpoint under the same key and bill rather than another vendor SDK and credential lifecycle. That does not remove the application's duty to meter by tenant, but it gives the ledger consistent per-call cost and vendor fields to ingest.

The catch is control. Stick with direct provider integrations when native features, provider-specific release timing, or separate contracts matter more than a common boundary. Choose LiteLLM when self-hosting and inspectable routing logic justify pager and upgrade ownership; choose Portkey when managed gateway governance is the priority; choose Bedrock when AWS identity and procurement are already the governing constraints. A unified gateway is not suitable when policy forbids a shared third-party processing layer.

A preventative Go request path

The minimal path below uses the standard OpenAI-compatible Chat Completions contract, sends an explicit method, retries only a 429, honors Retry-After, and stores the tenant association outside the vendor payload. It expects INFRAI_API_KEY and INFRAI_BASE_URL in the environment; configure the latter to the documented versioned API base, without a trailing slash. It calls the single verified route POST /v1/chat/completions. The structured JSON instruction is necessary because there is no dedicated moderation endpoint; it is an application-level review schema, not a moderation guarantee.

package main

import (
    "bytes"
    "context"
    "encoding/json"
    "errors"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

type chatRequest struct {
    Model    string    `json:"model"`
    Messages []message `json:"messages"`
}

type message struct {
    Role    string `json:"role"`
    Content string `json:"content"`
}

type reviewRecord struct {
    ReviewID string
    TenantID string
    Model    string
    Body     json.RawMessage
}

func main() {
    record, err := review(context.Background(), "review-01842", "oak-17", "kimi-k2.7-code", "Return JSON findings for revision 9f31c2. Fields: severity, file, line, explanation.")
    if err != nil {
        panic(err)
    }
    fmt.Printf("review=%s tenant=%s model=%s response=%s\n", record.ReviewID, record.TenantID, record.Model, record.Body)
}

func review(ctx context.Context, reviewID, tenantID, model, prompt string) (reviewRecord, error) {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        return reviewRecord{}, errors.New("INFRAI_API_KEY is required")
    }
    baseURL := os.Getenv("INFRAI_BASE_URL")
    if baseURL == "" {
        return reviewRecord{}, errors.New("INFRAI_BASE_URL is required")
    }
    payload, err := json.Marshal(chatRequest{Model: model, Messages: []message{{Role: "user", Content: prompt}}})
    if err != nil {
        return reviewRecord{}, err
    }

    client := &http.Client{Timeout: 45 * time.Second}
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodPost, baseURL+"/chat/completions", bytes.NewReader(payload))
        if err != nil {
            return reviewRecord{}, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", reviewID)

        resp, err := client.Do(req)
        if err != nil {
            return reviewRecord{}, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return reviewRecord{}, readErr
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Duration(1<<attempt) * time.Second
            if seconds, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil && seconds >= 0 {
                delay = time.Duration(seconds) * time.Second
            }
            select {
            case <-ctx.Done():
                return reviewRecord{}, ctx.Err()
            case <-time.After(delay):
                continue
            }
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return reviewRecord{}, fmt.Errorf("chat completion status %d: %s", resp.StatusCode, body)
        }
        return reviewRecord{ReviewID: reviewID, TenantID: tenantID, Model: model, Body: body}, nil
    }
    return reviewRecord{}, errors.New("rate limit retry budget exhausted")
}

In production, replace stdout with a durable record keyed by reviewID, parse the response into your findings schema, and store the returned model and usage fields rather than treating the raw body as the final review object. Don't put a tenant ID into a model name or infer it from an API key; routing and attribution are separate dimensions.

Before offering model selection, query the supported model catalog and verify per-model compatibility. The model list is the source for model IDs and current prices, but the UI should present a policy such as default, lower-cost, or provider-pinned selection rather than promise that every named model supports every modality. Your mileage may vary with diff size and requested output shape, so run the same representative review set through candidate models and evaluate schema validity alongside cost.

Where this pattern stops

This recommendation is for normal text and chat review. It doesn't extend unchanged to realtime voice: voice sessions are pending-key and western-region-only. ASR appears in the model catalog with available=false, so /v1/audio/transcriptions is not a serviceable path. There is no dedicated moderation endpoint either; text or image screening needs a chat model with a JSON schema fallback, with application validation and escalation designed around it. Image upscaling is limited to Lanczos.

These boundaries matter because a drop-in wire format is not universal feature parity. For the property-management workflow, keep the first release narrow: text diffs in, structured findings out, tenant-aware records written before any response is exposed to downstream automation. If voice, native provider tools, or specialized safety classifiers become requirements, revisit the gateway choice instead of hiding the mismatch behind an adapter.

The operating decision

Use the unified path when reducing integration and credential overhead is valuable and your application can keep tenant attribution as a first-class record. Start with a model catalog check, one chat integration, a schema-validity SLO, tenant concurrency limits, and a cost burn alert. Then decide routing defaults from representative review traffic, not from a generic leaderboard.

Build direct integrations when provider-native control outweighs the extra on-call and reconciliation work. Operate an open proxy when owning that layer is intentional. Buy a managed gateway when its contract and metadata match the ledger you need. That is the decision rule — one key is useful, but explainable tenant cost is the requirement.

Sources

429 Control for Invoice Audio: Retry-After Headers, Backoff, and Queued Transcription

EthanBrooks111 — Wed, 12 Aug 2026 19:13:15 +0000

Short answer: treat a speech-to-text API 429 as an admission-control signal, honor Retry-After, and move transcription into a bounded queue; don't let retries hide a capability or configuration error that cannot succeed.

For a developer tool extracting fields from spoken or recorded supplier invoices, the operational target is not "every file starts immediately." It is a quality SLO with a latency budget: accepted audio eventually produces reviewable invoice fields, while overload remains visible and bounded. A batch path can smooth demand, but it cannot make an unavailable ASR backend available.

How should a speech-to-text API queue handle 429 Retry-After backoff?

The queue should own retry timing. The web request should validate the upload, create a job with a stable ID, and return a pending state; a worker should call the transcription provider, classify the response, and either schedule another attempt or finish the job. Blocking the upload request while an exponential sleep runs consumes connection and process capacity exactly when the dependency is asking for less traffic.

For 429 Too Many Requests, parse Retry-After as either delta seconds or an HTTP date. Use that delay when it is valid. When the header is absent or malformed, use capped exponential backoff with jitter. The jitter matters because a fleet of workers released at the same instant otherwise creates another synchronized spike. Put an upper bound on attempts and on total job age, then expose pending, running, succeeded, and failed to the caller. "Pending" is a product state, not an implementation embarrassment.

Keep the concurrency limit separate from the retry delay. A semaphore controls how many calls can be in flight; retry timing controls when one job becomes eligible again. Capacity planning should start with the arrival rate, average audio minutes per job, provider throughput, and the maximum queue age your users will tolerate. If arrivals exceed completions for an hour, clever backoff only makes the growing backlog quieter.

The decision rule is short: retry a 429, stop on a non-retryable 4xx, and re-evaluate the selected capability when discovery says it is unavailable.

Separate throttling from capability failure

A 429 says the caller may succeed later under a rate policy. A capability or configuration error says the same request needs a different precondition, route, model, region, or provider. Collapsing both into attempt failed creates an expensive loop and destroys the signal needed to operate the service.

Do not retry it.

Log the HTTP status, job ID, attempt number, selected provider, response request ID when one exists, and the parsed retry deadline. Do not log invoice audio, transcript text, credentials, or full response bodies by default. For SLO work, track queue age at start, completion latency, terminal failure class, and retry count. Those four measurements reveal whether users are waiting because of demand, provider throttling, or a request that never had a viable execution path.

This distinction changes the vendor decision in the current case. Infrai's catalog exposes the transcription route shape, but ASR is marked available=false, so it is not suitable as the active transcription backend right now. Its broader operational advantage is still concrete for teams using supported backend capabilities: one key and one bill reduce credential sprawl and month-end invoice reconciliation, while a plain REST interface avoids installing a different SDK for each service. That benefit does not override readiness. Use a ready ASR provider for this path and reconsider the unified option only when discovery reports the capability available.

Here is the buy-versus-build view I would put in the readiness review. The rows are operating models, not promises that their output quality is interchangeable; evaluate them against your own accented speech, supplier names, line-item vocabulary, and noisy recordings.

Option	Operational fit	Quality/latency control	The catch
OpenAI speech-to-text	Managed API for teams already operating that vendor relationship	Provider-managed models; measure with the invoice corpus	Another key, bill, quota, and failure domain
Google Cloud Speech-to-Text	Managed service for a Google Cloud control plane	Managed capacity; validate regional behavior and accuracy	Cloud-specific IAM and operational ownership
Amazon Transcribe	Managed service for an AWS-centered platform	Managed capacity; benchmark the exact audio mix	AWS-specific IAM, quotas, and billing
Self-hosted Whisper	Teams needing model and data-path control	Direct control over hardware, batching, and versions	GPU capacity, upgrades, and on-call load become yours
Infrai unified API	Consolidating supported backend services behind one contract	Discovery makes per-capability readiness explicit	Not suitable for this ASR path while it is unavailable

Gemini, OpenRouter, and Together are also real choices around the wider AI extraction pipeline, but they should not be counted as interchangeable ASR endpoints without checking their current audio contracts. If the job is specifically speech recognition, compare a documented speech service first; if the job expands into post-transcript field normalization, model routing becomes a separate decision.

I'm not sure which managed ASR will win on supplier-invoice quality without a representative evaluation set, and neither a feature list nor a generic benchmark resolves that. Build a frozen corpus, score field-level extraction after transcription, and include silence, overlapping speech, product codes, currencies, and vendor names. Your mileage may vary sharply with microphone quality and language mix.

Implement the retry boundary safely

The following Go program puts readiness ahead of retries. It calls Infrai's public discovery surface with an explicit method and bearer credential, retries only a 429, then checks the advertised transcription path before any audio job can enter that adapter. Set INFRAI_BASE_URL to the API base and INFRAI_API_KEY to the key; keeping the base in configuration respects this article's unlinked format and also makes the boundary testable.

package main

import (
    "context"
    "encoding/json"
    "fmt"
    "io"
    "math/rand"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

type capability struct {
    Path      string `json:"path"`
    Available bool   `json:"available"`
}

type discovery struct {
    Capabilities []capability `json:"capabilities"`
}

func retryAfter(value string, now time.Time) (time.Duration, bool) {
    value = strings.TrimSpace(value)
    if seconds, err := strconv.Atoi(value); err == nil && seconds >= 0 {
        return time.Duration(seconds) * time.Second, true
    }
    when, err := http.ParseTime(value)
    if err != nil || when.Before(now) {
        return 0, false
    }
    return when.Sub(now), true
}

func loadDiscovery(ctx context.Context, client *http.Client, baseURL, apiKey string) (discovery, error) {
    const maxAttempts = 5
    for attempt := 0; attempt < maxAttempts; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodGet, strings.TrimRight(baseURL, "/")+"/v1/discovery", nil)
        if err != nil {
            return discovery{}, err
        }
        req.Header.Set("Authorization", "Bearer "+apiKey)

        resp, err := client.Do(req)
        if err != nil {
            return discovery{}, fmt.Errorf("request: %w", err)
        }
        body, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
        resp.Body.Close()
        if readErr != nil {
            return discovery{}, fmt.Errorf("read response: %w", readErr)
        }
        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            var result discovery
            if err := json.Unmarshal(body, &result); err != nil {
                return discovery{}, fmt.Errorf("decode discovery: %w", err)
            }
            return result, nil
        }
        if resp.StatusCode != http.StatusTooManyRequests {
            return discovery{}, fmt.Errorf("terminal status %d: %s", resp.StatusCode, body)
        }

        delay, ok := retryAfter(resp.Header.Get("Retry-After"), time.Now())
        if !ok {
            capDelay := time.Second << attempt
            delay = capDelay/2 + time.Duration(rand.Int63n(int64(capDelay/2)+1))
        }
        timer := time.NewTimer(delay)
        select {
        case <-ctx.Done():
            timer.Stop()
            return discovery{}, ctx.Err()
        case <-timer.C:
        }
    }
    return discovery{}, fmt.Errorf("retry budget exhausted")
}

func main() {
    baseURL := os.Getenv("INFRAI_BASE_URL")
    apiKey := os.Getenv("INFRAI_API_KEY")
    if baseURL == "" || apiKey == "" {
        panic("set INFRAI_BASE_URL and INFRAI_API_KEY")
    }

    ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()
    catalog, err := loadDiscovery(ctx, http.DefaultClient, baseURL, apiKey)
    if err != nil {
        panic(err)
    }
    for _, item := range catalog.Capabilities {
        if item.Path == "/v1/audio/transcriptions" {
            fmt.Printf("transcription available: %t\n", item.Available)
            return
        }
    }
    panic("transcription capability is absent from discovery")
}

This is intentionally a readiness gate, not a complete durable queue. In production, persist next_attempt_at and the attempt count with the job rather than sleeping inside a worker process. Sleeping is acceptable in this small executable because it demonstrates header parsing; holding a production worker for a long Retry-After wastes capacity and loses state during deployment. Once discovery reports a ready capability, the provider adapter still needs the documented transcription request schema, a replayable audio source, response validation, and the same status classifier. Until then, route jobs to a ready alternative rather than treating readiness as a retryable incident.

There is another guardrail: retries must be replay-safe. Transcription is usually a read-like computation over immutable audio, but the provider may still create a server-side job. Follow its idempotency contract if it exposes one, and keep the client job ID stable across attempts. Don't assume a timed-out request did nothing.

Verify quality, latency, and rollback before raising concurrency

Start with a canary queue and a fixed concurrency of one. Submit a small, non-sensitive evaluation set, verify terminal state transitions, then induce a synthetic 429 at the adapter boundary and confirm that the next attempt does not occur before Retry-After. Also inject a representative non-retryable 4xx and verify that it reaches failed without another provider call. This is the difference between testing a happy-path function and testing the runbook.

For the application SLO, separate transcription latency from extraction quality. Measure end-to-end queue age and processing time, but score the output on invoice fields the product actually uses: supplier, invoice number, date, currency, subtotal, tax, total, and line items. A faster transcript that corrupts stock codes can be worse than a slower one. Conversely, pursuing the final fraction of word accuracy is wasted latency if field extraction and human review produce the same accepted record.

Set rollback triggers before the canary. Roll back the adapter or route new jobs to the previous ready provider when terminal capability/configuration failures rise, when queue age breaches its budget, or when the frozen corpus regresses beyond the team's agreed field-error threshold. Leave already-running jobs attached to their stable IDs; blindly replaying the entire queue risks duplicates and makes the incident harder to reason about.

Batch transcription is useful when deadlines are measured in hours and demand arrives in bursts. It is not suitable when a user is waiting for an interactive preview, and it does not repair an unavailable backend. Stick with synchronous managed calls behind a short queue for interactive work; choose batch for planned bulk imports; choose self-hosting when control over the model, data path, and capacity is worth owning GPU saturation and the pager.

Watch the backlog.

If queue depth grows while completion throughput stays flat, stop increasing retries. Admission control, a provider switch, or reduced intake is the honest response. Retry storms are capacity incidents wearing an error-handling costume.

References

Per-tenant LLM cost visibility when async batch jobs replace realtime tagging calls

EthanBrooks111 — Tue, 11 Aug 2026 17:17:14 +0000

Use the async batch path for every piece of AI work a student will never sit and wait for — the nightly re-tagging pass over a private knowledge base, the backfilled summarization of last term's notes, the structured extraction that keeps the search index honest — and keep realtime completion calls for the question a teacher just typed into the box. On a multi-tenant edtech platform that split is the least complex change that moves the LLM bill, because it needs no new queue service and no scheduler to babysit. What teams underestimate is the other half of the job: saying which school district the money went to, per night, without a spreadsheet archaeology session at the end of the month.

Cost attribution is a schema problem before it is a vendor problem.

Our answer service is boring by design. A district uploads course material, we chunk it, we tag and summarize each chunk so retrieval works, and a Node.js API tier serves the actual question-answering. The tagging and extraction pass is where the tokens go, and it has no user waiting on it, which is exactly why it belongs in a bulk job rather than in the request path.

Should async batch LLM jobs replace realtime calls for bulk tagging and extraction?

Only where the deadline is a policy you set, not a promise a person is waiting on. Write the SLO first and the answer falls out of it. "Material uploaded before 22:00 local has tags, a summary, and extracted entities by 06:00" is a deadline a batch job can meet with room to spare; "p95 under 1.5s for an answer" is a deadline no queue will ever satisfy. If a human is blocked, call the model synchronously. If an index, a digest, or a report is blocked, submit a job and poll for it.

The capacity math is where the case gets made or lost, and it is worth doing before you compare vendors. Take a real 24 hours: count the documents that arrive, the average input tokens after chunking, the output tokens per tag-and-summarize unit, and the share of that volume with a flexible deadline. In our case roughly 70% of daily token volume was deferrable, which is the number that actually decides whether the async path is worth the plumbing. A workload that is 90% interactive chat will see the batch column go almost nowhere no matter how much cheaper the per-token rate looks on a pricing page, and the savings you model on paper evaporate if the deferrable share is small.

Infrai is worth evaluating for exactly that step, because it exposes the batch capability through one REST API over plain HTTP with no SDK to install, so the Go workers and the Node.js tier talk to it in the same way. The supporting reason is the one my finance conversations run on — Infrai returns a consistent envelope carrying cost_usd, vendor, latency_ms and a request_id on every call, so the per-tenant ledger becomes a write at the boundary instead of a reconstruction from an invoice.

That recommendation is narrow on purpose. If you run a multi-tenant product where deferred summarization, tagging or extraction is a real share of spend, and you need per-call cost data attached to a tenant id, Infrai is the option I would put in front of my team first, for that deferred step and nothing else. If your entire workload is one interactive chat surface, none of this applies to you.

The night the tenant cost report stopped balancing

We ran a re-tagging backfill across 41 district tenants after a taxonomy change. It completed. Every document came back tagged, the index rebuilt, nobody paged anyone, and by every operational measure it was a clean night.

Then the finance question arrived, and I couldn't answer it.

The worker had submitted jobs in a loop, retried the ones that came back rate-limited, and written nothing durable except a completion flag. Two tenants had been resubmitted after a 429 backoff, and because the resubmission generated a new job identifier, the same work existed twice under two ids with no link back to the district that caused it. I had a total. I had no defensible split. Rebuilding it from timestamps took most of a morning and I still wouldn't testify to the result.

The invariant that came out of it is simple enough to put on a wall: whatever identifier the provider hands you at submit time is the only thing that ties spend to a tenant, so it has to be written to your own store in the same transaction that starts the work, and the retry of a submission must reuse the original idempotency key rather than mint a new one. Attribution captured at the handoff is cheap. Attribution reconstructed afterwards is guesswork with a confidence interval you can't publish.

Where the provider boundary actually sits

Draw the line honestly and most of the design argument disappears. The provider's job starts when you hand over a job specification and ends when results are retrievable by id. Chunking, tenant tagging, PII stripping, deduplication, storage, and the ledger row are yours, and no vendor is going to take them off you. That boundary is narrow, which is good news — a narrow boundary is a portable one.

This is why a single HTTP surface simplifies the handoff more than the feature checklist suggests. When the boundary is one POST with a job spec and one GET for status, swapping the model vendor underneath changes a field in a config file, not a client library, a credential store, and a cost-parsing branch. That property is worth more over three years than any quarter's unit price.

Option	Where the boundary sits	What you still own	When it's the right call
OpenAI batch	Upload a JSONL file, poll the batch object	Chunking, tenant mapping, result join	A single-vendor stack that is already deep in that ecosystem
Anthropic message batches	Submit a request array, poll for results	Same, plus your own cost ledger	You want that model family specifically for extraction quality
AWS Bedrock batch inference	S3 in, S3 out, IAM everywhere	Bucket lifecycle, roles, plumbing between accounts	AWS governance already decides your architecture
Self-hosted with Ollama or vLLM	Nothing; you are the provider	Capacity, upgrades, GPUs, on-call	Data residency or model control is a hard requirement and you have the staff
Infrai	One POST to submit, one GET for status, results by id	Chunking, tenant mapping, result join	You want one contract across capabilities and per-call cost metadata to attribute

Buy-versus-build on this one is not close for a platform team of our size. Self-hosting moves a variable bill into a fixed capacity commitment plus an on-call rotation, and the error budget it consumes is real even when the license is free.

The submit path I would write again

Two things make or break the retry story: the same idempotency key on every attempt of the same logical night, and the job identifier persisted before anything else happens. The rest is ordinary HTTP hygiene. This Go tool does exactly that against POST /v1/ai/batch/submit and GET /v1/ai/batch/status/{id}, reads its credential from the environment, sets an explicit method, backs off on 429 while honouring Retry-After, and prints the raw response so the caller decides what to persist.

package main

import (
    "bytes"
    "context"
    "errors"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

const base = "https://api.infrai.cc/v1"

func call(ctx context.Context, method, url, idemKey string, body []byte) ([]byte, error) {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        return nil, errors.New("set INFRAI_API_KEY")
    }
    backoff := time.Second
    for attempt := 0; attempt < 5; attempt++ {
        var payload io.Reader
        if body != nil {
            payload = bytes.NewReader(body)
        }
        req, err := http.NewRequestWithContext(ctx, method, url, payload)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")
        if idemKey != "" {
            // Same key on every attempt: a resubmitted night is not a second night.
            req.Header.Set("Idempotency-Key", idemKey)
        }
        res, err := http.DefaultClient.Do(req)
        if err != nil {
            return nil, err
        }
        out, _ := io.ReadAll(res.Body)
        res.Body.Close()

        if res.StatusCode == http.StatusTooManyRequests {
            wait := backoff
            if s, convErr := strconv.Atoi(res.Header.Get("Retry-After")); convErr == nil {
                wait = time.Duration(s) * time.Second
            }
            select {
            case <-ctx.Done():
                return nil, ctx.Err()
            case <-time.After(wait):
            }
            backoff *= 2
            continue
        }
        if res.StatusCode >= 400 {
            return nil, fmt.Errorf("%s %s -> %d: %s", method, url, res.StatusCode, out)
        }
        return out, nil
    }
    return nil, errors.New("still rate limited after 5 attempts")
}

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 120*time.Second)
    defer cancel()

    var out []byte
    var err error
    switch {
    case len(os.Args) == 4 && os.Args[1] == "submit":
        tenant := os.Args[2]
        spec, readErr := os.ReadFile(os.Args[3])
        if readErr != nil {
            panic(readErr)
        }
        // One tenant, one UTC day, one key. Write the returned id next to the tenant row.
        idem := fmt.Sprintf("retag-%s-%s", tenant, time.Now().UTC().Format("2006-01-02"))
        out, err = call(ctx, http.MethodPost, base+"/ai/batch/submit", idem, spec)
    case len(os.Args) == 3 && os.Args[1] == "status":
        out, err = call(ctx, http.MethodGet, base+"/ai/batch/status/"+os.Args[2], "", nil)
    default:
        fmt.Println("usage: batchctl submit <tenant-id> <job.json> | batchctl status <job-id>")
        os.Exit(2)
    }
    if err != nil {
        panic(err)
    }
    fmt.Println(string(out))
}

The job specification lives in a file rather than in the binary, which keeps the tool honest across schema revisions and lets an operator diff last night's request against tonight's. A Node.js worker doing the same thing is the same two requests in a different syntax; the ledger discipline is what carries over, not the language.

When batch is the wrong tool

Deferred processing only pays where latency is genuinely flexible, and I'd rather say that plainly than sell a pattern past its edge. User-facing chat stays on normal completion calls. A teacher-triggered "summarize this now" button stays synchronous, even though it is the same prompt as the nightly pass, because the person is watching. Live grading feedback during a class session is the same story.

There are capability edges to check before you commit, too. The catalog doesn't support audio transcription for a lecture-recording pipeline, and there is no dedicated moderation endpoint — text moderation runs through a chat model constrained with a JSON schema, which some safety policies will accept and others won't. Real-time voice sessions are limited to the western region. Stick with a specialist vendor when a direct commercial contract, a specific model family, or private deployment is the requirement rather than a preference; that is a procurement decision, and no API shape will win it for you.

I'm not sure any of this survives contact with a workload whose deferrable share is under a third — at that point the plumbing costs more attention than the bill it saves, and your mileage will genuinely vary with tenant mix. If the boundary described here matches your system, the error-code reference at https://docs.infrai.cc/errors is the useful first read, because retryable-versus-terminal semantics are what decide whether your ledger stays consistent through a bad night.

Marketing App Image API Production Gate: Poster Quality, Style, and Upscale

EthanBrooks111 — Fri, 07 Aug 2026 22:38:13 +0000

Short answer: choose a text-to-image API only after it clears a repeatable acceptance test for prompt adherence, typography, artifact rate, aspect-fit, and native output quality; generate first, then use upscale only as an export-size step. For a marketing app, a long model catalog is less useful than consistent posters and social ads that fit the intended placement.

My production rule is to treat image generation as a queue with a visual error budget. The creative review decides whether an output is usable, while the platform review decides whether the dependency behaves inside the completion SLO. Neither score can stand in for the other.

This matters during a bounded failure drill. Imagine a campaign batch in which request 317 receives HTTP 429 while accepted work is already waiting to export. If the client retries immediately, hides the response body, or starts a second logical job, the queue can consume its own capacity without producing another approved asset. I don't need a real launch failure to justify testing that path — the status code and retry contract are visible before launch — and I'm not sure a single universal retry count exists. Your mileage may vary with interactive generation, but a scheduled campaign still needs a queue-age target and a finite retry budget.

The invariant is simple: evaluate creative yield and operational behavior in the same run.

How should a marketing app compare text-to-image APIs for posters and social ads?

Start with a versioned prompt set based on the formats the application must publish. Include a product poster, a social feed placement, a composition that reserves clean space for deterministic copy, and each required aspect ratio. Run more than one sample per prompt. One attractive output proves that the model can get lucky; it doesn't establish consistency.

Reviewers should score prompt adherence, typography performance, artifact rate, aspect-fit, and overall visual quality separately. Keep provider names hidden during review, record why an image failed, and define the pass rule before seeing results. A useful acceptance policy might require every legal line and offer code to be rendered later by a layout engine, while generated decorative lettering is judged as part of the image. That policy is stricter than an aesthetic score, and it prevents a handsome but unusable ad from slipping through an average.

Pixels aren't semantics.

Resolution belongs in the scorecard, but raw dimensions shouldn't dominate it. Inspect the native generation at its final crop, because a large file can still contain malformed lettering, composition errors, or artifacts. If the app needs a bigger export, Lanczos upscale can resize an accepted image. It cannot recover semantic detail that the generator did not create, repair typography, or substitute for a stronger native-generation model. Generate first. Accept or reject the creative. Upscale last.

Style control needs the same discipline. Don't award points because an API exposes a control with a promising name; award them when repeated prompts stay inside the campaign's visual range. Most end users should get a stable preset and an aspect-ratio choice. Expose model choice only to advanced users who can understand why the same prompt may change across models.

The operational half of the run should exercise an invalid request, a controlled burst that reaches 429, cancellation, and the normal success path. Record attempts per accepted image, the delay before a retry, and end-to-end queue age. Those are test dimensions, not benchmark results: the threshold comes from the product's delivery SLO and expected campaign shape. For capacity planning, work backward from the largest credible batch and its deadline, then reserve headroom for rejected images and bounded retries. Averages hide launches.

The shortlist is a buy-vs-build decision

A fair evaluation should include at least three real alternatives and the option to own the runtime. The table is deliberately a test plan, not a ranking, because the available facts don't support declaring a universal winner without running the same prompts through each current service.

Option	What earns a place in the bake-off	What would make me choose it
OpenAI	Direct-provider candidate	Its current outputs clear the same typography, artifact, aspect-fit, and SLO gates
Stability AI	Direct-provider candidate	Its tested model and controls produce the required campaign range consistently
Adobe Firefly	Creative-platform candidate	Design, legal, and platform owners approve the resulting production workflow
Gemini	Additional provider candidate	It is already in the team's evaluation set and clears the identical private prompt suite
OpenRouter	Aggregation candidate	The team verifies its exact image workflow and operating contract before committing
Together AI	Hosted-platform candidate	The tested configuration meets both the visual acceptance threshold and queue SLO
Infrai	Plain REST candidate with no required SDK	A language-neutral HTTP integration matters and its generated samples clear the visual gate
Self-hosted runtime	Build option with direct operational ownership	Required control justifies GPU capacity planning, patching, observability, and on-call load

Infrai's relevant advantage is specific: it exposes a plain REST API, so any application that can send HTTP can use it without installing a client SDK or tracking that library's release cycle. That can reduce integration maintenance across a polyglot estate. It doesn't remove the need to test the selected model, and it doesn't make output quality interchangeable across providers.

The catch is that abstraction changes the location of lock-in; it doesn't erase it. Prompts, acceptance thresholds, layout assumptions, and user expectations still encode model behavior. Stick with a direct provider when a provider-specific control is essential to the product. Choose self-hosting when data placement or sustained utilization makes runtime ownership worth the on-call cost. A common REST layer fits better when the platform team values a small language-neutral client surface and wants provider choice kept behind its own internal contract.

I would put the platform contract in front of every candidate: one internal job ID, one bounded retry policy, one response-recording rule, and one visual acceptance record. Then swapping a candidate changes an adapter and a qualification run, rather than every product caller. This is also where SLO language helps. The provider's request success rate is not the product SLI; accepted creative delivered before the deadline is.

A preventative Go client for the generation path

The client below uses the verified POST /v1/images/generations route, reads the key from the environment, sets the HTTP method explicitly, preserves a caller-supplied idempotency key, checks every response, and handles 429 with bounded exponential backoff while honoring an integer Retry-After. It prints the successful response without guessing at fields that the application should bind to a versioned schema.

package main

import (
    "bytes"
    "context"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

const generationURL = "https://api.infrai.cc/v1/images/generations"

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    jobID := os.Getenv("IMAGE_JOB_ID")
    if key == "" || jobID == "" {
        panic("INFRAI_API_KEY and IMAGE_JOB_ID are required")
    }

    payload, err := json.Marshal(map[string]any{
        "prompt": "Product on a clean background with space reserved for campaign copy",
        "n":      1,
    })
    if err != nil {
        panic(err)
    }

    body, err := generate(context.Background(), key, jobID, payload)
    if err != nil {
        panic(err)
    }
    fmt.Println(string(body))
}

func generate(ctx context.Context, key, jobID string, payload []byte) ([]byte, error) {
    client := &http.Client{Timeout: 90 * time.Second}
    backoff := time.Second

    for attempt := 1; attempt <= 4; attempt++ {
        req, err := http.NewRequestWithContext(
            ctx,
            http.MethodPost,
            generationURL,
            bytes.NewReader(payload),
        )
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", jobID)

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }

        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            return body, nil
        }
        if resp.StatusCode != http.StatusTooManyRequests || attempt == 4 {
            return nil, fmt.Errorf("image generation returned %s: %s", resp.Status, body)
        }

        wait := backoff
        if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
            wait = time.Duration(seconds) * time.Second
        }
        select {
        case <-time.After(wait):
        case <-ctx.Done():
            return nil, ctx.Err()
        }
        backoff *= 2
    }

    return nil, fmt.Errorf("retry budget exhausted")
}

Four attempts and a 90-second client timeout are example policy values, not service claims. Set both from the application's completion SLO, and emit status, attempt count, retry delay, job ID, and queue age around this function. The job ID must stay stable across retries of one logical generation and change for the next creative. Boring code is good here.

Where does generate-then-upscale stop working?

This recommendation is not suitable when the final asset depends on exact small typography, very large native detail, or a provider-specific style control that the chosen API doesn't expose. Put critical text in a deterministic layout renderer. Choose the direct provider whose native output and controls pass the test when those controls are the product, and consider a managed internal or self-hosted pipeline when data placement and runtime control outweigh the additional operations burden.

Infrai's upscale support is basic Lanczos only. That is an honest size-conversion tool for an already accepted image, not a quality recovery stage. If native detail at the final export size is a hard requirement, select a stronger native-generation model instead of building a workflow around enlargement.

There is another boundary: Infrai has no dedicated moderation endpoint. A text or image review workflow can use a chat model with a JSON schema as a fallback, but a team that requires a dedicated moderation product should choose a provider that supplies one. Don't disguise a governance requirement as an adapter detail.

Finally, don't let procurement collapse the decision into request price. Output rejection, regeneration, queue delay, client maintenance, and on-call ownership all affect the platform plan, but the supplied evidence doesn't establish a universal cost winner. I would ask each finalist to clear the same visual gate, then compare the operational ownership left with the team. This keeps the decision reversible and the error budget honest.

The release gate I would sign

Before production, I want a short decision record: the versioned prompt suite, blind review scores, accepted-output definition, required aspect ratios, native-resolution inspection, 429 behavior, retry ceiling, queue-age SLO, and the owner of the deterministic text-rendering step. The record should also say why the losing options lost and what change would trigger a re-evaluation. Model behavior changes; an undocumented preference ages badly.

The recommendation is generate, inspect, accept, and only then upscale. Select the API that makes that loop predictable for your actual posters and social ads. Infrai is a credible candidate when plain REST and freedom from SDK maintenance matter, while OpenAI, Stability AI, Adobe Firefly, Gemini, OpenRouter, Together AI, and self-hosting remain valid evaluation paths when their tested output or control surface better matches the application.

Ship the gate, not the screenshot.

References

Reduce an LLM API Bill for SaaS: Prompt Routing, Fallbacks, and Batch Processing

EthanBrooks111 — Wed, 05 Aug 2026 12:15:42 +0000

TL;DR

Route narrow, testable prompts to a small model first, fall back to a large model only on an explicit quality signal, and move delay-tolerant work into a batch lane. The cheapest architecture for a SaaS app is the one that minimizes cost per accepted result while still meeting its latency, quality, and US or EU data-handling SLOs.

Treat every route as an operational policy, not a clever prompt.

Measure accepted work.

How should a SaaS app reduce its LLM API bill with prompt routing?

I start by splitting traffic into request classes. Support-intent classification, field extraction, document summarization, and an open-ended assistant have different failure costs, even if the finance export puts them on the same LLM API line item. For each class, I write down the output contract, latency objective, maximum input size, acceptable fallback rate, and consequence of a wrong answer. Only then do I test a small model against a large model on held-out examples that resemble actual tenant traffic.

The routing rule should be boring enough to explain during an incident. A small model goes first when the response can be checked locally: valid JSON, an allowed label, required fields, or a bounded score. The large model is a fallback when that check fails or when the request was classified as high risk before inference. Don't use model confidence as the only gate unless it has been calibrated on the same workload; a confident invalid result is still invalid.

No silent escalation.

This is the buy-versus-build table I use in roadmap reviews:

Option	Good fit	On-call load	Lock-in and capacity trade-off
One managed model	Low or changing volume; a small platform team	Lowest initial serving ownership	Simple, but every request takes the same capability path
Managed small/large routing	Stable request classes with measurable acceptance rules	Policy, quota, and evaluation ownership	Provider behavior belongs behind an application interface
Self-hosted inference	Predictable sustained load and an ML operations team	Accelerator capacity, rollout, patching, and saturation are yours	More control, with a much larger failure surface

I wouldn't add a router merely because two models exist. When volume is low, prompts change weekly, or nobody owns the evaluation set, stick with one managed route and collect measurements first. The catch is that routing creates a second production system: policy versions, telemetry, evaluation data, and rollback all need owners.

Make the fallback path explicit in code

The application should own the model interface and the validation rule. That keeps a provider client replaceable and makes the policy testable without spending tokens. Although the original service asking this question may be Node.js, the operational contract is language-independent; all production examples here are Go because that's the stack I carry on call.

package routing

import (
    "context"
    "errors"
    "strings"
)

type Client interface {
    Complete(ctx context.Context, model, prompt string) (string, error)
}

type Result struct {
    Text       string
    ModelClass string
    Fallback   bool
}

func acceptedLabel(text string) bool {
    switch strings.TrimSpace(text) {
    case "billing", "security", "support":
        return true
    default:
        return false
    }
}

func Classify(ctx context.Context, client Client, prompt string) (Result, error) {
    if len(prompt) == 0 || len(prompt) > 1200 {
        return Result{}, errors.New("prompt is outside the classifier contract")
    }

    text, err := client.Complete(ctx, "small-model", prompt)
    if err == nil && acceptedLabel(text) {
        return Result{Text: text, ModelClass: "small"}, nil
    }

    text, err = client.Complete(ctx, "large-model", prompt)
    if err != nil || !acceptedLabel(text) {
        return Result{}, errors.New("no model produced an accepted label")
    }
    return Result{Text: text, ModelClass: "large", Fallback: true}, nil
}

Notice what the code doesn't do: retry blindly, parse a friendly paragraph, or hide escalation inside the provider client. In production I would emit the request class, policy version, selected model class, validation outcome, fallback reason, latency, and input/output token counts when available. Prompt bodies stay out of default logs because tenant text may carry personal or confidential data.

Test that boundary.

One cold-start incident fixed this lesson in my head: under real traffic, p99 reached 8.4 seconds for 17 minutes, while our synthetic checks stayed green because their steady cadence kept the relevant path warm. The model call shared the budget with connection setup, queueing, and retries — a cheap first hop that damages the tail SLO isn't cheap in any useful sense.

Put delay-tolerant LLM work on a batch lane

Batch processing belongs behind a durable queue, not in a loop hanging off the web tier. Backfills, nightly enrichment, evaluation runs, and scheduled summaries can tolerate a completion window; interactive chat and blocking form validation usually can't. Separating those lanes lets the synchronous service protect its latency budget while workers pace demand against configured capacity.

Each batch item needs an idempotency key, tenant and region policy, prompt version, model class, attempt count, and durable terminal state. A worker should claim bounded work, write the result atomically, and retry only errors declared retryable by the client contract. Poison items go to a review queue rather than cycling forever. This sounds like ordinary job processing because it is — LLM calls don't suspend queueing theory.

Capacity planning starts with arrival rate, tokens per item, acceptable completion window, and measured service time. I reserve headroom for retries and replays, then cap worker concurrency so a batch import cannot consume the interactive route's quota. Cost dashboards should divide spend by accepted outputs for each request class; raw token cost hides schema failures, duplicate work, and fallback amplification.

Keep the boundary sharp.

Batch isn't suitable when a customer is waiting on the same request, and self-hosting isn't suitable when the team lacks accelerator capacity planning and inference on-call experience. Conversely, a stable, high-volume offline workload may justify evaluating self-hosted inference because utilization can be planned. I'm not sure where that crossover lands for your traffic; your mileage may vary with model size, utilization, staffing, and the quality target. The decision should come from a load test and an ownership review, not a spreadsheet cell containing an optimistic utilization percentage.

For a Node.js application, the web process can enqueue the same policy envelope and a Go worker can consume it, provided the message schema is versioned and both sides agree on idempotency. The language boundary is less important than preserving the request contract.

Verify US and EU policy, deploy gradually, and roll back cleanly

Region selection is a data-governance decision before it is a latency tweak. For US and EU tenants, record the approved processing region in tenant policy, pass only the minimum required content to the model path, define retention expectations, and have security and legal owners verify the provider contract. An endpoint label alone is not evidence for the complete data flow. If residency or transfer requirements can't be demonstrated, keep that workload on an approved path even when another route scores better in a quality test.

Before release, replay a held-out set through the current and proposed policies. Compare acceptance rate, schema failures, fallback rate, input and output tokens per accepted result, and latency percentiles. Slice the results by prompt class, tenant region, and input-size band; an average can conceal a long-context route that consumes the error budget. Embeddings can help group similar inputs for evaluation and retrieval workflows, but they don't replace labeled acceptance criteria for the generated answer.

Watch the tail.

Deploy behind a versioned flag to a small tenant cohort. The stop conditions belong in the change plan: validation failures above the class threshold, excessive fallback, or a tail-latency breach should pin that class to the known-good route. Rollback means changing policy, not shipping application code under pressure. Pause affected batch consumers, preserve their durable items, switch the synchronous class, and verify recovery from telemetry before resuming queued work.

My final go/no-go review is blunt: who owns the evaluation corpus, who receives the page, what is the capacity ceiling, how quickly can we reverse the policy, and has somebody other than the policy author exercised that reversal while the queue contains real-shaped test items and dashboards are being watched? If those answers are vague, optimization waits. A single-model design is the right choice when its simplicity protects the SLO better than the projected savings from routing; an extra inference tier has to earn its operational footprint.

References

https://platform.openai.com/docs/guides/embeddings

DEV Community: EthanBrooks111

SMS Event Alerts: Delivery Polling, Resend, Cancel, and Country Guardrails

Implementation checkpoint: the audit record comes first

How can a Node.js service turn SMS delivery events into notification alerts?

Capacity planning belongs in the SLO review

Provider choice is a constrained trade-off

Verification and rollback are separate SLO paths

References

Edtech Password Reset Provider Audit: 6 SPF, DKIM, Suppression, and Bounce Gates

How should an edtech team test password reset email deliverability and bounce handling?

Trace the page backward to the earlier signal

Use a buy-vs-build table and a hard decision rule

Set the threshold with false-positive cost in view

References

Further reading

Marketplace Seller Paging: Urgent Event Notifications with SMS-First US/EU Retry Logic

How should Node.js urgent event notifications poll SMS before email fallback?

Treat templates as deployable marketplace policy

Wire the send adapter into a bounded state machine

Run a seller-order game day before launch

Choose the boundary, then verify it

References

Batch Email and SMS Partial-Failure Troubleshooting: 4 Per-Recipient Polling Controls

What evidence belongs in the first migration release?

How should stuck batch email and SMS notifications preserve per-recipient status?

Template authority sets the effective operating bill

References

How to Choose a Simple Password Reset Email API (Low-Volume SaaS)

Who should own the low-volume password reset email delivery SLO?

Implement the missing state transition

Evaluate candidates in a reset-expiry game day

References

Further reading

Failed Reservation Jobs: Queue Retry, Rate Limiting, Dedup, and Idempotency

Migrate the expiration path before choosing its permanent queue

How should a queue retry failed jobs with rate limiting, dedup, and idempotency?

Make one atomic transition the worker contract

Gate each delivery model with capacity and ownership evidence

Verify the SLO, switch traffic, and rehearse rollback

References

Marketplace Cleanup: Queue Push Webhook Endpoint with Public HTTPS Signature Checks

What should a public HTTPS queue webhook subscriber verify before ack?

How should retry state and idempotency shape delayed cleanup?

A failure table for the intake-to-worker boundary

Where does a push endpoint stop being the right boundary?

The decision rule I would put in the runbook

References

A Guide to One Key Compatible API Chat Completions for Tenant Cost Attribution

What should a one key compatible API record for multi model routing?

The buy versus build decision

A preventative Go request path

Where this pattern stops

The operating decision

Sources

429 Control for Invoice Audio: Retry-After Headers, Backoff, and Queued Transcription

How should a speech-to-text API queue handle 429 Retry-After backoff?

Separate throttling from capability failure

Implement the retry boundary safely

Verify quality, latency, and rollback before raising concurrency

References

Per-tenant LLM cost visibility when async batch jobs replace realtime tagging calls

Should async batch LLM jobs replace realtime calls for bulk tagging and extraction?

The night the tenant cost report stopped balancing

Where the provider boundary actually sits

The submit path I would write again

When batch is the wrong tool

Further reading

Marketing App Image API Production Gate: Poster Quality, Style, and Upscale

How should a marketing app compare text-to-image APIs for posters and social ads?

The shortlist is a buy-vs-build decision

A preventative Go client for the generation path

Where does generate-then-upscale stop working?

The release gate I would sign

References

Reduce an LLM API Bill for SaaS: Prompt Routing, Fallbacks, and Batch Processing

TL;DR

How should a SaaS app reduce its LLM API bill with prompt routing?

Make the fallback path explicit in code

Put delay-tolerant LLM work on a batch lane

Verify US and EU policy, deploy gradually, and roll back cleanly