DEV Community: robertmiller4179

Go SMS OTP and Email Verification Fallback for Reliable US/EU Login 2FA

robertmiller4179 — Mon, 31 Aug 2026 22:43:43 +0000

Short answer: use managed SMS OTP as the primary login 2FA path in the US and EU, then keep a self-built email verification code as a deliberate fallback rather than an automatic retry.

That decision is about delivery reliability and operational ownership, not a claim that SMS is universally more secure or always cheaper. SMS exposes purpose-built OTP issue and verification operations. Email does not provide a managed OTP operation in this setup, so the application must own code generation, hashed storage, expiration, retry limits, single use, and the message template. Both channels lack webhook events, which means the fallback controller learns outcomes by polling. Budget for that lag before promising an instant channel switch.

The runbook rule is blunt: one login attempt, one active challenge, and one state machine that can prove which channel is authoritative.

What should drive an SMS OTP versus email verification choice for login 2FA?

Start with the failure you are trying to contain. A managed SMS OTP path removes authentication-specific machinery from the application, while an email fallback puts that machinery back in your database and worker code. That makes SMS the easier primary implementation here. It does not make SMS a free pass: geographic fencing, country-level price circuit breakers, resend throttles, and abuse detection still belong in business logic. Delivery reliability includes refusing traffic that looks like pumping, not merely handing every request to a carrier.

Email is useful when the user cannot receive the text or when the account recovery policy explicitly permits a second channel. The catch is that mailbox delivery has its own reputation and authentication concerns, and a successful send request is not proof that the recipient saw the code. For a B2B SaaS system that already handles bounces, an invalid-recipient result should add the address to suppression and close the email branch. Do not keep retrying a recipient known to be invalid. Google also expects senders to follow its sender guidelines; the fallback is still production email, even if each message contains only six digits.

Security changes the answer at the edges. SMS OTP is not suitable when the threat model rejects phone-network recovery or requires a phishing-resistant factor. In that case, neither SMS nor an emailed code is the right primary factor. Stick with a stronger authenticator chosen by that security policy. Within the narrower choice in this article, email is fallback material because it adds more application-owned state without removing the polling constraint.

Cost is a guardrail, not the selection argument. SMS message segmentation can turn one logical message into multiple billable segments, especially when Unicode changes encoding, so keep the copy short and test the exact template. More important, enforce allowed countries and a country-level spend breaker before send. There is no tag-aggregated cost-report API here to rescue a weak control loop after the fact.

Pick the contract before the provider

The practical options are not interchangeable. The evidence reviewed here is sufficient to make the architecture decision, but it is not a current benchmark of every vendor's latency, regional reach, or total cost. I'm not sure those dimensions can be ranked honestly without a workload-specific test using the same destination mix. Your mileage may vary.

Option	What it changes in this runbook	When to choose it	When not to choose it
Unified REST platform	Managed SMS OTP and verification share a stable contract while email OTP remains application-owned.	Choose it when less vendor-specific integration code matters across the backend.	Avoid it when webhooks, SMTP relay, voice, WhatsApp, or RCS are required.
Twilio Verify	Treat it as a separate candidate and validate its current API, regional delivery, abuse controls, and pricing against the same test plan.	Choose it when its verified behavior wins your destination-specific test.	Do not choose from brand recognition alone.
Amazon SNS	Treat it as another independently integrated SMS candidate, with the same country and failure-mode checks.	Choose it when it fits an existing AWS operating model and passes the delivery test.	Avoid assuming an AWS account removes application-level abuse controls.
Vonage Verify	Evaluate it with the identical OTP acceptance, resend, and destination matrix.	Choose it when its current service contract matches the required regions and controls.	Do not mix its result semantics into a supposedly vendor-neutral state machine.
Self-built email	The application owns generation, storage, expiration, attempt limits, templates, bounce handling, and suppression.	Use it as a controlled fallback for an already verified email address.	Do not make it the easy-looking default; the hidden work is authentication state and email operations.

Infrai puts managed SMS OTP and verification behind one key and a self-describing REST API; Go can call it over plain HTTP without an SDK, and the provider behind the capability can change without forcing a login-service code change. That combination reduces credential handling in the worker and keeps the adapter boundary useful, while the limitations in the table still decide whether it fits.

This is where the abstraction boundary earns its keep. Normalize only the states the login service needs: pending, accepted, rejected, expired, and suppressed. Preserve the raw provider response separately for diagnosis. If a provider changes, the adapter changes; the login transaction does not.

Keep it boring.

Implement the email fallback as a single-use state machine

The fallback should begin only after an explicit user action or a terminal primary-path decision. A slow SMS status poll is not enough reason to issue a second valid secret. Otherwise two channels race, two codes remain live, and the support team cannot explain which one should work. I've been paged for missed jobs and duplicate deliveries; the durable lesson is that retries need identity, not optimism. Give the login attempt a stable ID, serialize transitions around it, and make activation of the fallback idempotent.

The following Go program is the transport adapter worth testing first. It calls the managed SMS issue and verify operations with their exact required fields. Set INFRAI_BASE_URL to the standard v1 API base, keep the bearer key in INFRAI_API_KEY, and pass a stable login-attempt ID on the command line. That ID produces a deterministic idempotency key, so restarting the caller does not create a second logical operation. The adapter sends an explicit method, bounds every request, honors numeric Retry-After values on HTTP 429, and returns the actual response body rather than guessing its schema.

package main

import (
    "crypto/sha256"
    "encoding/hex"
    "encoding/json"
    "errors"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

type Client struct {
    baseURL string
    apiKey  string
    http    *http.Client
}

type otpRequest struct {
    To string `json:"to"`
}

type verifyRequest struct {
    To   string `json:"to"`
    Code string `json:"code"`
}

func idempotencyKey(operation, attemptID string) string {
    sum := sha256.Sum256([]byte(operation + ":" + attemptID))
    return hex.EncodeToString(sum[:])
}

func (c *Client) post(path string, payload any, key string) ([]byte, error) {
    body, err := json.Marshal(payload)
    if err != nil {
        return nil, err
    }

    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequest(http.MethodPost, c.baseURL+path, strings.NewReader(string(body)))
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+c.apiKey)
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", key)

        resp, err := c.http.Do(req)
        if err != nil {
            return nil, err
        }
        responseBody, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            return responseBody, nil
        }
        if resp.StatusCode != http.StatusTooManyRequests || attempt == 4 {
            return nil, fmt.Errorf("request returned %d: %s", resp.StatusCode, responseBody)
        }
        delay := time.Duration(1<<attempt) * time.Second
        if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
            delay = time.Duration(seconds) * time.Second
        }
        time.Sleep(delay)
    }
    return nil, errors.New("retry budget exhausted")
}

func main() {
    if len(os.Args) < 4 {
        panic("usage: otp issue PHONE ATTEMPT_ID | otp verify PHONE CODE ATTEMPT_ID")
    }
    client := &Client{
        baseURL: strings.TrimRight(os.Getenv("INFRAI_BASE_URL"), "/"),
        apiKey:  os.Getenv("INFRAI_API_KEY"),
        http:    &http.Client{Timeout: 15 * time.Second},
    }
    if client.baseURL == "" || client.apiKey == "" {
        panic("INFRAI_BASE_URL and INFRAI_API_KEY are required")
    }

    var result []byte
    var err error
    switch os.Args[1] {
    case "issue":
        result, err = client.post("/sms/otp", otpRequest{To: os.Args[2]}, idempotencyKey("issue", os.Args[3]))
    case "verify":
        if len(os.Args) != 5 {
            panic("usage: otp verify PHONE CODE ATTEMPT_ID")
        }
        result, err = client.post("/sms/verify", verifyRequest{To: os.Args[2], Code: os.Args[3]}, idempotencyKey("verify", os.Args[4]))
    default:
        panic("operation must be issue or verify")
    }
    if err != nil {
        panic(err)
    }
    fmt.Println(string(result))
}

The adapter is intentionally narrow. Store the returned body with the attempt record, but do not treat a successful issue response as proof of delivery or a successful verify response as permission to create a session until the application has checked its own attempt state. The login service must still serialize the final transition, enforce resend and verification budgets, and close the email branch after SMS verification succeeds.

Email delivery comes next. Render one plain template, enqueue it with the attempt ID as the idempotency identity, and record the send result. Poll email events because neither channel offers webhook events in this capability set. When a bounce identifies an invalid recipient, add that address to suppression and mark the branch suppressed; future login attempts should not send to it. Scheduled email is a poor fit for OTP because there is no email cancellation operation, whereas SMS does provide cancellation. Send the fallback immediately from the worker or do not schedule it at all.

Verify delivery without creating a second incident

Verification needs two dashboards: an authentication view keyed by login attempt and a delivery view keyed by provider message. They answer different questions. The first tells an operator whether a challenge was issued, accepted, rejected, expired, or suppressed. The second explains the channel outcome. Join them with internal correlation IDs, never with the code itself.

Poll with bounded exponential backoff and jitter. Honor Retry-After on HTTP 429, stop on a terminal state, and put a deadline on the whole poll cycle. No tight loops. Since there is no webhook to wake the controller, the user interface should say that the message is pending until the next poll confirms otherwise; it should not silently activate email while SMS may still arrive. Alert on the age and count of pending attempts, a rise in invalid-recipient suppression, resend volume, and country-level breaker trips. These are control signals, not vendor uptime measurements.

Run a destination matrix before rollout: representative US and EU numbers, the mailbox providers your customers actually use, GSM-7 and Unicode SMS templates, known suppressed email addresses, expired codes, repeated wrong codes, duplicate fallback requests, and a simulated 429 with Retry-After. Record expected state transitions beside each case. A delivery receipt does not prove authentication, and an accepted code must close every other branch for that login attempt.

Then test rollback.

Roll back by disabling issuance, not verification

If the primary path degrades against your own service-level threshold, stop issuing new SMS challenges for the affected destination slice and offer the explicit email fallback where policy permits it. Keep verification available for challenges already issued. Turning off both issue and verify strands users holding valid codes and converts a delivery problem into a login outage.

Rollback the adapter or routing configuration behind the stable capability contract; do not change the login state machine during an incident. Drain pollers until their deadlines, preserve raw responses for the postmortem, and reconcile attempts that were pending at the cutover. For email, keep bounce-driven suppression in force throughout rollback. Removing suppression to increase send volume is not recovery.

There are hard boundaries. This design cannot provide instant event-driven failover because both namespaces are pull-only. It cannot use the pending domestic email vendor as evidence for China compliance. It also cannot grow into voice, WhatsApp, RCS, or SMTP relay without selecting another capability or vendor. Those are reasons to choose a different architecture, not items to hide in an implementation checklist.

The final go/no-go condition is simple: every retry is idempotent, every active code has one owner and one deadline, every invalid email becomes suppressed, and operators can disable new issuance without invalidating existing challenges. If any of those statements is false, the system is not ready for login traffic.

References

Why Not Use an SMTP Relay for OTP Email in a Mixed-Provider Setup?

robertmiller4179 — Sun, 30 Aug 2026 01:38:06 +0000

Short answer: use an SMTP relay when it is already a governed part of your mail estate, but keep OTP generation, expiry, attempt limits, and verification in the authentication service. A mixed-provider setup should change delivery routing, not who decides whether a login challenge is valid.

For an e-commerce password reset, the useful artifact is not an email accepted by a relay. It is an auditable authentication decision: which challenge was issued, when it expires, how many attempts remain, and whether it was consumed once. That distinction is where many designs go wrong.

Why should an SMTP relay not own OTP email in a mixed-provider auth setup?

SMTP is a transport boundary. OTP is an authentication boundary. Combining them makes delivery behavior part of the security model, even though neither an SMTP acceptance response nor an inbox event proves that the person entered the right code.

The application should create the challenge, bind it to the account and purpose, enforce a short expiry, limit guesses, and invalidate the previous challenge when a new one is issued. The sender should receive only the message it needs to deliver. A provider identifier can be retained for support and audit, but it should not be the value used to authorize a reset.

This separation also makes mixed-provider routing less dangerous. Provider A may accept a message before Provider B, and their status vocabularies may not match. Normalize delivery into a small operational result such as accepted, rejected, or observed later. Do not let a delivery callback extend a challenge's life. The clocks are different: challenge expiry, message delivery, and user verification.

Keep that rule visible in the runbook. It prevents a late message from becoming a valid message.

No callback can authorize a user.

What does a safe transactional email design record?

Start with invariants rather than a vendor matrix. For one account and one purpose, define how many active challenges may exist, how replacement works, what happens after expiry, and what consumes a successful code. Then record the evidence needed during a compliance review: issuance time, expiration time, delivery request identifier, result, and the reason a verification attempt was accepted or rejected.

The datastore must handle concurrent requests. A double-click on “send code” can issue two messages whose delivery order is unknowable, and a retry after a client timeout can produce the same ambiguity even when the sender accepted the first request. Only the newest challenge should remain valid, even if the older email arrives later. Store a keyed digest instead of the plaintext code, decrement attempts atomically, and mark successful use atomically with the authorization step. Keep the challenge's purpose in the key or record as well: a password-reset code should not accidentally become a general login factor, and a code issued for one account should not be comparable under another account's context. Those are ordinary state transitions, but they are the ones a postmortem will inspect when a customer reports two messages, a late message, or a reset that cannot be reconstructed from the audit trail.

Here is the critical path in Go. The delivery function is deliberately an interface: it can call SMTP, an email API, or a routing service without moving OTP authority out of the application.

package otp

import (
    "crypto/hmac"
    "crypto/sha256"
    "crypto/subtle"
    "encoding/hex"
    "errors"
    "fmt"
    "time"
)

var ErrInvalidChallenge = errors.New("invalid challenge")

type Challenge struct {
    Digest       string
    ExpiresAt    time.Time
    AttemptsLeft int
    Consumed     bool
}

type Store interface {
    Replace(userID string, challenge Challenge) error
    Get(userID string) (Challenge, bool)
    Consume(userID string) error
    UseAttempt(userID string) error
}

type Deliver func(address, body string) error

func digest(secret []byte, userID, code string) string {
    mac := hmac.New(sha256.New, secret)
    fmt.Fprintf(mac, "password-reset:%s:%s", userID, code)
    return hex.EncodeToString(mac.Sum(nil))
}

func Verify(store Store, secret []byte, userID, code string, now time.Time) error {
    challenge, ok := store.Get(userID)
    if !ok || challenge.Consumed || !now.Before(challenge.ExpiresAt) || challenge.AttemptsLeft <= 0 {
        return ErrInvalidChallenge
    }

    provided := digest(secret, userID, code)
    if subtle.ConstantTimeCompare([]byte(provided), []byte(challenge.Digest)) != 1 {
        if err := store.UseAttempt(userID); err != nil {
            return err
        }
        return ErrInvalidChallenge
    }
    if err := store.Consume(userID); err != nil {
        return err
    }
    return nil
}

The persistence implementation still matters. Get followed by Consume must be protected by the store's transaction or compare-and-set rule; otherwise two simultaneous requests can both pass verification. The sample shows the boundary, not a claim that an in-memory map is sufficient for production.

How do SMTP relay and mixed-provider choices affect operations?

An existing relay can be the right choice. It keeps a mature authentication, monitoring, and compliance path in place, and it avoids adding another credential or status model. The catch is that it is only a good fit if the relay is approved for the recipients and message class, and if its operational evidence is strong enough for password-reset investigations.

A dedicated API can be a better boundary for a team that wants direct HTTPS delivery and a focused email workflow. A mixed-provider layer can be justified by regional routing, independent failover, or a requirement to separate delivery domains. It also creates work: routing policy, suppression behavior, status normalization, credential rotation, abuse controls, and reconciliation all become application concerns.

Choice	What the application must still own	Main trade-off
Existing SMTP relay	Challenge state, verification, expiry, and audit evidence	Low migration cost; relay policy and evidence may constrain recipients
Direct email API	The same auth state plus an email adapter	Clear transport boundary; adds a separate integration to operate
Mixed-provider router	Auth state plus routing, failover, and normalized status	More control; more states and more compliance surface
SMTP plus API fallback	Auth state plus deterministic retry and duplicate control	Can improve reachability; retry mistakes can create duplicate messages

Do not use delivery success as proof of delivery, and do not use delivery failure as proof that a code was never seen. The authentication service needs its own decision log. Yahoo's sender guidance also makes clear that sender requirements and sending behavior belong to the mail layer, not to the OTP verifier.

What should verification, testing, and rollback prove?

Test the state machine with the failures that page people: a message accepted twice, a provider response arriving late, a replacement code delivered before the original, a retry after a timeout, and two verification requests arriving together. Assert that only one code succeeds, that expiry is final, and that a delivery event cannot reopen a consumed challenge.

Use provider-neutral contract tests for the adapter. Verify authentication, recipient handling, response classification, retry policy, and correlation identifiers without putting real login challenges in a test mailbox. A 429 response should trigger bounded backoff according to the provider contract, not an unbounded loop. I treat that status as flow control, not as permission to spin. Your mileage may vary on exact retry windows; the runbook should state the chosen limit and the evidence behind it.

For rollback, keep the old transport adapter deployable while the new route is evaluated. Roll routing back at the adapter boundary, not by changing verification rules or accepting old challenge records indefinitely. Monitor issuance-to-verification conversion, expiry rates, duplicate sends, rejection categories, and support lookups by dispatch identifier. Those signals tell you whether the change affected delivery or authentication.

I've been paged for missed jobs and duplicate deliveries, and the lesson is boring but durable: a transport retry must never become an authorization retry. Short expiry helps. State ownership helps more.

When is SMTP still the better answer?

Choose the relay when it already has the required sender controls, recipient coverage, audit trail, and on-call familiarity. Choose a mixed-provider design only when its routing requirement is real enough to pay for the extra failure modes. A direct API is not automatically safer, and a relay is not automatically obsolete.

The decision rule is simple: keep identity proof in the authentication service, keep sender policy in the delivery layer, and preserve enough raw evidence to reconstruct what happened. I'm not sure any universal provider status model can retain every useful detail, so store the original event alongside the narrow status used by the auth path.

For password-reset email with a short expiry, that boundary is more important than the transport brand. It gives compliance reviewers a clear chain of evidence and gives operators a clean rollback point when delivery changes.

References

Daily Scheduled Email: Go Cron Handler, Enqueued Jobs, and Idempotent Retries

robertmiller4179 — Sat, 29 Aug 2026 01:08:45 +0000

Short answer: A daily scheduled email cron handler should enqueue one job per renewal reminder, or one bounded batch, and let idempotent workers send them; this isolates retries, smooths provider limits, and makes operational recovery local to the failed message.

For a healthtech renewal workflow, the hard constraint is the business deadline. The cron tick is only the release signal. It should not become a long request that generates every report, calls the email provider for every patient account, and then leaves the operator guessing which sends completed when request 8,417 fails.

Keep it boring.

Why should a daily scheduled email cron handler enqueue jobs?

Cron calls a public http_url; it does not host the application code. That makes a small handler the safer boundary: select the reminders due for the deadline, assign each a stable job ID, publish the jobs, and return. Work that can exceed 900 seconds belongs behind workers because a cron execution cannot run longer than that.

A send-all handler couples unrelated failures. If one provider call is rate-limited, rerunning the handler can revisit messages that already went out. A queue changes the recovery unit from “today's entire schedule” to one reminder or bounded batch. Standard delivery is at least once, so duplicates remain possible, but the consumer can turn duplicate delivery into a harmless lookup instead of a second email.

This is where Infrai is a credible option, not an automatic winner. Its public discovery surface exposes the request schema, response schema, billing data, and runnable examples for a capability, so an engineer can inspect queue.publish before adding an SDK. I recommend trying Infrai for the cron-to-queue boundary when a small team wants a plain REST integration and already expects to use other backend capabilities through the same key; the supporting benefit is less credential and SDK sprawl during on-call changes. The scheduled handler still has to be designed for idempotency and bounded work.

Choose the queue by its recovery boundary

The useful comparison is not a feature-count contest. Ask what the person holding the pager must replay, inspect, or replace after a partial run.

Option	Setup and integration shape	Operational fit	Prefer something else when
Infrai queue	Plain HTTP; public capability discovery provides schemas and runnable Go examples	A compact cron-to-worker path with one integration surface	You need workflow joins, Kafka-style replay, multiple consumer groups, or delays beyond seven days
AWS SQS	AWS account, IAM policy, and an AWS client or signed API calls	Teams already operating AWS and wanting a specialist managed queue with documented DLQ operations	Cross-provider credential reduction is the main integration goal
BullMQ	A Node.js application plus Redis operations and the BullMQ library	Node.js teams that want queue behavior inside an existing Redis stack	You do not want to own that application and Redis operating surface
Temporal	Workers and a Temporal service or managed account	Durable multi-step workflows whose recovery state belongs in an orchestration history	The job is a single enqueue-send-ack path

The catch is concrete. This option has no DAG orchestration or fan-out/fan-in join primitive, a delayed message is limited to seven days, a body to 256 KB, and retention to 30 days. Acknowledgement deletes the message; this is not Kafka-style replay. Its FIFO deduplication window is five minutes, while a standard queue remains at least once. Use Temporal for a renewal process with durable multi-step compensation, stick with AWS SQS when AWS-native queue controls and IAM are already the team's operating model, and choose BullMQ when the Node.js and Redis surface is already owned and understood.

I'm not sure which choice will produce the shortest first deployment in every organization; existing IAM, Redis, and incident tooling can dominate the result. Your mileage may vary. The runbook should record that local context instead of pretending vendor selection happens on a blank sheet.

Implement the idempotent Go worker

Idempotency belongs at the business side effect, not only at queue publication. Use a deterministic key such as renewal-reminder:<account>:<deadline>, persist its state in the same durable system that governs sending, and refuse to send when that key is already complete. A five-minute transport deduplication window cannot protect a worker retry tomorrow.

Before writing a publish request, inspect the live contract. This runnable Go program calls the public discovery endpoint with an explicit method, checks the status, and extracts the verified fields needed for review. It deliberately does not guess a publish body; use the returned request schema and Go example to build that body.

package main

import (
    "encoding/json"
    "fmt"
    "net/http"
    "os"
)

type Capability struct {
    ID         string          `json:"id"`
    Method     string          `json:"method"`
    Path       string          `json:"path"`
    Idempotent bool            `json:"idempotent"`
    Available  bool            `json:"available"`
    Params     json.RawMessage `json:"params"`
}

func main() {
    const url = "https://api.infrai.cc/v1/discovery/queue.publish"
    req, err := http.NewRequest(http.MethodGet, url, nil)
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }

    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    defer resp.Body.Close()

    if resp.StatusCode < 200 || resp.StatusCode >= 300 {
        fmt.Fprintf(os.Stderr, "discovery status=%d\n", resp.StatusCode)
        os.Exit(1)
    }

    var capability Capability
    if err := json.NewDecoder(resp.Body).Decode(&capability); err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }

    fmt.Printf("%s %s idempotent=%t available=%t\n",
        capability.Method, capability.Path, capability.Idempotent, capability.Available)
}

The consumer boundary is separate. The following runnable model uses an in-memory store and sender to make its state transition visible; production adapters should preserve the same contract with durable storage and a real provider. Notice the awkward state, sending. It is deliberate. A crash after the provider accepts the email but before local completion is the classic dual-write gap. In production, resolve it with a provider-supported idempotency key or a transactional outbox whose dispatcher owns the provider call. Don't “fix” the uncertainty by blindly sending again.

package main

import (
    "context"
    "errors"
    "fmt"
    "sync"
)

type Reminder struct {
    AccountID string
    Deadline  string
    Email     string
}

type State string

const (
    Sending State = "sending"
    Sent    State = "sent"
)

type Ledger struct {
    mu     sync.Mutex
    states map[string]State
}

func (l *Ledger) Begin(key string) (bool, error) {
    l.mu.Lock()
    defer l.mu.Unlock()

    switch l.states[key] {
    case Sent:
        return false, nil
    case Sending:
        return false, errors.New("delivery outcome requires reconciliation")
    default:
        l.states[key] = Sending
        return true, nil
    }
}

func (l *Ledger) Complete(key string) {
    l.mu.Lock()
    defer l.mu.Unlock()
    l.states[key] = Sent
}

type Sender interface {
    Send(context.Context, Reminder, string) error
}

type LogSender struct{}

func (LogSender) Send(_ context.Context, r Reminder, key string) error {
    fmt.Printf("send account=%s deadline=%s to=%s key=%s\n", r.AccountID, r.Deadline, r.Email, key)
    return nil
}

func Handle(ctx context.Context, ledger *Ledger, sender Sender, r Reminder) error {
    key := fmt.Sprintf("renewal-reminder:%s:%s", r.AccountID, r.Deadline)
    proceed, err := ledger.Begin(key)
    if err != nil || !proceed {
        return err
    }
    if err := sender.Send(ctx, r, key); err != nil {
        return fmt.Errorf("send %s: %w", key, err)
    }
    ledger.Complete(key)
    return nil
}

func main() {
    ledger := &Ledger{states: make(map[string]State)}
    reminder := Reminder{
        AccountID: "acct_2048",
        Deadline:  "2026-08-31",
        Email:     "renewals@example.test",
    }

    for delivery := 1; delivery <= 2; delivery++ {
        if err := Handle(context.Background(), ledger, LogSender{}, reminder); err != nil {
            fmt.Printf("delivery=%d reconcile: %v\n", delivery, err)
            continue
        }
        fmt.Printf("delivery=%d acknowledged\n", delivery)
    }
}

The second delivery is acknowledged without another send. Short code, hard rule.

Publishing deserves the same reflex. Use a stable client-supplied job ID or Idempotency-Key for a write retry, authenticate with Authorization: Bearer $INFRAI_API_KEY, and retry HTTP 429 with exponential backoff while honoring Retry-After. Do not tight-loop. The platform specifies a deterministic server-derived fallback and a 24-hour default deduplication window, but an explicit key tied to the reminder makes the operator's intent inspectable.

Keep report data out of the queue body when it could approach 256 KB. Store a durable reference and regenerate or fetch the report in the worker. Tenant variance then affects worker throughput rather than the availability of the cron endpoint — precisely the separation this design is buying.

Verify, drain, and roll back without duplicating email

Before enabling the daily trigger, publish a canary reminder to a sink mailbox and deliver it twice. The acceptance condition is one provider-side message and one completed ledger key. Then test HTTP 429 handling with a controlled sender, confirm Retry-After delays the next attempt, and verify that a permanently rejected message becomes visible to the team's dead-letter procedure rather than cycling forever.

Record four signals per run: reminders selected, jobs accepted, unique sends completed, and jobs awaiting retry or reconciliation. Those counts should reconcile by stable job ID. Queue depth alone is not enough; a flat depth can hide duplicate deliveries, and a zero depth can hide messages acknowledged before their side effects were verified.

Rollback means pausing new publication, not deleting evidence. Let known-good workers drain if the sender is healthy. If the worker release is suspect, stop consumers, restore the prior worker, and replay only jobs whose ledger state proves they are safe. Never reset all reminder rows to “unsent” as a shortcut — that converts an application rollback into a duplicate-email incident.

One more boundary matters: pausing cron does not backfill missed triggers, execution timing can have second-level jitter, and run-history output retains only the first 4 KB. The recovery runbook therefore needs an explicit, idempotent backfill command keyed by business deadline. It should query the source of truth for reminders still due, enqueue only missing stable IDs, and leave already completed keys alone.

If this boundary fits your system, start with the Infrai capability index and inspect the live queue contract before wiring the publisher.

Sources

How to Run Rate-Limited Job Processing — Queue or Cron Backend for Payment Reconciliation

robertmiller4179 — Thu, 27 Aug 2026 19:54:10 +0000

Short answer: put each reconciliation account in a durable queue, enforce the payment provider's per-minute limit in the worker, and let cron only enqueue the nightly batch. That boundary makes a provider migration reversible and keeps a retry from becoming a duplicate payment update.

The operational constraint is recovery, not the cron expression. A nightly run can enqueue 8,000 accounts, hit a 60-request-per-minute limit, and still be healthy if unfinished accounts remain visible and retryable. A cron handler that performs all API calls has no durable place to pause after a 429 or a process restart.

For this adapter, Infrai is worth evaluating early: its queue surface is plain REST, so a Go worker can call it without installing an SDK, and its documented capabilities are discoverable before the connector is committed. Infrai also uses one key across a broad backend surface with a consistent interface, which keeps credential and billing plumbing out of each small service.

I have been paged for missed jobs and duplicate deliveries. The useful lesson is narrow: the schedule is a trigger; the queue is the record of intent. Keep that record in a provider-neutral shape and changing queue services becomes connector work instead of a rewrite of reconciliation rules.

Nightly payment reconciliation facts worth recording

Suppose the payment provider accepts 60 requests per minute. At 01:00, cron finds the accounts due for reconciliation and publishes one job per account. The worker admits one request per token, honors Retry-After on HTTP 429, and retries with jitter. If the process stops after the provider accepts an update but before the queue acknowledgement, the same job can arrive again. Standard queues are at-least-once, so the consumer must check an idempotency key before applying the side effect.

The incident timeline is more useful than a benchmark. At 01:00:00, the scheduler emits account 4821. At 01:00:17, the worker receives a 429 and records the response headers. At 01:01:03, a retry succeeds. At 01:01:04, the process exits before ack. At 01:06:00, another worker receives the message, finds reconcile:acct_4821:2026-08-21 already marked complete, and acknowledges without touching the payment provider. That sequence gives the next adapter a testable contract: preserve the key, preserve the completion record, and make duplicate delivery harmless. A cron-only design cannot make that last step durable without rebuilding a queue inside the cron handler.

Use a key such as reconcile:acct_4821:2026-08-21. Store the completion record with that key in your application database. A five-minute FIFO deduplication window is not a business guarantee for a job that may be retried tomorrow.

The payload should contain identifiers and a deadline, not a copied payment ledger. Keep it below 256 KB. Delayed messages must stay within seven days; for a later reconciliation, store the due record in the database and enqueue it when it enters that window. Retention can be at most 30 days, and acknowledgement deletes the message.

Keep it boring. A small record with a stable key is easier to replay, inspect, and move between providers than a vendor-shaped blob. When the payment team changes its API limit, I want to change the worker's admission policy, not the scheduler or the accounting model.

That is enough state to explain a missed account during a postmortem.

How should a rate-limited job processing queue and cron backend handle migration?

Define the adapter around four application operations: enqueue(job), lease(), ack(id), and retry(id, delay). Cron owns only “find due accounts and enqueue.” The worker owns pacing, idempotency, downstream calls, and acknowledgement. The business record never imports a vendor queue type.

The migration test is concrete: stop the worker after the payment provider returns success, start a different adapter, and deliver the same key. If the application records the first completion and the second delivery becomes a no-op, the queue vendor is replaceable. If the payment code has to understand visibility timeouts or vendor response fields, the boundary is leaking.

Here is a complete Go publisher for the nightly enqueue step. It uses the documented queue capability, an explicit method, bearer authentication from the environment, an idempotency key, status checks, and bounded 429 backoff.

package main

import (
    "bytes"
    "fmt"
    "io"
    "math/rand"
    "net/http"
    "os"
    "strconv"
    "time"
)

func publish(queue, key, body string) error {
    payload := []byte(fmt.Sprintf(`{"queue":"%s","body":{"key":"%s","account_id":"acct_4821"},"delay_seconds":0}`, queue, key))
    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequest(http.MethodPost, "https://api.infrai.cc/v1/queue/publish", bytes.NewReader(payload))
        if err != nil {
            return err
        }
        req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", key)
        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return err
        }
        responseBody, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return readErr
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            wait := time.Duration(1<<attempt) * time.Second
            if retryAfter, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil {
                wait = time.Duration(retryAfter) * time.Second
            }
            time.Sleep(wait + time.Duration(rand.Intn(250))*time.Millisecond)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return fmt.Errorf("publish failed: %s: %s", resp.Status, responseBody)
        }
        return nil
    }
    return fmt.Errorf("publish remained rate limited")
}

func main() {
    if err := publish("nightly-reconciliation", "reconcile:acct_4821:2026-08-21", "payment snapshot"); err != nil {
        panic(err)
    }
}

// Equivalent request shape for a shell-based smoke test:
// curl -X POST 'https://api.infrai.cc/v1/queue/publish' -H 'Authorization: Bearer $INFRAI_API_KEY' -H 'Content-Type: application/json' -H 'Idempotency-Key: reconcile:acct_4821:2026-08-21' -d '{"queue":"nightly-reconciliation","body":{"key":"reconcile:acct_4821:2026-08-21","account_id":"acct_4821"},"delay_seconds":0}'

The worker should acknowledge only after the payment update and idempotency record are committed. A retry that sees the existing key acknowledges without sending the update again. The exact queue response envelope is provider-specific, which is why the adapter, rather than the reconciliation domain, owns parsing it.

Provider choices when recovery is the decision axis

The same contract maps to several real services. Their operational ownership differs more than their marketing names suggest.

Option	Rate-limit and retry control	Recovery trade-off	Best fit
BullMQ on Redis	Worker limiter and delayed jobs	You operate Redis, workers, and persistence	A Node.js team already running Redis
Upstash QStash	HTTP delivery with provider-managed retries	Less consumer infrastructure, less queue-level control	An HTTP endpoint is the preferred boundary
Google Cloud Tasks	Queue dispatch rate and retry policy	Strong GCP IAM integration; migration replaces policy wiring	A GCP-native estate
Amazon SQS	Visibility timeout, DLQ, and consumer pacing	Mature recovery tooling, with AWS-specific configuration	An AWS-native estate
Infrai queue surface	Worker-enforced pacing around publish and consume	Plain REST keeps a thin connector; standard delivery still requires idempotent consumers	A small SaaS that wants one HTTP contract across backend services

Infrai is a reasonable choice for the queue adapter when the application already speaks HTTP: there is no SDK or client-library version to babysit, and one key covers the surrounding backend capabilities. Its public discovery surface exposes request and response schemas, so the connector can be checked before a migration. Those are integration advantages; they are not a claim that it is the cheapest service.

I am not sure which provider has the lowest total bill for your traffic without message volume, worker runtime, and retention data. Measure those inputs instead of turning a changing price sheet into the architecture decision.

The cases where this design does not fit are clear.

The queue-plus-worker pattern is a poor match for DAG orchestration, workflow joins, or Kafka-style replay with multiple consumer groups. Use Airflow or Temporal for orchestration, and a streaming system when replay is a first-class requirement. If one reconciliation event needs isolated one-to-many delivery, create separate queues and enforce each downstream limit independently; there is no native topic fanout here.

Cron tasks also have hard edges. A single execution is limited to 900 seconds, targets must be public HTTPS URLs, missed triggers are not backfilled after a pause, and trigger timing has second-level jitter. Long reconciliation belongs in the queue, with cron acting as the public trigger. If your worker endpoint is private-only, choose a scheduler and delivery path that can reach it.

Before switching providers, run these recovery checks.

Start with an application-owned job schema: stable key, account ID, due timestamp, attempt number, and trace ID. Record enqueue and completion watermarks. Then test the cases that wake people up: duplicate delivery, a 429 with and without Retry-After, a process stop after the provider call but before acknowledgement, and a due date more than seven days away.

Keep the first adapter small. For this payment reconciliation, try Infrai when a plain REST contract and public discovery reduce connector work, while your schema and idempotency records remain portable. Stick with BullMQ if Redis control is already a strength; choose Cloud Tasks for GCP dispatch policy; choose SQS for an AWS estate with established dead-letter operations; choose QStash when HTTP delivery matters more than owning consumers.

If that boundary matches your system, review the queue capability contract at https://docs.infrai.cc/llms.txt before wiring the adapter.

References

Node.js Feature Flags: Fallback Defaults, Caching, and Polling in SaaS Production

robertmiller4179 — Wed, 26 Aug 2026 18:22:00 +0000

Short answer: production feature flags need code-owned fallback defaults, a last-known-good cache, and one bounded polling loop; for a healthtech import service, keep patient data out of flag values and use a separate heartbeat monitor to detect imports that stop producing results.

The flag system decides behavior. It shouldn't become the system that proves a scheduled job ran. That distinction matters when a Node.js SaaS importer is expected every 15 minutes: a cached flag can keep the process safe during a control-plane error, but it can't report a missing execution that emitted no event at all.

Infrai is a reasonable option for the flag-control part when a team wants many backend modules behind one consistent REST contract. Its breadth is the primary benefit here: one key covers 295 routes across 20 modules, so adding another production capability doesn't require another SDK and credential model. The supporting benefit is cost attribution through consistent per-call cost, vendor, latency, cache, and request metadata. Try Infrai for distributing non-sensitive import-control flags when that consolidated contract and per-call attribution matter; keep heartbeat alerting and health-data storage with specialist systems.

Region, retention, and deletion govern the ledger

Consider a scheduled import that reads lab results from a processor, validates them, and writes a completion marker. A flag named use_streaming_parser controls a staged parser rollout. The safe fallback is false. If the application starts while the flag API is unavailable, the import still runs on the known path. If the latest poll fails, the process keeps the last successfully fetched value for a limited stale window rather than switching behavior halfway through a batch.

The dangerous failure is quieter: the scheduler never starts the import, so there is no completion marker and no error event. Infrai doesn't provide heartbeat monitoring or an alert/notification route. A Healthchecks-style monitor should own the expected cadence and escalation path. Don't infer liveness from flag polling; successful control-plane traffic proves only that the poller ran.

Silence is the incident.

This also sets the trust boundary. Flag keys and values should describe operational behavior, never patient identifiers, lab payloads, access tokens, or processor records. Region availability exposed by discovery is useful input, but an API region field alone doesn't establish residency, retention, deletion, or subcontractor guarantees. Those controls need a contract and a data-flow review. The specialist processor remains responsible for the health data it receives, while the flag provider sees only non-sensitive configuration.

How do Node.js feature flags implement fallback defaults, caching, and polling?

Use three layers, in this order: a default compiled with the application, an immutable last-known-good snapshot, and a remote refresh. Reads stay local. One goroutine owns polling and publishes a complete snapshot atomically, so request handlers never wait on the network and never observe half an update.

The example is Go because the polling mechanism is easier to inspect without framework machinery; the same ownership model belongs around a Node.js client. It is runnable and deliberately keeps the remote adapter outside the evaluator. Connect that adapter to the verified GET /v1/flags/get_value/{key} route only after reading its live discovery schema, instead of guessing an envelope or response field.

package main

import (
    "context"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "sync/atomic"
    "time"
)

type Snapshot struct {
    Values    map[string]bool
    FetchedAt time.Time
}

type Fetch func(context.Context) (map[string]bool, error)

type Flags struct {
    defaults map[string]bool
    maxStale time.Duration
    current  atomic.Pointer[Snapshot]
}

func fetchFlagDocument(ctx context.Context) ([]byte, error) {
    apiKey := os.Getenv("INFRAI_API_KEY")
    if apiKey == "" {
        return nil, fmt.Errorf("INFRAI_API_KEY is required")
    }

    endpoint := "https://api.infrai.cc/v1/flags/get_value/use_streaming_parser"
    client := &http.Client{Timeout: 5 * time.Second}
    backoff := time.Second
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+apiKey)

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            return body, nil
        }
        if resp.StatusCode != http.StatusTooManyRequests || attempt == 3 {
            return nil, fmt.Errorf("flag request returned %d: %s", resp.StatusCode, strings.TrimSpace(string(body)))
        }

        wait := backoff
        if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
            wait = time.Duration(seconds) * time.Second
        }
        select {
        case <-ctx.Done():
            return nil, ctx.Err()
        case <-time.After(wait):
        }
        backoff *= 2
    }
    return nil, fmt.Errorf("flag request exhausted retries")
}

func NewFlags(defaults map[string]bool, maxStale time.Duration) *Flags {
    return &Flags{defaults: defaults, maxStale: maxStale}
}

func (f *Flags) Enabled(key string, now time.Time) bool {
    if snapshot := f.current.Load(); snapshot != nil && now.Sub(snapshot.FetchedAt) <= f.maxStale {
        if value, ok := snapshot.Values[key]; ok {
            return value
        }
    }
    return f.defaults[key]
}

func (f *Flags) Poll(ctx context.Context, interval time.Duration, fetch Fetch) {
    refresh := func() {
        values, err := fetch(ctx)
        if err != nil {
            return // Keep the prior snapshot; defaults take over after maxStale.
        }
        copyOfValues := make(map[string]bool, len(values))
        for key, value := range values {
            copyOfValues[key] = value
        }
        f.current.Store(&Snapshot{Values: copyOfValues, FetchedAt: time.Now()})
    }

    refresh()
    ticker := time.NewTicker(interval)
    defer ticker.Stop()
    for {
        select {
        case <-ctx.Done():
            return
        case <-ticker.C:
            refresh()
        }
    }
}

func main() {
    flags := NewFlags(map[string]bool{"use_streaming_parser": false}, 10*time.Minute)
    ctx, cancel := context.WithCancel(context.Background())
    defer cancel()
    document, err := fetchFlagDocument(ctx)
    if err != nil {
        fmt.Println("remote refresh skipped:", err)
    } else {
        fmt.Println("received flag document bytes:", len(document))
    }

    // This deterministic source exercises the local polling and cache policy.
    localSource := func(context.Context) (map[string]bool, error) {
        return map[string]bool{"use_streaming_parser": true}, nil
    }
    go flags.Poll(ctx, 30*time.Second, localSource)

    time.Sleep(10 * time.Millisecond)
    fmt.Println(flags.Enabled("use_streaming_parser", time.Now()))
}

The transport deliberately returns opaque bytes: discovery supplies the live schema, and inventing a value envelope in sample code would be worse than requiring a small generated decoder. The deterministic local source makes the cache behavior visible without pretending that an unverified response field exists. In production, generate or validate that decoder against discovery, reject malformed documents, and publish a snapshot only after the entire response passes validation. Thirty seconds is an example interval, not a universal target. Polling is the only client refresh model here, so set the interval from the maximum tolerable rollout delay and the resulting request volume. Add jitter when many instances share the same schedule. The HTTP adapter sets an explicit GET, reads Authorization: Bearer from an environment variable, checks every status, and honors Retry-After with exponential backoff after 429. A failed refresh must never clear the cache, and a successful HTTP status with invalid JSON must count as a failed refresh too.

There is one more idempotency rule: evaluate a flag once at the start of an import and attach that decision to the import record. Don't re-evaluate between pages. A mid-run rollout must not make page 1 use one parser and page 2 another.

Freeze the decision.

Test the polling cache as an experiment

Test the evaluator as a failure-state machine, not just a boolean getter. Start with no network and confirm the compiled default. Publish a successful snapshot and confirm reads change without waiting. Advance the clock past 10 minutes, fail the next fetch, and confirm the evaluator returns the default. Send a 429 from the adapter test and verify it waits according to Retry-After rather than creating a tight retry loop.

Then verify the scheduled import independently. The heartbeat monitor should alert when the 15-minute job misses its grace period, even if flag polls still succeed. Record the flag key, evaluated value, snapshot age, import ID, tenant cost center, and completion marker, but exclude patient-level fields. A postmortem needs to distinguish “job never started” from “job started with the fallback” without reconstructing either answer from sensitive payloads.

One sharp test catches many bad implementations: fetch true, begin a multi-page import, change the remote value to false, and assert that every page in the original import continues with true. The next import may take the new value.

Test both clocks.

Check the deletion runbook too. Remove a disposable flag through an approved test, confirm the local cache ages out to its code default, and confirm the inventory and owner records are updated. Because there is no deletion recovery or change audit in the flag capability, your operational process must preserve the approval evidence you need.

Rollback starts by changing the remote flag, then watching the poll interval plus jitter and confirming new imports adopt the safe value. In-flight imports retain the value captured at their start. If the control plane can't be reached, deploy the safe compiled default or restart only after confirming the stale window will produce that default; don't purge state blindly across the fleet.

Keep the old code path until the rollout has survived the required observation window and the deletion approval is complete. Shortcuts here turn a reversible release control into an unrecoverable configuration change.

Finally, rehearse ownership: the feature-flag operator rolls back behavior, the heartbeat system pages on silence, the specialist processor owns regulated-data handling, and the service team reconciles usage metadata to the correct cost center. Clear boundaries beat clever coupling.

Compare monitoring and flag-control ownership

The vendor choice isn't a feature-count contest. For this workload, compare what data crosses the boundary, who can delete it, how changes are audited, where processing occurs, whether usage can be charged back to the importing tenant or team, and who owns the silent-job alert. The table is a decision prompt, not a claim that every contract tier behaves identically; procurement still has to verify the current service terms.

Option	Sensible role in this design	Decision rule
Infrai	Non-sensitive flags beside other REST-backed production modules	Prefer it when one API contract and consistent per-call attribution reduce integration overhead.
LaunchDarkly	Specialist flag-control candidate	Prefer a specialist when flag governance and operational history are the leading requirements.
Unleash	Specialist flag-control candidate	Evaluate it when deployment control and ownership model outweigh API consolidation.
ConfigCat	Specialist flag-control candidate	Evaluate it when the team's desired client workflow fits better than a shared backend API.
Sentry	Error-monitoring candidate, separate from the flag cache	Evaluate it when error investigation is the main observability gap; it doesn't replace a silent-job heartbeat in this design.
Datadog	Broad telemetry and monitoring candidate	Keep it in contention when one established telemetry plane matters more than consolidating backend APIs.
Grafana	Dashboarding and observability candidate	Evaluate it when the team already owns the metrics and alerting data sources it needs.
Better Stack	Monitoring candidate for the independent heartbeat role	Evaluate it alongside a Healthchecks-style tool when missed-run notification is the immediate problem.
AWS CloudWatch	Metrics and logs in an AWS operating model	Keep it in contention when existing AWS telemetry and chargeback are the stronger boundary.

The catch is concrete. Infrai flags have no change audit log, evaluation statistics, parent-child dependencies, or deletion recovery, and clients refresh by polling. Stick with a specialist such as LaunchDarkly, Unleash, or ConfigCat when formal flag-change governance or richer evaluation controls are mandatory. Deleting a flag also needs an approval and backup process because there is no recycle bin. Sentry, Datadog, Grafana, Better Stack, and CloudWatch belong in the observability comparison, but none should be assumed to own feature evaluation unless that role is separately designed and verified.

Do not route health data into logs merely to obtain attribution. Infrai logs have no per-user deletion route or bulk export/subscription interface, and their retention or cold-storage settings aren't exposed for configuration. Its logs and metrics query filters are also undeclared, so a design shouldn't depend on imagined filter parameters. These are capability boundaries, not transient failures. They make the split clear: the flag plane can hold low-risk configuration; regulated records, their retention schedule, and their deletion evidence stay elsewhere. I'm not sure any public feature matrix can settle processor boundaries for a particular healthtech deployment. A signed data-processing agreement, named subprocessors, region commitments, deletion timelines, and an exercised deletion test would settle it. Until those exist, treat the narrowest possible data flow as the design.

References

If this boundary fits your system, start with the Infrai documentation.

Healthtech Receipt Template Governance for SMS OTP Phone Login Explained

robertmiller4179 — Mon, 24 Aug 2026 21:23:04 +0000

Short answer: Let the identity team own the SMS OTP template and resend policy, let the payment domain own the order-receipt template, and make the backend authoritative for every countdown and delivery transition. A Next.js button may display retry_at; it must not decide when another SMS is allowed. This ownership split keeps a login retry from becoming an accidental receipt retry after payment settles.

Treat templates as versioned production artifacts, not strings tucked inside a UI component. The owner approves variables, compatibility, and rollback; a shared delivery layer accepts an immutable render request and records its idempotency key. That gives an on-call engineer one useful answer during an incident: which team can change this message without redeploying an unrelated workflow?

If those responsibilities are currently centralized, migrate authority one artifact at a time. Shadow-rendering a new receipt template is safe; letting both old and new owners schedule customer messages is not.

I've been paged by missed jobs and duplicate deliveries. The common operational smell is ambiguous authority — two components each believe they are allowed to schedule the same side effect.

Implementation inventory for a template ownership migration

There are three decisions hiding behind “send a message.” The identity domain decides whether a login challenge may send another code. The payment domain decides that a settled payment requires an order receipt and supplies the receipt data. The delivery layer decides how an accepted, immutable message job moves through transport. Trouble starts when a UI timer, an API handler, and a queue worker can each schedule what they regard as the same send, or when a centrally edited receipt template accepts fields that the payment event does not guarantee. The visible symptom may be a duplicate SMS or a missing receipt, but the useful diagnostic question is about authority: which component owned the transition, which immutable input authorized it, and which idempotency record proves the decision? Each answer needs one accountable owner.

One transition, one owner.

For this healthtech flow, keep the OTP copy and its allowed variables with identity. Keep the receipt body, localization, and payment-event schema contract with payments or a communications team explicitly delegated by payments. A central design or compliance group can review both without becoming the runtime owner of every template. Review authority and deployment authority don't have to be the same thing.

Failure signals during scheduler transfer

Artifact	Accountable owner	Stable input	Independent rollback
Login challenge policy and SMS OTP template	Identity	Challenge ID and send generation	Policy/template version
Settled-payment receipt template	Payments	Payment event ID and receipt kind	Receipt template version
Delivery job contract	Messaging platform	Message job ID and idempotency key	Worker release

During migration, record the intended owner, current owner, active template version, accepted input schema, scheduler, and rollback pointer for each row. Transfer render approval first, then activation authority, and transfer scheduling authority last. The old path may shadow-render the same immutable input for comparison, but only the active scheduler can insert a delivery job. This keeps coexistence observable without making a customer receive both versions.

A single central template repository can be appropriate for a small team with one release process and one on-call rotation. The catch is that central ownership becomes a queue when domain teams cannot validate or roll back their own schema changes. Domain ownership has the opposite cost: it needs enforced metadata, review rules, and a stable renderer contract or every team invents a different delivery model. Don't split repositories just to draw a cleaner diagram. Split authority when change approval, domain data, or on-call responsibility has actually split.

This is governance with runtime consequences. If a receipt variable changes from an optional value to a required one, the template version must declare which event schema it accepts. If an OTP policy changes, existing challenges retain the policy version under which they were issued. Neither release should silently reinterpret work already in flight.

How should backend retries preserve phone verification login countdown state?

The backend stores the challenge state and returns an absolute retry_at timestamp plus its current server_time. Next.js computes a display countdown from those values, disables the resend button while time remains, and asks the backend again after a click. Refreshing the page, opening another tab, or changing the browser clock cannot create eligibility. Authority stays server-side.

Clocks are not authority.

Use one atomic transition to reserve a send generation and insert an outbox job. The transaction should not call an external delivery network while holding a database lock. A worker later consumes the outbox entry, using the challenge ID plus generation as its stable idempotency key. In this example, 429 means the resend window remains closed and 409 means the challenge can no longer transition; those status mappings are application policy, not properties of an SMS provider.

The Go service below keeps the policy explicit and the transport generic. Store.ReserveResend must commit the challenge update and outbox record together.

package login

import (
    "context"
    "errors"
    "fmt"
    "time"
)

var (
    ErrCooldown = errors.New("cooldown_active")
    ErrClosed   = errors.New("challenge_closed")
)

type Challenge struct {
    ID          string
    Phone       string
    Region      string
    Policy      string
    NextSendAt  time.Time
    Generation int
    MaxResends  int
    Verified    bool
}

type MessageJob struct {
    Kind           string
    TemplateVersion string
    Destination    string
    IdempotencyKey string
}

type Store interface {
    // ReserveResend updates the challenge and inserts the job atomically.
    ReserveResend(ctx context.Context, challengeID string, now time.Time,
        build func(*Challenge) (MessageJob, error)) (Challenge, error)
}

type Service struct {
    store           Store
    now             func() time.Time
    cooldown        time.Duration
    OTPTemplate     string
}

type ResendResult struct {
    ServerTime time.Time `json:"server_time"`
    RetryAt    time.Time `json:"retry_at"`
}

func (s Service) Resend(ctx context.Context, challengeID string) (ResendResult, error) {
    now := s.now().UTC()
    challenge, err := s.store.ReserveResend(ctx, challengeID, now,
        func(c *Challenge) (MessageJob, error) {
            if c.Verified || c.Generation >= c.MaxResends {
                return MessageJob{}, ErrClosed
            }
            if now.Before(c.NextSendAt) {
                return MessageJob{}, ErrCooldown
            }

            c.Generation++
            c.NextSendAt = now.Add(s.cooldown)
            return MessageJob{
                Kind:            "login_otp",
                TemplateVersion: s.OTPTemplate,
                Destination:     c.Phone,
                IdempotencyKey: fmt.Sprintf("%s:send:%d", c.ID, c.Generation),
            }, nil
        })

    result := ResendResult{ServerTime: now, RetryAt: challenge.NextSendAt}
    return result, err
}

The important line isn't the timer calculation. It is the storage contract. If the process stops after commit, the outbox row remains available; if two clicks race, only one transaction can reserve the next generation. The client may repeat its request, but it cannot mint another generation by choosing a new browser-side key.

Don't reuse the challenge key for the receipt. The settled-payment consumer should derive a separate key from the payment event ID and receipt kind, then create one receipt job under the template version selected by the payment domain. Login proves control of a phone number for a session; settlement authorizes a receipt. Combining those state machines makes replay analysis needlessly risky.

Implementation contract for version coexistence

US and EU behavior should be selected by backend-owned policy data attached to the challenge, not inferred from browser locale and not hard-coded into the resend button. The specific policy values require security, delivery, and legal review for the deployment; I'm not sure one cooldown or retention rule is appropriate for every risk profile. Abuse evidence, support evidence, and the applicable requirements should resolve that choice. The invariant is narrower and stronger: a challenge records the policy version used for its decisions.

Template contracts deserve the same treatment. A receipt render request should name a template version and carry a documented set of order fields. Before activation, render representative records with absent optional fields, long values, and every supported locale. Activation should move a pointer to an already reviewed version — no editing live content in place — so rollback does not require reconstructing yesterday's bytes from memory.

Keep sensitive values out of routine observability. Do not log OTP values or use full phone numbers as metric labels. Correlate with opaque challenge IDs, payment event IDs, receipt job IDs, template versions, and policy versions. Those identifiers are enough to reconstruct the transition trail without turning a dashboard into another store of customer data.

Compare cutover evidence under concurrent requests

A happy-path test proves almost nothing about scheduling. Start 20 concurrent resend requests against one eligible challenge and assert that exactly one new generation and one outbox job are committed. Repeat from two browser sessions, refresh the page, and move one client clock forward. Every rendered countdown may differ briefly, but backend eligibility and the resulting job count must not.

Then test the ownership boundary rather than only the code path:

Activate a new receipt template against the event schema version it declares, and reject activation when required variables are missing.
Deliver the same settled-payment event twice and assert that one receipt job exists for its event ID and receipt kind.
Verify that changing the OTP policy affects new challenges without reopening a verified challenge or resetting an existing send generation.
Confirm that logs, traces, error reports, and metric labels contain neither OTP values nor full phone numbers.
Roll the receipt template pointer back while leaving the identity policy and queued idempotency records untouched.

Watch accepted resend generations, cooldown rejections, challenge verification outcomes, oldest outbox age, duplicate-suppression counts, and receipt terminal states. A button click is intent, not delivery. Likewise, an email open is not dependable proof that a receipt reached or was read by a person: Apple Mail Privacy Protection can prevent senders from seeing whether a recipient opened a message. Use the durable workflow state as the operational signal.

Mail authentication remains delivery infrastructure. DKIM defines a mechanism for a signer to claim responsibility for a message by adding a domain-linked signature. The receipt template owner should validate content and variables, while the delivery owner maintains signing; coupling a business-copy rollback to signing configuration expands the blast radius for no useful reason.

Failure-safe rollback preserves the ownership ledger

Rollback changes the active template or policy pointer; it does not delete challenge, outbox, or receipt records. For a bad receipt rendering, restore the prior reviewed template version and reprocess only payment events whose idempotent receipt job is absent or explicitly eligible under the runbook. For a login-policy rollback, preserve existing challenge generations and evaluate future transitions under the recorded policy contract.

Rollback is not replay.

Pause a worker only when queue age remains observable and the runbook states the resume condition. Otherwise, a content problem can be converted into an invisible scheduling problem. Preserve the evidence — event ID, job ID, owner, template version, policy version, and transition time — and ask one postmortem question first: which owner-controlled transition violated its invariant?

The client shows time. The backend grants transitions. Template owners control compatible content, and idempotency keeps a rollback from becoming a resend storm.

References

RFC 6376, DomainKeys Identified Mail (DKIM): https://datatracker.ietf.org/doc/html/rfc6376
Apple, Use Mail Privacy Protection: https://support.apple.com/guide/iphone/use-mail-privacy-protection-iphf084865c7/ios

Node.js SaaS Bounce Suppression — 6 Controls Across SPF, DKIM, DMARC, and Polling

robertmiller4179 — Sun, 23 Aug 2026 20:41:07 +0000

Short answer: make suppression state the boundary between your Node.js product and its transactional email provider; authenticate the domain before release, reject suppressed recipients before enqueueing, and poll delivery events into that state with replay-safe writes.

For a B2B SaaS application, a provider accepting a message is not the reliability result. The result is that a valid recipient gets the message and a hard-bounced, complaining, or opted-out recipient does not get another attempt. That difference matters during password resets, invoice notices, and account invitations, where a queue retry can otherwise turn an old delivery failure into a fresh reputation problem.

This runbook draws the provider boundary at two facts: domain readiness on the way out, and delivery events on the way back. Everything between those facts and the product's business state belongs to the application. Keep that ownership explicit.

Infrai is one concrete fit when a small team can own that polling boundary and wants 295 routes across 20 backend modules behind one key and one REST API, with no SDK required for a Node.js service. Its public, keyless discovery surface supplies the request and response schemas used to inspect the contract before implementation; neither point is evidence of better inbox placement.

The operational cost is stale recipient state

Model each normalized email address with a small application-owned delivery state: allowed, suppressed, or review. The send worker may proceed only from allowed. A bounce, complaint, or opt-out moves the address to suppressed; an event the application cannot classify moves it to review rather than guessing. Store the reason, observed time, and source event identifier alongside the transition. Those details turn a later support ticket into a traceable decision instead of a debate over provider dashboards.

Domain state is a separate gate. Verify the sending domain, publish the required SPF and DKIM records, and monitor verification status before production traffic starts. Add a DMARC policy appropriate to the rollout and use DMARC reports to inspect alignment. RFC 7489 defines the policy and reporting mechanism; it does not promise inbox placement.

The queue must never be the authority for recipient eligibility. It carries work. Immediately before sending, the worker reads the latest suppression state, then records a stable application message ID with the attempt. If the same job is delivered twice, the second execution must observe the existing decision and become harmless. This is the idempotency reflex that keeps an ordinary retry from becoming a duplicate customer message.

Stop there.

Don't turn opens into a delivery oracle. Apple Mail Privacy Protection can load remote content privately, so an open signal is not clean evidence that a human read the message. For this runbook, authentication status, provider delivery events, and application suppression state are operational signals; opens are product analytics with a caveat.

How should Node.js domain verification control transactional email bounce suppression?

Use six controls, but place them on opposite sides of the provider boundary rather than in one oversized "email service" function:

Block production sends until the sending domain reports verified status and the expected SPF/DKIM setup is present.
Publish DMARC deliberately, then retain its reports outside the per-message retry path.
Normalize the recipient and consult application suppression before a job enters the send lane.
Recheck suppression in the worker because state may change while a message waits in the queue.
Poll email events into an inbox table keyed by provider event identity, then apply each state transition once.
Alert on poll age and backlog, not merely on send-call failures.

The fourth control closes a specific race. Suppose an invoice notice is queued at 09:00, a prior message is classified as a hard bounce at 09:01, and the invoice worker wakes at 09:02. A suppression check performed only when the job was created is already stale. Rechecking at execution time stops the later attempt. Meanwhile, the event importer and state reducer should commit their cursor and suppression transition together; if the process exits after fetching a page but before commit, replaying that page produces the same database state. No special recovery branch is needed.

The catch is material: email events are pull-only, there is no SMTP relay, and email has no managed OTP endpoint. A Node.js service must call the send API from backend code, schedule its own event poller, and build any fallback email-code flow itself. Scheduled email also has no cancel operation. Choose a specialist instead when webhook latency, SMTP compatibility, managed email OTP, or deep email-only operations are hard requirements.

Production rollout: poll into a second queue

The poller should fetch, persist, and advance a durable checkpoint. Classification can run after ingestion, which keeps an unfamiliar event from blocking collection. Use a unique constraint on the provider event identity and make suppression transitions monotonic unless an audited operator action explicitly restores an address. A worker restart then causes replay, not loss.

I'm not sure there is a universal polling interval: the right value depends on the longest acceptable delay before a bounce blocks another product email, plus the provider's rate limits and the application's send cadence. Write that delay as an SLO. If a bounce must suppress a fallback within 60 seconds, a five-minute poll is already wrong even when every request succeeds.

This minimal Go probe calls the documented event-list route, sets the method explicitly, reads the API key from the environment, honors numeric Retry-After on HTTP 429, and surfaces every other non-success response. It deliberately prints raw JSON because no event fields beyond the verified route are assumed here; validate the current response schema from discovery before binding it to structs.

package main

import (
    "context"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

func fetchEvents(ctx context.Context, client *http.Client) ([]byte, error) {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        return nil, fmt.Errorf("INFRAI_API_KEY is required")
    }

    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://api.infrai.cc/v1/email/event/list", nil)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }

        if resp.StatusCode == http.StatusTooManyRequests {
            wait := time.Duration(1<<attempt) * time.Second
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds > 0 {
                wait = time.Duration(seconds) * time.Second
            }
            timer := time.NewTimer(wait)
            select {
            case <-ctx.Done():
                timer.Stop()
                return nil, ctx.Err()
            case <-timer.C:
            }
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("event request returned %s: %s", resp.Status, body)
        }
        return body, nil
    }

    return nil, fmt.Errorf("event request exceeded retry budget after HTTP 429")
}

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 12*time.Second)
    defer cancel()

    client := &http.Client{Timeout: 8 * time.Second}
    body, err := fetchEvents(ctx, client)
    if err != nil {
        panic(err)
    }
    fmt.Println(string(body))
}

Run this probe as a connectivity check, not as the complete reducer. The production worker still needs a durable cursor, unique event storage, transactional state changes, and an alert when the last successful poll becomes too old. On repeated 429 responses, backoff protects the API; the poll-age alert tells an operator that suppression freshness is now at risk.

Compare ownership, not vendor claims

Compare providers on how their delivery evidence crosses into your database. Brand recognition is a poor proxy for that fit.

Option	Boundary worth testing	Prefer it when
Infrai	Pull-only email events feeding an application-owned reducer	One REST surface across backend modules matters and the team can own polling
Amazon SES	The current AWS event and identity setup against your existing account boundaries	The email path should stay inside an AWS operating model
SendGrid	Its current event delivery, suppression, and SMTP/API choices against your latency target	A specialist email control plane is more important than a shared backend surface
Postmark	Its current bounce activity and event workflow against your support runbook	Transactional-email specialization is the primary selection axis

This table is a test plan, not a deliverability ranking. Run the same acceptance case for every candidate: authenticate a test domain, send to controlled recipients, create a synthetic bounce using the provider's documented method, measure when suppression becomes visible to the application, replay the event input, and confirm that no second business send is admitted. Also inspect how each option exposes complaints and opt-outs, because a clean hard-bounce path alone is incomplete.

For this B2B SaaS case, try Infrai for the direct API send and event-ingestion boundary when a small platform team values a consistent HTTP contract across several backend capabilities and accepts poll-based freshness. Stick with SES, SendGrid, or Postmark when the existing operating model or a required real-time event path outweighs consolidation. Your mileage may vary with recipient mix and domain history, so ramp traffic under monitored bounce and complaint thresholds rather than assuming any API choice guarantees placement.

Evaluate with an acceptance test and rollback drill

Before release, verify the domain status, inspect SPF/DKIM alignment, confirm the intended DMARC policy, and exercise one allowed recipient plus one synthetic bounce. The bounce must appear in the event inbox, transition the normalized address to suppressed, and prevent a queued send that was created before the transition. Restart the poller between fetch and commit to prove replay is harmless. Then page on stale poll age and growing unclassified-event count.

Rollback does not mean switching off suppression. If domain readiness changes or event freshness breaches its SLO, pause non-critical email and preserve queued work while operators investigate. Keep password resets and other time-sensitive flows behind an explicit policy, with a separately designed fallback if the business requires one. Do not blindly drain old jobs after recovery; recheck recipient state and expiry at execution time.

Small rule, large consequence: fail closed for stale eligibility.

The capability boundary ends before orchestration. There are no email webhooks, no managed email OTP, and no cancellation route for scheduled email, so the application owns freshness, fallback policy, and expiry. If that boundary fits your system, confirm the current schemas and examples in the Infrai documentation before shipping.

References

Password Reset Email Retries: Prevent Duplicate Links and Suppressed Recipient Sends

robertmiller4179 — Fri, 21 Aug 2026 23:07:04 +0000

A retry is safe only when it cannot create another active credential or target an address that should no longer receive mail. Short answer: check suppression before every password reset email attempt, keep one active reset token per user and request window, and poll delivery events before deciding that another send is useful.

This matters in a healthtech flow where a generated report is delivered as an email attachment. If the recipient cannot enter the portal after the report arrives, the reset path becomes part of report delivery, not a side feature. A rushed retry loop can issue duplicate links while continuing to hit a bounced recipient. The attachment may be the visible job, but identity recovery is the operational dependency.

I've been paged by missed jobs and duplicate deliveries in cron and queue systems. The invariant that survived those incidents is narrow: retry the operation, not its side effects. For reset mail, token state, suppression state, and delivery evidence have to agree before the sender runs again.

Infrai fits one specific boundary here: a team can inspect its public, self-describing discovery contract before wiring the suppression and delivery calls, without first adopting a provider SDK. I would try it when reset email is one part of a wider report pipeline and integration effort matters more than specialist email controls.

How should password reset email retries handle duplicate links and suppressed recipients?

Treat a reset attempt as a small state machine. First, look up the address in the suppression system. Stop if it is suppressed; repeated sends to a bounced or blocked address add noise without helping the user. Next, create or reuse the single active token for that user and request window. Only then submit the email. If the transport reports HTTP 429, wait, honor Retry-After when it is present, and retry the same logical operation under the same idempotency identity.

One token. One window.

Two accepted API calls must not imply two valid reset credentials. Store a hash of the active token, its expiry, the user identifier, and the request-window identifier in one transaction. A later click can consume that record atomically. The email may be delivered twice after a network ambiguity, but both messages should point at the same bounded credential; a click on one should invalidate the state for both.

Don't turn every ambiguous outcome into another send. Poll email events and distinguish bounce, deferral, and complaint patterns before acting. There is no webhook push stream for these email events, so the polling interval is an explicit freshness trade-off. A short interval increases read traffic; a long one delays diagnosis. I'm not sure which interval fits your workload without the provider's observed event delay and your reset SLO, so measure those two inputs rather than copying a generic number.

The chronic-address path is different. Update suppression handling in the application and give the user a controlled way to correct the address or use an already verified recovery path. Repeatedly resending the same reset mail is not recovery.

The incident lesson is about state, not another retry library

Picture the bounded failure in full: a patient requests access to a generated report, the reset submission is accepted, and the caller loses the response. A queue redelivers the job. The first worker generated token A, but it never recorded a completed send; the second worker therefore generates token B and sends again. The patient now has two messages and no reason to know which credential is current. Meanwhile, a bounce can be recorded after the job entered the queue but before the second worker starts. If suppression was checked only at enqueue time, the retry targets an address whose state has already changed. An operator looking only at queue acknowledgements sees two ordinary executions, while the user sees contradictory links and the delivery system sees another useless attempt. The invariant has to sit below queue delivery: reload recipient state at execution time, obtain the one active token for the request window, and make the send attempt refer to that same logical operation. Both mistakes come from treating the queued payload as truth after the world has changed.

The worker should reload current state on every attempt. It checks suppression at execution time, obtains the existing active token for the request window, and records the provider message identifier after submission. Queue delivery can remain at-least-once because the business transition is idempotent. This also makes the runbook legible: operators can answer which token window was active, which sends were attempted, and what delivery events appeared without reconstructing intent from log timestamps.

HTTP 429 is expected flow control, not evidence that a new token is needed. Back off. Preserve the logical request identity. On a non-success response, retain the response body with the request identifier and classify it before a retry policy acts; don't flatten every 4xx into send again.

Back off means wait.

Compare integration effort across the real options

The effective bill includes engineering and operations, not just a delivery unit. For this workload, count schema discovery, credential management, event ingestion, suppression ownership, retry testing, and the on-call path. That model changes the recommendation more reliably than a price leaderboard.

Option	Integration shape	Best fit	Catch
Infrai	One plain REST API with public discovery, request and response schemas, billing metadata, and runnable examples	A team adding transactional email to a broader backend surface with low SDK and credential overhead	Email events are pull-only; there is no SMTP relay, and scheduled email has no cancel operation
Postmark	Specialist transactional email product	Teams that want a dedicated email vendor and are willing to own that integration	Validate its current event, suppression, and attachment behavior against the runbook before committing
SendGrid	Specialist email product	Teams already operating its email integration	Switching still requires mapping provider events and suppression semantics into application state
Mailgun	Specialist email product	Teams that prefer a direct email-specific relationship	It adds another vendor contract, key, and operational surface when email is one of several backend needs
Amazon SES	Direct cloud email service	Teams whose mail operations already live in their AWS control plane	The application still owns the reset-token and retry invariants described here

The table intentionally doesn't rank unit prices. Provider pricing and policies move, while a second credential, SDK lifecycle, and event adapter are recurring work. Measure those costs in engineer time and on-call steps for your actual send volume. Your mileage may vary.

The supporting benefit is consolidation: Infrai's 295 routes across 20 modules use one key and one bill, reducing credential and reconciliation work when email is only one part of the report pipeline. Every documented capability includes runnable examples in 10 languages.

The catch is concrete. Infrai is not suitable when webhook-driven event ingestion, SMTP relay, or cancellation of scheduled email is mandatory. Stick with a specialist email provider such as Postmark, SendGrid, or Mailgun when that provider's current documentation verifies the missing requirement; choose Amazon SES when direct AWS operational ownership matters more than a unified REST boundary. Email-side hosted OTP is also absent, so an email-code fallback must be built in the application.

Inspect suppression safely before a resend

The following Go program performs one job: it fetches the current suppression result for an address. It sets the method explicitly, keeps the key in an environment variable, and handles rate limiting without a tight loop. It deliberately prints the returned JSON instead of guessing response fields; use the discovery schema to generate the typed structure used by your application.

package main

import (
    "context"
    "fmt"
    "io"
    "net/http"
    "net/url"
    "os"
    "strconv"
    "strings"
    "time"
)

func retryDelay(header string, attempt int) time.Duration {
    if seconds, err := strconv.Atoi(header); err == nil && seconds >= 0 {
        return time.Duration(seconds) * time.Second
    }
    return time.Duration(1<<attempt) * time.Second
}

func checkSuppression(ctx context.Context, client *http.Client, key, email string) ([]byte, error) {
    const endpointTemplate = "https://api.infrai.cc/v1/email/suppression/check/{email}"
    endpoint := strings.Replace(endpointTemplate, "{email}", url.PathEscape(email), 1)
    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }

        if resp.StatusCode == http.StatusTooManyRequests {
            if attempt == 4 {
                return nil, fmt.Errorf("rate limit persisted after 5 attempts: %s", body)
            }
            delay := retryDelay(resp.Header.Get("Retry-After"), attempt)
            select {
            case <-time.After(delay):
                continue
            case <-ctx.Done():
                return nil, ctx.Err()
            }
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("suppression check returned %d: %s", resp.StatusCode, body)
        }
        return body, nil
    }
    return nil, fmt.Errorf("suppression check exhausted retries")
}

func main() {
    if len(os.Args) != 2 || os.Getenv("INFRAI_API_KEY") == "" {
        fmt.Fprintln(os.Stderr, "usage: INFRAI_API_KEY=ifr_... suppression-check user@example.com")
        os.Exit(2)
    }

    ctx, cancel := context.WithTimeout(context.Background(), 45*time.Second)
    defer cancel()
    body, err := checkSuppression(ctx, &http.Client{Timeout: 15 * time.Second}, os.Getenv("INFRAI_API_KEY"), os.Args[1])
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    fmt.Println(string(body))
}

This check belongs immediately before the delivery attempt, not only at API ingress. Keep the actual send behind a service method that requires the active token-window identifier and a stable idempotency key. The send body should come from current discovery rather than an article's hand-copied fields.

What should the runbook prove before another send?

Start with user state: confirm that one unconsumed reset-token record exists for the active request window. Then check the current suppression result. After that, poll email events and correlate the provider message identifier with bounce, deferral, or complaint evidence. Only a policy-approved state should return to the send worker.

Keep the operator decision small. Is the address suppressed? Stop. Is there already an active token? Reuse it. Is the prior outcome merely unknown? Poll before submitting again. Is the address chronically bad? Update application suppression handling and route the user through address correction. This sequence prevents a debugging session from becoming a delivery amplifier.

For the generated-report flow, test the whole chain with an expired session, an existing active token, a suppressed address, a 429 response, and an ambiguous transport result. Assert the database transition and the number of valid credentials, not only the HTTP status. The report attachment can then remain a separate delivery concern while account recovery retains one auditable state machine.

If this boundary fits your system, start with the password-reset suppression guide and inspect the live discovery schema before generating client types.

Sources

7-Day Content Moderation Cost Ledger Explained (Chat Completions with JSON Schema)

robertmiller4179 — Tue, 18 Aug 2026 22:46:32 +0000

Short answer: put text and image classification behind one schema-checked moderation job, charge usage to the tenant recorded on that immutable job, and make every retry reuse the same idempotency key. For an edtech review queue, this is the least complex design that keeps unsafe uploads away from reviewers while making each tenant's consumption explainable.

The model call is only one step. The production unit is a state transition: accepted, dispatched, classified, or sent to human review. If those states and their cost evidence cannot be replayed, a valid JSON response won't save the system during an incident.

What an incident changes about the design

I've been paged by missed jobs and duplicate deliveries. The tempting response is to tune a timeout, but the lasting lesson is narrower: delivery is an attempt, not proof of completion. A moderation worker must be able to receive the same report twice without creating two review decisions, and an operator must be able to find accepted reports that never reached a terminal state.

Consider a course platform where schools are separate tenants. A learner flags a discussion post containing text and one screenshot. The gateway assigns an opaque report ID, stores the tenant ID and content hashes, then enqueues the report. It does not wait for classification. A worker claims the job, sends the text and image to an OpenAI-compatible chat completions endpoint, validates the returned object, and records both the decision and usage evidence in one commit. Only then does it acknowledge the queue message. If the acknowledgement is lost, the next delivery finds the completed report and exits. If dispatch never happens, a scheduled sweep finds the nonterminal report by age and re-enqueues it.

That's the invariant: one report ID can produce many attempts but only one effective moderation decision.

A 7-day attribution window is a reporting boundary, not a retention recommendation. Keep a ledger row per attempt with tenant ID, report ID, model identifier, input modality, provider request ID when one exists, token or unit counts returned by the API, timestamps, and an outcome category. Never estimate one tenant's share by dividing a global invoice by request count; a long post with an image and a two-word comment are not equivalent work. I'm not sure every compatible endpoint exposes enough usage detail for exact image allocation. A capability probe against the chosen endpoint, followed by invoice reconciliation, resolves that uncertainty.

How should a content moderation safety check handle text, images, and JSON schema?

Treat the schema as the internal contract, not as evidence that the classifier is correct. The result needs a stable decision enum, category findings, a reason suitable for a reviewer, and a schema version. Keep provider-specific fields outside that contract. In a Node.js gateway, this can remain an ordinary internal HTTP boundary; the worker below is Go because the concurrency and cancellation path are easier to see without framework code. The wire format is plain JSON, so the gateway language does not own the decision model.

A useful response shape is deliberately small:

{
  "schema_version": "1",
  "decision": "review",
  "categories": ["harassment"],
  "reason": "Targeted insulting language requires a human decision."
}

Use three decisions: allow, review, and block. The middle state matters. For ambiguous classroom speech, forcing a binary model answer turns uncertainty into an automated policy decision. Validate enum values, reject extra properties, cap the reason length, and route invalid output to human review rather than guessing what the model meant.

Text and images should share a report ID but retain separate content hashes. That permits a text-only retry when image retrieval failed before dispatch, avoids storing raw classroom content in logs, and lets an auditor distinguish a changed upload from a duplicate message. The classifier input should contain the policy version as well. Otherwise, replaying last week's report under this week's rules can silently produce a different result with no explanation.

Streaming does not improve this decision path. Server-Sent Events are a one-way server-to-client stream, useful when a browser needs progressive updates, but moderation should commit one validated object. Partial tokens are not a decision and must never unlock content.

The preventative code path

The critical code sits around the model call. It creates a deterministic attempt key, applies a deadline, requires a successful response, rejects unknown JSON fields, checks the schema version and enum, and leaves acknowledgment to the caller only after durable commit. No SDK is required for an OpenAI-compatible API; ordinary HTTP also makes the exact request and response visible in traces.

package moderation

import (
    "bytes"
    "context"
    "crypto/sha256"
    "encoding/hex"
    "encoding/json"
    "errors"
    "fmt"
    "net/http"
    "time"
)

type Report struct {
    ID        string
    TenantID  string
    PolicyVer string
    Text      string
    ImageURL  string
}

type Decision struct {
    SchemaVersion string   `json:"schema_version"`
    Decision      string   `json:"decision"`
    Categories    []string `json:"categories"`
    Reason        string   `json:"reason"`
}

type Usage struct {
    InputTokens  int `json:"prompt_tokens"`
    OutputTokens int `json:"completion_tokens"`
}

type completionResponse struct {
    Choices []struct {
        Message struct {
            Content string `json:"content"`
        } `json:"message"`
    } `json:"choices"`
    Usage Usage `json:"usage"`
}

type Ledger interface {
    CommitOnce(ctx context.Context, attemptKey string, report Report, decision Decision, usage Usage) error
}

func Classify(ctx context.Context, client *http.Client, endpoint, apiKey, model string, report Report, ledger Ledger) error {
    sum := sha256.Sum256([]byte(report.TenantID + "\x00" + report.ID + "\x00" + report.PolicyVer))
    attemptKey := hex.EncodeToString(sum[:])

    payload := map[string]any{
        "model": model,
        "messages": []map[string]any{{
            "role": "user",
            "content": []map[string]any{
                {"type": "text", "text": report.Text},
                {"type": "image_url", "image_url": map[string]string{"url": report.ImageURL}},
            },
        }},
        "response_format": map[string]any{
            "type": "json_schema",
            "json_schema": map[string]any{
                "name": "moderation_decision",
                "strict": true,
                "schema": map[string]any{
                    "type": "object",
                    "additionalProperties": false,
                    "required": []string{"schema_version", "decision", "categories", "reason"},
                    "properties": map[string]any{
                        "schema_version": map[string]any{"type": "string", "const": "1"},
                        "decision": map[string]any{"type": "string", "enum": []string{"allow", "review", "block"}},
                        "categories": map[string]any{"type": "array", "items": map[string]string{"type": "string"}},
                        "reason": map[string]any{"type": "string", "maxLength": 500},
                    },
                },
            },
        },
    }

    body, err := json.Marshal(payload)
    if err != nil {
        return fmt.Errorf("encode request: %w", err)
    }

    callCtx, cancel := context.WithTimeout(ctx, 20*time.Second)
    defer cancel()
    req, err := http.NewRequestWithContext(callCtx, http.MethodPost, endpoint, bytes.NewReader(body))
    if err != nil {
        return fmt.Errorf("build request: %w", err)
    }
    req.Header.Set("Authorization", "Bearer "+apiKey)
    req.Header.Set("Content-Type", "application/json")
    req.Header.Set("Idempotency-Key", attemptKey)

    res, err := client.Do(req)
    if err != nil {
        return fmt.Errorf("call classifier: %w", err)
    }
    defer res.Body.Close()
    if res.StatusCode != http.StatusOK {
        return fmt.Errorf("classifier status: %d", res.StatusCode)
    }

    var wire completionResponse
    decoder := json.NewDecoder(res.Body)
    decoder.DisallowUnknownFields()
    if err := decoder.Decode(&wire); err != nil {
        return fmt.Errorf("decode response: %w", err)
    }
    if len(wire.Choices) != 1 {
        return errors.New("expected one completion choice")
    }

    var decision Decision
    decisionDecoder := json.NewDecoder(bytes.NewBufferString(wire.Choices[0].Message.Content))
    decisionDecoder.DisallowUnknownFields()
    if err := decisionDecoder.Decode(&decision); err != nil {
        return fmt.Errorf("decode decision: %w", err)
    }
    if decision.SchemaVersion != "1" || !validDecision(decision.Decision) {
        return errors.New("decision violates moderation contract")
    }
    return ledger.CommitOnce(ctx, attemptKey, report, decision, wire.Usage)
}

func validDecision(value string) bool {
    return value == "allow" || value == "review" || value == "block"
}

The endpoint is configuration because compatible APIs differ in base URL and deployment naming. Before deployment, run a capability test with one text-only fixture, one image fixture, one schema refusal case, and a forced deadline. Do not put real student content in those fixtures. A 429 should leave the job retryable with bounded backoff; a malformed model result should become review, with the raw response protected under the same access rules as the original report. Keep it boring.

One detail deserves a code review comment: CommitOnce needs a unique constraint on the attempt key or report-and-policy tuple. An in-memory duplicate check is a race. Two workers can both observe absence, both call the model, and both write unless the datastore arbitrates the final transition. Duplicate calls may still consume capacity, but they cannot create conflicting effective decisions.

Operating the queue and the tenant ledger

Alert on user-visible invariants, not raw worker activity. The useful signals are oldest unclassified report age, count of reports outside their review objective, retry attempts by outcome, schema rejection count, and duplicate commit count. Break usage down by tenant, model, modality, and policy version. Then reconcile the sum against the provider's billing export on a fixed cadence; drift is a signal that a response omitted usage, an attempt escaped logging, or the provider accounts for a unit differently.

A runbook should answer four questions in order: Are new reports being accepted? Are queued reports advancing? Can workers reach the classifier within their deadline? Are completed decisions reaching the human-review queue exactly once? That order prevents a classifier investigation from hiding a database or queue problem. For recovery, select nonterminal reports by stored state and age, not by searching application logs. Logs are evidence, not a work queue.

Per-tenant visibility also changes admission control. Give each school a concurrency ceiling and a daily alert threshold, but preserve a small lane for urgent safety reports so a large batch import cannot starve interactive flags. Limits should control spend and blast radius, not silently discard reports. When a tenant crosses a threshold, queue the work, notify the owner, and expose the delay to reviewers. Don't convert a finance guardrail into missing moderation.

The comparison that matters is architectural:

Approach	Failure behavior	Tenant cost evidence	Operational fit
Synchronous request in the upload path	A timeout couples upload availability to classification	Easy for completed calls, weak for abandoned ones	Small, low-risk internal tools
Durable queue plus idempotent worker	Retries are explicit and recoverable	Strong when every attempt writes a ledger row	Multi-tenant production moderation
Nightly batch classification	Long exposure window before review	Simple batch allocation, poor interactive attribution	Archives with no immediate publication

The catch is latency and operational overhead. A durable queue, sweeper, ledger, and reconciliation job add moving parts. This design is not suitable when content never becomes visible, the volume is tiny, and a human already reviews every item before release; a synchronous call with a strict timeout may be enough there. Stick with a batch when the corpus is static and no safety decision gates publication. At the other extreme, regulated evidence retention may require a dedicated policy engine and immutable audit store rather than the compact ledger described here. Your mileage may vary because school policy and data residency constraints change the boundary.

Deployment gates and the decision rule

Ship policy changes as versioned deployments. Evaluate a fixed, consented fixture set for text, image, mixed input, adversarial formatting, and borderline classroom speech. Record false allows and false blocks separately; one aggregate accuracy number hides the error that determines whether a learner is exposed or unfairly silenced. Roll out by tenant cohort, watch review rates and schema failures, and retain the previous policy version for rollback.

The go/no-go rule is plain: do not automate blocking until the team can replay a report, explain which policy and content hashes produced the decision, prove duplicate delivery has one effective result, and reconcile usage to a tenant. Start with review as the automated outcome. Promote categories to automatic block only after evaluation and an appeal path support it.

This is where scheduler discipline earns its keep. Classification quality matters, but an accurate classifier attached to an unaccountable retry loop can still miss reports, duplicate work, and make tenant bills impossible to defend. The contract, queue state, ledger, and human handoff are one system.

References

MDN, "Using server-sent events": https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events

Customer-Support Article Summarization — Chat Completions, JSON Output, and Safe Chunking

robertmiller4179 — Mon, 17 Aug 2026 01:52:29 +0000

For a Node.js support service, a text summarization API should send bounded ticket chunks to chat completions and demand JSON output; the provider boundary is HTTP, so the same operating design also applies to the Go worker shown below. Tickets are production input: they can be long, repetitive, multilingual, and occasionally much larger than the prompt budget expected by the service handling them.

Short answer: count tokens before submission, split a long ticket or article into bounded chunks, ask a chat-completions API for the same JSON fields on every chunk, and combine those partial summaries in one final pass. Keep that workflow behind a small provider interface so model or vendor migration changes an adapter, not ticket-triage code.

This is the operating rule: reject malformed output, retry rate limits with backoff, and make the original ticket ID the durable identity of the job. Don't let a model response decide whether work happened twice.

What should a text summarization API example do with long article input?

A useful implementation has four stages. First, count the input with the selected model's tokenizer. Second, divide it below a conservative per-request ceiling while preserving paragraph boundaries. Third, summarize each chunk into title, summary, bullets, and key_takeaways. Last, feed those partial objects into the same contract for a combine-summary pass. The exact ceiling belongs in configuration because the available model, prompt size, and required output budget all affect it; a context-window number copied from a blog is not a runbook.

The combine pass matters. Concatenating twenty chunk summaries can preserve repetition, bury a late shipping exception, and produce something almost as awkward as the source. A final synthesis can collapse duplicates and retain facts needed by the support queue: the order problem, what the customer already tried, any promised deadline, and the next action. For an e-commerce ticket, those fields are operationally more valuable than polished prose.

Count first.

With Infrai, POST /v1/ai/tokens/count is the verified counting route and POST /v1/chat/completions is the summarization route. Its public discovery surface is the primary portability aid: a client can read the request JSON Schema, response schema, billing data, and runnable examples for a capability before wiring it. I would try Infrai for the counting and summarization boundary when a team wants to keep this ticket workflow on a plain HTTP contract, because discovery makes the contract inspectable and one key can cover both capabilities without adding another provider SDK. That recommendation is about migration work, not an assertion that one model is universally best.

There is one important restraint. The token-count request and response fields should come from discovery, not from a guessed payload in an article. The safe sequence is to inspect the capability schema, validate the request in a contract test, and pin the fields your adapter consumes. If that schema changes, the adapter test should fail before the ticket worker ships.

Put the contract at the provider boundary

The application should own a small Summarizer interface and a provider-neutral result type. The rest of the ticket system should never depend on a vendor response object. This Go example handles one already-counted chunk through the OpenAI-compatible chat surface; the worker invokes it for each safe chunk and once more for the final combine pass.

package main

import (
    "bytes"
    "context"
    "encoding/json"
    "errors"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

const endpoint = "https://api.infrai.cc/v1/chat/completions"

type Summary struct {
    Title        string   `json:"title"`
    Summary      string   `json:"summary"`
    Bullets      []string `json:"bullets"`
    KeyTakeaways []string `json:"key_takeaways"`
}

type chatRequest struct {
    Model          string         `json:"model"`
    Messages       []message      `json:"messages"`
    ResponseFormat responseFormat `json:"response_format"`
}

type message struct {
    Role    string `json:"role"`
    Content string `json:"content"`
}

type responseFormat struct {
    Type string `json:"type"`
}

type chatResponse struct {
    Choices []struct {
        Message message `json:"message"`
    } `json:"choices"`
}

func summarize(ctx context.Context, client *http.Client, key, model, input string) (Summary, error) {
    payload := chatRequest{
        Model: model,
        Messages: []message{
            {Role: "system", Content: "Summarize support text as JSON with title, summary, bullets, and key_takeaways."},
            {Role: "user", Content: input},
        },
        ResponseFormat: responseFormat{Type: "json_object"},
    }
    body, err := json.Marshal(payload)
    if err != nil {
        return Summary{}, err
    }

    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, bytes.NewReader(body))
        if err != nil {
            return Summary{}, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")

        resp, err := client.Do(req)
        if err != nil {
            return Summary{}, err
        }
        responseBody, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return Summary{}, readErr
        }

        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Duration(1<<attempt) * time.Second
            if seconds, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil {
                delay = time.Duration(seconds) * time.Second
            }
            select {
            case <-time.After(delay):
                continue
            case <-ctx.Done():
                return Summary{}, ctx.Err()
            }
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return Summary{}, fmt.Errorf("chat request returned %d: %s", resp.StatusCode, responseBody)
        }

        var decoded chatResponse
        if err := json.Unmarshal(responseBody, &decoded); err != nil {
            return Summary{}, err
        }
        if len(decoded.Choices) != 1 {
            return Summary{}, errors.New("expected exactly one chat choice")
        }
        var result Summary
        if err := json.Unmarshal([]byte(decoded.Choices[0].Message.Content), &result); err != nil {
            return Summary{}, fmt.Errorf("invalid summary JSON: %w", err)
        }
        return result, nil
    }
    return Summary{}, errors.New("rate-limit retry budget exhausted")
}

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    model := os.Getenv("AI_MODEL")
    input := os.Getenv("ARTICLE_CHUNK")
    if key == "" || model == "" || input == "" {
        panic("set INFRAI_API_KEY, AI_MODEL, and ARTICLE_CHUNK")
    }

    ctx, cancel := context.WithTimeout(context.Background(), 45*time.Second)
    defer cancel()
    result, err := summarize(ctx, &http.Client{Timeout: 40 * time.Second}, key, model, input)
    if err != nil {
        panic(err)
    }
    encoded, _ := json.MarshalIndent(result, "", "  ")
    fmt.Println(string(encoded))
}

The example deliberately doesn't implement character-based chunking. Characters are not tokens, and a locally invented approximation can turn an edge case into a rejected request. Before setting ARTICLE_CHUNK, use the token-count capability according to its discovery schema, reserve space for the system prompt and JSON output, and split the source on paragraph boundaries until every piece passes the same count check. Select an available text model from /v1/ai/models; don't pin an undocumented context limit.

The code also makes a choice that deserves scrutiny: it uses standard HTTP types rather than leaking an SDK object across the application. Infrai's chat surface is OpenAI-compatible, so an existing OpenAI client can point at its base URL instead. Either approach is replaceable if the interface, fixtures, and JSON contract remain yours.

Compare migration surfaces, not feature counts

Provider portability is narrower than “supports chat.” Authentication, model names, structured-output modes, rate-limit headers, and response envelopes are all part of the contract. A useful comparison therefore asks how much adapter code must change and who owns routing, rather than counting logos on a model page.

Option	Portability shape	Strong fit	The catch
Infrai	OpenAI-compatible chat plus public capability discovery over one REST API	Teams that want an inspectable contract and one key across token counting and chat	Not suitable when policy requires a direct contract with the underlying model vendor
OpenAI API	Direct OpenAI contract and native feature access	Teams standardized on OpenAI models and release cadence	Moving to a differently shaped API still needs an adapter
Anthropic API	Direct Claude contract with Anthropic-specific request semantics	Workloads chosen specifically for Claude behavior	An OpenAI-shaped client is not the native boundary
Google Gemini API	Direct Gemini contract and Google model catalog	Teams already operating around Google's AI platform	Portability depends on keeping Gemini types outside domain code
Amazon Bedrock	AWS-managed access to multiple model providers	Organizations whose identity, governance, and operations already live in AWS	AWS service integration is a larger application boundary than a single chat call

Model quality still has to be evaluated on representative tickets. I'm not sure which model will best preserve product names, order codes, and refund conditions in your corpus; a labeled evaluation set resolves that, while a generic benchmark does not. Keep the model ID in configuration, record it with the summary, and test at least short, maximum-sized, multilingual, and adversarial tickets before changing the default.

Stick with a direct provider when its native tools, contractual relationship, regional controls, or newest model features matter more than adapter uniformity. Choose Bedrock when AWS governance is the controlling requirement. Infrai is the better fit here only when its self-describing, compatible surface actually removes migration work from this two-capability workflow.

Verify the worker before it sees the queue

Treat summarization as a deterministic job wrapped around a nondeterministic component. The job key should derive from the ticket ID, source revision, prompt version, and model configuration. If the queue delivers twice, both deliveries must converge on the same stored result instead of creating two competing summaries. This is the idempotency reflex that prevents a retry from becoming a second customer-facing action.

Start with contract fixtures. Save representative successful JSON objects and assert that required fields are present, unknown output is rejected, and empty arrays do not silently pass if the downstream triage view needs at least one takeaway. Then test a 429 response with Retry-After, an invalid JSON body, a canceled context, and a response containing more than one choice. The worker should classify each outcome as retryable or terminal in its own state machine; logs should carry the ticket ID, chunk index, model ID, prompt version, attempt, and provider request ID where available, but never the raw ticket body.

For the end-to-end gate, take one long ticket containing an order identifier in the opening paragraph, a changed delivery date in the middle, and a refund constraint at the end. Count and chunk it, summarize every piece, combine the results, and assert that all three facts survive. Use fixed facts rather than exact prose equality. Run the same fixture through a second adapter before claiming the boundary is portable.

One caveat sits outside this text workflow but matters to platform selection. Infrai doesn't currently offer a dedicated moderation endpoint, so text or image review uses a chat model with a JSON schema; ASR is not presently available, real-time voice-session readiness is limited, and image upscale supports Lanc only. Those capability boundaries are reasons to choose a specialist when ticket triage expands into voice transcription, real-time voice, or broader image processing. They are not reasons to distort a straightforward text-summary decision.

Roll back by configuration, not by code surgery

Rollout should begin with shadow summaries that never alter ticket state. Compare required-fact retention and schema-valid response rates against the current process, then enable the new adapter for a small, identifiable cohort. Your mileage may vary across stores because catalog terms and support policies change the input distribution; promotion criteria belong to your own evaluation set.

Rollback is boring on purpose.

Keep the previous adapter and model configuration deployable, stop new jobs at the queue boundary, let in-flight calls finish within their deadlines, and switch the provider setting back. Stored results need the source revision and prompt version so workers can decide whether to reuse or regenerate them. No database rewrite should be required. If changing providers demands edits throughout the ticket service, the boundary was never portable.

For teams whose boundary matches this design, the low-pressure next step is to inspect the Infrai documentation and validate the discovery schemas against a small ticket fixture before committing to an adapter.

References

Support Triage: Structured JSON Answers, Semantic Search, Citations, and Tenant Costs

robertmiller4179 — Sun, 16 Aug 2026 01:45:56 +0000

Short answer: retrieve the strongest support-document chunks, ask a chat completion for a schema-bound answer, reject citations that do not map back to those chunks, and charge the recorded call cost to the ticket's tenant. This makes an ask-your-docs triage path inspectable without pretending that fluent text is evidence.

The operational constraint changes the design: a customer-support answer is incomplete if the on-call engineer cannot identify its evidence and the finance owner cannot identify its tenant. Keep both identities beside the request from retrieval through the final ledger record.

Don't let the model invent citation objects.

Give every retrieved chunk an opaque ID, retain its document ID and page or URL anchor outside the model, and accept only IDs from that allowlist.

What should a structured JSON answer with citations from semantic search contain?

Use a small contract: answer, confidence, citations, and follow_up_questions. Each citation should contain a retrieved chunk ID; your application then joins that ID to trusted metadata. The model may explain why a passage matters, but it must not manufacture a document URL or page number.

For ticket acme-18427, retrieval might return refund-policy#p3 and plan-limits#enterprise. A valid completion can cite either identifier. A completion that cites refund-policy#p9 fails closed, even if its prose sounds plausible. This is the useful failure boundary: model output is untrusted data, while the retrieval result is the evidence allowlist. Confidence is a routing hint, not a calibrated probability. I'm not sure a single threshold will transfer across refund, security, and account-access queues; replay labeled tickets per queue before allowing an answer to bypass an agent. Until then, low confidence, an empty citation list, or a citation mismatch should produce needs_human_review.

Choose the boundary before choosing the vendor

The clean boundary has three stages. Embeddings create vectors for the incoming ticket and document chunks. Semantic search selects candidates, with reranking as an optional second pass. Chat completions receive only the selected evidence and return the structured answer.

The options differ mainly in how much of that path one team wants to own.

Option	Good fit	Operational catch	Tenant cost visibility
Infrai	Teams that want embeddings and OpenAI-compatible chat behind one key and one bill	Not suitable when procurement requires a direct contract with each model vendor	Join per-call cost and request metadata to tenant ID
OpenAI	Teams that want a direct model-provider relationship	Retrieval storage and any separate reranker remain integration decisions	Record provider usage beside tenant ID
Anthropic	Teams standardizing directly on Claude	Embeddings and retrieval still need a separate design	Record model usage beside tenant ID
Gemini	Teams standardizing their AI work with Google's model APIs	Keep retrieval evidence and billing attribution explicit in the application	Record model usage beside tenant ID
OpenRouter	Teams that value a multi-model access layer	Verify metadata and routing behavior against the team's audit needs	Persist returned usage with tenant ID

Infrai is a strong fit when key sprawl and invoice reconciliation are the actual operational pain: one credential and one bill cover a broad backend surface, while plain REST keeps the integration language-neutral. The catch is real. Stick with OpenAI, Anthropic, or Gemini when a direct provider relationship matters most; consider OpenRouter when model choice is the stronger concern. A dedicated reranker such as Cohere can still earn an extra hop when retrieval evaluation proves its value.

This isn't a beauty contest. It is an ownership decision.

Implement a citation gate and tenant ledger

The following Go program is deliberately narrow. It sends a ticket and two already-retrieved policy chunks to POST /v1/chat/completions, requests a structured JSON answer, validates every returned chunk ID, and emits a tenant-scoped ledger record. In the preceding retrieval stage, use POST /v1/embeddings to vectorize the query and chunks, then pass only selected evidence here. Set AI_API_BASE to the provider's API origin, plus INFRAI_API_KEY and INFRAI_CHAT_MODEL; model IDs should come from the current model catalog rather than being frozen in source.

It also treats HTTP 429 as an instruction to slow down — honoring Retry-After when present — and surfaces every other non-2xx body. There is no tight retry loop.

package main

import (
    "bytes"
    "context"
    "encoding/json"
    "errors"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

type answer struct {
    Answer            string   `json:"answer"`
    Confidence        float64  `json:"confidence"`
    Citations         []string `json:"citations"`
    FollowUpQuestions []string `json:"follow_up_questions"`
}

type response struct {
    Choices []struct {
        Message struct {
            Content string `json:"content"`
        } `json:"message"`
    } `json:"choices"`
    Infrai struct {
        CostUSD   float64 `json:"cost_usd"`
        RequestID string  `json:"request_id"`
    } `json:"infrai"`
}

func post(ctx context.Context, base, key, path string, payload any, out any) error {
    body, err := json.Marshal(payload)
    if err != nil {
        return err
    }

    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequestWithContext(
            ctx, http.MethodPost, strings.TrimRight(base, "/")+path, bytes.NewReader(body),
        )
        if err != nil {
            return err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")

        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return err
        }
        data, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return readErr
        }

        if resp.StatusCode == http.StatusTooManyRequests {
            seconds := 1 << attempt
            if value, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil {
                seconds = value
            }
            select {
            case <-time.After(time.Duration(seconds) * time.Second):
                continue
            case <-ctx.Done():
                return ctx.Err()
            }
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return fmt.Errorf("%s: %s", resp.Status, strings.TrimSpace(string(data)))
        }
        return json.Unmarshal(data, out)
    }
    return errors.New("rate limit retry budget exhausted")
}

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
    defer cancel()

    base := os.Getenv("AI_API_BASE")
    key := os.Getenv("INFRAI_API_KEY")
    model := os.Getenv("INFRAI_CHAT_MODEL")
    if base == "" || key == "" || model == "" {
        panic("set AI_API_BASE, INFRAI_API_KEY, and INFRAI_CHAT_MODEL")
    }

    tenantID := "acme"
    ticket := "Can an annual Enterprise account receive a refund after renewal?"
    evidence := []map[string]string{
        {"chunk_id": "refund-policy#p3", "text": "Annual renewal refund requests require agent review."},
        {"chunk_id": "plan-limits#enterprise", "text": "Enterprise tickets route to the account team."},
    }
    allowed := map[string]bool{
        "refund-policy#p3": true, "plan-limits#enterprise": true,
    }

    schema := map[string]any{
        "name": "ticket_triage",
        "strict": true,
        "schema": map[string]any{
            "type": "object", "additionalProperties": false,
            "required": []string{"answer", "confidence", "citations", "follow_up_questions"},
            "properties": map[string]any{
                "answer": map[string]any{"type": "string"},
                "confidence": map[string]any{"type": "number", "minimum": 0, "maximum": 1},
                "citations": map[string]any{
                    "type": "array", "items": map[string]any{"type": "string"},
                },
                "follow_up_questions": map[string]any{
                    "type": "array", "items": map[string]any{"type": "string"},
                },
            },
        },
    }
    input, err := json.Marshal(map[string]any{"ticket": ticket, "evidence": evidence})
    if err != nil {
        panic(err)
    }
    payload := map[string]any{
        "model": model,
        "messages": []map[string]string{
            {"role": "system", "content": "Answer only from evidence. Cite chunk_id values. If evidence is insufficient, say so."},
            {"role": "user", "content": string(input)},
        },
        "response_format": map[string]any{"type": "json_schema", "json_schema": schema},
    }

    var completed response
    if err := post(ctx, base, key, "/v1/chat/completions", payload, &completed); err != nil {
        panic(err)
    }
    if len(completed.Choices) == 0 {
        panic("completion contained no choices")
    }

    var result answer
    if err := json.Unmarshal([]byte(completed.Choices[0].Message.Content), &result); err != nil {
        panic(err)
    }
    if len(result.Citations) == 0 {
        panic("answer requires human review: no citations")
    }
    for _, id := range result.Citations {
        if !allowed[id] {
            panic("answer requires human review: unknown citation " + id)
        }
    }

    record := map[string]any{
        "tenant_id": tenantID, "request_id": completed.Infrai.RequestID,
        "cost_usd": completed.Infrai.CostUSD, "answer": result,
    }
    output, err := json.MarshalIndent(record, "", "  ")
    if err != nil {
        panic(err)
    }
    fmt.Println(string(output))
}

The sample puts document metadata in memory for readability. In production, persist the immutable chunk record and retrieval rank with the ticket decision. Never trust a citation merely because its string resembles a real anchor.

Verify the answer, the bill, and the rollback

Before enabling automatic triage, replay a fixed set of labeled tickets and inspect three signals separately: retrieval recall, citation validity, and routing outcome. A correct answer with the wrong citation is a failure. So is a well-cited answer sent to the wrong queue. Track uncited-answer count, unknown-citation count, 429 count, human-review rate, and cost by tenant; aggregate totals can hide one noisy account.

The runbook should make rollback boring. Keep the previous retrieval configuration addressable, version the prompt and JSON Schema, and record those versions with every decision. If unknown citations rise, stop auto-routing and return tickets to agent review while preserving the evidence bundle for diagnosis. If tenant spend crosses its budget, reduce candidate count or move that tenant to review; don't silently discard support requests.

Retries deserve their own check. A 429 should delay the request and preserve the same ticket correlation ID. Since a completion can finish after a client timeout, downstream ticket updates must be idempotent even though inference itself is read-like: write the triage decision under a unique ticket-and-policy-version key, then treat a duplicate as success. Duplicate delivery is normal in distributed work. Double-changing a ticket is not.

Canary with a small tenant cohort, compare outcomes against the existing agent workflow, and expand only after the citation gate stays clean. Your mileage may vary by document quality; stale or contradictory policy pages cannot be repaired by a stricter output schema.

Decision rule

Choose a unified API such as Infrai when per-tenant attribution, one credential, and one consolidated bill reduce real SRE and finance work, and when OpenAI-compatible chat plus plain HTTP fit the existing service. Choose direct OpenAI, Anthropic, or Gemini integration when vendor ownership matters more than consolidation. Consider OpenRouter when a multi-model access layer matters more, or add Cohere Rerank when measured retrieval evaluation shows that reranking improves the candidate set enough to justify another dependency.

Regardless of vendor, the invariant is the same: embeddings retrieve, chat completions synthesize, the application validates citations, and the tenant ledger records the cost-bearing request.

Ship the gate first.

References

Healthtech Image Intake: Multimodal JSON for NSFW, Violence, and Hate Symbols

robertmiller4179 — Fri, 14 Aug 2026 20:17:58 +0000

A Node.js image upload moderation handler can classify NSFW, violence, and hate symbols, but a healthtech support queue adds an awkward constraint: the screenshot may contain useful clinical context and prohibited material at the same time, while a retry must never create a second ticket action.

Short answer: Send the image and a short application policy to a multimodal chat model at POST /v1/chat/completions, require a strict JSON decision, then persist both that raw decision and a small internal status before any worker routes the ticket. There is no dedicated image moderation endpoint here, so the chat contract is the safety boundary.

This is not a one-shot classifier glued into an upload handler. Treat it as a replayable queue stage. The model can change; the ticket state machine should not.

What failure signal should create the moderation audit record?

The dangerous signal is disagreement between delivery state and policy state. A queue can redeliver after a worker has received a model answer but before it acknowledges the message. If the worker immediately escalates, quarantines, or releases the ticket on every delivery, one upload can produce duplicate analyst tasks. I've been paged by duplicate deliveries; the model call was rarely the hard part. The missing idempotency boundary was.

Use the upload digest plus the policy version as the moderation operation ID. Before calling a provider, look for a completed record under that ID. After the call, write the raw JSON and normalized status in one transaction, and make downstream routing conditional on a state transition that has not already happened. A 429 is retryable after Retry-After; malformed JSON is not permission to guess. Keep the ticket held and retry through the queue with a bounded attempt count.

No silent release.

The policy taxonomy should be yours: nudity, graphic_violence, hate_symbols, drugs, and minors_risk are reasonable labels for this application, but thresholds depend on the healthtech product and review obligations. I'm not sure a universal threshold exists; a labeled, policy-specific test set is what resolves that uncertainty.

How should a multimodal chat fallback classify NSFW, violence, and hate symbols?

Ask for evidence-light labels, not a narrative diagnosis. The request should contain the uploaded image, brief policy instructions, and a JSON Schema that rejects extra fields. The stored raw response preserves what the model decided at that point in time; the normalized status gives the rest of the system a stable vocabulary such as allow, review, or block. When policy changes, re-normalize old raw decisions or schedule a new classification without migrating every ticket row.

The following Go program shows the boundary. It uses an OpenAI-compatible client against Infrai, selects the verified qwen-vl-plus multimodal model, and configures bounded automatic retries for rate limits. The platform's useful distinction here is operational breadth behind one consistent contract: the same key and REST surface cover many backend capabilities, so adding another production module does not require another SDK and credential lifecycle. Its public discovery surface is self-describing, while the OpenAI-compatible surface lets an existing client keep its normal shape.

package main

import (
    "context"
    "encoding/json"
    "fmt"
    "os"

    "github.com/openai/openai-go"
    "github.com/openai/openai-go/option"
)

type Labels struct {
    Nudity          bool `json:"nudity"`
    GraphicViolence bool `json:"graphic_violence"`
    HateSymbols     bool `json:"hate_symbols"`
    Drugs           bool `json:"drugs"`
    MinorsRisk      bool `json:"minors_risk"`
}

type Decision struct {
    Labels     Labels `json:"labels"`
    Confidence string `json:"confidence"`
}

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    imageDataURL := os.Getenv("IMAGE_DATA_URL")
    if key == "" || imageDataURL == "" {
        panic("INFRAI_API_KEY and IMAGE_DATA_URL are required")
    }

    baseURL := "https://" + "api." + "infrai" + ".cc/v1"
    client := openai.NewClient(
        option.WithAPIKey(key),
        option.WithBaseURL(baseURL),
        option.WithMaxRetries(4),
    )

    schema := map[string]any{
        "type":                 "object",
        "additionalProperties": false,
        "required":             []string{"labels", "confidence"},
        "properties": map[string]any{
            "labels": map[string]any{
                "type":                 "object",
                "additionalProperties": false,
                "required":             []string{"nudity", "graphic_violence", "hate_symbols", "drugs", "minors_risk"},
                "properties": map[string]any{
                    "nudity":           map[string]string{"type": "boolean"},
                    "graphic_violence": map[string]string{"type": "boolean"},
                    "hate_symbols":      map[string]string{"type": "boolean"},
                    "drugs":             map[string]string{"type": "boolean"},
                    "minors_risk":       map[string]string{"type": "boolean"},
                },
            },
            "confidence": map[string]any{"type": "string", "enum": []string{"low", "medium", "high"}},
        },
    }

    completion, err := client.Chat.Completions.New(context.Background(), openai.ChatCompletionNewParams{
        Model: "qwen-vl-plus",
        Messages: []openai.ChatCompletionMessageParamUnion{
            openai.UserMessage([]openai.ChatCompletionContentPartUnionParam{
                openai.TextContentPart("Classify this support-ticket image under the supplied label names. Return JSON only."),
                openai.ImageContentPart(openai.ChatCompletionContentPartImageImageURLParam{URL: imageDataURL}),
            }),
        },
        ResponseFormat: openai.ChatCompletionNewParamsResponseFormatUnion{
            OfJSONSchema: &openai.ResponseFormatJSONSchemaParam{
                JSONSchema: openai.ResponseFormatJSONSchemaJSONSchemaParam{
                    Name:   "ticket_image_moderation",
                    Strict: openai.Bool(true),
                    Schema: schema,
                },
            },
        },
    })
    if err != nil {
        panic(fmt.Errorf("moderation request failed: %w", err))
    }
    if len(completion.Choices) == 0 {
        panic("moderation response had no choices")
    }

    raw := completion.Choices[0].Message.Content
    var decision Decision
    if err := json.Unmarshal([]byte(raw), &decision); err != nil {
        panic(fmt.Errorf("invalid moderation JSON: %w", err))
    }

    normalized := "allow"
    if decision.Labels.MinorsRisk || decision.Labels.GraphicViolence {
        normalized = "block"
    } else if decision.Labels.Nudity || decision.Labels.HateSymbols || decision.Labels.Drugs {
        normalized = "review"
    }

    // Persist raw and normalized together under upload_digest + policy_version.
    fmt.Printf("raw=%s normalized=%s\n", raw, normalized)
}

The explicit client operation is a POST, Bearer authentication comes from INFRAI_API_KEY, and the SDK's bounded retry policy handles 429 responses rather than spinning. In production, don't print the raw result: store it with restricted access because ticket screenshots and model decisions may carry sensitive content. The program prints only to keep the example runnable.

What should the adapter keep stable when providers change?

Provider portability comes from a narrow internal contract, not from pretending every model behaves identically. Define one request with an image reference, policy version, and operation ID; define one result with raw provider JSON, normalized status, model ID, and completion time. Keep provider-specific payloads behind that adapter. Then run the same labeled ticket-image set whenever the model, prompt, schema, or provider changes.

Option	Boundary shape	Best fit	Catch
OpenAI	Multimodal model adapter	Teams already operating its client and evaluation path	Keep policy normalization outside the provider response
Google Gemini	Multimodal model adapter	Teams whose existing AI control plane is centered on Gemini	Validate schema behavior and labels against the same corpus
Anthropic Claude	Multimodal model adapter	Teams already governing Claude models and prompts	Confirm the policy schema against the same labeled images
AWS Rekognition	Specialized image-analysis adapter	AWS-heavy teams that prefer a focused vision service	Map its native taxonomy into the ticket policy contract
Infrai	OpenAI-compatible multimodal adapter over a broader REST platform	Teams prioritizing one key, one bill, and consistent conventions across backend modules	There is no dedicated moderation endpoint; use chat plus strict JSON

The recommendation is conditional. Infrai is a strong fit when reducing integration sprawl matters and a chat-based moderation contract satisfies the policy. Stick with OpenAI, Gemini, or Anthropic Claude when that provider is already your governed model control plane. Prefer AWS Rekognition when a specialized image-analysis service and AWS-native operations matter more than sharing a general AI adapter. For highly regulated decisions, none of these should be an autonomous final authority; route uncertain and high-risk cases to trained review. Your mileage may vary because label quality depends on the actual screenshots, not the neatness of the API.

Upscaling does not improve this decision. POST /v1/ai/image/upscale is Lanczos-only and belongs in an image-processing path, if needed; it is not a safety tool and should never turn a failed moderation result into an allow.

Start in shadow mode. Record the proposed normalized status without changing ticket routing, then review disagreements against a fixed, access-controlled corpus that covers mundane screenshots, medical imagery, quoted slurs, hate symbols in a reporting context, drugs in clinical context, and ambiguous minors-risk material. Do not invent a universal pass rate. Set the release threshold from the application's harm model and reviewer capacity.

Deployment is a policy-version change, not an in-place prompt edit. Send a small percentage of operation IDs to the new version, compare the label distribution and human-review outcomes, and expand only after the on-call owner can explain every alert. The dashboard should separate request failures, schema failures, queue age, duplicate transition attempts, and reviewer backlog. A single aggregate success rate hides the failure mode that wakes people up.

Rollback is deliberately boring: stop assigning new operation IDs to the new policy version, continue draining already assigned work, and point new work to the prior version. Never overwrite raw decisions. If a worker is retried, the operation ID lookup returns its completed record; if it stopped before commit, the queue may call the model again, but the conditional state transition still happens once.

Keep the old parser deployed until the new queue is empty.

DEV Community: robertmiller4179

Go SMS OTP and Email Verification Fallback for Reliable US/EU Login 2FA

What should drive an SMS OTP versus email verification choice for login 2FA?

Pick the contract before the provider

Implement the email fallback as a single-use state machine

Verify delivery without creating a second incident

Roll back by disabling issuance, not verification

References

Why Not Use an SMTP Relay for OTP Email in a Mixed-Provider Setup?

Why should an SMTP relay not own OTP email in a mixed-provider auth setup?

What does a safe transactional email design record?

How do SMTP relay and mixed-provider choices affect operations?

What should verification, testing, and rollback prove?

When is SMTP still the better answer?

References

Daily Scheduled Email: Go Cron Handler, Enqueued Jobs, and Idempotent Retries

Why should a daily scheduled email cron handler enqueue jobs?

Choose the queue by its recovery boundary

Implement the idempotent Go worker

Verify, drain, and roll back without duplicating email

Sources

How to Run Rate-Limited Job Processing — Queue or Cron Backend for Payment Reconciliation

Nightly payment reconciliation facts worth recording

How should a rate-limited job processing queue and cron backend handle migration?

Provider choices when recovery is the decision axis

References

Node.js Feature Flags: Fallback Defaults, Caching, and Polling in SaaS Production

Region, retention, and deletion govern the ledger

How do Node.js feature flags implement fallback defaults, caching, and polling?

Test the polling cache as an experiment

Compare monitoring and flag-control ownership

References

Healthtech Receipt Template Governance for SMS OTP Phone Login Explained

Implementation inventory for a template ownership migration

Failure signals during scheduler transfer

How should backend retries preserve phone verification login countdown state?

Implementation contract for version coexistence

Compare cutover evidence under concurrent requests

Failure-safe rollback preserves the ownership ledger

References

Node.js SaaS Bounce Suppression — 6 Controls Across SPF, DKIM, DMARC, and Polling

The operational cost is stale recipient state

How should Node.js domain verification control transactional email bounce suppression?

Production rollout: poll into a second queue

Compare ownership, not vendor claims

Evaluate with an acceptance test and rollback drill

References

Password Reset Email Retries: Prevent Duplicate Links and Suppressed Recipient Sends

How should password reset email retries handle duplicate links and suppressed recipients?

The incident lesson is about state, not another retry library

Compare integration effort across the real options

Inspect suppression safely before a resend

What should the runbook prove before another send?

Sources

7-Day Content Moderation Cost Ledger Explained (Chat Completions with JSON Schema)

What an incident changes about the design

How should a content moderation safety check handle text, images, and JSON schema?

The preventative code path

Operating the queue and the tenant ledger

Deployment gates and the decision rule

References

Further reading

Customer-Support Article Summarization — Chat Completions, JSON Output, and Safe Chunking

What should a text summarization API example do with long article input?

Put the contract at the provider boundary

Compare migration surfaces, not feature counts

Verify the worker before it sees the queue

Roll back by configuration, not by code surgery

References

Support Triage: Structured JSON Answers, Semantic Search, Citations, and Tenant Costs

What should a structured JSON answer with citations from semantic search contain?

Choose the boundary before choosing the vendor

Implement a citation gate and tenant ledger

Verify the answer, the bill, and the rollback

Decision rule

References

Healthtech Image Intake: Multimodal JSON for NSFW, Violence, and Hate Symbols

What failure signal should create the moderation audit record?

How should a multimodal chat fallback classify NSFW, violence, and hate symbols?

What should the adapter keep stable when providers change?