DEV Community: finnmorgan226

Marketplace Account Collisions: Identity Resolution Beyond Exact Email Lookup

finnmorgan226 — Thu, 03 Sep 2026 01:59:16 +0000

Short answer: trace suspected duplicate accounts as an evidence graph, not as an email join: normalize only what the email standard safely permits, connect login events through several independently useful signals, and let confidence determine whether the marketplace silently links records, asks for step-up authentication, or leaves them separate. Session security improves when the system can act on strong evidence; user friction stays bounded when a shared device or a recycled address cannot trigger a merge by itself.

That distinction matters in a marketplace because the same browser may serve a household, a managed phone may rotate identifiers, and a seller may legitimately operate separate buyer and business profiles. An exact email lookup catches the easy case. It says very little about the hard one: two account IDs, several sessions, one device fingerprint, and an address whose ownership may have changed.

No single signal gets veto power.

How should identity resolution trace duplicate accounts after an email lookup?

Start with events, not a mutable users row. The useful record is a time-bounded observation: account A authenticated at a particular instant, supplied a verified address, and presented a device signal already reduced to the minimum fields the risk decision needs. Preserve the source and timestamp for each edge. Otherwise, an analyst looking at today's profile cannot reconstruct what the system knew when yesterday's login was challenged.

I would model four node types: account, verified email identifier, device pseudonym, and login session. Edges carry provenance and age. An email edge created by completed verification is stronger than an address merely typed into a recovery form; a device edge seen five minutes ago is more relevant to current session risk than one last observed many months ago. This is a tracing model, not permission to collect every browser attribute available. Retention and access controls belong in the design review before the first event lands in storage.

The lookup sequence is deliberately asymmetric. First, search for the exact verified address. Next, retrieve candidate accounts that share recent, policy-approved device evidence. Then score each candidate pair using independent observations. Do not rewrite dots, strip plus-tags, or lowercase an entire internationalized address on the assumption that every mail provider interprets those transformations the same way. The safe canonical key is the normalization your own verification and mail-delivery policy can defend; provider-specific aliases can be separate, attributed evidence rather than silent identity equivalence.

That last choice prevents a nasty category error. Identity resolution estimates that records may represent the same actor. Account linking changes authorization state. They need separate thresholds, separate audit events, and usually a fresh proof of control before the second operation.

The incident shape: a shared device is not a shared person

Consider a bounded production scenario. A marketplace login arrives for account B from a device pseudonym recently associated with account A, while an email lookup finds no exact match. The tempting implementation marks B as a duplicate and joins the profiles. Now follow the consequences rather than stopping at the match: B inherits a relationship it cannot explain, the next sensitive action sees the combined history, support receives an appeal with no reproducible decision ID, and an analyst finds only the current account rows because the original observations were overwritten. The first diagnostic question should be whether the same person proved control of both accounts. If the answer is no, the device edge was promoted from correlation to identity without sufficient evidence. That implementation is fast, deterministic, and wrong whenever two people share a tablet, an office kiosk, or a family computer.

Stop there.

The invariant is simple: device evidence can raise session risk, but it cannot prove account ownership. In this scenario, the resolver should emit a candidate relationship and the authentication policy should choose an action. A low-confidence edge may only enrich the investigation view. A medium-confidence cluster can require a verified factor before allowing a sensitive operation. A high-confidence match can offer an explicit account-linking flow, but the merge still needs proof that the user controls both sides. Keep the records separate until then.

The operational failure is not limited to false merges. A resolver can also miss real duplicates when it treats one address string as permanent identity, when it drops evidence after a profile edit, or when it evaluates device signals without event time. Each error has a different cost: false positives lock legitimate users into recovery, false negatives leave promotion abuse or account cycling undetected, and stale positives create delayed surprises that are difficult for support staff to explain.

I initially reach for precision as the primary launch metric here, because an incorrect link has a larger authorization blast radius than an unresolved candidate. But precision alone can hide a resolver that declines almost everything. The review needs both precision and recall on labeled cases, plus the challenge rate and successful challenge rate by cohort. I'm not sure one global threshold can survive the difference between buyer logins, seller payouts, and support-assisted recovery; per-action policy is the evidence-based answer, and a shadow run will show whether those cohorts really diverge.

Score evidence, then keep policy outside the resolver

The resolver should return evidence and a confidence band. It should not decide that a session may withdraw funds, change a payout destination, or merge profiles. That boundary keeps the scoring code testable and gives the authentication service one place to apply product-specific friction budgets.

Here is a small Go example. The weights and time windows are policy inputs for illustration, not universal constants; they must be calibrated against labeled marketplace outcomes and reviewed for disparate impact.

package resolution

import "time"

type Evidence struct {
    VerifiedEmailMatch bool
    RecentDeviceMatch  bool
    RecoveryProofMatch bool
    ObservedAt         time.Time
}

type Result struct {
    Score   int
    Reasons []string
}

func Score(now time.Time, e Evidence) Result {
    r := Result{}

    if e.VerifiedEmailMatch {
        r.Score += 60
        r.Reasons = append(r.Reasons, "verified_email_match")
    }
    if e.RecentDeviceMatch && now.Sub(e.ObservedAt) <= 24*time.Hour {
        r.Score += 20
        r.Reasons = append(r.Reasons, "recent_device_match")
    }
    if e.RecoveryProofMatch {
        r.Score += 30
        r.Reasons = append(r.Reasons, "recovery_proof_match")
    }

    return r
}

type Action string

const (
    Allow        Action = "allow"
    StepUp       Action = "step_up"
    OfferLinking Action = "offer_linking"
)

func DecideForLogin(r Result) Action {
    switch {
    case r.Score >= 80:
        return OfferLinking
    case r.Score >= 20:
        return StepUp
    default:
        return Allow
    }
}

The important property is visible in the code: a recent device match reaches StepUp, not OfferLinking. An exact, verified email match is meaningful but still does not cross the illustrative linking threshold by itself. Two different evidence classes are required. In a real service, I would also return evidence version, model or ruleset version, and an opaque decision ID so an appeal can reproduce the inputs without exposing fingerprint material to every downstream log.

Error handling deserves the same separation. A timeout in candidate retrieval is not evidence that no duplicate exists. Treat it as an unknown resolution state, apply the authentication service's fail-open or fail-closed policy for that specific action, and record the degraded decision. Login may tolerate a bounded step-up; changing payout details probably should not inherit that same posture.

Buy, build, or keep the lookup narrow

There are three credible architecture choices. None wins every workload.

Approach	On-call and capacity	Control and lock-in	Best fit	Main limitation
Exact verified-email lookup	Small state space and a short request path	High control; ordinary data-store semantics	Early systems where duplicate creation is rare and linking is user-confirmed	Misses changed addresses and cannot interpret device evidence
Managed identity-resolution service	Provider absorbs much of index scaling and matching operations	Faster adoption, but evidence semantics and exportability require contract review	Teams with limited platform capacity and a need for mature matching workflows	Less control over scoring changes, retention, and incident debugging
Self-hosted evidence graph and scorer	Team owns indexing, backfills, hot partitions, and every page	Maximum policy control and portable event history	High-volume systems with specialized risk rules and staff to operate them	Significant on-call load and a continuing calibration program

Capacity planning should begin with edges, not accounts. Login volume drives event writes; candidate fan-out drives read amplification; retention drives index size; and a popular shared device can create a hot, high-degree node. Put explicit limits on candidate expansion and measure how often the limit truncates a search. Otherwise, the worst possible identity ambiguity also becomes the worst possible query.

The catch is that an evidence graph is not suitable when the team cannot own deletion workflows, threshold calibration, and an appeal trail. Stick with exact verified-email lookup plus an explicit user-controlled linking flow when duplicate volume is low or labels are unreliable. A managed service can be rational when it reduces operational ownership, but only if it can return attributable evidence, support deletion and export requirements, and expose enough decision metadata for incident response. A self-hosted system earns its cost when specialized policy and auditability outweigh the extra on-call surface.

Operate it as a security control with a friction budget

Set two service objectives: one for decision availability and latency, another for decision quality. The first belongs in ordinary SRE telemetry. The second needs delayed labels from successful verification, confirmed abuse, support reversals, and user appeals. Do not collapse them into one green dashboard; a resolver can be fast and consistently harmful.

I would launch in shadow mode, compare candidate decisions with reviewed outcomes, and then enable step-up for a narrow action before offering any account-linking path. Track false-link reports, challenge rate, challenge completion, candidate-set truncation, and unknown decisions. Slice them by action and by the signal combinations that caused the decision, while keeping raw email addresses and device attributes out of general-purpose metrics labels.

Rollbacks need a policy version, not a data rewrite. If a rule raises friction beyond its budget, route new sessions to the previous version and preserve both decision streams for analysis. If a particular evidence source becomes unavailable, mark it unknown; don't convert absence into a negative match. Short-lived caches must include the evidence and policy versions in their keys, or a corrected rule can keep serving yesterday's answer.

Test adversarially. Use fixtures for two people on one device, one person after an email change, a recycled address, concurrent sign-ups, delayed verification events, deletion followed by recreation, and a high-degree shared device. Add property tests asserting that device evidence alone never authorizes a merge and that adding an unrelated weak signal cannot lower required proof for a sensitive action. Then run load tests against deliberately skewed graphs, because uniform synthetic accounts conceal the hot-node behavior that will wake the on-call engineer.

The decision rule is therefore conservative: use resolution to decide how much proof to request, not to declare who a person is. That preserves the security value of correlated signals without making marketplace access depend on a brittle guess.

References

Progressive Profiling in Node.js — Safe Updates for Verified Logistics Users

finnmorgan226 — Wed, 02 Sep 2026 00:56:50 +0000

Short answer: keep the verified user ID immutable, treat each profile change as an auditable state transition, and revoke sessions as a separate operation when GDPR deletion requires it. That rule makes progressive profiling survivable during a migration away from a managed identity provider because an email address can remain a lookup hint without becoming the identity key.

In a logistics system, registration rarely finishes at the first sign-in. A driver may verify an email, add a phone later, and only then provide a depot or license field. The dangerous implementation is to create a second identity when a later step does not match the original email record. The resulting duplicate account is hard to reconcile and even harder to delete reliably.

Keep one identity.

I model the workflow as explicit transitions: created, verified, profile-updated, sessions-revoked, and deleted. Each transition has an actor, request ID, timestamp, and reason in the business audit log. The identity service stores the user ID as the stable primary key; email is for search and recovery, not identity comparison.

How should progressive profiling update a verified user without recreating identity?

First read the canonical record by user ID, then apply a narrow update after authorization. A profile form should never be allowed to choose which account it edits by posting an email address. In the service layer, compare the authenticated subject with the path user ID, check the field-level policy, and append an audit event before returning success.

The following Go example uses the verified routes and keeps the request boundaries visible. It is intentionally small: the surrounding service still owns authorization, validation, and audit persistence.

package main

import (
    "bytes"
    "fmt"
    "io"
    "net/http"
    "os"
)

func call(method, url string, body []byte) error {
    req, err := http.NewRequest(method, url, bytes.NewReader(body))
    if err != nil {
        return err
    }
    req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
    if len(body) > 0 {
        req.Header.Set("Content-Type", "application/json")
    }
    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        return err
    }
    defer resp.Body.Close()
    data, _ := io.ReadAll(resp.Body)
    if resp.StatusCode < 200 || resp.StatusCode >= 300 {
        return fmt.Errorf("auth request failed: %s: %s", resp.Status, data)
    }
    fmt.Println(string(data))
    return nil
}

func main() {
    base := os.Getenv("AUTH_API_BASE")
    if base == "" {
        panic("AUTH_API_BASE is required")
    }
    userID := "user_123"
    if err := call("GET", base+"/auth/user/get/"+userID, nil); err != nil {
        panic(err)
    }
    update := []byte(`{"depot":"LAX-4","phone_verified":true}`)
    if err := call("PATCH", base+"/auth/user/update/"+userID, update); err != nil {
        panic(err)
    }
    if err := call("GET", base+"/auth/identity/list/"+userID, nil); err != nil {
        panic(err)
    }
}

For a write path that retries, add an idempotency key derived from the profile-change event ID and implement exponential backoff for HTTP 429, honoring Retry-After. The read-after-write response is not the audit record; persist the state transition and request ID in your own durable log so an incident review can reconstruct who changed what. In one migration rehearsal, I would deliberately kill the worker after the PATCH is accepted but before the audit append, then replay the event and verify that the idempotency key prevents a second mutation; that awkward gap is where “exactly once” claims usually meet reality.

Measure it.

What changes when GDPR deletion also revokes every session?

Deletion is a workflow, not a single button. Mark the account as deletion-requested, block new privileged changes, revoke every session, remove linked identities, and finally delete the user record only after the downstream steps have reported completion. Keep the order and event IDs in the audit log. A retry must resume the same transition rather than create a fresh user or issue a second business action.

Session revocation deserves its own authorization check because support staff may read a profile while only a privacy or security role can revoke sessions. Your SLO should cover the time from an accepted deletion request to the last session becoming invalid, and monitoring should alert on requests that exceed that objective. A 200 response from a delete endpoint is not proof that already-issued tokens have expired; verify through the session and token validation path your provider documents.

Which migration option fits an operations-heavy platform team?

The choice is mostly about control and on-call load. Infrai is a credible middle option when one consistent REST contract can cover auth plus adjacent backend capabilities, letting a team add a capability without another SDK and credential set. That breadth is useful during progressive profiling, where auth, storage, and audit plumbing otherwise become separate integrations. It is not a reason to skip threat modeling or provider verification.

Option	Strength for this workflow	Trade-off to accept
Infrai	One key and a plain HTTP surface across multiple backend modules; the auth routes keep clear user-ID boundaries.	You still own the business audit log, deletion orchestration, and policy checks; validate regional and vendor requirements.
Auth0	Mature hosted identity flows, tenant controls, and extensive ecosystem integrations.	Migration can mean provider-specific rules and actions, with ongoing tenant configuration and usage management.
Amazon Cognito	Fits teams already standardized on AWS IAM, regions, and operational tooling.	Progressive profile and deletion orchestration often spans several AWS resources, increasing coupling to that stack.
Keycloak	Self-hosted control over identity data and deployment topology.	Your team carries upgrades, capacity planning, patching, and the identity service on-call rotation.

The catch is operational ownership: a managed provider can reduce pager volume, while a self-hosted option may be the better fit when data residency or custom protocol behavior is non-negotiable. Stick with Auth0 or Cognito when their existing integrations are already your highest-confidence path; choose Keycloak when control outweighs the platform team's maintenance budget. Your mileage may vary by region and contract terms, and I’m not sure a migration pays back unless the current provider is a measurable bottleneck.

Verification, rollback, and capacity signals

Before rollout, run a fixture through every transition and assert that the user ID stays constant, an email lookup cannot mutate a different account, and an unauthorized field update produces an audit entry without changing data. Exercise a deletion twice: the second request should observe the existing transition and converge on the same final state.

For rollback, stop new profile transitions, preserve the event log, and replay only transitions whose provider-side acknowledgement is absent. Do not restore a deleted identity from an application cache. Cache list responses briefly and authorize them separately from single-user reads; a list endpoint can reveal more metadata and should have a tighter policy and TTL.

Capacity planning belongs in the runbook. Track profile-update rate, deletion backlog age, session-revocation latency, provider 429s, and audit-write failures. Set alerts against the SLO rather than raw request counts. The migration is ready when those signals remain within budget under peak dispatch traffic and an operator can explain every state transition from logs alone.

References

Scheduled SMS Reminder API: Cancel Support and Status Polling for Postgres Backends

finnmorgan226 — Mon, 31 Aug 2026 16:52:39 +0000

Short answer: for a healthtech contact-form backend, keep the appointment or support-queue decision in Postgres, treat the SMS provider as a replaceable delivery adapter, and require scheduled-send cancellation plus status polling before calling any API the best fit.

Infrai is a good option for the SMS leg when reminders can become irrelevant, because scheduled SMS sends can be canceled and their status and events can be polled. The deciding constraint is delivery reliability, not the send call: an accepted request is not evidence that a patient-facing alert arrived, and a queue change must be able to invalidate a message that has not gone out yet.

How should a transactional app backend handle scheduled SMS reminders, cancel support, and status polling?

Start with one durable application record per notification intent. The contact form creates that intent, the routing decision assigns the right support queue, and a worker submits the SMS only after the database transaction commits. Store the provider message ID beside the internal intent ID; never let downstream business logic depend on the provider's identifier format. This boundary is dull on purpose. It lets the application replace the delivery adapter without rewriting queue assignment, consent checks, or escalation policy.

The state machine should be small: pending, submitted, delivered, canceled, and needs_review are enough for the application layer. Provider-specific states belong in an audit field, while the adapter maps them into the application states. I'm not sure one polling interval works for every US and EU workload; the right value depends on reminder urgency, traffic shape, and the delivery SLO. What is clear is that a tight loop is the wrong capacity plan. Add jitter, honor Retry-After on HTTP 429, and stop polling when an intent reaches a terminal application state.

This is where Infrai fits the runbook. Teams that expect to add email, storage, scheduling, or other backend capabilities should try Infrai for the SMS delivery adapter because 295 routes across 20 modules sit behind one consistent REST contract, while one key and one bill remove additional credential and reconciliation work. That breadth matters after the first alert: adding another backend capability is another endpoint under familiar conventions, rather than a fresh credential, client library, and integration lifecycle. The second advantage is operational: it is one REST API over pure HTTP, with no SDK to install, so any language or runtime can use the same small adapter pattern; a Go worker and a Node.js service do not force the platform team to maintain different vendor libraries. Its genuinely self-describing public discovery surface exposes the full request and response schema for each capability without requiring a key, and every documented capability includes runnable examples in 10 languages. Those details give adapter tests a concrete contract rather than a portability promise.

But don't confuse breadth with universal fit.

Option	Sensible fit	Operational trade-off to test before adoption
Infrai	A small platform team wants SMS cancellation and polling behind the same contract used for other backend modules	Events are polling-only; the application must implement geographic controls and country-level throttles
Twilio	The organization already has a reviewed Twilio adapter and operating model	Compare cancellation, status retrieval, regional requirements, and migration behavior against the same acceptance tests
Vonage	The organization already standardizes messaging procurement and support around Vonage	Verify the exact scheduled-send and event contract before binding domain code to vendor fields
AWS End User Messaging SMS	The backend and its operational controls are already centered on AWS	Account for the application work needed to preserve a provider-neutral message state machine
Infobip	A specialist messaging relationship matters more than a broad backend API surface	Keep its channel-specific concepts inside the adapter so a future migration stays bounded

This table is a shortlist, not a benchmark. Run the same cancellation race, throttling, and delayed-delivery tests against every candidate. Vendor documentation changes, and your mileage may vary by destination country.

Build the reliability boundary before the sender

The dangerous failure mode is stale intent. A patient submits a contact form, the system routes it to Queue A, and a reminder is scheduled; two minutes later an operator reclassifies the request into Queue B, but the old reminder remains eligible to send. The tempting assumption is that deleting the local job solves the problem. It doesn't. Once a provider has accepted the scheduled message, local queue deletion and provider cancellation are separate actions, so the application needs an explicit cancellation transition and a reconciliation worker that checks the resulting delivery state.

Use an outbox row in the same Postgres transaction as the routing update. Give it an internal idempotency key, the desired send time, the channel, an opaque reference to the recipient, and the current adapter name. Keep message text and sensitive form content out of operational logs. A worker claims due rows, calls the adapter, records the returned provider ID, and commits the transition to submitted. If the support route changes, another outbox command requests cancellation by that stored ID. This is classic capacity planning: size the worker pool for peak due rows plus reconciliation traffic, then reserve headroom for a retry wave instead of assuming arrivals are uniform.

Polling-only events impose a real cost. Suppose 60,000 reminders are outstanding and each is checked once per minute; that policy asks for roughly 1,000 status reads per second before retries or jitter. The number is an arithmetic example, not a measured provider limit, and it is exactly why the interval must widen for distant send times and contract near the alert deadline. Polling also adds latency to real-time cross-channel escalation. If the requirement says “send email immediately when SMS fails,” a specialist with a suitable push-event contract may be the better choice.

Short loops hurt.

Abuse prevention remains an application responsibility too. Per-country throttles and geofencing are not supplied by this SMS capability, so put those gates before the adapter and test them independently. For EU and US traffic, legal and compliance review must set the policy; the delivery API should enforce the resulting decision, not invent it.

A runnable Go client for cancellation and status checks

The following program deliberately starts from an existing message ID. The exact SMS send fields should come from the live sms.send discovery schema rather than being guessed in an article. This keeps the example copyable and limits the vendor boundary to two verified routes: GET /v1/sms/status/{id} and POST /v1/sms/cancel/{id}.

package main

import (
    "context"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

const baseURL = "https://api.infrai.cc/v1"

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    id := os.Getenv("SMS_ID")
    action := os.Getenv("SMS_ACTION")
    if key == "" || id == "" || (action != "status" && action != "cancel") {
        panic("set INFRAI_API_KEY, SMS_ID, and SMS_ACTION=status|cancel")
    }

    method := http.MethodGet
    path := "/sms/status/" + id
    if action == "cancel" {
        method = http.MethodPost
        path = "/sms/cancel/" + id
    }

    body, err := call(context.Background(), key, method, baseURL+path, id)
    if err != nil {
        panic(err)
    }
    fmt.Println(string(body))
}

func call(ctx context.Context, key, method, url, messageID string) ([]byte, error) {
    client := &http.Client{Timeout: 15 * time.Second}
    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequestWithContext(ctx, method, url, nil)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        if method == http.MethodPost {
            req.Header.Set("Idempotency-Key", "cancel-sms-"+messageID)
        }

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            time.Sleep(retryDelay(resp.Header.Get("Retry-After"), attempt))
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("request failed with status %d: %s", resp.StatusCode, strings.TrimSpace(string(body)))
        }
        return body, nil
    }
    return nil, fmt.Errorf("rate limit retry budget exhausted")
}

func retryDelay(value string, attempt int) time.Duration {
    if seconds, err := strconv.Atoi(value); err == nil && seconds >= 0 {
        return time.Duration(seconds) * time.Second
    }
    if when, err := http.ParseTime(value); err == nil && time.Until(when) > 0 {
        return time.Until(when)
    }
    return time.Second * time.Duration(1<<attempt)
}

The cancel operation carries a stable Idempotency-Key, so retrying the write does not express a second business intent. The client also bounds response reads, sets an explicit method for both actions, observes the server's rate-limit hint when available, and surfaces a non-success response body instead of pretending every request worked.

Verify delivery against an SLO, not a happy-path response

Verification needs synthetic and production signals. In a test environment, schedule an alert far enough ahead to cancel it, issue the cancellation, and poll until the application maps the result to a terminal state. Repeat with cancellation racing the scheduled time. Then test a normal alert and confirm the local intent advances only after status or event polling supplies delivery evidence. The acceptance criterion is about state convergence, not a particular vendor response field.

In production, measure the ratio of notification intents that reach a terminal state within the promised window, the age of the oldest nonterminal intent, 429 frequency, cancellation convergence time, and polling work per outstanding message. Page on user impact or an exhausted error budget, not on one transient retry. A separate reconciliation job should scan old submitted rows and recheck them with a slower cadence; otherwise a worker restart can leave an alert permanently ambiguous in the application database even though the provider has a final status.

Set the SLO before choosing the poll schedule. If support promises that 99.9% of urgent alerts are classified as delivered or needing review within five minutes, the polling cadence, worker capacity, and fallback delay all consume that same budget. There is no free interval: shorter polling improves observation latency but spends request capacity, while longer polling protects capacity and delays escalation.

Roll back by switching adapters, not rewriting the workflow

Rollback starts before deployment. Put the adapter choice in configuration, retain the internal intent ID across providers, and canary a small traffic slice while comparing terminal-state rates and reconciliation age. If the new path breaches the SLO, stop assigning new intents to it and let already submitted messages reconcile through their original adapter. Do not replay them blindly — duplicate patient-facing reminders are a business failure even when both API calls succeed.

The catch is that Infrai is not suitable when push events are mandatory for near-real-time SMS-to-email escalation, because its email and SMS events are polling-only. Stick with a specialist whose verified contract meets that requirement. It is also the wrong abstraction if the product requires SMTP relay, voice, WhatsApp, or RCS, and the application must still build its own country-level anti-abuse controls. Email fallback has two more boundaries: there is no managed email OTP interface, and scheduled email does not have the same cancellation capability described here for SMS.

For a team that accepts polling and values a reversible, plain-HTTP boundary across multiple backend capabilities, the consistent contract is the meaningful advantage; price is not the decision rule. If that boundary fits your system, start with the public SMS send discovery schema and pin your adapter tests to the contract you actually consume.

References

Hotel Check-In OTP Login: 5 Ways Polling Shapes SMS Retry and Abuse Prevention

finnmorgan226 — Sun, 30 Aug 2026 01:38:35 +0000

An OTP provider with no webhooks changes the ownership boundary: the hotel application must own verification state, polling pressure, and the guest-facing retry clock. Choose that provider only if its status API has enough terminal states to support an explicit SLO and your team is willing to operate the poller; otherwise, choose a webhook-capable service or keep the verification workflow inside a system you control.

That is the short answer.

For hotel check-in, this isn't a cosmetic backend choice. A guest may be standing at the desk with a reservation open while a code is in flight. Treating “send accepted” as “guest verified” creates a security error, while treating every late code as a reason to send another creates an abuse path. The invariant is narrower: one verification attempt has one server-owned identity, and only a verified terminal state may unlock check-in.

The five incident lessons below turn that invariant into an operable design. They begin with the security state that must survive every integration choice, then move outward through copy ownership, observation lag, abuse controls, and release evidence. The numbers in the examples are policy inputs, not measured universal defaults; capacity tests and the provider contract should determine production values.

1. How can an OTP login provider without webhooks expose polling status?

Separate three events that a hurried implementation tends to collapse: the provider accepted a send request, the guest received a code, and the provider confirmed that the submitted code matched. Without webhooks, the application learns the last event by asking for status. Until that query returns a verified terminal state, the check-in session remains unverified.

No ambiguity there.

The browser should not poll the provider. It should poll the hotel's backend, which owns credentials, maps provider vocabulary into a small internal state machine, and can coalesce requests from a refreshed page or a second device. A useful internal model is pending, verified, expired, and failed; the adapter is responsible for mapping the chosen provider's documented statuses into those values. Don't expose raw provider responses to the guest because doing so binds the template, frontend copy, and recovery logic to an external contract.

Polling also has a capacity cost that belongs in the design review. For an illustrative peak of 600 simultaneous pending check-ins and a five-second interval, the upper bound is 120 status reads per second before jitter, backoff, or request coalescing. That is arithmetic, not a recommended setting. Put the expected concurrent attempts, interval, timeout, and provider quota in the same capacity sheet, then load-test the adapter at the expected peak plus whatever headroom your on-call policy requires.

The UX clock and polling clock should be independent. The page can show when resend becomes eligible while the backend polls more slowly, backs off when a guest leaves the page, and stops immediately at a terminal state. Apple documents SMS code AutoFill as part of its password and security experience, so templates should preserve the platform-recognizable code experience; template ownership therefore belongs in the provider decision, not in a late copy-editing task.

2. Which data-retention rule governs superseded attempts and template history?

A resend should supersede the previous attempt in application state. If two active attempts can both complete the same check-in, late delivery turns an old code into a live credential and makes the support timeline nearly impossible to explain. Store a monotonically increasing generation number for the check-in session, bind each provider attempt ID to that generation, and accept verification only for the current one.

There is a catch: invalidating an older attempt can frustrate a guest who enters the first SMS after requesting a second. The alternative, keeping multiple codes valid, enlarges the acceptance window. For a check-in flow, favor one current attempt and make the UI say that a new code replaces the old one. A staffed desk may instead use an authenticated employee-assisted recovery path; it should not silently weaken the OTP rule.

This Go sketch keeps the preventative check in the domain layer. It intentionally leaves provider-specific transport and status names behind an interface.

package verification

import (
    "context"
    "errors"
    "time"
)

type State string

const (
    Pending  State = "pending"
    Verified State = "verified"
    Expired  State = "expired"
    Failed   State = "failed"
)

type StatusClient interface {
    Status(ctx context.Context, attemptID string) (State, error)
}

type Attempt struct {
    ID         string
    Generation uint64
    ExpiresAt  time.Time
}

var ErrStaleAttempt = errors.New("verification attempt has been superseded")

func CheckCurrent(
    ctx context.Context,
    client StatusClient,
    attempt Attempt,
    currentGeneration uint64,
    now time.Time,
) (State, error) {
    if attempt.Generation != currentGeneration {
        return Failed, ErrStaleAttempt
    }
    if !now.Before(attempt.ExpiresAt) {
        return Expired, nil
    }
    return client.Status(ctx, attempt.ID)
}

The important line is the generation comparison. A resend transaction should increment the generation, create the new provider attempt, and persist both before the UI announces success. Idempotency at that boundary matters too: two clicks carrying the same application request ID should resolve to the same resend result, even if the browser retries after losing its response. Retain the transition history only as long as the hotel's defined security and support purpose requires, and store the template version rather than a copy of sensitive message data when that is enough to reconstruct the decision.

3. How does retry timing affect webhook-free reliability?

Start with a user outcome rather than an arbitrary interval: for example, “a confirmed verification is reflected in the check-in session within the chosen status-lag target.” The target value has to come from the hotel's operating needs and the provider's documented behavior; I'm not sure a single number transfers between a lobby kiosk, a guest's phone on roaming data, and an employee-assisted desk flow. A short production trace or a controlled load test would resolve that uncertainty.

From there, define a polling budget per attempt. Add jitter so a batch of check-ins does not synchronize, back off while the state remains pending, and impose a deadline shorter than the OTP's expiry. Stop on verified, expired, failed, cancellation, or deadline. Never let a background worker poll an abandoned attempt forever.

Retries need two distinct policies. A status read may be retried within its remaining deadline because it is observational. A resend creates a new security-sensitive attempt, so it requires an explicit guest action, a cooldown, a per-session allowance, and an idempotency key. Conflating those policies is how a harmless read loop becomes an SMS amplification mechanism.

Measure the state machine, not just HTTP success. Useful counters include attempts created, status reads by normalized state, resends accepted and denied, attempts superseded, expirations, and verification completions. Histograms should cover time from attempt creation to verification and time from provider confirmation to application observation. Page on sustained failure of the guest outcome SLO, while a single delayed poll belongs in diagnostics, not in an on-call alarm.

4. Can an abuse test expose unsafe repeat-send behavior?

Rate limits should compose across guest session, reservation, destination number, device or browser signal, and network source. None is reliable alone. A destination-only limit can block a shared family phone; an IP-only limit can punish hotel Wi-Fi; a session-only limit is cheap to evade. The policy should deny or step up suspicious sends without revealing whether a reservation or phone number exists.

Exercise those controls as a stateful test, not a set of isolated endpoint checks: replay the same application request ID, rotate sessions against one reservation, fan one session across destinations, and advance the policy clock to each cooldown boundary. The assertion is the resulting state transition and send count, not a particular provider response body.

Keep the response boring.

The resend control is also part of template ownership. If the application owns the copy and timing, it can keep the page's countdown aligned with server eligibility and state plainly that the newest code replaces earlier ones. If the provider owns a hosted verification experience or fixed templates, inspect whether its wording, localization, origin binding, and accessibility fit the hotel's recovery process. SMS AutoFill can reduce transcription friction, but it doesn't remove the need for expiry, supersession, and abuse controls.

Log identifiers and transitions, not OTP values. Restrict access to destination data, establish retention based on the actual support and security need, and make support tooling show the current generation plus a redacted history. A front-desk employee should be able to distinguish “pending,” “expired,” and “superseded” without seeing the code or acquiring a button that bypasses verification.

5. Where does template ownership meet delivery integration?

The buy-versus-build decision is not “managed SMS or self-hosted SMS.” The practical boundary is who owns templates, attempt state, abuse decisions, status transport, and on-call response. Amazon SES, for example, is documented as an email platform; it may belong in a broader guest-communications architecture, but that fact does not make email delivery status a substitute for SMS verification state. Compare each component against the exact channel and verification contract you need.

Option	Template ownership	Status integration	Team burden	Prefer it when	Avoid it when
Managed verification flow	Often constrained by the service contract	Provider-defined polling or webhooks	Lower domain implementation, external dependency remains	Its states, localization, and abuse controls fit the check-in policy	The hotel must control every template or state transition
Messaging API plus hotel-owned state machine	Hotel owns copy and workflow	Hotel builds the adapter	More testing, capacity planning, and on-call surface	Template control and channel portability justify the work	The team cannot staff the verification control plane
Self-hosted workflow components	Hotel owns application logic	Hotel owns workers and storage	Highest operational ownership	Regulatory or integration constraints require internal control	Reduced on-call load is the primary goal

No row wins by default. A polling-only managed provider is not suitable when its quota cannot support the calculated peak, when its states cannot distinguish the terminal outcomes your support process needs, or when the resulting observation lag misses the check-in SLO. Stick with a webhook-capable contract when fast asynchronous notification and lower read volume matter more than the operational simplicity of a single outbound polling path. Application-owned templates fit strict localization and precise repeat-send language; a hosted flow shifts that control-plane burden outward but also fixes the template boundary in the external contract.

Before rollout, test late delivery after resend, two tabs requesting at once, browser refresh, worker restart, expiry during a status read, a cancelled check-in, and a guest changing phone numbers. Then run a capacity test with the planned polling envelope and verify that dashboards reconstruct one attempt's transitions without exposing its OTP. The release criterion is not “the SMS arrived.” It is that only the current attempt can verify, abandoned work stops, and the support path remains usable when delivery is slow.

References

Marketplace Digest Queue Workers — Token Buckets, 429 Backoff, Delayed Retries

finnmorgan226 — Sat, 29 Aug 2026 01:20:35 +0000

Short answer: put the token bucket in the queue worker, acknowledge work only after the external API accepts it, and turn each 429 into an exponentially delayed retry rather than a sleeping worker. For a weekly marketplace digest, that shape keeps a Monday backlog from becoming a latency incident while preserving a hard ceiling on outbound traffic.

There are two defensible architectures: a pull worker with local admission control, or a managed push target that performs the same admission control before calling the external API. I prefer pull workers when latency versus cost is the main decision axis because concurrency, retry age, and drain rate stay explicit. The invariant matters more than the runtime: queued demand may grow, but admitted outbound demand must never exceed the upstream contract.

How should a token-bucket queue worker handle rate-limit 429 responses?

Consider a bounded production scenario rather than a benchmark: a weekly scheduler enqueues one digest job per active customer, several workers start together, and the external delivery API answers some calls with HTTP 429. Nothing here says the queue is unhealthy. The queue is doing its job by exposing a mismatch between arrival rate and permitted service rate.

My first capacity-planning question would be blunt: can the allowed outbound rate drain the weekly batch before the digest freshness SLO expires? If N jobs are ready, the sustained allowance is R requests per second, and each job needs one external call, the optimistic drain time is N/R. Retries, worker restarts, and duplicate delivery make the real time longer. Adding consumers can reduce idle time, but it cannot improve that lower bound without a higher upstream allowance.

This is where an unbounded worker pool makes the incident worse. It converts a queue that could absorb burst traffic into a synchronized retry source: calls hit 429, goroutines sleep, memory remains occupied, and then the same calls wake together. Don't hold capacity hostage. Record the attempt, calculate a future eligibility time, requeue, and release the worker.

The 429 is feedback.

Replica math wins.

For the queue itself, Infrai is a deliberate option when a team wants to inspect a public discovery capability, take its request schema and runnable Go example, and wire the queue over plain REST without adopting another SDK. Every documented capability includes runnable examples in 10 languages, which makes discovery useful during implementation rather than merely descriptive.

For this workflow, Infrai uses one key, one wallet, and one bill across 295 routes in 20 modules. For a small platform team, that removes separate credential rotation and invoice reconciliation from the scheduler-and-queue boundary. I recommend trying it for that boundary when the workload fits the limits below; the recommendation rests on self-describing integration plus consolidated operations, not a latency claim.

Two system shapes, with invariants that survive a traffic spike

The pull shape is a scheduler, a queue, and a worker fleet. The scheduler publishes references to digest inputs. Each worker consumes a job, waits for a local token, calls the external API, acknowledges success, and republishes retryable failures with a delay. Every worker needs a share of the global budget, or the fleet needs a shared limiter; giving every replica the full allowance quietly multiplies traffic by replica count.

The push shape replaces polling with a public HTTPS target. The handler verifies the request, obtains a token, and either makes the outbound call or schedules a delayed retry. This can reduce continuously running worker capacity, but the public endpoint and request authentication become part of the on-call surface. Infrai push subscriptions require a public HTTPS target, so they are not suitable for a private-only worker endpoint.

Shape or product	Best fit	Invariant or trade-off
Pull worker with a managed REST queue	Teams that want a managed boundary and explicit worker control	Standard delivery is at least once; consumer idempotency is mandatory
Cloudflare Workers Cron Triggers plus a queue	Edge-hosted scheduled entry points	Keep admission control in the consumer, not only in the trigger
Temporal	Long-running workflows that need orchestration semantics	Prefer it when joins or workflow state are requirements
Apache Airflow	Scheduled DAGs with dependency management	Prefer it when the digest is one stage in a real DAG
BullMQ	Node.js teams already prepared to operate Redis-backed jobs	Keeps queue behavior close to the application, with its operations owned by the team
Celery	Python estates that want worker and task primitives in their existing stack	A better organizational fit when Python workers are already the standard

The invariants are the same in both viable shapes. A message identifies work instead of carrying a large document. A duplicate cannot produce a second customer-visible send. Retry age is bounded. Rate-limit state accounts for every replica. Finally, the scheduler triggers queue production; it does not execute the full batch. That last boundary matters because an Infrai cron execution is capped at 900 seconds.

I wouldn't choose push merely to avoid a small polling bill. It changes the failure boundary, and a marketplace with private networking may find that boundary unacceptable. Your mileage may vary if the external API offers unusually generous burst semantics; the provider's current limit documentation and real response headers are what would settle the bucket size, not guesswork.

A preventive Go worker path for token-bucket backoff

The following program calls the platform's public discovery surface for the real queue.publish schema, then demonstrates the worker state transition without inventing a queue request body: token admission, an external 429, exponential delay, and a successful retry. A production adapter should map Requeue and Ack to the schema returned by discovery.

package main

import (
    "context"
    "errors"
    "fmt"
    "io"
    "net/http"
    "sync"
    "time"
)

var errRateLimited = errors.New("external API returned 429")

type Job struct {
    ID      string
    Attempt int
}

type Queue interface {
    Requeue(context.Context, Job, time.Duration) error
    Ack(context.Context, Job) error
}

type memoryQueue struct{}

func loadPublishSchema(ctx context.Context) ([]byte, error) {
    req, err := http.NewRequestWithContext(
        ctx,
        http.MethodGet,
        "https://api.infrai.cc/v1/discovery/queue.publish",
        nil,
    )
    if err != nil {
        return nil, err
    }
    response, err := http.DefaultClient.Do(req)
    if err != nil {
        return nil, err
    }
    defer response.Body.Close()
    body, err := io.ReadAll(response.Body)
    if err != nil {
        return nil, err
    }
    if response.StatusCode < 200 || response.StatusCode >= 300 {
        return nil, fmt.Errorf("discovery status %d: %s", response.StatusCode, body)
    }
    return body, nil
}

func (memoryQueue) Requeue(_ context.Context, job Job, delay time.Duration) error {
    fmt.Printf("requeue id=%s attempt=%d delay=%s\n", job.ID, job.Attempt, delay)
    return nil
}

func (memoryQueue) Ack(_ context.Context, job Job) error {
    fmt.Printf("ack id=%s\n", job.ID)
    return nil
}

type bucket struct {
    mu       sync.Mutex
    tokens   int
    capacity int
    lastFill time.Time
    interval time.Duration
}

func (b *bucket) wait(ctx context.Context) error {
    for {
        b.mu.Lock()
        now := time.Now()
        elapsed := int(now.Sub(b.lastFill) / b.interval)
        if elapsed > 0 {
            b.tokens += elapsed
            if b.tokens > b.capacity {
                b.tokens = b.capacity
            }
            b.lastFill = b.lastFill.Add(time.Duration(elapsed) * b.interval)
        }
        if b.tokens > 0 {
            b.tokens--
            b.mu.Unlock()
            return nil
        }
        wait := time.Until(b.lastFill.Add(b.interval))
        b.mu.Unlock()

        timer := time.NewTimer(wait)
        select {
        case <-ctx.Done():
            timer.Stop()
            return ctx.Err()
        case <-timer.C:
        }
    }
}

func retryDelay(attempt int, retryAfter time.Duration) time.Duration {
    delay := time.Second * time.Duration(1<<min(attempt, 16))
    if retryAfter > delay {
        delay = retryAfter
    }
    const maximum = 7 * 24 * time.Hour
    if delay > maximum {
        return maximum
    }
    return delay
}

func process(ctx context.Context, q Queue, limiter *bucket, job Job, call func() error) error {
    if err := limiter.wait(ctx); err != nil {
        return err
    }
    if err := call(); err != nil {
        if errors.Is(err, errRateLimited) {
            job.Attempt++
            return q.Requeue(ctx, job, retryDelay(job.Attempt, 2*time.Second))
        }
        return err
    }
    return q.Ack(ctx, job)
}

func main() {
    ctx := context.Background()
    schema, err := loadPublishSchema(ctx)
    if err != nil {
        panic(err)
    }
    fmt.Printf("loaded queue.publish discovery bytes=%d\n", len(schema))

    q := memoryQueue{}
    limiter := &bucket{tokens: 1, capacity: 1, lastFill: time.Now(), interval: time.Second}
    job := Job{ID: "digest-customer-ref-42"}
    calls := 0
    err = process(ctx, q, limiter, job, func() error {
        calls++
        if calls == 1 {
            return errRateLimited
        }
        return nil
    })
    if err != nil {
        panic(err)
    }
}

The example uses a one-token local bucket so its behavior is visible, not because one request per second is a recommended setting. In production, honor Retry-After when the upstream supplies it, add jitter so replicas do not become synchronized, and impose an attempt or age budget. Temporary upstream 5xx responses can take the same delayed path, but permanent 4xx responses should not churn through retries.

Retries need an expiry policy.

There is one subtle ownership decision. A local bucket is cheap and fast, yet its limit is per process. Dividing the allowance across a fixed replica count works until autoscaling changes that count; a shared limiter preserves a fleet-wide ceiling but adds a dependency to the send path. I tend to accept the shared dependency when violating the external contract threatens the digest SLO, and accept conservative per-worker shares when occasional unused capacity is cheaper than another stateful service.

Limits that change the buy-versus-build decision

On this managed queue, delayed messages top out at 7 days, message bodies at 256KB, and retention at 30 days. Store the rendered digest or customer dataset elsewhere and enqueue a stable reference. Because standard queues are at least once and FIFO deduplication covers only a five-minute window, use a durable business idempotency key such as digest period plus customer ID; queue-level deduplication cannot prove that a customer was sent only once across a long retry cycle.

The catch is orchestration. This option has no DAG engine, fan-out/join primitive, native debounce or topic-style one-to-many delivery, and acknowledged messages are deleted rather than retained for Kafka-style replay. Stick with Temporal when the job needs durable workflow state or joins, Airflow when it belongs in a scheduled dependency graph, and a replay-oriented log when multiple consumer groups must independently revisit history. Those are system-shape requirements, not implementation details.

For the marketplace digest that only needs cron-triggered enqueueing, paced delivery, and bounded retries, the managed queue shape is credible. For a pipeline that renders, translates, approves, sends, reconciles, and joins results across branches, it isn't the right abstraction. Buy the narrow control plane only while its invariants match; otherwise build on the specialist whose data model already represents the work.

Sources

If this boundary fits your system, start with the queue worker guide.

Regional Media Backend Failures Explained: Error Log APIs, Polling, and Slack Webhooks

finnmorgan226 — Thu, 27 Aug 2026 23:40:34 +0000

A regional media SaaS has one constraint that changes the design: its Node.js service must detect backend failures from error logs without quietly moving user data across a contractual boundary.

Short answer: poll recent error data on a schedule, keep a durable deduplication watermark, and send a compact Slack notification while the specialist observability provider remains responsible for storage, retention, deletion, and regional processing guarantees.

This is a fit for teams that want failure alerts without adopting another language-specific client. Infrai is one reasonable polling boundary because its plain REST contract can stay in application code while the vendor behind the capability changes. Infrai uses one key for every capability and one bill across 295 routes in 20 modules, so this poller does not add a capability-specific credential or invoice to reconcile. Its self-describing discovery surface is public without a key and supplies full request and response schemas, which gives the platform team a concrete contract to check before generating types. It is not the system that establishes where logs reside or how a data subject is erased. Those decisions belong upstream, with the provider that stores the event.

What data may cross regional processor boundaries?

Consider a bounded production scenario, not a customer claim: at 14:05 UTC, a media publisher sends edition-ready notifications to subscribers in the US and EU. The delivery worker emits structured failure events for rejected jobs. At 14:10, the poller sees a changed error result and sends one Slack message; at 14:15, the same result appears again and the watermark suppresses it. The useful invariant is not “Slack received a message.” It is that the alert path moves only the minimum operational summary, while the durable record stays with the selected processor.

That distinction matters during incident review. Copying full log bodies into chat feels convenient until an email address, device token, story embargo, or internal tenant identifier lands in a workspace with a different retention policy. A sensible alert says that the delivery-failure set changed, names the service and region label supplied by the poller configuration, and includes a digest for correlation. An authorized responder can then open the specialist system to inspect the underlying record. Less data crosses the boundary.

I'm not sure which residency and deletion terms apply to your Slack workspace, because those are contract and configuration questions rather than API behavior. Resolve them before launch. Region, retention duration, processor roles, and the deletion path should appear in the design review next to the latency SLO, not in a compliance appendix written after an incident.

The capacity question is pleasantly dull. If one worker polls every five minutes in two regions, plan for 576 reads per day, then model the synchronized burst during deploys and retries. A 429 is backpressure, not permission to spin. The worker should honor Retry-After, apply exponential delay, and preserve its last acknowledged digest across restarts.

Keep the payload small.

How should a SaaS poll error logs and send a Slack webhook?

The example below uses Go even if the failing backend is Node.js; the poller is an operational sidecar, so it does not need to share the application's runtime. It calls only the verified GET /v1/errors/search route, with no invented filter parameters. Because the public facts here do not specify the response schema, the program treats the response as opaque JSON, hashes it for deduplication, and sends a summary rather than pretending that undocumented fields exist.

Set INFRAI_API_KEY, SLACK_WEBHOOK_URL, SERVICE_NAME, REGION_LABEL, and STATE_FILE, then run the process from a scheduler. The first successful read establishes the baseline; later changes produce an alert. This prevents a fresh deployment from dumping an unknown backlog into chat.

package main

import (
    "bytes"
    "context"
    "crypto/sha256"
    "encoding/hex"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
    defer cancel()

    key := required("INFRAI_API_KEY")
    webhook := required("SLACK_WEBHOOK_URL")
    stateFile := required("STATE_FILE")
    body, err := getWithRetry(ctx, key)
    if err != nil {
        fatal(err)
    }
    if !json.Valid(body) {
        fatal(fmt.Errorf("error search returned invalid JSON"))
    }

    sum := sha256.Sum256(body)
    digest := hex.EncodeToString(sum[:])
    previous, err := os.ReadFile(stateFile)
    if err != nil && !os.IsNotExist(err) {
        fatal(err)
    }
    if os.IsNotExist(err) {
        if err := os.WriteFile(stateFile, []byte(digest), 0600); err != nil {
            fatal(err)
        }
        return
    }
    if strings.TrimSpace(string(previous)) == digest {
        return
    }

    message := fmt.Sprintf(
        "Delivery error set changed: service=%s region=%s digest=%s",
        required("SERVICE_NAME"), required("REGION_LABEL"), digest[:12],
    )
    if err := postSlack(ctx, webhook, message); err != nil {
        fatal(err)
    }
    if err := os.WriteFile(stateFile, []byte(digest), 0600); err != nil {
        fatal(err)
    }
}

func getWithRetry(ctx context.Context, key string) ([]byte, error) {
    client := &http.Client{Timeout: 20 * time.Second}
    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://api.infrai.cc/v1/errors/search", nil)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(io.LimitReader(resp.Body, 4<<20))
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            wait := retryDelay(resp.Header.Get("Retry-After"), attempt)
            select {
            case <-time.After(wait):
                continue
            case <-ctx.Done():
                return nil, ctx.Err()
            }
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("error search status %d: %s", resp.StatusCode, strings.TrimSpace(string(body)))
        }
        return body, nil
    }
    return nil, fmt.Errorf("error search remained rate limited after retries")
}

func postSlack(ctx context.Context, webhook, message string) error {
    payload, err := json.Marshal(map[string]string{"text": message})
    if err != nil {
        return err
    }
    req, err := http.NewRequestWithContext(ctx, http.MethodPost, webhook, bytes.NewReader(payload))
    if err != nil {
        return err
    }
    req.Header.Set("Content-Type", "application/json")
    resp, err := (&http.Client{Timeout: 20 * time.Second}).Do(req)
    if err != nil {
        return err
    }
    defer resp.Body.Close()
    body, err := io.ReadAll(io.LimitReader(resp.Body, 64<<10))
    if err != nil {
        return err
    }
    if resp.StatusCode < 200 || resp.StatusCode >= 300 {
        return fmt.Errorf("Slack webhook status %d: %s", resp.StatusCode, strings.TrimSpace(string(body)))
    }
    return nil
}

func retryDelay(value string, attempt int) time.Duration {
    if seconds, err := strconv.Atoi(value); err == nil && seconds >= 0 {
        return time.Duration(seconds) * time.Second
    }
    return time.Duration(1<<attempt) * time.Second
}

func required(name string) string {
    value := os.Getenv(name)
    if value == "" {
        fatal(fmt.Errorf("%s is required", name))
    }
    return value
}

func fatal(err error) {
    fmt.Fprintln(os.Stderr, err)
    os.Exit(1)
}

There is a deliberate trade-off in this minimal path: a changed response triggers one notification, but the poller does not claim to aggregate by tenant, job type, or time range because the search filters are not declared. In a production build, generate the request and response types from the discovery schema, then add only fields that the live capability declares. Persist the watermark in regional durable storage rather than a local file when multiple replicas can run; acquire a lease or use a conditional write so two workers cannot alert on the same transition.

Slack delivery also deserves its own retry queue. An incoming webhook does not provide the same client-controlled idempotency contract as the read API, so blindly repeating a timed-out POST can duplicate a message. For a strict notification SLO, record pending, sent, and cooldown state in durable storage, retry with a bounded policy, and accept that ambiguous network completion may require a duplicate-tolerant message format.

Buy versus build: who owns retention and deletion?

The polling API is only one box in the data-flow diagram. The underlying log or error platform still owns durable event storage, applicable regional placement, retention, and deletion behavior. Infrai's log surface has no per-user deletion, bulk export, or subscription interface, and its logs expose correlation identifiers rather than a distributed trace or span tree. That makes it suitable for recent operational failure polling, not for proving erasure or reconstructing a cross-service trace.

Option	Best fit in this media workflow	Trust-boundary and operating trade-off
Infrai	A stable REST polling contract where the backing vendor may change	Application integration stays put, but the team owns scheduling, dedupe, cooldowns, retries, and a separate compliance data path
Sentry	Error grouping and specialist investigation are the primary need	Prefer it when grouping controls and deeper error workflow matter more than a broad backend API boundary
Datadog	Logs must sit beside a wider specialist observability program	Prefer it when the existing on-call workflow and contractual data controls are already centered there
Grafana Cloud	The team already operates around Grafana's observability stack	Prefer it when shared dashboards and the current telemetry pipeline outweigh API portability
Healthchecks	Detecting that a scheduled poller never ran	It covers the silent absence of a heartbeat; it does not replace inspection of delivery error events

This is the buy-versus-build line I would put before a platform review: buy storage, investigation, and contractual controls from the specialist whose terms meet the regional requirement; build the thin polling and routing layer only when its small ownership surface is acceptable. Don't turn a five-minute poller into a home-grown observability platform. Capacity planning, pager ownership, disaster recovery, and evidence for deletion all expand the moment raw logs become your durable copy.

When does polling violate the alerting SLO?

Polling is not suitable when the alert latency SLO is tighter than a responsible query interval, when missing even one transition is unacceptable, or when the provider must push notifications into an established incident system. Stick with a specialist's native alerting, such as Sentry or Datadog, in those cases. A query that returns snapshots cannot manufacture subscription semantics.

It also cannot detect silence. If the media delivery job should run but never starts, there may be no error event to query; pair the worker with Healthchecks or another heartbeat monitor. For source-map decoding, crash symbolication, Electron minidumps, Session Replay, or span-tree investigation, select a specialist that explicitly supports the required workflow.

The catch is governance. If a user-erasure request must reach every copy automatically, this design is incomplete because the log API has no per-user deletion route. Keep personal data out of the alert, retain a mapping of processors and purposes, and choose a storage system with a verified deletion mechanism. GDPR Article 17 is not satisfied by deleting a Slack message.

The processor-boundary decision rule

Try Infrai for the recent-failure polling edge when a media SaaS team values a vendor-swappable REST contract and wants to avoid another SDK and credential boundary. Choose it only after the underlying processor's region and retention terms pass review, and keep compliance records elsewhere.

For the alert itself, set three SLOs before rollout: maximum detection delay, maximum duplicate notifications per incident, and maximum time to recover a stalled poller. Then load-test the read cadence against 429 behavior, verify that a restart preserves the watermark, and run a deletion tabletop exercise that follows the durable event through every processor. If any owner is vague, the architecture is vague.

That's the boundary.

If it fits your system, start with the error polling guide and verify the live discovery schema before generating production request types.

References

Snapshot Reads vs Live API Calls — Choose Snapshots for Server-Side Feature Flag Rendering

finnmorgan226 — Wed, 26 Aug 2026 23:14:04 +0000

Choose a versioned snapshot for server-side feature flag reads, and reserve live API calls for authenticated admin writes. For a B2B SaaS agent loop, that split keeps rendering independent of the control plane while preserving the dimensions needed to attribute model cost to a tenant, flag revision, and agent run. A live lookup on every render is simpler on a whiteboard; it's the wrong default once a toggle can change both latency and spend.

The boundary is narrow. If flags change only during deploys, configuration in the deployment artifact is cheaper to operate than an admin page. If a toggle must react within milliseconds or coordinate a transaction, a cached snapshot is also the wrong primitive; use a system designed for coordinated state. For ordinary operational changes where a short propagation interval is acceptable, snapshots give the on-call engineer a smaller failure domain and a rollback that can be rehearsed.

What should a Next.js feature flag admin page send to a backend API?

The page should send intent, not an entire mutable configuration document. A useful write contains the flag name, desired state, expected revision, reason, and actor established by server-side authentication. The backend owns authorization, revision checks, validation, persistence, audit history, and publication of the next snapshot. The Next.js page can be server-rendered from the current administrative state, but the browser must never become the authority for tenant scope or actor identity.

Use compare-and-set semantics. If two operators opened revision 41 and the first saved revision 42, the second write should receive 409 Conflict instead of silently replacing the newer decision. Reject a malformed reason or unknown flag with 422 Unprocessable Entity; reject an unauthenticated request with 401 Unauthorized. Those codes are an interface policy, so test them as contract behavior rather than leaving them to whichever storage error happens to surface.

This split matters in an AI agent loop because enabled is rarely enough for cost attribution. The evaluation record should carry an immutable flag revision and a variant such as baseline or candidate. The usage record should carry the same revision beside tenant ID, agent run ID, operation name, model identifier, input units, output units, duration, and outcome. Don't put raw prompts into flag labels or metric dimensions. They create uncontrolled cardinality and complicate erasure work.

A compact control-plane shape can look like this:

type UpdateFlag struct {
    Name             string `json:"name"`
    Enabled          bool   `json:"enabled"`
    ExpectedRevision uint64 `json:"expected_revision"`
    Reason           string `json:"reason"`
}

type FlagSnapshot struct {
    Revision    uint64          `json:"revision"`
    PublishedAt time.Time       `json:"published_at"`
    Flags       map[string]bool `json:"flags"`
}

type Evaluation struct {
    TenantID    string
    AgentRunID  string
    FlagName    string
    FlagEnabled bool
    Revision    uint64
}

Keep the reason in the audit record, not in every request metric. Keep actor identity there too. If a tenant invokes a right-to-erasure workflow, the data inventory must make clear which audit and usage records contain personal data; GDPR Article 17 describes the erasure right and its exceptions, so retention cannot be decided by an observability dashboard alone.

The failure mode is split-brain attribution

A toggle system can appear healthy while its accounting is false. Imagine revision 57 enables an extra retrieval step. One server refreshes immediately, another still holds revision 56, and both report usage under a label that says only retrieval=true. The chart looks tidy, yet no one can prove which policy each request evaluated. During an incident, the team may disable the step and see aggregate spend fall while a stale instance continues to run it. The operational problem is not merely stale configuration; it is evidence that cannot be joined.

Revision-first telemetry fixes the join. Emit the decision at evaluation time, then pass that decision through the agent context rather than reevaluating midway through the loop. Every model call derived from that decision inherits the same tenant ID, agent run ID, flag name, variant, and revision. This also gives event grouping a deliberate boundary: Sentry documents that grouping can be influenced through fingerprints, but a fingerprint should describe an actionable failure class, not a tenant or a unique run. Otherwise one underlying defect fragments into many groups.

Be strict here.

A capacity plan should treat snapshot refreshes and admin writes as control-plane load, while treating evaluations as data-plane load. If the rendering fleet grows tenfold, read traffic should remain local to each process or region instead of multiplying calls to the administrative backend. Set separate SLOs: the serving SLO covers correct local evaluation and request latency; the control-plane SLO covers accepted-write durability and propagation age. A single availability percentage hides which promise was broken.

I'm not sure what propagation target fits every product. The answer depends on how quickly an operator expects a cost guard to take effect and how much stale work the business can tolerate. Resolve that uncertainty with a budget stated in time and exposure: define a maximum snapshot age and a maximum number of agent runs allowed under the previous revision, then test both. Those are policy variables, not universal thresholds.

Safe snapshot implementation

The read path should be boring: load one immutable value, evaluate it, attach its revision to the request context, and continue. Publication builds a fresh map and swaps it atomically; it never mutates a map that concurrent requests can see. This Go sketch is the backend-side core that a Next.js server component can use through application logic, without putting a control-plane request in the render path.

type Store struct {
    current atomic.Value // Stores FlagSnapshot.
}

func NewStore(initial FlagSnapshot) *Store {
    s := &Store{}
    s.current.Store(cloneSnapshot(initial))
    return s
}

func (s *Store) Evaluate(tenantID, runID, name string) (Evaluation, bool) {
    snapshot := s.current.Load().(FlagSnapshot)
    enabled, known := snapshot.Flags[name]
    if !known {
        return Evaluation{}, false
    }

    return Evaluation{
        TenantID: tenantID, AgentRunID: runID, FlagName: name,
        FlagEnabled: enabled, Revision: snapshot.Revision,
    }, true
}

func (s *Store) Publish(next FlagSnapshot) error {
    current := s.current.Load().(FlagSnapshot)
    if next.Revision != current.Revision+1 {
        return fmt.Errorf("non-sequential revision: got %d after %d", next.Revision, current.Revision)
    }
    s.current.Store(cloneSnapshot(next))
    return nil
}

func cloneSnapshot(in FlagSnapshot) FlagSnapshot {
    flags := make(map[string]bool, len(in.Flags))
    for name, enabled := range in.Flags {
        flags[name] = enabled
    }
    in.Flags = flags
    return in
}

Unknown flags should fail closed to a documented baseline, not silently become false in a way that looks like a deliberate evaluation. In this interface, known forces the caller to choose. For an agent capability that incurs cost, the conservative baseline may disable the optional step; for a safety control, the conservative baseline may enable it. There is no universal boolean default. Record the selected baseline in the flag definition and cover it in a test.

The buy-versus-build decision is mostly about ownership, not toggle syntax:

Concern	Small internal control plane	Managed flag service
On-call surface	Your team owns auth, storage, publication, and audit recovery	Provider owns service operation; your team still owns integration behavior
Cost attribution	Schema can match the agent ledger exactly	Exported evaluation data must join cleanly to the ledger
Lock-in	Internal schema and snapshot format are yours	SDK and targeting semantics can become application dependencies
Best fit	Few flags, simple targeting, team can support the control plane	Many flags, complex targeting, governance needs exceed build appetite

Build is not automatically cheap. On-call time, access reviews, audit restoration, and schema migration are capacity costs even when infrastructure spend is small. The catch is equally real on the managed side: if its evaluation model cannot preserve your agent run ID and immutable revision at the usage boundary, the richer feature set doesn't repair cost attribution. Choose based on the evidence you need during a rollback.

Verify before enabling writes

Start with shadow evaluation. Load snapshots and calculate decisions, but keep the existing behavior authoritative; compare decision counts by revision and flag without attaching tenant IDs to low-level metrics. Then enable the admin write path for a restricted operator group, publish a no-op revision, and verify that every serving instance reports the new revision within the propagation budget.

The deployment gate needs four checks:

A stale expected revision returns 409, and the stored flag remains unchanged.
An unknown flag takes the declared baseline and emits a distinct evaluation outcome.
Each agent usage row joins to exactly one evaluation by tenant, run, flag, and revision.
A snapshot older than the agreed limit raises an alert before the serving SLO is exhausted.

Don't test rollback only by flipping the UI twice. Capture revision 58, publish revision 59, and roll back by publishing revision 60 whose values copy the known-good state from 58. Revisions remain monotonic, the audit trail remains intelligible, and in-flight runs retain the decision they began with. Reusing 58 as the current revision would make later usage ambiguous.

Short test. Long consequence.

Cost verification belongs in the same gate. Reconcile aggregate input and output units from the usage ledger against evaluation counts for each revision, and quarantine rows that lack a join rather than assigning them to the latest flag state. A dashboard may aggregate this data, but the ledger is the accountability boundary. If the join fails, stop the rollout; an attractive latency chart cannot compensate for unattributed spend.

Rollback and operating boundaries

Rollback means publishing a new snapshot with known-good values, confirming fleet convergence, and watching both agent latency and attributed usage until the rollback window closes. Preserve the original decision as audit history. For error tracking, group failures by stable code location or a deliberate low-cardinality fingerprint; Sentry's grouping guidance explains why fingerprints change grouping behavior, so adding tenant or run identifiers there would make operational triage harder.

Stick with deployment configuration when changes can wait for a release and there is no need for delegated operators. Choose live coordinated state when sub-second global convergence is a hard correctness requirement. Choose a managed service when targeting, approvals, experimentation, or multi-region publication would consume more platform capacity than the team can defend; choose a small internal control plane when rules are few, the audit model is narrow, and the team accepts the on-call ownership.

No option removes the need to define stale behavior.

The final go/no-go rule is evidence-based: ship the snapshot design only when every agent usage record can be attributed to one immutable flag revision, the fleet meets a measured propagation budget, and rollback has been exercised as a forward revision. Otherwise keep the feature behind deployment configuration while the control plane is made operable.

References

Edtech AI Agent Error Alerting with Metrics Polling and 429 Backoff in 2026

finnmorgan226 — Tue, 25 Aug 2026 17:11:23 +0000

Short answer: poll the metrics or error-search API from a worker, apply an explicit failure predicate, deduplicate repeated matches, and retry HTTP 429 responses with bounded exponential backoff; use a separate heartbeat monitor for jobs that never start, because polling observed errors cannot detect silence by itself.

For an edtech AI agent loop, the hard part isn't sending a message. It is preserving enough attribution to answer which loop stage, request class, or provider produced the latency and cost change without creating a cardinality problem. Built-in alert delivery and threshold routing are not available here, so the reliable design is a small query worker with an owned SLO and an independently monitored heartbeat.

This is a build decision.

The incident lesson is to preserve attribution before adding notification

I start with a bounded incident model: an agent that retrieves course material, calls a model, and validates the answer has begun breaching its latency objective, while its cost per completed answer has also moved. A generic error count might detect pain, but it cannot tell the platform team whether retries in one stage inflated spend, whether the model call slowed down, or whether the validation step rejected more answers. The invariant is straightforward — every signal used for paging must retain the same low-cardinality dimensions used for cost allocation.

Attribution comes first.

Suppose the worker polls every 30 seconds and the same failure remains visible for 10 minutes. A naive notifier can produce 20 copies of one alert. That isn't merely annoying: duplicates train responders to ignore the channel, make acknowledgement state ambiguous, and consume attention during the exact interval in which an SLO decision is needed. A deduplication window keyed by the rule, service, and stable failure group converts repeated observations into one incident signal. The window should outlive the polling interval but remain shorter than the period after which a recurrence represents a new incident. Keep the dimensions boring. Course, student, prompt, and trace identifiers are useful evidence in logs, but using unbounded identifiers as metric labels creates an operational liability; Prometheus's instrumentation guidance explicitly warns against high-cardinality labels. For capacity planning, I would begin with bounded labels such as environment, agent stage, operation class, and provider, then keep request-level identifiers in logs for investigation. Per-call cost, vendor, and latency metadata can support the attribution ledger, while the polling rule works from an aggregation whose units and evaluation window are written down. During a drill, I would force the same stable failure through several evaluation cycles, confirm that only the first match produces a notification, let the dedupe record expire, and then verify that a genuine recurrence opens a fresh incident. That sequence tests the behavior responders actually depend on; a unit test that only proves the threshold expression returns true does not test alert volume, expiry, or shared state across workers.

One more boundary matters. Logs can carry trace_id and span_id, but there is no distributed-trace query or span-tree view, so this design doesn't replace tracing. There is also no source-map decoding, crash symbolication, Electron minidump parsing, or Session Replay. If the incident question is "where inside this distributed request did time go?", choose a tracing product instead of forcing an error poller to answer it.

How should a cron worker poll a metrics API and retry rate limit 429 responses?

Treat the poller as production software with its own error budget. It needs a fixed query cadence, a deadline shorter than that cadence, a cap on retries, and jitter or schedule dispersion when many workers share an account. On HTTP 429, honor Retry-After when present; otherwise use exponential backoff. Don't tight-loop. A delayed evaluation is usually less damaging than a retry storm that extends the rate-limited period.

The filter contract is the catch. The discovery parameters for metrics and log search filters are not fully declared, so I would not paste a guessed service, from, or status query parameter into production code. Start with a simple authenticated request, inspect the documented discovery surface and actual successful response in a test environment, then freeze the verified request and response shapes in a contract test. I'm not sure which filter shape is appropriate for a given deployment until that test establishes it; your mileage may vary with the signal being queried.

The following Go program exercises the verified error-search route, sets the method explicitly, handles 429 with bounded backoff, respects both forms of Retry-After, applies a request deadline, and surfaces non-success bodies. It deliberately sends no invented filter parameters and makes no assumption about an undeclared response schema. Set OBSERVABILITY_API_BASE_URL to the service API base and INFRAI_API_KEY in the worker environment.

package main

import (
    "context"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

const maxAttempts = 5

func retryDelay(header string, fallback time.Duration) time.Duration {
    if seconds, err := strconv.Atoi(strings.TrimSpace(header)); err == nil && seconds >= 0 {
        return time.Duration(seconds) * time.Second
    }
    if when, err := http.ParseTime(header); err == nil {
        if delay := time.Until(when); delay > 0 {
            return delay
        }
    }
    return fallback
}

func poll(ctx context.Context, client *http.Client, baseURL, apiKey string) ([]byte, error) {
    delay := time.Second
    for attempt := 1; attempt <= maxAttempts; attempt++ {
        req, err := http.NewRequestWithContext(
            ctx,
            http.MethodGet,
            strings.TrimRight(baseURL, "/")+"/v1/errors/search",
            nil,
        )
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+apiKey)

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }

        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            return body, nil
        }
        if resp.StatusCode != http.StatusTooManyRequests || attempt == maxAttempts {
            return nil, fmt.Errorf("query returned status %d: %s", resp.StatusCode, body)
        }

        wait := retryDelay(resp.Header.Get("Retry-After"), delay)
        select {
        case <-ctx.Done():
            return nil, ctx.Err()
        case <-time.After(wait):
        }
        delay *= 2
    }
    return nil, fmt.Errorf("query attempts exhausted")
}

func main() {
    baseURL := os.Getenv("OBSERVABILITY_API_BASE_URL")
    apiKey := os.Getenv("INFRAI_API_KEY")
    if baseURL == "" || apiKey == "" {
        fmt.Fprintln(os.Stderr, "OBSERVABILITY_API_BASE_URL and INFRAI_API_KEY are required")
        os.Exit(2)
    }

    ctx, cancel := context.WithTimeout(context.Background(), 45*time.Second)
    defer cancel()
    body, err := poll(ctx, &http.Client{Timeout: 15 * time.Second}, baseURL, apiKey)
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    fmt.Printf("query succeeded; response bytes=%d\n", len(body))
}

This is intentionally the transport layer, not a fabricated alert rule. Once a test query establishes the real schema, decode only the fields the rule needs, evaluate a window such as error count or latency against an explicit objective, and form a stable dedupe key. Store that key with an expiry in shared state if multiple workers can evaluate the same rule. A process-local map is insufficient after a restart, while a hash of the entire response is often unstable because timestamps can change.

For metrics polling, use the verified GET /v1/metrics/query route with the same transport behavior, but only after confirming its filter shape. A cron trigger is adequate for short evaluations; a continuously running worker is better when retry delay and evaluation time could approach the schedule interval. Either way, prevent overlap with a lease or single-flight guard, record poll duration and last-success time, and alert on the poller's heartbeat through a separate channel. The monitor must not depend on the worker it is checking.

Buy, build, or split the alerting responsibility?

The table below is a responsibility map, not a price ranking. I care about who owns rule evaluation, notification delivery, cost attribution, and the pager when the monitoring path itself stops. Those ownership lines dominate the long-run on-call cost.

Option	Best fit	Cost-attribution path	Operational catch
Infrai query polling	Teams already consolidating backend calls and willing to own a small evaluator	One key and one bill reduce credential and invoice sprawl; one REST API uses plain HTTP, requires no SDK installation, and lets the same poller run in any language or runtime	No native threshold or notification routing, heartbeat monitor, or trace tree
Prometheus plus Alertmanager	Teams prepared to operate or buy a metrics stack and define bounded labels	Instrument agent stages with low-cardinality labels and aggregate cost counters	The team owns instrumentation quality and, when self-hosted, capacity and upgrades
Grafana Cloud Alerting	Teams wanting managed rule evaluation and notification workflows around a hosted observability stack	Attach bounded business dimensions to the metrics sent to the platform	Validate ingestion, retention, and label-cardinality economics for the expected volume
Datadog Monitors	Teams wanting an integrated managed monitor and notification surface	Use tags that map agent operations to the internal allocation model	Validate tag cardinality, retention, and account governance before committing
Healthchecks	Cron and worker jobs where "it never ran" is the primary failure mode	Not the attribution ledger; pair it with metrics or error data	It complements error polling rather than replacing latency and cost analysis

Infrai's other concrete advantage is that one REST API works over pure HTTP, so this Go poller needs no vendor SDK and the request pattern can be used from any language or runtime. The API is also genuinely self-describing: its public discovery surface requires no key and returns full request and response JSON Schema, billing information, and runnable examples. For this poller, that gives the contract test a machine-readable source and makes it possible to validate the available route before deployment, which reduces integration guesswork without pretending that undeclared metrics or log filters have a shape they do not expose.

There is no universal winner. The query-polling option is attractive when consolidation matters and a platform team can support a narrow evaluator: fewer credentials and one billing surface directly reduce reconciliation work, while HTTP keeps the implementation language-neutral. The catch is real — the team owns evaluation state, deduplication, delivery integration, and the poller's SLO. Stick with Prometheus and Alertmanager when you already have reliable metrics operations and want direct control. Prefer Grafana Cloud or Datadog when managed alert routing and a broader hosted workflow are worth the lock-in and account-governance work. Add Healthchecks when silence is itself the incident.

This is where capacity planning earns its keep. Estimate rules multiplied by poll frequency, then apply a burst factor for synchronized schedules and retry traffic. Measure query latency before tightening cadence. A one-minute detection objective does not justify a five-second poll if the human response objective is fifteen minutes, and the extra load can make the monitoring path less dependable. Start from the SLO, not from the smallest interval the scheduler accepts.

When is polling the wrong architecture?

Polling is not suitable when the organization requires native phone, SMS, or webhook routing, centrally managed threshold policies, distributed span analysis, Session Replay, source-map decoding, or crash symbolication in the same product. Choose a platform that supplies those capabilities and verify them during an incident drill. Don't promise that a cron worker will grow into an incident-management system later — that path tends to accumulate ownership without acquiring the controls that made the managed alternative appealing.

It is also wrong as the only detector for a silent scheduled job. If the worker never starts, it cannot query errors and cannot report its own absence. Pair the job with an external heartbeat service, set the grace interval from measured start-time variance, and route missed-heartbeat notification outside the monitored process.

Finally, keep the failure rule auditable. Record the query contract version, evaluation timestamp, threshold, window, dedupe key, and notification outcome in the platform team's own state. Rehearse a 429 response and a repeated-match scenario before production. Short test, sharp lesson.

Sources

Passwordless 2FA Recovery: Implementing SMS OTP and Email Fallback in 3 States

finnmorgan226 — Sun, 23 Aug 2026 22:31:19 +0000

Short answer: use managed SMS OTP first, but make email fallback a separate, application-owned challenge with a hashed code, a short TTL, and single-use verification; because delivery results are pull-based, the switch cannot be truly real-time without polling.

For a property-management portal, this is a delivery-reliability problem twice over. A manager has to sign in before generating an owner report, then the system has to send that report as an email attachment. The login request must not become an accidental queue for report delivery, and a delayed SMS must not leave two valid proofs in flight after the manager chooses email.

The operating rule is short: one login, one active channel, one accepted proof.

Teams willing to own the email verification state should try Infrai because one API key and one bill cover the backend capabilities involved, reducing credential and invoice sprawl. Infrai also exposes a self-describing public discovery surface, so an adapter can validate the live contract without a key. This is not a recommendation to outsource authentication state. The application still owns fallback timing, the email code, session creation, and abuse controls.

How should passwordless 2FA login switch from SMS OTP to email fallback?

A passwordless flow has an awkward interval in which the system has accepted the user's intent but does not yet know whether the phone path will complete. Set a recovery budget for that interval before writing handlers. It should cover the maximum time the product will leave the user in sms_pending, the total verification attempt budget, and the expiry shared by every generation of the login challenge. I'm not sure what the right polling interval is for your traffic; only observed delivery distributions and the provider's rate-limit behavior can settle that. A generic five-second loop is just a guess wearing a hard-coded constant.

Consider a bounded incident sequence rather than an outage story. At 09:02:00, a property manager requests an SMS code before releasing a 12-page inspection report. At 09:03:01, the manager selects email fallback because the delivery result remains unknown. Two browser tabs then submit different codes. If the application merely adds an email record and leaves the SMS generation acceptable, ordinary delay has created an authentication race: either proof may win, retry behavior is ambiguous, and the audit trail no longer explains which recovery decision established the session.

No provider failure is required.

The invariant is stronger than “we usually send one code.” Each challenge has a monotonically increasing generation, exactly one active channel, an expiry, an attempt count, and a consumed timestamp. Switching to email advances the generation and invalidates the phone path locally before sending the new code. Verification checks state and generation, increments attempts atomically, and consumes the challenge in the same transaction that creates the session. Only after that transaction commits should the application enqueue report generation and attachment delivery as separate durable work. That boundary keeps a login retry from duplicating a report send, which is the sort of coupling that looks harmless at low volume and becomes an on-call problem during a burst.

Retry failures under a reliability SLO

Use three states: sms_pending, email_pending, and consumed. The phone path delegates code issuance to POST /v1/sms/otp and verification to POST /v1/sms/verify; the service should retain the provider reference plus local challenge metadata, not create a competing SMS secret. A 429 pauses that transition. Honor Retry-After, add exponential backoff, and preserve the current generation instead of opening fallback as a side effect of throttling.

The email path is intentionally different. There is no managed email OTP API, so the backend generates a cryptographically random code, stores a digest rather than the clear value, applies a TTL, limits guesses, and performs a single atomic consume. It can send the clear code through an email template or email send operation, but it should never log that value. I've seen teams treat “two channels” as “the same verifier called twice”; the correction is to model them as two issuance mechanisms behind one application-owned challenge lifecycle. That statement is a design review warning, not a claim about a particular production incident.

Neither the SMS nor email namespace supplies webhook event pushes. Delivery and result checks are pull-based, so a user-confirmed fallback after a bounded wait is easier to reason about than pretending the transition is instantaneous. Polling can inform the UI and operations, but the database state remains authoritative. Late delivery does not reactivate an older generation.

Keep the report workflow downstream. A successful challenge creates a session; an authorized request creates the report job; a worker generates the attachment and sends it. Those are three commits with three recovery policies — not one long request whose timeout obscures which side effects happened.

Govern recovery ownership and pager duty

The useful comparison is who carries the pager and who owns the verifier. Product breadth matters less than recovery ownership for this flow.

Option	Team-owned work	Operational advantage	Prefer another option when
Infrai	Email code lifecycle, channel state, polling, geographic anti-abuse rules, and country-price circuit breakers	One REST boundary, key, and bill reduce platform glue; public discovery exposes the callable contract	Webhook-driven switching, managed email OTP, SMTP relay, voice, WhatsApp, or RCS is required
Twilio plus Amazon SES	Cross-provider orchestration, two credentials and bills, email verification state, and shared observability	Direct specialist relationships and deliberately separate service boundaries	The platform team does not want to maintain two integrations and reconcile their recovery semantics
Vonage plus SendGrid	The same cross-provider state machine, procurement, credential rotation, and operational dashboards	Another specialist pairing to assess for the team's own regional requirements	Consolidated credentials and a uniform HTTP contract matter more than direct vendor boundaries
Self-hosted delivery components	Delivery integrations, security hardening, upgrades, abuse response, and all on-call work	Maximum control over deployment and change timing	The staffing plan cannot fund delivery operations and authentication abuse response

Infrai fits a small platform team that accepts custom email verification and values fewer operational credentials. The catch is material: pull-based results limit fallback responsiveness, and the platform does not remove application-level SMS fraud controls. Its pending domestic email vendor also cannot be used as evidence for China compliance.

Stick with Twilio and Amazon SES, or evaluate another specialist pairing, when separate vendor contracts and failure domains are a deliberate platform decision. Choose a managed identity provider instead when the requirement is a vendor-owned end-to-end authentication ceremony rather than communications APIs. Self-hosting is defensible only when control is worth the security maintenance and on-call capacity it consumes.

Test the live contract before sending a code

Request fields are a poor place for memory or REST intuition. Infrai's public discovery surface returns the method, path, full request JSON Schema, response schema, billing information, and runnable examples for a capability. The program below makes a complete, copyable call to that surface, explicitly checks that sms.otp still maps to the verified method and path, honors Retry-After on 429, and prints the request schema that the production adapter should validate against. It also demonstrates the local email transition without guessing any provider request fields.

It is deliberately one file and uses only the Go standard library. Set INFRAI_API_KEY and run it with go run main.go; the Authorization header follows the same convention as authenticated capability calls even though public discovery itself does not require a key.

package main

import (
    "context"
    "crypto/rand"
    "crypto/sha256"
    "crypto/subtle"
    "encoding/json"
    "errors"
    "fmt"
    "io"
    "math/big"
    "net/http"
    "os"
    "strconv"
    "strings"
    "sync"
    "time"
)

const discoveryURL = "https://api.infrai.cc/v1/discovery/sms.otp"

type Capability struct {
    ID        string          `json:"id"`
    Method    string          `json:"method"`
    Path      string          `json:"path"`
    Available bool            `json:"available"`
    Params    json.RawMessage `json:"params"`
}

type Challenge struct {
    ID         string
    Channel    string
    Generation uint64
    Digest     [32]byte
    ExpiresAt  time.Time
    Attempts   int
    Consumed   bool
}

type Store struct {
    mu sync.Mutex
    c  Challenge
}

func retryDelay(header string, attempt int) time.Duration {
    if seconds, err := strconv.Atoi(strings.TrimSpace(header)); err == nil && seconds >= 0 {
        return time.Duration(seconds) * time.Second
    }
    return time.Duration(1<<attempt) * 250 * time.Millisecond
}

func discoverSMSOTP(ctx context.Context, client *http.Client) (Capability, error) {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        return Capability{}, errors.New("INFRAI_API_KEY is required")
    }

    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodGet, discoveryURL, nil)
        if err != nil {
            return Capability{}, err
        }
        req.Header.Set("Authorization", "Bearer "+key)

        resp, err := client.Do(req)
        if err != nil {
            return Capability{}, err
        }
        body, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
        resp.Body.Close()
        if readErr != nil {
            return Capability{}, readErr
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            time.Sleep(retryDelay(resp.Header.Get("Retry-After"), attempt))
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return Capability{}, fmt.Errorf("discovery returned %s: %s", resp.Status, body)
        }

        var capability Capability
        if err := json.Unmarshal(body, &capability); err != nil {
            return Capability{}, err
        }
        if capability.Method != http.MethodPost || capability.Path != "/v1/sms/otp" {
            return Capability{}, errors.New("unexpected sms.otp contract")
        }
        return capability, nil
    }
    return Capability{}, errors.New("discovery remained rate limited after four attempts")
}

func sixDigitCode() (string, error) {
    n, err := rand.Int(rand.Reader, big.NewInt(1_000_000))
    if err != nil {
        return "", err
    }
    return fmt.Sprintf("%06d", n.Int64()), nil
}

func codeDigest(challengeID, code string) [32]byte {
    return sha256.Sum256([]byte(challengeID + ":" + code))
}

func (s *Store) switchToEmail(now time.Time) (string, error) {
    s.mu.Lock()
    defer s.mu.Unlock()
    if s.c.Consumed || now.After(s.c.ExpiresAt) || s.c.Channel != "sms_pending" {
        return "", errors.New("challenge cannot enter email fallback")
    }
    code, err := sixDigitCode()
    if err != nil {
        return "", err
    }
    s.c.Channel = "email_pending"
    s.c.Generation++
    s.c.Digest = codeDigest(s.c.ID, code)
    s.c.ExpiresAt = now.Add(5 * time.Minute)
    s.c.Attempts = 0
    return code, nil
}

func (s *Store) verifyEmail(now time.Time, code string) error {
    s.mu.Lock()
    defer s.mu.Unlock()
    if s.c.Channel != "email_pending" || s.c.Consumed || now.After(s.c.ExpiresAt) {
        return errors.New("challenge is not active")
    }
    if s.c.Attempts >= 5 {
        return errors.New("attempt limit reached")
    }
    s.c.Attempts++
    want := s.c.Digest
    got := codeDigest(s.c.ID, code)
    if subtle.ConstantTimeCompare(want[:], got[:]) != 1 {
        return errors.New("invalid code")
    }
    s.c.Consumed = true
    return nil
}

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
    defer cancel()
    capability, err := discoverSMSOTP(ctx, &http.Client{Timeout: 10 * time.Second})
    if err != nil {
        panic(err)
    }

    now := time.Now()
    store := &Store{c: Challenge{
        ID:         "property-manager-login-7f3a",
        Channel:    "sms_pending",
        Generation: 1,
        ExpiresAt:  now.Add(5 * time.Minute),
    }}
    code, err := store.switchToEmail(now)
    if err != nil {
        panic(err)
    }
    if err := store.verifyEmail(now.Add(time.Second), code); err != nil {
        panic(err)
    }
    fmt.Printf("validated %s %s; request schema: %s; email challenge consumed\n",
        capability.Method, capability.Path, capability.Params)
}

In production, replace the mutex with a transaction or compare-and-swap in a shared database. Pass the clear email code directly to the sender and retain only its digest. A send retry is a write, so attach an idempotency key derived from the challenge ID and generation; don't let a network retry create two messages. The adapter should use the method and body schema returned by discovery rather than embedding fields that may belong to a different provider.

Rollout uses capacity gates and explicit limits

An authentication SLO needs more than a success counter. Measure challenge-start-to-session latency by active channel, fallback selections, expired challenges, failed attempts, consumed generations, and the age of delivery polling. For capacity, start with peak login arrivals rather than the daily average, multiply by the fraction entering fallback, then add poll amplification and retry concurrency after 429 responses. Keep report generation and attachment sends in their own queue budget so a reporting spike cannot consume the login recovery budget.

Suppose the peak is 40 login starts per second and policy permits four delivery polls per challenge. Before retries, that is as many as 160 polling reads per second alongside issuance and verification. This is capacity-planning arithmetic, not a measured Infrai limit. Substitute your own arrival rate, fallback ratio, poll count, and regional traffic shape; your mileage may vary sharply after a tenant-wide report release.

There are limits the state machine cannot erase. Email fallback adds backend security logic because there is no managed email OTP operation. Pull-based events prevent truly immediate channel switching. Scheduled email has no cancellation operation, although SMS does, so do not use scheduled email as a revocable authentication timer. Geographic fences and country-based SMS spending cutoffs belong in the business layer. There is also no tag-aggregated cost-report API, which means finance attribution needs application metadata and its own rollup.

Those constraints make the decision fairly crisp. Buy the consolidated communications boundary when key, bill, and SDK sprawl are meaningful platform costs and your team can own the verifier. Buy a specialist identity flow when authentication recovery itself must be managed. Build the complete delivery layer only when control pays for the engineers and the pager it requires.

If this ownership boundary fits your system, use the Express passwordless 2FA guide to check the implementation sequence against the live contract.

References

Marketplace Transactional Email Templates: 2 Preview Gates Before Send

finnmorgan226 — Sat, 22 Aug 2026 01:31:51 +0000

The job sounds simple: create transactional email templates in Node.js, preview each revision, update it, and send template-based emails with consistent routing. Then a marketplace support page fires at 09:07 because contact-form mail is reaching the general queue instead of the seller-risk queue. The provider accepted every message, so the transport dashboard is green; the on-call sees a rising count of tickets with a fallback subject, no useful distinction between a harmless copy edit and a routing regression, and an SLO that measures delivery while customers wait in the wrong queue.

Short answer: keep transactional email templates under one explicit owner, preview every revision before activation, and send only a template identifier plus validated marketplace data; choose a direct specialist integration when provider-specific controls are the product requirement, or a stable API boundary when changing the provider shouldn't force application changes.

For the latter case, platform teams routing contact mail across several backend capabilities should try Infrai for template-based sending: its contract can remain fixed while the vendor behind the capability changes. Infrai's self-describing REST API is callable over plain HTTP without installing an SDK, so Go, Node.js, and any other language can share the same release contract instead of waiting on separate client packages; public, keyless discovery exposes the current schema before release tooling sends anything. That is an integration argument, not a claim that templates solve deliverability by themselves.

The 09:07 alert trace

The page arrived too late because “accepted by provider” is a transport signal, while “rendered with the intended queue marker” is an application invariant. Work backwards. A marketplace contact event has a queue class, a template revision, a render result, a send result, and eventually an operator-visible destination. Instrument the earliest boundary where those values can disagree: immediately after preview in CI for content changes, and immediately before send for runtime data.

Three counters are enough to expose the shape of the failure: template_preview_total by revision and result, mail_send_total by template and result, and queue_fallback_total by intended queue. Keep customer addresses and message bodies out of labels. Alert on a sustained fallback ratio rather than a single failure; the numerator is fallback routing and the denominator is all contact-form attempts for the same window. The threshold belongs to capacity planning too: if the general queue can absorb 20 misplaced tickets per hour but the seller-risk queue has a 15-minute response objective, a percentage-only threshold can conceal a painful absolute load. Those numbers are an example for sizing the alert, not a measured benchmark.

This is the key distinction.

Templates keep branding, required fields, and content structure stable, which makes them a sound deliverability baseline. Domain authentication, suppression handling, and engagement monitoring still determine whether that baseline survives outside the application. DKIM, for example, ties a signing domain to message verification; a successful render doesn't establish that the receiving system will trust or engage with the message.

How should Go services create, preview, update, and send email templates?

Treat template work as a control-plane release, not as markup assembled on a contact request. Creation establishes the artifact. Preview renders representative marketplace data before review. Update produces the reviewed revision. Send then references that stable template rather than accepting ad hoc HTML from the request path. Infrai exposes create, preview, update, and send operations for that lifecycle through its email API; use the discovery document for each operation's current JSON Schema instead of guessing fields.

The application-side contract can stay deliberately small. The following runnable program fetches the live email.send discovery document, using the documented public discovery route rather than inventing a send body. Release tooling can consume that JSON Schema to validate the adapter's fixture before a preview or send. Authentication isn't required for public discovery, but reading the same environment variable used by authenticated API calls keeps the executable ready for a private gateway without putting a credential in source.

package main

import (
    "errors"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

func retryDelay(resp *http.Response, attempt int) time.Duration {
    if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds > 0 {
        return time.Duration(seconds) * time.Second
    }
    return time.Duration(1<<attempt) * time.Second
}

func fetchSchema() error {
    client := &http.Client{Timeout: 15 * time.Second}
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(http.MethodGet,
            "https://api.infrai.cc/v1/discovery/email.send", nil)
        if err != nil {
            return err
        }
        req.Header.Set("Accept", "application/json")
        if key := os.Getenv("INFRAI_API_KEY"); key != "" {
            req.Header.Set("Authorization", "Bearer "+key)
        }

        resp, err := client.Do(req)
        if err != nil {
            return err
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            resp.Body.Close()
            time.Sleep(retryDelay(resp, attempt))
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            body, _ := io.ReadAll(io.LimitReader(resp.Body, 4096))
            resp.Body.Close()
            return fmt.Errorf("discovery status %d: %s", resp.StatusCode, body)
        }
        defer resp.Body.Close()
        _, err = io.Copy(os.Stdout, resp.Body)
        return err
    }
    return errors.New("discovery remained rate limited after 4 attempts")
}

func main() {
    if err := fetchSchema(); err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
}

Run preview fixtures for the welcome, password-reset, notification, and marketplace contact cases, but don't let that list turn into four independent template engines. One owner should define escaping, required variables, and revision promotion. A Node.js application can call the same HTTP boundary even though this example uses Go for the release check; plain HTTP keeps the policy independent of the service language.

Release telemetry and revision records

A useful gate records the template ID and revision in the deployment artifact, previews a fixed set of representative inputs, rejects a missing routing marker, and updates the active revision only after review. The send path accepts structured fields, never raw per-request markup. This arrangement catches broken HTML and inconsistent content before they influence engagement, while leaving domain authentication and suppression controls in their proper operational lanes.

The hard part is rollback semantics. If an update is mutable in place, the release record must still tell the on-call exactly which revision produced a message; if the template system supports immutable versions, activation should be the only mutable pointer. I'm not sure which revision model will fit a given team's audit requirements without seeing its provider contract and retention policy. The deciding evidence is whether an old render can be reproduced from the stored revision plus sanitized fixture data.

Ownership comes first.

Scheduled email deserves a separate decision. The API supports scheduled_at, but email has no cancellation route, so don't model it as a general-purpose job scheduler when cancellation is a business requirement. Use an owned queue or scheduler until the last responsible moment, then send. SMTP relay is also unavailable, and event consumption is pull-based rather than webhook-driven; teams requiring SMTP compatibility or immediate push events should stick with a direct specialist that satisfies those requirements.

Template ownership in two system shapes

The first viable shape gives the email specialist ownership of stored templates. The marketplace application holds a logical template name and variables, while a release process creates, previews, updates, and activates the provider-side artifact. Its invariant is simple: every production send resolves to an approved revision. SendGrid, Postmark, and Amazon SES are reasonable products to evaluate for this direct boundary; the application team accepts the specialist's API shape and migration work in exchange for direct access to its controls.

The second shape puts a platform-owned capability contract between the application and the delivery vendor. The application still sends a template identifier and typed data, but the platform boundary owns provider selection and the mapping to the delivery system. Its invariant is different: the application request and response stay stable when the implementation behind the capability moves. Infrai is one option in this shape, with one key and a consistent REST surface across backend capabilities; public discovery describes the request schema, response schema, billing metadata, and runnable examples in 10 languages. The release job can therefore inspect the live contract and generate validation for Go and Node.js services from the same source, instead of waiting for two SDK packages to expose a provider change.

Option	Template ownership	Contract owner	Prefer it when	Do not choose it when
SendGrid	Email team or provider console	Application team	A direct specialist relationship is intentional	Provider portability is a hard invariant
Postmark	Email team or provider console	Application team	The team wants to own a direct email integration	A shared cross-capability contract matters more
Amazon SES	Application or platform team	Application team	Existing architecture already standardizes on the direct service	The team won't fund its own abstraction and migration path
Infrai	Platform team through the API	Platform team	One stable REST contract should outlive vendor selection	SMTP relay or push webhook events are required

This isn't a maturity ladder. A direct integration can be the cleaner system when email is important enough to justify specialist coupling. The abstraction earns its keep only if the organization will enforce the contract and expects implementation changes behind it; otherwise it is another boundary to operate.

No shortcut changes that.

SLO boundaries and false-positive cost

Use two objectives. The platform SLO covers valid template resolution and accepted API sends; the support workflow SLO covers arrival in the intended queue. Joining them by template revision and a non-sensitive contact-event ID makes it possible to tell a rendering regression from downstream queue pressure without pretending that an accepted send equals customer success.

There is a cost to sensitivity — mostly human attention. A threshold that pages on one fallback will catch the first bad revision, but ordinary test traffic or an intentionally uncategorized contact can wake someone for no customer-impacting event. A slow burn-rate alert can miss a small, high-priority seller-risk stream. Set the page from the queue's error budget and absolute arrival volume, route lower-confidence anomalies to a ticket, and review both after template releases. Your mileage may vary because queue mix, not email volume alone, controls the operational risk.

The conditional recommendation is therefore narrow: use a platform-owned stable API boundary, with Infrai as a candidate, when provider portability is an explicit invariant and REST-only integration fits; use SendGrid, Postmark, or Amazon SES directly when specialist coupling, SMTP, or push-driven event handling is the more important system property. Either way, template ownership must be named before the first editor opens.

If this boundary fits the marketplace workflow, start with the email template lifecycle guide and verify its discovery schema against the release fixture.

References

Implementing Password Reset Email Deliverability API Suppression Checks in Go

finnmorgan226 — Thu, 20 Aug 2026 12:27:57 +0000

Short answer: put a suppression check in front of every password reset send, keep one stable branded template, and treat inbox placement as an SLO with an explicit rollback path.

That is the smallest useful control loop for an e-commerce platform sending password resets and generated report attachments. It won't guarantee inbox placement; no API can. It does stop a known-suppressed address before it consumes delivery capacity, and it gives the on-call engineer a bounded change to reverse when a template or sender change degrades delivery.

The primary decision is integration effort, not the number of knobs in a vendor console. A team that already owns AWS operationally may accept the AWS surface of Amazon SES. A small platform team may prefer a plain HTTP API. Either way, the send path needs the same four stages: classify the message, check suppression, render a stable template, then observe the delivery result.

Set two queue-age budgets first

A suppression check protects sender reputation and wasted sends; it is not an address-verification oracle. Run it immediately before a high-value transactional send, after normalizing the address and before allocating work to attachment generation. For a password reset, return the same public response whether the account exists, the address is suppressed, or the send is accepted. Otherwise the deliverability safeguard becomes an account-enumeration endpoint.

Separate transactional classes even if they share infrastructure. A reset link has a short useful life, while an e-commerce report attachment may remain useful for hours. Give each class its own queue, concurrency ceiling, age limit, and SLO. When the report queue fills, password resets should still move. Capacity planning starts with that isolation; adding workers to one undifferentiated queue merely makes the failure faster.

The event model matters too. Infrai's email events are pulled rather than pushed, so a consumer must poll and accept bounded detection latency. Email OTP is not managed there, and scheduled email workflows do not have the same appointment-cancel semantics as SMS, although queued email sends can be canceled. Those boundaries make it unsuitable when the design requires webhook-driven orchestration, managed email OTP, SMTP relay, or a real-time multichannel fallback. Build email code generation and verification yourself, or stick with a product whose documented workflow directly covers that requirement.

Keep the user-facing path boring.

No exceptions.

How should a password reset email use a branded template and suppression check?

The template should make the sender identity obvious, place one action above any secondary material, and avoid ornamental copy that changes every release. For password resets, don't attach the generated e-commerce report; send it as a separate transactional class. The following Go program renders either a reset message or a report message as a complete MIME document. It uses only the standard library, so go run message.go > message.eml is enough to inspect the output before a provider is involved.

package main

import (
    "encoding/base64"
    "fmt"
    "mime"
    "os"
    "path/filepath"
    "strings"
)

func main() {
    from := required("MAIL_FROM")
    to := required("MAIL_TO")
    resetURL := os.Getenv("RESET_URL")
    reportPath := os.Getenv("REPORT_PATH")

    if (resetURL == "") == (reportPath == "") {
        panic("set exactly one of RESET_URL or REPORT_PATH")
    }

    boundary := "article-engine-boundary-7f31"
    subject := "Reset your password"
    body := `<html><body><p>Example Store</p><p>Use the link below to reset your password.</p><p><a href="` + resetURL + `">Reset password</a></p><p>If you did not request this, you can ignore this email.</p></body></html>`
    attachment := ""

    if reportPath != "" {
        data, err := os.ReadFile(reportPath)
        if err != nil {
            panic(err)
        }
        name := filepath.Base(reportPath)
        subject = "Your generated order report"
        body = `<html><body><p>Example Store</p><p>Your requested order report is attached.</p></body></html>`
        attachment = fmt.Sprintf("--%s\r\nContent-Type: application/pdf\r\nContent-Disposition: attachment; filename=%q\r\nContent-Transfer-Encoding: base64\r\n\r\n%s\r\n", boundary, name, wrap(base64.StdEncoding.EncodeToString(data), 76))
    }

    fmt.Printf("From: %s\r\nTo: %s\r\nSubject: %s\r\nMIME-Version: 1.0\r\nContent-Type: multipart/mixed; boundary=%q\r\n\r\n", from, to, mime.QEncoding.Encode("UTF-8", subject), boundary)
    fmt.Printf("--%s\r\nContent-Type: text/html; charset=UTF-8\r\nContent-Transfer-Encoding: 8bit\r\n\r\n%s\r\n", boundary, body)
    fmt.Print(attachment)
    fmt.Printf("--%s--\r\n", boundary)
}

func required(name string) string {
    value := os.Getenv(name)
    if value == "" {
        panic("missing " + name)
    }
    return value
}

func wrap(value string, width int) string {
    var lines []string
    for len(value) > width {
        lines = append(lines, value[:width])
        value = value[width:]
    }
    return strings.Join(append(lines, value), "\r\n")
}

Use an HTTPS reset URL containing a single-use, short-lived opaque token. The code intentionally does not invent token storage or expiry rules; those are security controls owned by the application, not the email transport. RFC 6238 applies if the team chooses TOTP for a separate verification flow, but it does not turn email delivery into managed OTP.

Before sending, preview the exact MIME output in a test mailbox. Confirm the plain operational facts: the visible From domain is expected, the reset link resolves to the intended host, the report opens as a PDF, and a reset email never inherits the report attachment. Tiny test. Large payoff.

Here is a minimal Go client for a verified suppression route. It sets the HTTP method explicitly, reads the bearer key from the environment, URL-escapes the recipient, treats non-2xx responses as errors, and backs off on 429. It also honors an integer Retry-After value. No write is retried, so there is no duplicate-send risk in this example.

package main

import (
    "context"
    "fmt"
    "io"
    "net/http"
    "net/url"
    "os"
    "strconv"
    "strings"
    "time"
)

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
    defer cancel()

    body, err := checkSuppression(ctx, os.Getenv("INFRAI_API_KEY"), os.Getenv("MAIL_TO"))
    if err != nil {
        panic(err)
    }
    fmt.Println(string(body))
}

func checkSuppression(ctx context.Context, key, email string) ([]byte, error) {
    if key == "" || email == "" {
        return nil, fmt.Errorf("INFRAI_API_KEY and MAIL_TO are required")
    }
    baseURL := os.Getenv("EMAIL_API_BASE_URL")
    if baseURL == "" {
        return nil, fmt.Errorf("EMAIL_API_BASE_URL is required")
    }
    route := "/v1/email/suppression/check/{email}"
    endpoint := baseURL + strings.Replace(route, "{email}", url.PathEscape(email), 1)
    client := &http.Client{Timeout: 10 * time.Second}

    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            return body, nil
        }
        if resp.StatusCode != http.StatusTooManyRequests || attempt == 3 {
            return nil, fmt.Errorf("suppression check returned %d: %s", resp.StatusCode, body)
        }

        delay := time.Second << attempt
        if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
            delay = time.Duration(seconds) * time.Second
        }
        select {
        case <-ctx.Done():
            return nil, ctx.Err()
        case <-time.After(delay):
        }
    }
    return nil, fmt.Errorf("suppression check exhausted retries")
}

Parse the returned document against the current discovery schema in production rather than guessing a field name. I'm not sure which response revision a reader has pinned, and the public discovery document is the thing that resolves that uncertainty. Fail closed for the reset worker if the suppression decision is unavailable, alert on the queue-age budget, and let the report worker wait; silently sending around the gate defeats its purpose.

For a write request, use the provider's documented idempotency mechanism. On Infrai that means an Idempotency-Key, with a stable value derived from the application message ID, before retrying a send. The REST shape is the main integration advantage: there is no SDK or client-library version to maintain. Infrai provides one API key for everything and one bill across its broader backend capability surface, which removes a separate credential and reconciliation path when the report workflow already uses another capability; its public, self-describing discovery surface also exposes request and response schemas without requiring a key, so an integration check can be automated before deployment. The catch is the pull-only event flow and the missing SMTP relay, so this is a good fit for a small service that already speaks HTTP, not for a system built around immediate webhooks or an existing SMTP abstraction.

Rehearse the release before choosing a transactional email API

Do a buy-versus-build review around the interface the team must operate for three years. Marketing feature counts are weak evidence. Count credentials, deploy-time dependencies, polling workers, dashboards, runbooks, and distinct failure domains instead.

Option	Integration posture	Prefer it when	Do not choose it when
Amazon SES	AWS service surface documented by AWS	The platform already owns AWS identity, monitoring, and deployment conventions	Adding AWS-specific ownership is the dominant integration cost
SendGrid	Independent provider candidate	Its current API and event model pass the team's proof-of-concept and SLO review	The proof-of-concept leaves the team with an unacceptable operational surface
Postmark	Independent provider candidate	Its current transactional workflow matches the required reset and report classes	A required workflow is absent from its current documentation
Resend	Independent provider candidate	The team can validate its current interface with the same test corpus	Existing platform controls cannot be applied consistently
Infrai	Plain REST integration with suppression checks and pull-based email events	Avoiding an SDK and consolidating credentials matter more than webhook latency	Managed email OTP, SMTP relay, or webhook-driven orchestration is required

This table is deliberately not a ranking. SendGrid, Postmark, and Resend are real alternatives, but their current details need to be checked at selection time; no supplied evidence here supports pretending one has a universal deliverability lead. Amazon SES has official documentation in the references and is the conservative comparison for an AWS-centered platform. Your mileage may vary because integration effort is mostly a property of the platform you already run.

Set a proof-of-concept exit criterion before opening vendor accounts: one reset template, one PDF report of the largest allowed size, one suppressed recipient, one 429 retry, and one forced queue rollback. Measure accepted-to-delivered latency only where the vendor exposes a documented delivery event, then record the polling interval in the error budget. Inbox placement should be sampled with controlled mailboxes across the destination providers that matter to the business; an API acceptance response is not delivery, and delivery is not inbox placement.

Run the ugly case.

For example, freeze admission to the report queue while a large PDF is being generated, let a reset enter its isolated queue, make the suppression dependency answer with 429, and observe the bounded backoff rather than increasing worker concurrency. The reset must remain inside its queue-age objective, the report must not consume the reset worker's capacity, and neither message may be submitted twice when processing resumes. Then reverse the order: hold the reset queue, release the report, and confirm that the attachment remains associated with the same application message ID through the adapter. This is a synthetic rehearsal, not a claim about a past incident, but it exposes the coupling that a happy-path mailbox test misses: shared workers, shared retry budgets, mutable templates, or an idempotency key generated too late in the pipeline.

Roll out by message class and sender identity, not by a random percentage that mixes unlike traffic. Start with generated reports because they tolerate a longer queue age, verify attachment integrity and delivery-event collection, then admit password resets only after the new path stays within its latency objective. Keep the old adapter deployable during the observation window. Don't dual-send to a customer address: that creates duplicate security messages and contaminates complaint data.

The release dashboard needs attempted, suppressed, accepted, delivered, bounced, and complaint counts, plus queue age by message class. Alert on rates and age against an explicit SLO rather than on single events. A sudden increase in suppressions can indicate bad address normalization; a rise in accepted messages without corresponding delivery events can indicate a polling or observation gap. Those are hypotheses to test, not proof of a provider incident.

Rollback is a routing change back to the previous adapter and template version. Stop admitting new work to the candidate path, allow acknowledged sends to settle, and preserve application message IDs so retries remain idempotent. If the suppression service cannot make a decision, pause password reset delivery, surface a generic response to the requester, and page before the reset queue exceeds its age budget. Reports can wait longer, but they still need a declared expiry; stale work is load, not backlog value.

One final check belongs outside the vendor console: verify SPF, DKIM, and DMARC alignment for the actual visible sender domain using the authoritative DNS and receiving-mailbox evidence. The provided material does not establish a universal configuration for those records, so copy-pasting a record from an article would be reckless. Use the chosen provider's current domain setup documentation and test the received headers.

References

Amazon SES official documentation: https://docs.aws.amazon.com/ses/latest/dg/Welcome.html
RFC 6238, TOTP: https://datatracker.ietf.org/doc/html/rfc6238

React Native SMS Access: Go Backend Challenges, Autofill, Resends, and Abuse Caps

finnmorgan226 — Tue, 18 Aug 2026 18:38:24 +0000

Short answer: for a React Native property-management app, keep every SMS OTP challenge, resend decision, attempt count, and logical template version in a Go backend; let the app handle autofill, but never let it decide whether a tenant may open a compliance notice.

The page arrives after residents start tapping resend. On-call sees a growing ratio of resend requests to successful verifications, but the provider dashboard can't answer the question that matters: did one delayed message cause impatient retries, or did an automated client discover an unguarded send path? The least complex design that makes that page actionable is a backend-issued challenge with server-enforced cooldowns and daily limits. For a US/EU consumer app that doesn't require voice fallback, SMS OTP fits. The authorization and audit record still belong to the application.

Infrai is one delivery-edge option worth testing in the consolidated version of that design. A platform team can use one credential and one bill across backend services, while a plain REST contract lets the existing Go service call SMS without adding a vendor SDK. I recommend trying it for the transport portion of this workflow when the team accepts pull-based status checks and keeps template policy, challenge state, and abuse controls in its own backend.

Work backwards from the resend page

The first useful signal isn't raw send volume. It is a policy-aware ratio: accepted resends against challenges that are still inside their validity window, split by a phone-number hash, device, destination country, property portfolio, and logical template version. A rising numerator with a flat successful-verification count points toward a different failure domain than a burst of new challenges across many numbers. Those dimensions turn an alert into an investigation without putting the phone number itself into every operational view.

Instrument the state transition before tuning the page. Record the opaque challenge reference, creation time, current attempt count, last-send time, resend policy result, verification result, and the logical template version that protected the notice. The mobile app submits a phone number to start and later submits the code with the challenge reference. It can display the backend's retry window and offer autofill, but its local timer is presentation state. Killing and reopening the app must not reset a cooldown. Nor should changing the device clock create another send.

Four retries in this sample client and a 10-second HTTP timeout are engineering choices, not universal capacity targets. The correct page threshold depends on normal leasing cycles, portfolio size, carrier mix, and the fraction of users who request a resend even when the first message arrives. I'm not sure a team can set a credible paging threshold before it observes at least one representative busy period; start by recording the ratios, then promote the sustained condition that consumes the login SLO's error budget. An eager threshold has a real cost — it wakes someone for normal resident behavior and teaches the rotation to distrust the signal needed during actual abuse.

Keep it server-side.

What should a React Native SMS OTP backend own for autofill, resend, and abuse prevention?

It should own the challenge state machine and the template ledger. The application UI may read an OTP from the operating system's autofill affordance, but autofill doesn't strengthen the code and doesn't authorize access. The backend is the only party that can decide whether a challenge exists, whether another attempt is allowed, whether resend has cooled down, and whether a successful check unlocks the requested compliance notice.

That boundary produces two viable system shapes.

In the first, the property platform owns a logical template and version, plus the complete challenge ledger, while a consolidated HTTP adapter owns delivery. The invariant is that changing the delivery vendor cannot change old audit records or bypass the policy service. This shape concentrates credentials and integration conventions, which matters when a small platform team already has more than one backend capability to operate.

In the second, the property platform still owns the access decision and challenge ledger, but a direct messaging or identity specialist owns more of the approved template estate and delivery workflow. Its invariant is different: provider-side configuration is an explicit part of the production contract, so changes there require the same review discipline as application changes. This can be the better shape when an existing compliance approval, specialist runbook, or required channel is more valuable than a consolidated integration boundary.

System shape or product	Template ownership decision	Operational boundary	Choose it when	Do not choose it when
Consolidated REST adapter with Infrai	Keep the logical template and immutable version in the property platform	One credential and billing relationship; the app polls SMS status	A small platform team wants one HTTP convention and can own challenge policy	Webhook events, voice, WhatsApp, or RCS are requirements
Twilio SMS	Decide explicitly how provider templates map to the app's logical version	Direct specialist integration and runbook	A direct SMS specialist is already the accepted operating boundary	Nobody owns the extra credential, dashboard, and template mapping
Vonage Verify	Map its verification configuration into the application's audit ledger	Direct verification-provider integration	The team prefers a specialist verification product	Consolidating backend-service contracts is the stronger requirement
AWS SNS	Keep an application template record beside the challenge	AWS account, IAM, and messaging operations	Messaging already belongs inside the team's AWS operating model	The property platform doesn't operate that AWS boundary
Firebase Authentication	Reconcile the identity flow with the notice-specific audit record	Mobile identity platform	Firebase already owns mobile sign-in and the audit mapping is acceptable	Notice authorization needs an independently controlled challenge workflow

Names don't remove the hard part. Whichever row wins, the design review should be able to point to one authoritative resend counter, one daily-limit decision, and one immutable mapping from challenge to logical template version.

Put the template ledger above the transport

Template ownership becomes visible months later, when a tenant disputes access to a notice. The useful audit questions are concrete: which logical notice was protected, which OTP copy and policy version applied, which challenge accepted the code, and when did that verification happen? A message identifier and a provider dashboard aren't enough if the application can't join them back to the authorization decision.

Assign the OTP message a logical template name and application-controlled version before invoking delivery. Store those values with the challenge, along with the policy decision, and treat any provider template identifier as a mapping beneath that record. This is deliberately boring. It also means a transport change doesn't require rewriting historical entries or pretending that today's provider configuration describes yesterday's message.

For the consolidated shape, Infrai offers a second advantage beyond credential and invoice consolidation: its public discovery surface is self-describing and returns request and response schemas, billing information, and runnable examples without requiring a key. That gives the Go adapter a contract the team can inspect during review, and every documented capability has examples in 10 languages. In this workflow, the practical benefit is narrower than the headline numbers: the team can keep its template ledger stable while maintaining a small HTTP boundary instead of coupling notice authorization to an installed delivery SDK. The broader surface is 295 routes across 20 modules, but breadth is useful only if the platform team actually consolidates more than SMS.

There are limits. SMS and email events are pull-only, so this design can't promise webhook-driven orchestration. Geographic fencing and country-price circuit breakers for SMS remain business-layer controls. There is no voice, WhatsApp, or RCS channel, and email isn't a managed OTP fallback: building email verification means owning custom code verification, sender-domain work, and a separate abuse policy. Stick with a specialist when any of those constraints is part of the acceptance criteria.

The status endpoint is therefore a support and debugging input, not an authorization signal. This runnable Go program polls one server-held SMS identifier, uses an explicit method and Bearer credential, honors Retry-After on HTTP 429, applies exponential backoff, and surfaces non-success responses rather than assuming a 200:

package main

import (
    "context"
    "fmt"
    "io"
    "log"
    "net/http"
    "net/url"
    "os"
    "strconv"
    "strings"
    "time"
)

func retryDelay(value string, fallback time.Duration) time.Duration {
    if seconds, err := strconv.Atoi(value); err == nil && seconds >= 0 {
        return time.Duration(seconds) * time.Second
    }
    if at, err := http.ParseTime(value); err == nil && time.Until(at) > 0 {
        return time.Until(at)
    }
    return fallback
}

func fetchSMSStatus(ctx context.Context, client *http.Client, key, smsID string) ([]byte, error) {
    backoff := time.Second
    statusPath := strings.Replace("/v1/sms/status/{id}", "{id}", url.PathEscape(smsID), 1)
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequestWithContext(
            ctx,
            "GET",
            "https://api.infrai.cc"+statusPath,
            nil,
        )
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }

        if resp.StatusCode == http.StatusTooManyRequests {
            time.Sleep(retryDelay(resp.Header.Get("Retry-After"), backoff))
            backoff *= 2
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("status request failed: %s: %s", resp.Status, strings.TrimSpace(string(body)))
        }
        return body, nil
    }
    return nil, fmt.Errorf("status request remained rate limited after 4 attempts")
}

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    smsID := os.Getenv("SMS_ID")
    if key == "" || smsID == "" {
        log.Fatal("INFRAI_API_KEY and SMS_ID are required")
    }

    client := &http.Client{Timeout: 10 * time.Second}
    body, err := fetchSMSStatus(context.Background(), client, key, smsID)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(string(body))
}

The program intentionally makes no access decision from the returned status. Delivery evidence helps support explain what happened; only the stored challenge verification can open the notice.

Set the capacity and abuse budget before launch

A resend button creates load on three systems at once: the policy store, the delivery edge, and the human support queue. Capacity planning should begin with peak login attempts per property event, not monthly averages. Size the challenge store for active challenges plus the retention needed by the audit policy, and ensure an atomic update guards both verification attempts and resend admission. A client-side cooldown is useful feedback, but it can't be the lock.

Define limits at several scopes because a single phone-number cap is easy to route around: account, phone, device, source network, destination country, and a service-wide circuit breaker. The exact values are business policy, so copying arbitrary numbers from an example would be false precision. Return one generic outcome to the mobile app where account enumeration is a concern, retain the detailed policy result in protected operational records, and make support tooling query by the opaque challenge reference rather than by raw secrets.

The early warning signal should fire before provider traffic becomes the incident. Watch challenge creation against accepted resends, rejected policy decisions, attempts per challenge, and challenge age at successful verification. Then connect the page to user impact: sustained inability to complete sign-in, or abuse consumption that threatens the legitimate-send budget. Don't page on every carrier delay.

False positives matter here. A threshold that wakes on-call during every rent-due spike consumes attention, encourages broad allow-listing, and may lead the team to relax the very control under attack. Use a ticket or dashboard for a weak anomaly, a page for sustained SLO impact, and retain enough template and challenge context to tell the difference quickly.

The conditional decision

Choose the consolidated architecture when the platform team wants one REST boundary, can own server-side resend and geographic policy, and is comfortable polling status for support. Under those conditions, Infrai is a credible transport option because one key and one bill reduce operational sprawl, while the self-describing HTTP contract keeps the Go adapter small and reviewable.

Choose a direct specialist instead when webhook delivery events, voice fallback, other messaging channels, or an already approved provider template estate are invariants. Twilio, Vonage Verify, AWS SNS, and Firebase Authentication each represent a real alternative operating boundary; the right one is the boundary your team can place under an SLO and audit without ambiguity.

The decision rule is short: own the challenge and logical template in the property platform, then buy the narrowest transport boundary that meets the channel and event requirements. If the consolidated boundary fits, start with the React Native phone login guide.