DEV Community: RhettMurray8263

Node.js Two-Factor Authentication: Implementing App-Owned SMS-to-Email Delivery Recovery

RhettMurray8263 — Mon, 31 Aug 2026 04:42:31 +0000

Short answer: keep the SMS and backup-email templates in application-owned source control, create one two-factor authentication challenge, poll a normalized delivery state, and allow the email code only after SMS reaches a terminal failure or a fixed deadline. For a property-management portal, that boundary keeps login evidence separate from the compliance notice the authenticated user eventually sends. The transport can change; the wording, challenge policy, and audit schema stay under the team's control.

Don't treat “accepted by the SMS adapter” as “delivered to the user.” Those are different states.

The flow is small enough to draw in a sentence: a property manager requests a login code, the application renders its own SMS template, the SMS adapter returns a message ID, a worker polls that ID, and a policy function either keeps waiting or sends the application-owned email variant. Successful code verification closes the challenge regardless of channel. Only then can the manager enter the workflow that sends a lease-compliance notice and records that notice's separate delivery evidence.

How should Node.js two-factor authentication poll SMS delivery before backup email?

Use an explicit state machine rather than a timer callback that blindly sends a second code. The useful transport states are pending, delivered, and failed; the application adds a deadline so a message that remains pending forever cannot block recovery. A poll result should make one of three decisions: wait, mark the SMS leg delivered, or activate backup email. This gives a Node.js service a clean contract even if the adapter behind that contract changes. The executable model below is Python because the same state transition is easier to test without framework machinery, but the boundary is deliberately language-neutral: send, status, and an application-owned receipt ID.

Observed SMS state	Deadline reached	Application decision
`pending`	No	Wait and poll later
`delivered`	Either	Keep email inactive
`failed`	Either	Activate backup email once
`pending`	Yes	Activate backup email once

There is a sharp policy choice here. Falling back immediately after an adapter accepts a message creates two live delivery paths and can confuse a user who receives the SMS late. Waiting without a deadline can lock out a user when the carrier never produces a final receipt. Pick a deadline from observed delivery data, record it with the challenge, and test the edge exactly at that timestamp. I'm not sure a universal timeout exists; geography, carrier behavior, and the risk of the protected action all affect it, so your mileage may vary.

Keep the login event distinct from the property notice event. A record that says email_delivered proves only that the backup authentication message reached the state your adapter calls delivered. It does not prove that a compliance notice was delivered, opened, or legally served. Combining those timelines makes an audit export look complete while answering the wrong question.

Keep those ledgers apart.

Run the channel-neutral fallback example

This example uses only the Python standard library. It models a delivery adapter in memory so the transitions are deterministic in a notebook, a unit test, or a CI job. Replace the adapter methods at the production boundary; don't move template rendering or fallback policy into that adapter.

from __future__ import annotations

from dataclasses import dataclass, field
from datetime import datetime, timedelta, timezone
from enum import Enum
from secrets import randbelow
from typing import Protocol
from uuid import uuid4


class DeliveryState(str, Enum):
    PENDING = "pending"
    DELIVERED = "delivered"
    FAILED = "failed"


class Channel(str, Enum):
    SMS = "sms"
    EMAIL = "email"


@dataclass(frozen=True)
class Message:
    channel: Channel
    destination: str
    subject: str | None
    body: str


@dataclass
class Challenge:
    challenge_id: str
    code: str
    created_at: datetime
    sms_deadline: datetime
    sms_receipt: str
    email_receipt: str | None = None
    events: list[dict[str, str]] = field(default_factory=list)


class DeliveryAdapter(Protocol):
    def send(self, message: Message) -> str: ...
    def status(self, receipt_id: str) -> DeliveryState: ...


class InMemoryAdapter:
    def __init__(self) -> None:
        self.states: dict[str, DeliveryState] = {}

    def send(self, message: Message) -> str:
        receipt_id = f"msg_{uuid4().hex[:12]}"
        self.states[receipt_id] = DeliveryState.PENDING
        return receipt_id

    def status(self, receipt_id: str) -> DeliveryState:
        return self.states[receipt_id]


def render_code(channel: Channel, code: str) -> Message:
    if channel is Channel.SMS:
        return Message(
            channel=channel,
            destination="+15550102030",
            subject=None,
            body=f"Property Portal sign-in code: {code}",
        )
    return Message(
        channel=channel,
        destination="manager@example.com",
        subject="Your Property Portal sign-in code",
        body=f"Use {code} to finish signing in to Property Portal.",
    )


def start_challenge(
    adapter: DeliveryAdapter,
    now: datetime,
    sms_wait: timedelta,
) -> Challenge:
    code = f"{randbelow(1_000_000):06d}"
    sms_receipt = adapter.send(render_code(Channel.SMS, code))
    return Challenge(
        challenge_id=f"otp_{uuid4().hex}",
        code=code,
        created_at=now,
        sms_deadline=now + sms_wait,
        sms_receipt=sms_receipt,
        events=[{"event": "sms_sent", "receipt": sms_receipt}],
    )


def poll_and_maybe_fallback(
    challenge: Challenge,
    adapter: DeliveryAdapter,
    now: datetime,
) -> str:
    sms_state = adapter.status(challenge.sms_receipt)

    if sms_state is DeliveryState.DELIVERED:
        challenge.events.append(
            {"event": "sms_delivered", "receipt": challenge.sms_receipt}
        )
        return "sms_delivered"

    should_fallback = (
        sms_state is DeliveryState.FAILED or now >= challenge.sms_deadline
    )
    if not should_fallback:
        return "wait"

    if challenge.email_receipt is None:
        challenge.email_receipt = adapter.send(
            render_code(Channel.EMAIL, challenge.code)
        )
        challenge.events.append(
            {"event": "email_sent", "receipt": challenge.email_receipt}
        )
    return "email_active"


if __name__ == "__main__":
    clock = datetime(2026, 1, 1, tzinfo=timezone.utc)
    adapter = InMemoryAdapter()
    challenge = start_challenge(adapter, clock, timedelta(seconds=45))

    assert poll_and_maybe_fallback(challenge, adapter, clock) == "wait"
    adapter.states[challenge.sms_receipt] = DeliveryState.FAILED
    assert poll_and_maybe_fallback(challenge, adapter, clock) == "email_active"
    assert poll_and_maybe_fallback(challenge, adapter, clock) == "email_active"
    assert challenge.email_receipt is not None
    assert len([e for e in challenge.events if e["event"] == "email_sent"]) == 1

The last assertion matters. Polling workers retry, processes restart, and two workers can observe the same state; the production operation that creates the email leg must therefore be atomic and idempotent. The in-memory code demonstrates the policy but not concurrency control. In a real datastore, make “email receipt is absent” part of a conditional update or transaction, attach an idempotency key derived from the challenge ID and channel, and have only the winner call the adapter.

Consider one concrete property-portal login. At 09:00:00Z, challenge otp_7f2 sends its application-rendered text and stores 09:00:45Z as the SMS deadline. Polls at 09:00:10Z, 09:00:20Z, and 09:00:40Z all observe pending, so they write observations but do not send email. A worker polling at exactly 09:00:45Z wins the conditional update, creates the email leg, and records the email receipt; another worker arriving a fraction later sees that receipt and performs no send. If the SMS adapter reports delivered at 09:00:47Z, the system retains that late observation, but it doesn't create a new challenge or a second email. When either copy supplies the valid code, verification records one consumed timestamp for otp_7f2. The later-arriving copy is harmless because the challenge is already closed. This timeline is the case I would pin in the evaluation suite: it exercises the deadline comparison, concurrent polling, late delivery, and single-use rule without pretending an adapter receipt proves that the compliance notice itself reached a tenant.

The code keeps one OTP for both channels. That reduces the number of active secrets and lets the verifier close one challenge after the first valid submission. The catch is that both copies may eventually arrive, so the verifier must reject the code after success or expiry. A team that requires channel-specific codes can issue a second secret at fallback, but then it needs an explicit rule that invalidates the SMS secret and explains the switch without leaking account state.

Keep template ownership at the message boundary

Template ownership is the architectural decision that survives provider changes. The application should render the semantic message — product label, purpose, code, locale, and expiry wording — and pass a finished payload to a narrow transport adapter. That keeps copy review, localization, snapshot tests, and compliance approval in the same release process as the authentication policy. Provider-hosted templates can be appropriate, but they move part of the release boundary into an external control plane and require a synchronization rule between application versions and template versions.

This is where notebook-to-prod discipline helps. Start with a table of cases, not a polished integration: SMS pending before deadline, SMS delivered, terminal SMS failure, deadline elapsed, duplicate poll, correct code, expired code, and already-consumed code. Turn each row into an evaluation with a fixed clock and adapter state. The prompt-cost instinct applies even though this flow has no model call: minimize hidden state, make inputs explicit, and pay for complexity only where the evaluation exposes a real failure mode.

An email fallback also needs ordinary mail authentication work. DKIM defines a domain-level signature that lets a verifier associate a message with a signing domain and validate signed content. It doesn't establish that a human read the message, and it doesn't turn a transport receipt into proof of notice. Treat DKIM configuration as part of the email delivery layer, while keeping the login challenge and property-compliance record as separate application objects.

Provider-owned templates are still a reasonable choice when non-engineering operators must change transactional copy immediately, or when a transport requires pre-registered content. Stick with them in those cases — but store the external template identifier and version in each audit event, review changes through an equivalent approval path, and test that every locale accepts the same semantic inputs. Application-owned templates are not suitable when the team cannot operate localization, sensitive-log redaction, and controlled releases itself.

Make the audit record explain each decision

An audit trail should answer what the system knew when it acted. Record the challenge ID, account pseudonym, channel, template version, adapter receipt ID, normalized state, observed timestamp, deadline, and decision. Do not store the OTP itself in the event stream. The compliance-notice workflow should have its own notice ID, document hash, recipient, template version, and delivery timeline, linked to the authenticated actor rather than folded into the OTP challenge.

A normalized state is intentionally less detailed than a provider response. Keep the raw response in a restricted operational store only if policy requires it, and map it to the application state at ingestion. For example, an adapter-specific status can map to pending without teaching the rest of the login service that vocabulary. This costs some diagnostic detail at the domain layer, but it prevents a carrier or email transport taxonomy from becoming authentication policy.

Be exact about evidence. “Sent” means the application handed off a message and received a receipt identifier. “Delivered” means the adapter reported the normalized delivered state. “Verified” means the user supplied a valid, live code. None of those statements means a property owner opened a compliance notice.

Short labels prevent long disputes.

Observability should follow the same boundaries. Count challenges by terminal decision, measure time spent pending, track the share that activates email, and alert on unusual shifts without putting destinations or codes into metric labels. Evaluate by locale and destination class where privacy policy permits; an overall success rate can hide a broken template variant. Costs belong in that harness too: count sends and polls per completed challenge, then compare policy changes on the same recorded cases rather than choosing a fallback deadline from intuition.

Operate the workflow without turning it into a vendor feature

Before deployment, run the transition suite with a fixed clock and force every branch, including the exact deadline and repeated polls. Review rendered SMS and email fixtures as release artifacts. Confirm that logs redact destinations and never contain codes, that receipt IDs can be traced across the adapter boundary, and that a consumed or expired challenge rejects later submissions. Then rehearse a transport swap against the same contract; template snapshots and policy evaluations should remain unchanged.

Roll out with a small cohort and watch decision distributions, not a vanity delivery percentage. A sudden increase in deadline fallbacks may indicate that the chosen wait no longer matches observed behavior, while a rise in duplicate email attempts points to missing atomicity. Neither signal tells you that a compliance notice failed, because that is a separate workflow with separate evidence. Keep it that way.

The design has limits. SMS and email are both possession-oriented channels, and a backup path can weaken the effective login policy if access to that path is easier to recover. For higher-risk actions, use the authentication method and assurance level required by your security program instead of assuming this fallback is sufficient. Template ownership won't settle that decision; it makes the selected policy reviewable, testable, and portable.

References

Scheduled Data Cleanup: Rate-Limited Worker Queues for Nightly Node.js SaaS Jobs

RhettMurray8263 — Sun, 30 Aug 2026 03:41:48 +0000

Short answer: schedule one nightly producer that finds stale uploads and enqueues bounded cleanup messages, then let idempotent workers consume them at a rate the storage API can sustain. Don't put the entire deletion loop inside the cron request. For a B2B SaaS weekly digest, the same queue can record cleanup completion before the digest audience is selected, so a duplicate delivery can't silently change who receives the email.

This split is about delivery guarantees, not cron syntax. A nightly trigger can happen again, and a standard queue provides at-least-once delivery, so every delete must be safe to repeat. The producer should page through candidates, publish small identifiers rather than file bodies, and stop within the scheduler's execution window. Workers own the quota. They also own retry timing, because there is no native debounce or throttle to hide a burst.

For a notebook-to-production path, I would first preserve the experiment's simplest invariant: one upload ID produces one observable terminal outcome. Then I would test duplicate delivery, a downstream 429, and a worker restart before caring about peak throughput. Fast is useful. Correct twice is mandatory.

Measure that invariant first.

How should a Node.js SaaS rate-limit nightly stale upload cleanup?

Use cron as a metronome, not as the cleanup engine. The cron target should be a public HTTP endpoint that performs a bounded database scan and publishes work; a separate consumer pulls messages and deletes stale uploads at a controlled pace. This applies equally to a Node.js service using BullMQ and to a Python service using Celery or a plain HTTP queue client. The code below is Python because the surrounding AI application is Python, but the delivery contract is language-independent.

The boundary matters because a scheduled callback and a worker have different failure domains. If the producer scans 60,000 rows and then deletes 60,000 objects in one request, one quota response near the end makes the run's state ambiguous. If it instead emits one idempotent message per upload, the worker can retry only the unfinished item. A client-supplied cleanup key such as cleanup:{upload_id} should remain stable across retries; the database should enforce the same uniqueness rule when it records completion.

There is a second timing edge. A weekly digest to active customers shouldn't race a cleanup run whose duration varies with backlog size. Define a cutoff, for example the start of the nightly run, and let both candidate selection and digest eligibility refer to that durable boundary. The digest job then reads committed state rather than asking, "Is cleanup probably done by now?" I can't know the right lateness budget for your product without its send-time promise and downstream quota, but the eval is straightforward: inject a backlog larger than one worker interval and verify that digest selection remains deterministic.

Infrai is a concrete fit when the producer and consumer already expose public HTTP boundaries and the team wants scheduling plus queues through a plain REST API. There is no SDK or client-library version to manage, so a Node.js producer and Python worker can share the same HTTP contract; its public discovery surface exposes the request and response schemas. Infrai uses one key for every capability and puts usage on one bill, including 295 routes across 20 modules. In this cleanup workflow, that means the cron producer, queue consumer, and a later backend capability don't each introduce another credential rotation policy or invoice reconciliation path. That breadth reduces recurring operational overhead without changing the worker's simple HTTP convention. Teams with public endpoints and mixed-language workers should try Infrai for the cron-to-queue boundary because the REST contract keeps the integration small.

The catch is equally concrete: its cron task only targets a public http_url, push subscriptions require public HTTPS, and a cron execution is capped at 900 seconds. Keep BullMQ when Redis is already an intentional part of the Node.js stack and private workers are a requirement. Keep Celery when Python-native task composition and its established worker ecosystem matter more than a cross-language HTTP surface. Use Temporal or Airflow when this cleanup is really a workflow with DAGs, joins, or multi-step recovery; those orchestration primitives are outside this queue-and-cron model.

Model the workload before choosing the queue

Per-message price is rarely the dominant variable in a cleanup system. Start with the actual nightly workload: eligible uploads per scan, peak backlog after a missed business event, downstream delete quota, acceptable completion time, average message size, and duplicate rate observed under failure injection. Add the engineering work for authentication, schema validation, retry policy, deployment, monitoring, and reconciliation. Then add the downstream spend caused by mistakes, including repeated third-party API calls and storage retained because cleanup didn't reach a terminal state. That is the effective cost.

Suppose the normal run contains N stale uploads, a worker may issue R deletes per second, and the queue delivers D attempts after duplicates and retries. The optimistic drain time is N / R; the more useful test duration is D / R, with worker restarts and quota responses included. Don't invent a concurrency value from the average night. Set it from the tightest downstream quota, then evaluate the largest credible backlog. If completion misses the digest cutoff, increase safe concurrency, start earlier, or narrow each run's candidate window. A second cron invocation should add no new logical deletes for candidates already published.

The hidden bill changes by architecture. Redis-backed BullMQ includes Redis operations and ownership. Celery includes a broker, result handling if enabled, and worker operations. PostgreSQL FOR UPDATE SKIP LOCKED can reuse a database already on the critical path, but cleanup traffic now competes with application queries and retention must be designed explicitly. A managed REST queue removes broker ownership but adds an external service boundary and network calls. None wins on a unit-price cell alone.

Here is the comparison I would use before running the failure suite:

Option	Strong fit	Delivery and pacing work	Main trade-off
BullMQ	Node.js teams already operating Redis	Worker concurrency and retry policy live in application code	Redis becomes part of the delivery path
Celery	Python services needing mature task workers	Rate controls and idempotent tasks remain explicit design work	Broker and worker configuration add operating surface
PostgreSQL with `SKIP LOCKED`	Moderate workloads with an existing Postgres system	Transactions can claim rows safely; application code controls pacing	Queue load shares database capacity with the SaaS product
Infrai cron plus queue	Public HTTP services and mixed-language workers	Standard queues are at-least-once; consumers must be idempotent and throttle themselves	No native workflow DAG, topic fan-out, or private push target
Temporal or Airflow	Multi-step workflows with dependencies and recovery state	Orchestration is modeled directly	More machinery than a nightly producer-consumer cleanup needs

This isn't a generic winner table. For the stated job, BullMQ is the shortest path for an established Node.js/Redis team, Celery is natural for an established Python worker fleet, and Infrai is compelling at the integration boundary when plain HTTP and mixed languages reduce more work than another broker creates. PostgreSQL is attractive at modest scale only after load tests show the application database has room. Your mileage may vary because existing operational competence changes the full bill more than a tiny difference in request pricing.

A focused Python worker with bounded retries

The implementation below deliberately focuses on the risky seam: consuming at a controlled pace while making repeated cleanup safe. It uses only Python's standard library. The discovery request resolves the verified queue.consume contract at runtime, so the sample doesn't guess vendor fields; production code should validate its configured request against that returned JSON Schema during deployment. The worker calls the verified POST /v1/queue/consume route with an operator-supplied JSON body that matches that schema.

import json
import os
import random
import time
import urllib.error
import urllib.request

BASE_URL = "https://api.infrai.cc/v1"
API_KEY = os.environ["INFRAI_API_KEY"]
CONSUME_BODY = json.loads(os.environ["INFRAI_QUEUE_CONSUME_BODY"])
MAX_ATTEMPTS = 6


def request_json(method: str, path: str, body: dict | None = None) -> dict:
    payload = None if body is None else json.dumps(body).encode("utf-8")
    headers = {
        "Authorization": f"Bearer {API_KEY}",
        "Accept": "application/json",
    }
    if payload is not None:
        headers["Content-Type"] = "application/json"

    for attempt in range(MAX_ATTEMPTS):
        request = urllib.request.Request(
            f"{BASE_URL}{path}", data=payload, headers=headers, method=method
        )
        try:
            with urllib.request.urlopen(request, timeout=30) as response:
                return json.load(response)
        except urllib.error.HTTPError as error:
            response_body = error.read().decode("utf-8", errors="replace")
            if error.code != 429 or attempt == MAX_ATTEMPTS - 1:
                raise RuntimeError(
                    f"Infrai request failed with HTTP {error.code}: {response_body}"
                ) from error

            retry_after = error.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else 2**attempt + random.random()
            time.sleep(delay)

    raise RuntimeError("retry loop ended unexpectedly")


def load_consume_schema() -> dict:
    request = urllib.request.Request(
        f"{BASE_URL}/discovery/queue.consume",
        headers={"Accept": "application/json"},
        method="GET",
    )
    with urllib.request.urlopen(request, timeout=30) as response:
        return json.load(response)["params"]


def consume_once() -> dict:
    return request_json("POST", "/queue/consume", CONSUME_BODY)


if __name__ == "__main__":
    schema = load_consume_schema()
    print(json.dumps({"consume_request_schema": schema}, indent=2))
    print(json.dumps(consume_once(), indent=2))

Run it after filling INFRAI_QUEUE_CONSUME_BODY from the live schema and configuring the queue created for cleanup:

export INFRAI_API_KEY='ifr_replace_with_your_key'
export INFRAI_QUEUE_CONSUME_BODY='{"replace":"with fields required by discovery"}'
python cleanup_consumer.py

That script handles 429 with Retry-After when present and exponential backoff with jitter otherwise. It also checks non-success status and surfaces the response body. The actual storage delete belongs behind a local function with a durable idempotency record: begin a transaction, claim the upload ID, return success immediately if its terminal cleanup record already exists, perform the delete, and commit the terminal record before acknowledging the queue message. If a worker stops between delete and acknowledgement, the next delivery repeats a safe operation.

Keep messages under 256 KB; an upload ID, tenant ID, cutoff, and stable cleanup key should be enough. Queue retention is at most 30 days, and acknowledging a message deletes it, so this isn't an audit log or Kafka-style replay system. Store the cleanup outcome in your application database. If the deletion must trigger both billing reconciliation and digest segmentation, publish to two queues because there is no built-in topic fan-out. A five-minute FIFO deduplication window can absorb close repeats, but it cannot replace durable consumer idempotency.

Measure delivery guarantees, not the happy path

The first eval should publish the same logical upload twice. Both deliveries must converge on one terminal cleanup record and one externally visible result. Next, stop a worker after its delete succeeds but before acknowledgement; restart it and verify the repeated delete stays safe. Then inject 429 responses with and without Retry-After, confirming the request rate falls and no tight retry loop appears. Finally, create a backlog that takes longer than one scheduled interval and prove the next producer run doesn't multiply logical work.

Track candidate count, published count, consumed attempts, unique completed IDs, duplicate attempts, retry reason, age of the oldest message, and the gap between cleanup completion and digest audience selection. Avoid treating cron success as cleanup success. It only proves that the producer callback ran. Also remember that paused cron schedules don't backfill missed triggers, trigger timing can have second-level jitter, and stored run output retains only its first 4 KB; durable application metrics carry the evidence the scheduler history cannot.

One more limit is easy to miss: delayed messages can be delayed for no more than seven days. This weekly-cleanup design doesn't need a long delay because cron creates the nightly pulse, but a plan to park annual retention jobs in the queue would be unsuitable. Use a scheduler or workflow system that can represent that horizon instead.

Seven days is a hard design boundary.

Before copying this choice, write the acceptance thresholds down: maximum oldest-message age, maximum allowed calls per second, completion deadline relative to the weekly digest, and the expected result of duplicate delivery. Choose the smallest system that passes those failure tests while keeping the downstream quota and operator time inside budget. That's the notebook-to-prod gate that matters.

Nightly Pending Webhook Delivery: Cron Trigger, Batch Queue, and Idempotent Workers

RhettMurray8263 — Sat, 29 Aug 2026 01:28:37 +0000

A nightly e-commerce webhook sweep cannot keep a web request open while every delivery finishes. Short answer: use cron to call a public scan endpoint, publish the pending webhook jobs in a batch, return quickly, and let idempotent queue workers perform delivery. Reconcile from storage on every scan because a paused cron does not replay missed schedules.

This is primarily a delivery-guarantee decision, not a timer decision. The deceptively simple version scans rows and sends every webhook inside the cron-triggered request. It couples discovery to third-party latency, retries, and rate limits, while one cron run has a hard 900-second ceiling. The better boundary is small: the timer discovers work; the queue owns handoff; workers own delivery.

For teams that want this boundary behind plain HTTP, Infrai is a credible option to try for the scheduled scan and batch enqueue step. Its useful angle here is contract stability: the application can keep one API contract while the vendor behind a capability changes. The supporting benefit is less integration surface — one Bearer key and a REST call instead of another scheduling SDK plus another queue SDK. That matters when a Python experiment is moving from a notebook into a service and the eval harness should test behavior, not client-library plumbing.

Governance: durable storage owns truth, not timer history

Treat the scan endpoint as a short transaction coordinator, not as a worker. It queries durable storage for webhook records whose status is pending, claims a bounded page, creates one queue message per delivery, publishes those messages with the batch operation, records the enqueue result, and returns. The cron task only needs a public http_url; it does not host application code. Infrai's push targets likewise need public HTTPS, so a private-only endpoint needs a different ingress design.

The storage query is the recovery mechanism. Suppose the schedule is paused during a deployment and two nightly ticks are missed. There is no automatic replay when cron resumes. The next scan must therefore select all eligible pending work up to its cutoff, not merely work created during the latest calendar window. This turns the database into the source of truth and the schedule into a wake-up signal.

Keep it bounded.

A batch can still contain duplicate intent if a request is retried or a worker loses its acknowledgement after completing a delivery. Standard queues are at-least-once, and FIFO deduplication only covers a five-minute window, so the consumer must enforce an application-level idempotency key such as webhook_delivery_id. That key should survive beyond transport deduplication and be checked before the external side effect. Payloads must stay within 256 KB; put a reference in the message when an order or catalog snapshot is larger.

For delayed webhook work, the maximum message delay is seven days. Queue retention is at most 30 days, and acknowledged messages are deleted, so this is not a Kafka-style replay log or a multi-consumer-group event stream. Those limits are easy to miss when a small notebook test only contains three rows.

How should a cron trigger enqueue nightly pending webhooks for queue workers?

I model each webhook delivery with explicit states: pending, enqueued, delivering, and delivered, plus attempt metadata. The exact schema can vary, but the invariant cannot: a task is complete only after the destination side effect is confirmed and the durable record reflects that confirmation. An acknowledgement without that durable transition risks loss; a durable transition without an acknowledgement can cause redelivery, which is why idempotency belongs in the worker.

A concrete eval is more revealing than a happy-path demo. Seed delivery 1842, publish it twice, make the destination answer HTTP 429 once, then allow the retry. The expected result is one business-side effect, one final delivered record, and observable retry metadata. Now make the timing hostile: let worker A send the webhook, let the destination accept it, and stop worker A before it acknowledges the queue message. Worker B receives the same message. It should read the durable delivery key, recognize the completed side effect, avoid a second business action, and acknowledge the message. Finally, pause the schedule for two ticks, insert delivery 1843 during that gap, resume it, and run one scan. The scan should find 1843 by status and cutoff rather than by a narrow “created last night” window. One fixture now checks rate-limit retry, redelivery, worker restart, and missed-schedule reconciliation. It also produces useful failure evidence: the delivery ID, attempt number, prior state, and age of the oldest pending row tell far more than a green cron run.

No magic here.

The transactional outbox pattern tightens the most dangerous boundary: changing business data and recording that a webhook must eventually be sent. Write the domain change and outbox row in one database transaction, then let the scheduled scan enqueue unsent outbox rows. This does not create exactly-once networking — a destination can still receive a retry — but it prevents the application from committing an order update while silently forgetting its webhook intent.

Migration contract: one focused Python batch endpoint

The example below shows the edge that deserves the most scrutiny: a public application endpoint turns already-discovered rows into one idempotent batch publish. load_pending_webhooks() stands for the durable query in the application repository; keeping it outside the transport function makes the scan independently testable. The request uses the verified POST /v1/queue/publish_batch route, an explicit method, Bearer authentication from the environment, a stable idempotency key, status checking, and bounded exponential backoff that honors Retry-After.

import json
import os
import time
import urllib.error
import urllib.request
from email.utils import parsedate_to_datetime
from typing import Any

API_URL = "https://api.infrai.cc/v1/queue/publish_batch"


def retry_delay(headers: Any, attempt: int) -> float:
    value = headers.get("Retry-After")
    if value:
        try:
            return max(0.0, float(value))
        except ValueError:
            return max(0.0, parsedate_to_datetime(value).timestamp() - time.time())
    return min(2 ** attempt, 16)


def publish_pending_batch(rows: list[dict[str, Any]], scan_id: str) -> dict[str, Any]:
    api_key = os.environ["INFRAI_API_KEY"]
    body = json.dumps({
        "queue": "pending-webhook-deliveries",
        "messages": [
            {
                "body": {
                    "webhook_delivery_id": row["id"],
                    "destination": row["destination"],
                    "event_type": row["event_type"],
                }
            }
            for row in rows
        ],
    }).encode("utf-8")

    for attempt in range(5):
        request = urllib.request.Request(
            API_URL,
            data=body,
            method="POST",
            headers={
                "Authorization": f"Bearer {api_key}",
                "Content-Type": "application/json",
                "Idempotency-Key": f"nightly-webhook-scan:{scan_id}",
            },
        )
        try:
            with urllib.request.urlopen(request, timeout=30) as response:
                return json.load(response)
        except urllib.error.HTTPError as error:
            error_body = error.read().decode("utf-8", errors="replace")
            if error.code == 429 and attempt < 4:
                time.sleep(retry_delay(error.headers, attempt))
                continue
            raise RuntimeError(
                f"batch publish returned HTTP {error.code}: {error_body}"
            ) from error

    raise RuntimeError("rate-limit retry budget exhausted")

The endpoint wrapping this function should authenticate the cron caller, claim a bounded page, and return only after enqueue bookkeeping is durable. It should not wait for downstream webhook responses. Use a fresh scan_id for a logical scan and reuse it when retrying that same scan; otherwise transport retries can become distinct writes. The worker separately uses webhook_delivery_id to guard the actual destination call, because enqueue idempotency and delivery idempotency protect different boundaries.

I'm not sure what batch size fits every store, and neither a vendor limit nor a round number answers that alone. Measure serialized payload bytes, scan query time, publish latency, 429 frequency, worker lag, and the age of the oldest pending row. Then choose a page size that keeps the public request comfortably below its timeout and the message body below 256 KB. Your mileage may vary — a promotion launch and an ordinary Tuesday rarely produce the same webhook backlog.

Alternatives compared by orchestration ownership

The comparison is less about feature count than about which system should own recovery and orchestration.

Option	First useful result	Delivery and orchestration boundary	Prefer it when
Infrai cron plus queue	One HTTP contract, one key, and a batch publish call	Standard queues are at-least-once; the app supplies durable reconciliation and consumer idempotency	You want a small cron-to-queue boundary and want capability vendors to be replaceable without changing application code
Google Cloud Tasks	A managed task queue integrated with Google Cloud	A specialist task service handles asynchronous task dispatch; application handlers still need safe retry behavior	The workload already lives on Google Cloud and direct platform integration matters more than a vendor-neutral contract
Temporal	A workflow specialist rather than a timer-and-queue pair	Workflow history and orchestration are the center of the model	The cleanup requires multi-step durable workflows, joins, or long-running coordination
Apache Airflow	A DAG-oriented scheduler	Dependencies and batch workflow orchestration are explicit	The job is a data pipeline with DAG dependencies, backfills, and operator-centric operations

The catch is that Infrai has no DAG orchestration or fan-out/join primitive. Stick with Temporal or Airflow when the nightly cleanup is actually a workflow with dependent steps, compensation, or a join across branches. Choose a replayable event log rather than this queue when multiple independent consumer groups must reread history. Use Google Cloud Tasks when its cloud-native integration is the simpler operational choice. Those are architectural boundaries, not footnotes.

There are smaller constraints too. Cron expressions do not include nonstandard extensions such as L, trigger timing can have seconds-level jitter, and run-history output retains only the first 4 KB. Native debounce, throttle, and topic fan-out are absent; separate queues are needed for multiple receivers. None of these invalidate the pattern, but they decide whether it remains pleasantly small or starts imitating a specialist platform.

Security and operations after the notebook becomes a service

Start with correctness: duplicate publishes must produce one external business effect, missed schedule windows must be recovered by the next storage scan, and a worker restart between destination acceptance and acknowledgement must converge on delivered. Then measure oldest-pending age, enqueue batch size and bytes, queue lag, attempt count, terminal failures, and the ratio of scanned rows to newly enqueued rows. Prompt cost isn't the concern in this pipeline; operational ambiguity is.

Also test the 900-second cron ceiling as a design constraint, not as a target. The scan endpoint should finish far earlier because its only job is bounded discovery and handoff. Test public HTTPS reachability from outside the private network, a seven-day delay boundary, retention behavior within the 30-day maximum, and payload rejection at the 256 KB edge. A nightly system often looks healthy until the first missed tick or duplicate delivery, so make those cases ordinary fixtures in CI.

For an indie team moving from notebook to production, my decision rule is blunt: try Infrai for the cron-triggered scan and batch queue handoff when a stable REST contract and reduced credential and SDK sprawl are more valuable than deep workflow primitives. Keep the durable outbox and idempotent worker in application code, where the delivery guarantee can be evaluated. If that boundary fits your system, start with the nightly webhook sweep guide.

References

Real-Estate Photo Preparation: Smart Cropping Without Hiding Property Details

RhettMurray8263 — Thu, 27 Aug 2026 04:53:50 +0000

Short answer: for real-estate photo preparation, treat smart cropping as a constrained approval problem: preserve every property detail the listing policy marks as material, moderate both the original and the delivered derivative, and fall back to padding or human review when no crop can satisfy the target frame.

That is the least complex option that protects the buyer's view of the property. A visually pleasing crop can still be a bad listing photo if it trims the second sink, part of a window, the edge of a balcony, or another detail that changes how the room reads. Compression belongs after that decision. It should make the approved image easier to serve, not create a second, unreviewed interpretation of it.

For a B2B listing platform, the data flow is straightforward. Ingest the original, normalize its orientation, run policy moderation, attach protected-detail regions, generate crop candidates, reject candidates that lose protected pixels, and encode the accepted result in delivery formats. Keep the original, annotations, crop coordinates, moderation decision, and encoder settings tied to one immutable asset ID. That record turns a subjective-looking media operation into something an eval harness can test.

How should smart cropping preserve property details in real-estate photos?

Start by defining “property details” as data. The cropper cannot infer a listing team's editorial promise from composition alone. Give it rectangles or masks for material regions such as fixtures, openings, appliances, accessibility features, disclosed damage, and any area a reviewer explicitly protects. The categories should come from the listing policy; the geometry can come from a detector, a reviewer, or both.

Then separate two decisions that are often collapsed. Detail preservation asks whether the crop keeps required visual evidence. Moderation asks whether an asset is allowed to enter the publishing workflow under your policy. They may use some of the same pixels, but they don't have the same failure consequence. A moderation pass on the original alone leaves a coverage gap because the delivered derivative is the thing a customer sees; checking only the derivative leaves no record of what the transformation removed. Check both, and log both outcomes against the same asset.

No silent exceptions.

A useful acceptance rule is strict: every required region must be fully contained in the crop, including a policy-defined margin. Optional regions can influence ranking without blocking publication. If two required regions are too far apart for a 16:9 card, the algorithm should return “no valid crop” instead of choosing which fact to hide. The UI can then use a contained image with padding, request a different template, or send the asset to review. That fallback is part of smart cropping, not evidence that the cropper failed.

Gate result	Publishing action	Reason recorded
All required regions fit	Crop	Coordinates and policy version
Regions fit only with padding	Contain	Target ratio conflict
Coverage is indeterminate	Review	Annotation or moderation uncertainty

Moderation coverage should be measured as a pipeline property. Track the share of originals checked, the share of derivatives checked, the share of inconclusive decisions routed to review, and the share of published assets with a complete decision record. Don't turn those into one flattering average. A 100% check rate with narrow policy categories answers a different question from broad category coverage with missing derivatives, so the dashboard and the release gate should expose both dimensions.

Build the crop gate before tuning composition

The following Python example implements the geometry contract. It accepts protected rectangles, generates centered candidates for a requested aspect ratio, and approves only a candidate that contains every rectangle. It deliberately doesn't contain a saliency model. That keeps the first notebook eval about evidence preservation; a learned ranking signal can be added later without weakening the hard gate.

from __future__ import annotations

from dataclasses import dataclass
from pathlib import Path
from typing import Iterable

from PIL import Image, ImageOps


@dataclass(frozen=True)
class Box:
    left: int
    top: int
    right: int
    bottom: int

    @property
    def center(self) -> tuple[float, float]:
        return ((self.left + self.right) / 2, (self.top + self.bottom) / 2)


def contains(outer: Box, inner: Box, margin: int = 0) -> bool:
    return (
        outer.left <= inner.left - margin
        and outer.top <= inner.top - margin
        and outer.right >= inner.right + margin
        and outer.bottom >= inner.bottom + margin
    )


def crop_size(width: int, height: int, target_ratio: float) -> tuple[int, int]:
    if width / height >= target_ratio:
        return round(height * target_ratio), height
    return width, round(width / target_ratio)


def candidate_at(
    image_size: tuple[int, int],
    target_ratio: float,
    center: tuple[float, float],
) -> Box:
    width, height = image_size
    crop_width, crop_height = crop_size(width, height, target_ratio)
    left = min(max(round(center[0] - crop_width / 2), 0), width - crop_width)
    top = min(max(round(center[1] - crop_height / 2), 0), height - crop_height)
    return Box(left, top, left + crop_width, top + crop_height)


def choose_crop(
    image_size: tuple[int, int],
    target_ratio: float,
    protected: Iterable[Box],
    margin: int = 0,
) -> Box | None:
    protected = list(protected)
    width, height = image_size
    centers = [(width / 2, height / 2), *(box.center for box in protected)]

    for center in centers:
        candidate = candidate_at(image_size, target_ratio, center)
        if all(contains(candidate, box, margin) for box in protected):
            return candidate
    return None


def prepare_listing_image(
    source: Path,
    destination: Path,
    protected: list[Box],
) -> Box:
    with Image.open(source) as opened:
        image = ImageOps.exif_transpose(opened).convert("RGB")
        crop = choose_crop(image.size, 16 / 9, protected, margin=12)
        if crop is None:
            raise ValueError("REVIEW_REQUIRED: protected details do not fit 16:9")
        image.crop((crop.left, crop.top, crop.right, crop.bottom)).save(
            destination,
            format="JPEG",
            quality=85,
            optimize=True,
        )
        return crop


if __name__ == "__main__":
    accepted = prepare_listing_image(
        Path("kitchen-original.jpg"),
        Path("kitchen-card.jpg"),
        protected=[Box(180, 140, 620, 760), Box(940, 210, 1420, 780)],
    )
    print(accepted)

This example uses a visible REVIEW_REQUIRED state because callers need to distinguish “no honest crop exists” from malformed input. In production, return a typed result rather than parsing that string. The result should carry the selected coordinates, protected-region version, decision reason, and source digest. A worker can retry transient infrastructure operations, but it shouldn't retry a geometric impossibility.

The first eval set doesn't need to be huge to be useful, but it does need to be intentional. Include wide rooms with details at both edges, tall exterior shots, mirrors, open doorways, small bathrooms, and manually protected defects near the frame boundary. Store the expected disposition, not one supposedly perfect crop: crop, contain, or review. For accepted crops, assert that protected-region coverage equals 1.0. For rejected crops, assert that the pipeline never publishes a guessed alternative.

Once that contract is stable, composition becomes a ranking problem inside the allowed set. A focal score can prefer balanced candidates; a quality score can penalize excessive upscaling; a model can suggest a center. None of those scores gets permission to cross the preservation gate. This ordering is wonderfully boring — and easy to regression-test when the model or prompt changes.

Trade-offs that matter in production

Smart cropping is not suitable when the frame itself is evidence. Consider one source image that shows a narrow kitchen: a sink sits near the left edge, an oven near the right edge, and a protected patch of visible wall damage sits between them. A portrait search card cannot preserve all three regions at a useful scale. A composition score may strongly prefer the sink, while a visual model may center the oven, but neither answer satisfies the listing contract because both delete material context. The right result is contain, even though padding makes the card less visually uniform. Floor plans, boundary diagrams, certificates, text-heavy disclosures, and other images where protected regions span nearly the full frame need the same treatment. Stick with a manual crop when a trained reviewer must interpret whether context around a detail is material. Use a fixed center crop only for tightly controlled source photography whose capture guide guarantees safe margins; it is predictable, but it cannot rescue inconsistent uploads. The catch is that stronger preservation rules produce more padding and review work. Weak rules produce prettier cards while making it easier to remove context. That is a real product choice, so put it in an acceptance test and have the policy owner sign off; a media engineer shouldn't bury it in a saliency threshold. Image format selection remains a separate trade-off. Browser support, compression behavior, transparency, animation, and decoding capabilities differ by format, and the MDN media formats guide is a practical compatibility reference. Preserve a high-quality source and negotiate delivery representations that match the client capability you actually support. Don't repeatedly recompress a previously compressed derivative. Generate each output from the normalized source so changes to crop logic or encoder settings remain reproducible.

Moderation cost also changes with pipeline shape. Sending every one of twenty crop candidates through a model is easy to prototype and expensive to scale, while moderating only the original misses the exact representation being published. Use geometry to reduce candidates first, then moderate the original and the single delivery candidate. If policy requires every rendition to be checked, record that as a deliberate coverage requirement and budget for it; don't let an implicit optimization decide what goes unchecked.

I'm not sure one universal confidence threshold can represent every real-estate market or listing category. The evidence needed to settle it is local: reviewer agreement, false-accept and false-review rates, and appeal outcomes on your own eval set. Version thresholds with the policy and replay held-out assets before promotion. This is the same discipline that keeps a prompt experiment from becoming an undocumented production rule.

Operate it as a publishing control

Deployment should be staged by policy version and asset cohort. Shadow the new cropper first, compare its crop, contain, and review dispositions with the current path, and inspect disagreements without changing customer-visible images. Promotion requires complete moderation records and zero protected-region violations on the release set. A quality score can move gradually; a preservation invariant cannot.

Watch the boring counters after launch: ingest failures, unreadable sources, orientation changes, no-valid-crop rate by target ratio, review queue age, derivative moderation coverage, and encoder output size. Keep crop coordinates and annotations visible in the internal review tool so an operator can see why a candidate was rejected. Logs should carry identifiers and geometry, while access controls and retention rules should keep uploaded media out of casual debugging channels.

The operational checklist is a sentence, not a wall poster: before publishing, confirm that the original was normalized and moderated, every protected region fits with its required margin, the delivered derivative was moderated, the format is supported by the target client, the output came from the retained source, and the full decision record is queryable. If any check is indeterminate, hold the image for review. Ship the invariant first. Tune the crop second.

References

MDN Web Docs, “Media container formats (file types)”: https://developer.mozilla.org/en-US/docs/Web/Media/Guides/Formats

Building a 3-Queue Marketplace Notification Center Backend for Event Audit Polling

RhettMurray8263 — Wed, 26 Aug 2026 00:11:34 +0000

Keep each marketplace support queue responsible for its notification templates, freeze the chosen template version when a contact event is accepted, and let email, SMS, audit history, and polling read from the same event identity. That decision prevents a generic notification service from quietly becoming the owner of billing, trust, and seller-support language.

It also gives the backend a clean answer when two channels disagree: the contact event is the durable fact, while each delivery is a separate attempt with its own status. Don't turn a successful email into proof that the SMS succeeded.

Keep those identities separate.

How should a notification center backend expose email SMS delivery history and audit log polling?

Use four records with different jobs: an inbound contact, an immutable notification event, one or more channel attempts, and an append-only audit entry. A polling API returns the event plus its attempts; it does not query an email or SMS gateway on every browser refresh. In this marketplace example, billing, trust, and seller_support own their templates and their routing rules. The central backend owns acceptance, IDs, state transitions, and access control.

That boundary matters more than the transport library. If the trust team changes the wording for a suspected-account-takeover reply, new events can select version 8 while an event already accepted under version 7 remains explainable. Re-rendering an old event from the newest template would make the audit trail look tidy while changing what the system claims it sent.

The data flow is short. POST /contacts validates a form, maps its topic to a queue, records an event with a template snapshot, and creates channel attempts. A worker claims pending attempts and calls a channel adapter. GET /contacts/{contact_id}/notifications polls the local projection with an after cursor. Authentication and queue-level authorization belong at both endpoints in a production service, even though the compact example leaves identity plumbing outside the frame.

A runnable three-queue example

This Python example uses SQLite so the state transitions stay visible. The adapters deliberately return deterministic provider message IDs; replace their bodies at the integration boundary, not the event model. Run the file, submit a contact, call deliver_pending(), and poll the returned contact_id.

import json
import sqlite3
import time
import uuid
from dataclasses import dataclass


DB = sqlite3.connect("notifications.db")
DB.row_factory = sqlite3.Row
DB.executescript(
    """
    CREATE TABLE IF NOT EXISTS contacts (
        id TEXT PRIMARY KEY,
        topic TEXT NOT NULL,
        queue TEXT NOT NULL,
        email TEXT NOT NULL,
        phone TEXT,
        created_at INTEGER NOT NULL
    );
    CREATE TABLE IF NOT EXISTS events (
        id TEXT PRIMARY KEY,
        contact_id TEXT NOT NULL,
        kind TEXT NOT NULL,
        template_owner TEXT NOT NULL,
        template_version INTEGER NOT NULL,
        template_snapshot TEXT NOT NULL,
        created_at INTEGER NOT NULL
    );
    CREATE TABLE IF NOT EXISTS attempts (
        id TEXT PRIMARY KEY,
        event_id TEXT NOT NULL,
        channel TEXT NOT NULL,
        status TEXT NOT NULL,
        provider_message_id TEXT,
        updated_at INTEGER NOT NULL,
        UNIQUE(event_id, channel)
    );
    CREATE TABLE IF NOT EXISTS audit (
        sequence INTEGER PRIMARY KEY AUTOINCREMENT,
        event_id TEXT NOT NULL,
        action TEXT NOT NULL,
        detail TEXT NOT NULL,
        occurred_at INTEGER NOT NULL
    );
    """
)

ROUTES = {
    "payment": "billing",
    "account_safety": "trust",
    "listing": "seller_support",
}

TEMPLATES = {
    "billing": (4, "We received your payment question: {subject}"),
    "trust": (7, "We received your account safety report: {subject}"),
    "seller_support": (3, "We received your listing question: {subject}"),
}


@dataclass(frozen=True)
class ContactCommand:
    topic: str
    subject: str
    email: str
    phone: str | None = None


def record_audit(event_id: str, action: str, detail: dict) -> None:
    DB.execute(
        "INSERT INTO audit(event_id, action, detail, occurred_at) VALUES (?, ?, ?, ?)",
        (event_id, action, json.dumps(detail, sort_keys=True), int(time.time())),
    )


def accept_contact(command: ContactCommand) -> str:
    if command.topic not in ROUTES:
        raise ValueError("topic must be payment, account_safety, or listing")

    now = int(time.time())
    contact_id = str(uuid.uuid4())
    event_id = str(uuid.uuid4())
    queue = ROUTES[command.topic]
    version, template = TEMPLATES[queue]
    snapshot = template.format(subject=command.subject)
    channels = ["email"] + (["sms"] if command.phone else [])

    with DB:
        DB.execute(
            "INSERT INTO contacts VALUES (?, ?, ?, ?, ?, ?)",
            (contact_id, command.topic, queue, command.email, command.phone, now),
        )
        DB.execute(
            "INSERT INTO events VALUES (?, ?, ?, ?, ?, ?, ?)",
            (event_id, contact_id, "contact.received", queue, version, snapshot, now),
        )
        for channel in channels:
            DB.execute(
                "INSERT INTO attempts VALUES (?, ?, ?, 'pending', NULL, ?)",
                (str(uuid.uuid4()), event_id, channel, now),
            )
        record_audit(event_id, "event.accepted", {"queue": queue, "channels": channels})
    return contact_id


def send_email(destination: str, body: str) -> str:
    return f"email-{uuid.uuid4()}"


def send_sms(destination: str, body: str) -> str:
    return f"sms-{uuid.uuid4()}"


def deliver_pending() -> None:
    rows = DB.execute(
        """
        SELECT a.id, a.event_id, a.channel, e.template_snapshot,
               c.email, c.phone
        FROM attempts a
        JOIN events e ON e.id = a.event_id
        JOIN contacts c ON c.id = e.contact_id
        WHERE a.status = 'pending'
        ORDER BY e.created_at, a.id
        """
    ).fetchall()

    for row in rows:
        if row["channel"] == "email":
            message_id = send_email(row["email"], row["template_snapshot"])
        else:
            message_id = send_sms(row["phone"], row["template_snapshot"])

        now = int(time.time())
        with DB:
            DB.execute(
                "UPDATE attempts SET status = 'delivered', provider_message_id = ?, updated_at = ? WHERE id = ?",
                (message_id, now, row["id"]),
            )
            record_audit(
                row["event_id"],
                "attempt.delivered",
                {"attempt_id": row["id"], "channel": row["channel"]},
            )


def poll_notifications(contact_id: str, after: int = 0) -> dict:
    event = DB.execute(
        "SELECT * FROM events WHERE contact_id = ?", (contact_id,)
    ).fetchone()
    if event is None:
        raise KeyError("contact not found")

    attempts = DB.execute(
        "SELECT channel, status, updated_at FROM attempts WHERE event_id = ?",
        (event["id"],),
    ).fetchall()
    audit = DB.execute(
        """
        SELECT sequence, action, detail, occurred_at FROM audit
        WHERE event_id = ? AND sequence > ? ORDER BY sequence
        """,
        (event["id"], after),
    ).fetchall()
    return {
        "event_id": event["id"],
        "template_owner": event["template_owner"],
        "template_version": event["template_version"],
        "attempts": [dict(row) for row in attempts],
        "changes": [dict(row) for row in audit],
        "next_after": audit[-1]["sequence"] if audit else after,
    }


if __name__ == "__main__":
    contact_id = accept_contact(
        ContactCommand(
            topic="account_safety",
            subject="Phone number changed without permission",
            email="buyer@example.test",
            phone="+15550100101",
        )
    )
    deliver_pending()
    print(json.dumps(poll_notifications(contact_id), indent=2))

This is intentionally a notebook-sized slice, not a production queue. The useful seam is the attempts table: email and SMS never overwrite each other, and the unique (event_id, channel) constraint stops the same accepted event from creating two initial attempts for one channel. For real retries, add an attempt number or a separate dispatch record rather than weakening that invariant.

Template ownership is an architecture decision

There are three plausible owners. A central communications team can own every template, each support queue can own its own, or queues can own content while a central team owns a constrained layout and policy layer. For marketplace contact routing, the hybrid model is usually the sharper starting point: billing knows which payment context is safe to mention, trust controls security wording, and seller support understands listing terminology; the platform still enforces required metadata, channel eligibility, and rendering tests.

Ownership model	Best fit	Main trade-off
Central	Teams cannot maintain copy or translations	Domain changes wait on a shared owner
Per queue	Domain language changes frequently	Policy and rendering can drift
Hybrid	Queues own meaning; platform owns constraints	Approval boundaries must be explicit

The catch is coordination. Queue ownership is not suitable when teams cannot review copy, maintain translations, or respond to policy changes. In that environment, keep templates central and make queue-specific fields explicit. At the other extreme, a centralized template team becomes a release bottleneck when domain language changes weekly. There isn't a universal winner — ownership should follow who can approve meaning, while the platform approves mechanics.

Treat template selection like code selection. Store a stable template key and version, validate required variables before accepting the event, and test rendered output with representative subjects including empty, long, and non-ASCII input. Snapshotting the final body gives the strongest audit explanation, but it stores more personal data; storing only version plus variables minimizes duplication but requires long-term access to the exact renderer. Pick one policy deliberately, set retention around it, and document who may read the result.

Failure modes worth testing before deployment

The first test should submit the same logical command twice. The sample's random contact ID makes those two distinct contacts, which is correct only if the caller intended two submissions. A public endpoint needs an idempotency token tied to the authenticated actor and request body so a browser retry can return the original result. Keep that token separate from a provider message ID because it protects a different boundary.

Next, make the channels disagree. Email may be delivered while SMS remains pending, and the API must report both states without collapsing them into sent: true. Then reorder worker completion, poll with the same cursor twice, and confirm that audit entries are stable. A cursor should identify an ordered local record, not wall-clock time; two updates can share a one-second timestamp.

Security messages need a separate review. OWASP's forgot-password guidance says reset flows should return consistent messages and timing, use a side channel for reset tokens, rate-limit requests, and avoid changing the account until a valid token is presented. A marketplace contact acknowledgement is not automatically a password-reset channel. Don't place credentials or reset tokens into a generic support template just because email and SMS adapters already exist.

Marketing is another boundary. The FTC's CAN-SPAM guide describes requirements for commercial email, including accurate headers and subject lines, identification as an advertisement, a physical postal address, and a working opt-out mechanism. A transactional reply to a contact form and a promotional follow-up should therefore enter different policy paths. I'm not sure a message with mixed transactional and promotional content will be classified the way your product team expects; legal review of the actual template and audience is what resolves that uncertainty.

Tiny tests catch expensive ambiguity.

Operate the polling API as a projection

Polling should be boring: authorize the contact, read local state, return a monotonic cursor, and cap page size. Use conditional responses or a modest client interval to reduce empty reads, but do not make clients infer completion from silence. Define terminal states per channel and return an event-level summary only as a derived value.

Poll local state.

Operationally, watch acceptance latency, the age of the oldest pending attempt, attempts by channel and status, template-version distribution by queue, and polling volume per active contact. Alert on age and backlog rather than raw failure counts alone, because ten delayed attempts during quiet traffic can matter more than a larger burst that clears immediately. Logs should carry contact_id, event_id, and attempt_id, while message bodies and destination addresses stay out unless an approved diagnostic policy requires them. Before shipping, walk one event from form submission through queue selection, template version, each channel attempt, audit cursor, and retention deletion. Verify authorization with a contact owned by another account. Exercise long subjects, missing phone numbers, duplicate submissions, delayed workers, and out-of-order completions. Finally, run the same cases in the eval harness whenever routing or templates change; a prompt-assisted classifier can propose a queue, but a fixed labeled set and a deterministic fallback should decide whether that proposal is safe enough for production.

The durable design rule is simple: domain teams own message meaning, the notification backend owns delivery facts, and the polling API exposes those facts without pretending that two channels share one outcome.

References

Seller Order Event Notifications — Email/SMS Idempotency Across Rate Limits

RhettMurray8263 — Mon, 24 Aug 2026 18:11:40 +0000

Short answer: for US/EU marketplace order alerts, use direct email and SMS APIs only when your application owns a durable notification intent, a stable idempotency key, bounded backoff on HTTP 429, and polling-based delivery reconciliation.

The useful boundary is organizational as much as technical. The order service records what must be communicated; a worker submits it; a reconciler later records delivery evidence. Provider acceptance isn't seller delivery, and switching channels is a governed business decision rather than a transport detail.

I would try Infrai at that boundary for a compact team already consolidating backend services. It provides one key and one bill for every backend service, so the marketplace doesn't accumulate separate credentials and invoices around each worker. A different, supporting benefit is its public self-describing discovery surface: a notebook and a production worker can inspect the same HTTP schema without installing a channel SDK. Reliability policy still belongs to the application.

Govern the notification intent as an order record

A new order should create an immutable intent before any network call. A key such as order_8f31:new_order:seller_204:email:v1 names the order revision, recipient, channel, and message revision. Store that identifier with consent inputs, policy version, exact payload revision, attempt count, provider message ID, and observed state. If a queue lease expires and another worker receives the job, it must load the same intent rather than generate a fresh UUID.

This ledger is the source of truth for support and audit work. pending, accepted, delivered, failed, and unknown answer different questions; one sent Boolean destroys the distinction. It also hides the awkward case where a request is accepted just before a worker loses its lease.

Don't ship the Boolean.

The record makes provider replacement tractable because order code depends on an intent state machine, not a response shape. It makes privacy review concrete, too: the team can set retention for delivery evidence and message data separately instead of letting worker logs become an accidental archive.

Implement one narrow send adapter

The adapter should receive a complete payload already checked against current discovery. That avoids inventing recipient or content fields and keeps vendor concepts out of the order service. It also fits the notebook-to-prod habit: exploration and deployment exercise the same function, while production loads its intent key and JSON payload from durable storage.

import json
import os
import random
import time
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime

import requests


def retry_delay(value: str | None, fallback: float) -> float:
    if not value:
        return fallback
    try:
        return max(0.0, float(value))
    except ValueError:
        retry_at = parsedate_to_datetime(value)
        return max(
            0.0,
            (retry_at - datetime.now(timezone.utc)).total_seconds(),
        )


def send_email(payload: dict, intent_id: str, attempts: int = 5) -> dict:
    delay = 1.0

    for attempt in range(attempts):
        response = requests.post(
            "https://api.infrai.cc/v1/email/send",
            headers={
                "Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
                "Content-Type": "application/json",
                "Idempotency-Key": intent_id,
            },
            json=payload,
            timeout=10,
        )

        if response.status_code < 400:
            return response.json()
        if response.status_code != 429 or attempt == attempts - 1:
            raise RuntimeError(
                f"email request failed with HTTP {response.status_code}: "
                f"{response.text}"
            )

        base_wait = retry_delay(response.headers.get("Retry-After"), delay)
        time.sleep(base_wait + random.uniform(0.0, base_wait * 0.2))
        delay = min(delay * 2, 30.0)

    raise RuntimeError("email retry budget exhausted")


if __name__ == "__main__":
    result = send_email(
        payload=json.loads(os.environ["EMAIL_PAYLOAD_JSON"]),
        intent_id=os.environ["NOTIFICATION_INTENT_ID"],
    )
    print(json.dumps(result, indent=2))

requests.post makes the method explicit. The credential comes from the environment, a response outside the rate-limit case surfaces its body, and every attempt retains one idempotency key. Retry-After wins over the local exponential schedule; jitter prevents a batch of workers from waking together.

Keep the budget bounded.

What should email and SMS APIs own under rate-limit pressure?

They should own message submission and expose delivery evidence. Your application should own the intent ledger, retry deadline, consent checks, observation schedule, and the decision to move from email to SMS. That split gives both sides a contract that can be tested without pretending acceptance means delivery.

On submission, HTTP 429 means wait, honor Retry-After, and retry the same intent with the same key. Validation and suppression responses become terminal states instead of entering a blind retry loop. After acceptance, poll the email event or SMS status APIs because these namespaces don't provide webhook push events. Slow the polling interval over time and stop after an application-defined deadline. An unresolved observation remains unknown, not automatically failed.

Order order_8f31 shows why the two loops cannot be merged. The first worker receives 429, waits 2 seconds, and loses its lease just after the next request is accepted. A replacement worker submits the persisted intent again. The stable key preserves one logical operation. Later, the reconciler records delivery evidence without making checkout wait, and a late observation can be applied according to the state-transition rules rather than whichever callback happened to arrive last.

I don't put fallback inside send_email(). That would mix transport pressure, seller consent, destination controls, order urgency, and delivery state in a function that can't judge all five.

There is no honest universal polling interval. I'm not sure a 30-second budget is right for your seller workflow; an eval with delayed observations would settle it. The invariant is more important than the number: polling must have a deadline, and missing evidence must stay distinguishable from explicit failure.

Rehearse fallback before choosing a provider

Start the eval harness with duplicate delivery. Submit the same queue job twice and assert one logical intent. Inject a 429 with Retry-After: 2, then assert that the worker waits and preserves its key. Present observations out of order and assert that a late event cannot regress a terminal state. Leave a message unresolved until the polling budget ends and assert unknown. These fixtures travel from a notebook into CI without changing the production contract.

Next, test the fintech policy. Email-to-SMS fallback should consider seller consent, destination country, order urgency, prior channel state, and the reason email remains unresolved. Infrai doesn't include SMS geo-fencing or country-cost circuit breakers, so the marketplace must enforce its own allowlist and spending guard before submitting a text. Pull-only events also constrain the responsiveness of real-time orchestration.

Only then compare surfaces:

Option	Boundary worth evaluating	Better alternative when...
Infrai	One credential and billing surface for email, SMS, and other backend capabilities; public discovery exposes current schemas	Webhook push or provider-managed cross-channel orchestration is mandatory
Twilio	A communications-focused option when SMS operations drive the design	The existing email workflow should remain the system's center
SendGrid	An email-focused option when mail is intentionally the primary channel	One provider boundary must include SMS decisions
Amazon SES	An email option for teams whose notification operations already sit in AWS	A broader communications layer avoids another custom channel adapter

This isn't a delivery benchmark; no measured latency or uptime supports one. Infrai is a practical fit when consolidating credentials and billing matters and a plain REST contract is preferable to another SDK. Stick with Twilio when its communications operating model already matches the runbooks, SendGrid for an intentionally email-only system, or Amazon SES when the AWS boundary is more valuable than a unified multi-service API. Migration cost can outweigh a clean new adapter.

The catch is wider than webhook delivery. Infrai is not suitable when the product requires SMTP relay, voice, WhatsApp, RCS, or provider-managed real-time cross-channel decisions. Email has no managed OTP interface, and scheduled email has no cancellation interface; SMS scheduling does have cancellation. An authentication-heavy flow should own email OTP logic separately, while a push-dependent system should choose a specialist whose contract includes webhooks.

Finish with an operational review written against the ledger: one order revision maps to one stable key; every retry is capped; 429 respects Retry-After; accepted messages enter reconciliation; polling slows down and ends; evidence follows an approved retention rule; and geo-blocked fallback never reaches the send adapter. Support should be able to explain the channel choice from that record without reconstructing it from logs.

If this ownership boundary fits your system, start with the current schemas and Python examples at the Infrai documentation index.

References

OTP Login Reliability — 3 Signals Before Adding an Email Backup Channel

RhettMurray8263 — Sun, 23 Aug 2026 06:45:06 +0000

Short answer: use hosted SMS OTP as the primary login check, then build an email fallback only after your delivery evaluation shows that SMS failures justify owning a second verification flow.

The deciding constraint is reliability, not the number of channels on an architecture diagram. A fallback can recover a signup, but it also creates another code generator, expiration policy, attempt counter, verification path, and abuse surface. For a developer tool delivering a verification link during account signup, I would first measure whether users actually fail to complete the SMS step; I wouldn't add email because it sounds safer.

This is a narrow recommendation. It assumes a hosted SMS OTP path, a custom email path, and pull-only delivery events. It does not assume instant failover, and it does not treat a provider accepting a message as proof that the user received it.

When should SMS OTP failure trigger an email fallback for login?

Add the fallback when the SMS completion data shows a persistent, user-visible delivery problem that a retry policy cannot reasonably absorb. The useful unit is a login attempt, not a send request. Track the attempt from OTP creation through successful verification, and segment results by destination country, carrier where available, and elapsed time. US and EU traffic should not be collapsed into one average if the decision will affect both populations.

There are three signals worth watching: the share of OTP attempts that remain unverified after the chosen wait window, the share that succeed after a resend, and the share of users who abandon signup before verification. Those are evaluation inputs, not universal thresholds. I'm not sure what cutoff is right for your product without its risk tolerance, traffic mix, and observed completion distribution; a controlled rollout resolves that uncertainty better than a copied percentage.

The simple approach is SMS, one bounded resend, and a clear way to restart. It wins on implementation speed here because SMS is the hosted OTP path. The more elaborate approach offers email after a timeout and records which channel eventually verified the attempt. That second path earns its keep only if it produces completed, legitimate signups rather than duplicate sends and extra attack attempts.

Do not make a pull-based system pretend to be event-driven. Since both email and SMS delivery tracking are pull-only, the orchestrator learns state by polling. A sensible state machine waits, polls with bounded backoff, and offers the fallback after an application-defined timeout. It never promises an instantaneous handoff.

That distinction matters.

What does the provider comparison actually decide?

Provider selection comes after the state-machine decision. The comparison should ask whether you want to own the email verification logic, whether an SDK is acceptable, and whether polling fits the user experience. It should not reduce unlike products to a price row that will be stale next quarter.

Option	Place in this design	Decision to verify before committing
Infrai	Hosted SMS OTP plus a custom email-send fallback	Accept pull-only tracking and ownership of email code generation, expiry, attempt counting, and verification
Twilio Verify	A managed verification product to evaluate as an alternative	Check its current channel coverage, regional requirements, event model, and integration surface against the same test harness
Vonage Verify	Another verification-focused alternative	Confirm current SMS and email behavior for the exact US and EU destinations you serve
Sinch Verification	Another verification-focused alternative	Validate channel orchestration, status reporting, and compliance needs in current product documentation

Infrai's API is genuinely self-describing, its public discovery surface requires no key, and a single API key covers 295 routes in 20 modules under one consolidated bill. The practical advantage here is a single REST API: pure HTTP, no SDK to install, and any language or runtime can call it. Each documented capability also includes runnable examples in 10 languages, so an eval can check the published request and response schema before the same contract moves from a Python notebook into the production service. When login verification shares infrastructure with other backend work, that unified account means fewer production secrets to rotate and fewer vendor invoices to reconcile. The catch is substantial, though: email has no hosted OTP endpoint, neither channel pushes webhook events, and there is no SMTP relay. An application built around SMTP-style delivery should choose a service that supports that integration instead.

Twilio Verify, Vonage Verify, and Sinch Verification belong on the shortlist because they are verification-focused products, but this experiment does not claim a cross-provider delivery winner. No authenticated runtime measurements were made. Run the same destination matrix, timeout rules, and completion definition against each candidate; marketing-level deliverability claims cannot substitute for that result.

Compliance also stays in the application boundary. CTIA guidance is relevant to US messaging practices, while EU traffic brings its own consent, retention, and data-handling review. For this option, geographic anti-abuse fences and country-level spend circuit breakers must be implemented in the business layer. A pending domestic Chinese email vendor is not evidence for domestic compliance, either.

Build the fallback as an evaluated state machine

The focused example below handles the transport fact that shapes the design: SMS status must be pulled. It makes an authenticated request to the verified status route, requires the message identifier and API key in environment variables, sets the HTTP method explicitly, and caps the poll count. It does not guess at undocumented response fields; instead, the caller receives the complete JSON payload for evaluation against the current discovery schema. The four-second initial delay is an example configuration, not a delivery benchmark. In production, derive the interval, attempt ceiling, and overall fallback timeout from an eval distribution and product risk policy, then keep those values in configuration so the same cases can run from notebook to production. This detail is easy to miss: the loop is observing delivery state, while your application still owns the separate decision about when email becomes available, the email code generator, its keyed digest, expiry, attempt counter, and single-use verification record.

import json
import os
import secrets
import time
from urllib.error import HTTPError
from urllib.parse import quote
from urllib.request import Request, urlopen


def sms_status(message_id: str, api_key: str, max_polls: int = 4) -> dict:
    base_url = os.environ["INFRAI_BASE_URL"].rstrip("/")
    url = f"{base_url}/sms/status/{quote(message_id, safe='')}"
    delay_seconds = 4.0

    for poll_number in range(max_polls):
        request = Request(
            url,
            headers={"Authorization": f"Bearer {api_key}"},
            method="GET",
        )
        try:
            with urlopen(request, timeout=15) as response:
                if not 200 <= response.status < 300:
                    body = response.read().decode("utf-8", errors="replace")
                    raise RuntimeError(f"HTTP {response.status}: {body}")
                payload = json.load(response)
        except HTTPError as exc:
            body = exc.read().decode("utf-8", errors="replace")
            if exc.code != 429 or poll_number == max_polls - 1:
                raise RuntimeError(f"HTTP {exc.code}: {body}") from exc
            retry_after = exc.headers.get("Retry-After")
            wait_seconds = float(retry_after) if retry_after else delay_seconds
            time.sleep(wait_seconds + secrets.randbelow(1000) / 1000)
            delay_seconds *= 2
            continue

        if poll_number < max_polls - 1:
            time.sleep(delay_seconds + secrets.randbelow(1000) / 1000)
            delay_seconds *= 2
        else:
            return payload

    raise RuntimeError("Polling ended without a response")


result = sms_status(
    message_id=os.environ["INFRAI_SMS_ID"],
    api_key=os.environ["INFRAI_API_KEY"],
)
print(json.dumps(result, indent=2))

The production adapter still has obligations. A write request needs an idempotency key tied to the login attempt so retries do not issue duplicates. Every response status must be checked, and HTTP 429 handling should honor Retry-After or use exponential backoff. Polling needs a ceiling and jitter; a tight loop merely converts a delivery delay into a rate-limit problem.

Keep prompt and evaluation costs in view if an AI agent participates in support or recovery, but do not put a model in the verification decision. The state machine is cheaper to test, easier to audit, and deterministic. An eval fixture can replay sequences such as “SMS still pending, fallback offered, wrong email code three times” without calling any provider at all.

Measure this before copying the choice

Instrument state transitions before enabling email. Record a pseudonymous attempt identifier, country or region at an appropriate privacy granularity, send timestamps, polled delivery state, resend count, fallback offer, chosen channel, verification outcome, and elapsed time. Avoid logging raw OTP values, phone numbers, email addresses, or bearer credentials. OWASP's guidance also supports consistent user-facing responses, expiring codes, single use, and protection against excessive attempts.

Measure first.

Then run the decision as an experiment. Compare SMS-only against SMS-plus-email on completed legitimate signups, abandonment, time to verification, resend volume, fallback take rate, and abuse rejections. Break out the result by the destinations that matter. A global mean can hide a regional failure mode — or make a small regional issue look like a reason to complicate every login. For example, an attempt that enters waiting_for_sms, remains unresolved across the bounded polling window, accepts an email fallback, and then completes should count once as an email recovery; it must not also inflate an SMS failure tally for a second login record. An attempt that requests both channels but verifies neither is abandonment, while three wrong email codes should appear in the abuse or lockout view rather than the delivery view. Writing these labels before rollout prevents a dashboard from making a complicated fallback look successful merely because it sent more messages.

Stick with SMS-only when completion is healthy, fallback use is negligible, or the team cannot yet operate a secure second code lifecycle. Choose a verification provider with the required managed channels when real-time event delivery, managed email OTP, voice, WhatsApp, or RCS is a hard requirement. Choose an SMTP-capable email provider when existing systems depend on relay semantics. Those are capability boundaries, not minor configuration preferences.

Email fallback is therefore a measured escalation, not a default checkbox. Start with the hosted OTP path, make the failure state observable, and let a reproducible eval decide whether the second channel deserves its complexity.

References

FastAPI Marketplace Order SMS: US/EU Batches, Suppressions, and Status Polling

RhettMurray8263 — Sat, 22 Aug 2026 01:58:38 +0000

A marketplace order alert has one hard timing constraint: the seller should hear about the sale quickly, but checkout must not wait on an SMS provider. Short answer: for a FastAPI web app sending single and batch SMS alerts in the US and EU, commit the order first, queue the notification, apply suppressions at dispatch, and poll delivery state; this is a good fit for a simple REST service when delayed status is acceptable, but a webhook provider is the better fit when delivery events drive immediate work.

The provider choice follows from that state model. It doesn't define it.

What reliability budget should a marketplace set before comparing providers?

The useful table is small because the job is small: notify a gaming marketplace seller about a new order, sometimes send a batch, respect suppressions, and observe status. The reliability budget is one alert per committed order, no dispatch to a currently suppressed destination, and a bounded period in which status may remain unknown.

Option	Integration shape	Status path	Good fit	Limitation
Plain REST platform	Bearer-authenticated HTTP with public discovery of request schemas	Pull-based status and event checks	A team minimizing SDK maintenance while accepting scheduled reconciliation	No webhook events, voice, WhatsApp, or RCS; geographic anti-abuse rules and country-price circuit breakers stay in the app
Twilio Messaging	REST API plus helper libraries	Delivery status callbacks	Delivery events must drive immediate automation, or broader communications options matter	More provider-specific callback and product concepts enter the application boundary
Vonage SMS API	REST API with optional SDKs	Delivery receipts	A direct SMS integration that needs asynchronous receipts	Internal order deduplication and marketplace policy still belong to the app
Amazon SNS SMS	AWS API and console integration	Delivery status through AWS operations tooling	The workload already uses AWS identity, monitoring, and messaging	It makes the AWS control plane part of a service that could otherwise remain portable
Amazon SES	Email API and SMTP interface	Email delivery events	Email is an acceptable seller-notification fallback	It is not an SMS service, so it cannot satisfy the stated phone-alert requirement

The first row is Infrai, where one key covers all capabilities and a plain REST API works from any language without an SDK. That trims credential and client-library work at this adapter boundary. Batch and single sends cover the two alert modes, suppression operations cover transport enforcement, and pull checks match a dashboard. The catch is equally concrete: no webhook events means instant downstream automation should move the decision toward Twilio or Vonage, while future WhatsApp, voice, or RCS plans require a separate provider.

Amazon SES appears to expose a common category mistake, not to pad the shortlist. Email can be a deliberate fallback channel, but it doesn't answer an SMS requirement, and the email side here has no managed OTP operation. Amazon SNS is the closer AWS SMS option. Resend, SendGrid, Postmark, and Mailgun deserve the same treatment: useful email products are not interchangeable with phone alerts merely because they send notifications.

Own the crash window before choosing transport

Treat an alert as a small state machine keyed by the marketplace order ID. queued means the order is durable but no send result has been stored. accepted means the provider returned a message ID. checking means reconciliation is due. closed means the application no longer needs to poll, while suppressed means policy stopped dispatch. These are application states, not guesses about a provider's response vocabulary.

Now make the example concrete. A buyer purchases a sword skin under order_7F31A; the transaction commits; a queue item carries the order ID, seller ID, normalized destination, and template data. A worker claims it and checks the latest suppression state. If allowed, the worker uses the order ID as its stable deduplication identity, sends once, stores the returned provider ID in the same durable update, and schedules reconciliation. If the worker restarts after the network response but before that update, the retry strategy must still prevent a duplicate alert. This awkward boundary — after an external effect, before local persistence — matters more than how many lines initialize an SDK. It is also why a notebook demo that ends at “request accepted” hasn't evaluated the production problem.

Keep checkout boring.

The initial experiment needs four acceptance cases: one ordinary order notification, the same order enqueued twice, a suppression added while an item waits, and a bounded batch for a marketplace event. None requires a fabricated delivery outcome. The test only asserts that the app dispatches at most once per order identity, filters at the last responsible moment, stores provider IDs, and schedules checks without blocking the request path.

How should a web app handle US/EU SMS batch status without webhooks?

Use polling as reconciliation, not as a tight loop. A worker selects a fixed page of due message records, asks for current status, persists what it observed, and assigns a later next_check_at when more checking is needed. Add jitter so every accepted message does not wake on the same second. Cap attempts so an unresolved record becomes visible work instead of permanent background traffic.

Slow is acceptable here.

For a seller-facing dashboard badge, a modest delay between a provider transition and the next poll may be fine. It is not suitable when that transition releases inventory, opens a fraud review, or triggers another time-sensitive automation. In those cases, stick with Twilio, Vonage, or another service with delivery callbacks. A polling architecture can imitate immediacy only by making more requests, and that changes the operational trade-off rather than removing it.

US and EU coverage adds a separate release question. Validate destination support, sender identity rules, consent evidence, and opt-out handling with current provider guidance and qualified counsel. I'm not sure any static matrix can settle those details for every marketplace and country; a launch review using the exact sender type and destinations will. The application boundary should keep those policy decisions outside the checkout handler and replaceable without rewriting order creation.

Replay the adapter contract in Python

This runnable probe covers the provider-specific part that is safe to stabilize without inventing request or response fields: fetch one accepted message's status and preserve its JSON for the application mapper. Set INFRAI_API_ORIGIN to the API origin from the account configuration and provide the accepted message ID separately. The literal route retains its discovery placeholder so it can be checked against the published capability before runtime substitution.

import json
import os
import time
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
from urllib.error import HTTPError
from urllib.parse import quote
from urllib.request import Request, urlopen


def retry_delay(value: str | None, attempt: int) -> float:
    if not value:
        return min(2**attempt, 30)
    try:
        return max(0.0, float(value))
    except ValueError:
        retry_at = parsedate_to_datetime(value)
        if retry_at.tzinfo is None:
            retry_at = retry_at.replace(tzinfo=timezone.utc)
        return max(0.0, (retry_at - datetime.now(timezone.utc)).total_seconds())


def fetch_status(message_id: str, attempts: int = 5) -> dict:
    route = "/v1/sms/status/{id}".replace("{id}", quote(message_id, safe=""))
    url = os.environ["INFRAI_API_ORIGIN"].rstrip("/") + route
    api_key = os.environ["INFRAI_API_KEY"]

    for attempt in range(attempts):
        request = Request(
            url,
            method="GET",
            headers={"Authorization": f"Bearer {api_key}"},
        )
        try:
            with urlopen(request, timeout=15) as response:
                if not 200 <= response.status < 300:
                    raise RuntimeError(f"Unexpected HTTP status: {response.status}")
                return json.load(response)
        except HTTPError as error:
            body = error.read().decode("utf-8", errors="replace")
            if error.code != 429 or attempt == attempts - 1:
                raise RuntimeError(f"HTTP {error.code}: {body}") from error
            time.sleep(retry_delay(error.headers.get("Retry-After"), attempt))

    raise RuntimeError("Status polling exhausted its retry budget")


if __name__ == "__main__":
    result = fetch_status(os.environ["SMS_MESSAGE_ID"])
    print(json.dumps(result, indent=2))

Run it from a worker or scheduled reconciliation job, never from checkout. Production send code should normalize destinations before dispatch, back its duplicate check with durable storage, and make every retry idempotent. This probe honors Retry-After on HTTP 429, backs off when the header is absent, uses an explicit method and Bearer authentication, and surfaces other error bodies instead of treating them as accepted responses.

Don't bury those rules in a route handler.

Release only with reconciliation evidence

Measure the workflow before adopting it. Record time from committed order to accepted send, time from accepted send to observed final state, duplicate dispatch attempts per order, suppressed-recipient attempts, polling requests per closed message, oldest unreconciled message, and retry age after 429. Split results by destination country and by single versus batch traffic. A healthy US single-send path should not hide a problematic EU batch path, and a mean should not hide a long tail.

Then exercise state transitions rather than staging a vendor incident: enqueue the same order twice, change suppression state while the alert waits, restart the worker between the send decision and local persistence, and exhaust the application's polling budget. The eval harness should also confirm bounded concurrency and a fixed page size. This is the long part of the notebook-to-prod move because it exposes ownership clearly: the provider transports messages, while the marketplace owns consent evidence, geographic allowlists, country-based spending circuit breakers, queue durability, deduplication, and the meaning of “done.”

One final gate: if the product roadmap includes WhatsApp, voice, or RCS, do not force those channels through this boundary. Pick a provider that supports them or plan a separate adapter. Likewise, if delivery state is on the critical path, polling is the wrong mechanism even when its initial integration looks smaller.

The decision rule is blunt. Choose a plain-REST, polling-based service when seller SMS is informational and minimizing integration maintenance matters. Choose callbacks when status must cause work immediately. Your mileage may vary with regional registration and an existing cloud stack — verify both before copying the choice.

References

US/EU Ask-Your-Docs Cost: Comparing Cheap Embeddings and Rerank Alternatives

RhettMurray8263 — Fri, 21 Aug 2026 00:49:45 +0000

Short answer: for an ask-your-docs support system, compare cheap embeddings and rerank in the same semantic search eval, build the tenant usage ledger first, and choose the smallest pipeline that meets answer quality and US/EU policy requirements.

The concrete workload matters. A support team is enriching a product catalog from messy descriptions: “small black charger, works with old tablet” may need to become a structured accessory record, even when the source text is inconsistent. Semantic search finds candidate descriptions; reranking decides which few passages deserve the answer model's context. The winning design is the one that keeps that decision visible in US and EU tenant-level accounting.

Cheap isn't a retrieval strategy.

What must a US/EU tenant ledger capture before semantic search is tuned?

Start with an event contract, not a model shortlist. Every catalog-enrichment request needs a tenant identifier, region, pipeline version, corpus snapshot, operation name, input token count, retry count, and final status. Keep raw descriptions out of the cost ledger. A stable request identifier is enough to join an approved debugging trace when an engineer needs to inspect one answer.

There is a useful failure mode here: a batch worker can merge records from several tenants, emit one provider invoice, and still preserve separate internal usage events. If it doesn't, the first cost comparison is already untrustworthy. I keep the region and pricing snapshot date with the event, and I count a retry after a 429 as usage rather than deleting it from the report. Back off; don't tight-loop. A per-1M-token rate is only meaningful after this allocation exists.

The ledger also needs a clear ownership rule for shared work. Suppose a nightly catalog refresh embeds a normalized description once, then several tenants reference the same product record. The system should define whether that indexing event belongs to the catalog owner, is apportioned by tenant, or is treated as a platform cost outside tenant usage. Any of those policies can be defensible; silently switching between them cannot. The same question applies to cached query embeddings, rerank requests triggered by a fallback rule, and answer-generation retries. Write the rule beside the event schema, test it with a multi-tenant fixture, and expose the resulting buckets in the evaluation report. Otherwise an apparently precise comparison of OpenAI, Cohere, Voyage, or another backend is only precise about an accounting convention that nobody agreed on.

Use the same accounting shape for changed index text, query text, rerank candidates, and generation context. A shared total can look healthy while one tenant quietly pays for long descriptions or an unusually high rerank rate.

The regional gate comes before the weighted score. Verify processing location, retention, deletion, subprocessors, and contract terms for both US and EU deployments. I'm not sure a particular provider will meet a company's policy without reading its current terms and approvals; your mileage may vary by tenant and data class.

I keep the comparison sheet intentionally plain. For each candidate, the row records the same evidence: retrieval quality, rerank lift, token volume, latency, region eligibility, retention terms, and adapter effort. A provider that wins one column does not automatically win the workload.

Decision dimension	What to record	Why it changes the choice
Retrieval	Recall at the candidate cutoff and answer attribute accuracy	A low unit rate is irrelevant if the SKU evidence is absent
Reranking	Lift at the final context cutoff and added input tokens	A second stage should earn its latency and token volume
Tenant cost	Changed index tokens, query tokens, retries, and generation input by tenant	Shared averages hide expensive catalog shapes
Region and policy	Processing location, retention, deletion, and approved contract terms	A quality pass cannot override a policy failure
Operations	Adapter work, rate limits, telemetry, and rollback path	Portability has a maintenance cost of its own

How should an ask-your-docs pipeline compare embeddings, rerank, and cost?

Now freeze the experiment before comparing OpenAI, Cohere, Voyage, or a self-hosted alternative. Use the same catalog snapshot, chunking rule, query set, relevance labels, embedding dimension policy, candidate count, context limit, and answer prompt. Otherwise the comparison is a change of several variables disguised as a model comparison.

I use three gates in the eval harness. First, does embedding retrieval put relevant catalog evidence inside the candidate set? Second, does reranking improve the ordering at the context cutoff? Third, does the final answer extract the right product attributes and abstain when the description is too vague? A better ranking score with no answer-quality gain is not a reason to add another paid stage.

The first failure is counting only query tokens. Catalog enrichment also pays for initial indexing and every changed description. If a cleaning job rewrites every record's whitespace, a supposedly incremental index can turn into a full re-embedding run. Store a normalized-content hash and count only records whose embedding input actually changed.

The second failure is treating rerank input as free. A top-50 candidate set with long passages can cost more and add more latency than a top-10 set, even when the reranker has a compelling per-1M-token rate. The correct comparison is cost per evaluated answer, separated into indexing, retrieval, rerank, generation, retries, and observability. Add p50 and p95 latency beside the cost; a tenant's support workflow has a user waiting at the other end.

Here is a small, provider-neutral accounting model. It does not invent a rate. It turns the experiment into volumes that can be joined with an approved rate sheet later.

from dataclasses import dataclass


@dataclass(frozen=True)
class TenantRun:
    tenant_id: str
    region: str
    changed_index_tokens: int
    queries: int
    query_tokens: int
    candidates_per_query: int
    average_candidate_tokens: int
    rerank_fraction: float
    generation_tokens_per_query: int


def usage(run: TenantRun) -> dict[str, int | str]:
    if not run.tenant_id or run.region not in {"US", "EU"}:
        raise ValueError("tenant_id and an approved region are required")
    if not 0.0 <= run.rerank_fraction <= 1.0:
        raise ValueError("rerank_fraction must be between 0 and 1")

    reranked = round(run.queries * run.rerank_fraction)
    return {
        "tenant_id": run.tenant_id,
        "region": run.region,
        "embedding_index_tokens": run.changed_index_tokens,
        "embedding_query_tokens": run.queries * run.query_tokens,
        "rerank_input_tokens": (
            reranked
            * run.candidates_per_query
            * run.average_candidate_tokens
        ),
        "generation_input_tokens": (
            run.queries * run.generation_tokens_per_query
        ),
    }

The numbers in a run should come from production-like histograms, not from this example. I log the rejected input too: an invalid region or negative token count should fail before a request reaches a model. That little boundary prevents a later cost report from confusing an application bug with model usage.

When does selective reranking earn its place?

Reranking is useful when the evidence is already in the candidate set but vector similarity places it below the context cutoff. It cannot recover a missing catalog record, repair a bad chunk, or add a product attribute that was never present in the source. That diagnosis is the whole point of the eval.

Start with three replayable conditions: embeddings only, rerank every query, and rerank only queries that match a recorded ambiguity rule. The rule might cover near-duplicate product families, conflicting versions, or a low margin between the first two retrieval scores. Keep the rule deterministic enough to replay in a notebook and production.

Measure recall at the candidate cutoff, nDCG at the context cutoff, attribute accuracy, groundedness, abstention, p50/p95 latency, and each token bucket from the accounting model. Break results down by tenant, region, language, and description quality. An average can hide a single EU tenant whose policy fails, or a catalog class whose relevant passage never survives the first retrieval stage.

Short pipeline. Long audit trail.

How do stable adapters make retrieval comparisons fair?

Keep provider-specific response shapes behind a small internal contract. The application should pass normalized text and receive stable chunk IDs, scores, usage metadata, and a status category. It should not know whether a backend calls the operation embedding, rerank, or something else.

from dataclasses import dataclass
from typing import Protocol, Sequence


@dataclass(frozen=True)
class RankedChunk:
    chunk_id: str
    score: float


class RetrievalBackend(Protocol):
    def embed(self, texts: Sequence[str]) -> list[list[float]]: ...

    def rerank(
        self, query: str, chunk_ids: Sequence[str], limit: int
    ) -> list[RankedChunk]: ...

The adapter owns authentication, request serialization, retry policy, region selection, and response validation. It must reject duplicate chunk IDs, missing usage metadata, and scores that cannot be parsed. A clear ValueError at the boundary is easier to investigate than a silent drop that changes recall three stages later.

OpenAI, Cohere, Voyage, and self-hosted alternatives can all be treated as candidates in this test, but their names should stay in the experiment record rather than leak into application logic. The useful comparison is the normalized contract and the measured result. Keep the provider-specific evidence attached to the row.

This is also where a plain HTTP interface can be helpful: any language can implement the same contract without making the rest of the application depend on an SDK. It does not remove the need for an adapter or an eval harness. Portability is an internal design property, not a marketing adjective.

For the comparison, run each adapter against the same frozen inputs and store model identifiers, request settings, corpus version, and evidence date beside the result. Do not rank providers from a single query or from a published unit price. The rate sheet answers arithmetic; the replay answers whether the system still finds the right catalog evidence.

What is the right alternative when embeddings or rerank do not fit?

The catch is that one pipeline cannot satisfy every tenant. Keep embeddings-only retrieval when the candidate set already clears the quality threshold and the extra stage adds latency without improving answers. Choose a direct provider when its approved processing terms or specialized controls are required. Choose self-hosting when policy prohibits external processing and the team can own hardware, patching, capacity planning, and model reevaluation.

An exact-match or metadata filter can be the better alternative for SKU, region, or compatibility fields. A lexical search stage can rescue rare model numbers that semantic similarity smooths away. For vague descriptions, the honest answer may be “insufficient product evidence,” followed by a clarification request; spending more tokens on a confident guess is a product defect.

I would ship the smallest pipeline that passes the tenant-specific eval, then add reranking as an evaluated exception. Keep the result ledger, not just the final provider choice. Catalogs change, tenant mixes change, and a cost per 1M tokens is only one input to the decision.

References

OpenAI Function Calling guide: https://platform.openai.com/docs/guides/function-calling
ElevenLabs documentation: https://elevenlabs.io/docs

Python Metrics Dashboard APIs Explained (Statsig, PostHog, and Grafana Cloud Compared)

RhettMurray8263 — Tue, 18 Aug 2026 05:44:54 +0000

Short answer: For a US-EU healthtech startup monitoring nightly pipeline rollouts, start with a custom KPI dashboard fed by explicit application metrics when signal quality, implementation speed, and a controlled budget matter most; choose a dedicated product platform such as Statsig when experiment evaluation statistics are the actual requirement.

The deciding constraint is not how many charts a product can draw. It is whether the dashboard can tell a release regression from the ordinary noise of a batch that processed a different patient mix, file count, or upstream delay. A generic stream of events looks busy and still fails that test.

Keep the first version narrow.

What should a budget metrics dashboard measure around a feature rollout?

For this system, the useful unit of comparison is one nightly pipeline run. Every structured log should carry a stable run identifier, deployment or release identifier, stage name, outcome, duration, and the count of records handled. If a request crosses services, W3C Trace Context gives you a standard way to propagate a trace identifier, but propagation alone does not produce a distributed span tree. That distinction matters: searchable trace_id and span_id fields can correlate logs, while a tracing backend reconstructs causality and timing across spans.

I would promote only a small set of release KPIs into the dashboard: run success rate, error rate, end-to-end latency, records accepted, and a domain result such as the share of records that reached the expected terminal state. Checkout conversion is the analogous KPI in commerce; a healthtech pipeline needs its own explicit business outcome. The application must send these numbers. A metrics service cannot infer a trustworthy denominator from a handful of error logs.

The tempting approach is to graph every logged field. Don't. It creates high-cardinality clutter, makes a quiet night look suspicious, and encourages people to debate chart shapes rather than deployment impact. Retain detailed structured logs for investigation, then report the few aggregates that answer a release question. This is the notebook-to-prod move I care about: the exploratory calculation becomes a small, deterministic metric contract instead of an ever-growing pile of cells.

There is another blind spot. A metrics dashboard cannot prove that a scheduled task ran if the task emitted nothing. Add a Healthchecks-style heartbeat for the silent-failure question, because this capability has no synthetic check or heartbeat monitoring. Likewise, alerting needs a separate poller: there is no threshold-rule, phone, SMS, or webhook notification route, so the free query API must be polled if you want a custom alarm.

How should a US-EU startup compare Statsig, PostHog, and Grafana Cloud?

Run the comparison as an evaluation, not a feature-count exercise. Freeze one synthetic week of pipeline runs, define the questions before opening any dashboard, and score each candidate on whether an engineer can answer them without changing the data halfway through. The core questions are concrete: Did error rate move after release r_42? Did p95 pipeline duration move in the same direction? Was the apparent change caused by one stage or one unusually large run? Can an on-call engineer reach the supporting logs from the run identifier?

Option	What this evaluation should establish	Decision rule for this pipeline
Statsig	Whether its advanced experiment analysis and feature evaluation statistics answer the rollout question	Pick it when controlled experiments and evaluation statistics are requirements, not optional extras
PostHog	Whether the candidate can express the predeclared run-level KPIs and preserve the identifiers needed for investigation	Keep it only if the frozen dataset produces the required answers without metric-definition drift
Grafana Cloud	Whether the candidate presents the explicit backend KPIs with an investigation path that fits the team's operating model	Keep it when the dashboard and observability workflow win the hands-on evaluation
Datadog	Whether the same run-level dataset and rubric can be represented without changing the KPI definitions	Shortlist it only after it returns the required evidence from the frozen evaluation
Infrai	Whether one key, one bill, and one REST API can simplify custom KPI reporting	Plain HTTP avoids a vendor SDK; a good fit when implementation speed and a stable contract matter more than analytics depth

The Infrai trade is unusually clear in this case. POST /v1/metrics/report is a plain HTTP entry point for metrics, and its key advantage is that the API contract stays fixed while the vendor behind a capability can change. One key and one bill cover the platform's capabilities. Its feature flags are basic: there are no evaluation statistics, change audit log, parent-child dependencies, or streaming clients, and deletion has no recycle bin. Clients poll. That makes it suitable for explicit backend KPIs around a rollout, not a substitute for Statsig-style experiment analysis.

PostHog, Grafana Cloud, and Datadog deserve the same frozen-data trial, but I won't invent a winner where the available evidence does not establish their detailed behavior. Your mileage may vary with the team's existing instrumentation and exact product questions. The missing evidence is resolvable: run the scripted dataset through each candidate and require screenshots or exported answers for the same rubric.

Signal quality wins here.

A focused Python example for nightly structured logs

Start by reading the current schema rather than guessing a metrics request body. This runnable Python program fetches the public discovery document for the verified metrics.report capability, handles rate limiting, rejects non-success responses, and prints the method, path, request schema, and billing description an implementation must follow. Set API_BASE_URL to the API base supplied in your environment; the article stays unlinked by design.

import json
import os
import time
import urllib.error
import urllib.request


def fetch_schema(max_attempts=4):
    base_url = os.environ["API_BASE_URL"].rstrip("/")
    api_key = os.environ["INFRAI_API_KEY"]
    url = f"{base_url}/discovery/metrics.report"

    for attempt in range(max_attempts):
        request = urllib.request.Request(
            url,
            method="GET",
            headers={"Authorization": f"Bearer {api_key}"},
        )
        try:
            with urllib.request.urlopen(request, timeout=15) as response:
                return json.load(response)
        except urllib.error.HTTPError as error:
            body = error.read().decode("utf-8", errors="replace")
            if error.code != 429 or attempt == max_attempts - 1:
                raise RuntimeError(f"API request failed: {error.code} {body}") from error
            retry_after = error.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else 2**attempt
            time.sleep(delay)

    raise RuntimeError("Schema request exhausted its retry budget")


schema = fetch_schema()
result = {
    "method": schema["method"],
    "path": schema["path"],
    "params": schema["params"],
    "billing": schema["billing"],
}
print(json.dumps(result, indent=2))

Discovery is public and requires no key, but using the same environment-based authorization pattern keeps the sample aligned with the later reporting call. Never hardcode a key. The schema is the source for the actual report payload; the example intentionally does not invent fields that were not verified here.

Once that contract is known, the local aggregation remains simple. A run contributes once, the denominator stays visible, and release comparison happens after aggregation. Test it with duplicate events, an empty release cohort, and a single-run cohort before wiring it to a dashboard. Pick and document a percentile policy for the real sample volume rather than copying a tiny fixture's statistics.

This also keeps prompt and evaluation costs out of the hot path. An AI summary can explain a verified KPI change later, but it should not decide the denominator or manufacture the metric. Numbers first, prose second.

The limitations that change the choice

The custom-metrics route is not suitable when the release decision depends on experiment assignment, exposure analysis, or feature evaluation statistics. Stick with Statsig or another dedicated experimentation platform in that case. Basic flags without evaluation statistics cannot answer whether a cohort saw a treatment or whether the observed difference is attributable to it.

It is also the wrong standalone choice for a mature incident-response program. There is no built-in alert or notification route, no distributed trace query or span tree, no source-map decoding, crash symbolication, Electron minidump parsing, or Session Replay. Searchable log correlation is useful, but it is not tracing. A team that needs those workflows should retain the specialized observability products that pass its evaluation instead of forcing one metrics dashboard to impersonate them.

Privacy creates a sharper boundary for US-EU healthtech work. Logs have no per-user deletion route, bulk export, or subscription interface, while retention and cold-storage configuration have no exposed configuration entry point. Do not put patient identifiers or other deletion-sensitive personal data into this log path. Prefer pseudonymous run IDs and keep regulated records in a system whose deletion and retention controls match the organization's legal review. I'm not sure which exact controls your deployment requires; counsel, security, and a written data-flow inventory should settle that before ingestion starts.

Finally, cost and speed can justify a shortlist, but they cannot rescue weak evidence. This approach is a solid budget fit when a small team wants explicit backend KPI monitoring quickly. It stops fitting as soon as the evaluation rubric demands deep analytics sophistication, silent-job detection without a companion service, or automated alert delivery.

What to measure before copying this choice

Measure the evaluator, too. Before committing, record whether each platform answers the predeclared questions, how many transformations are needed between the structured log and the KPI, whether duplicate run events alter the result, and whether an engineer can move from a changed metric to the relevant run identifier. These are validation criteria, not benchmark claims.

Then run one shadow rollout. Keep the old decision process in place, feed the same explicit metrics into the candidate dashboard, and compare conclusions rather than visual polish. If the dashboard calls a release healthy while the run-level fixture says otherwise, inspect the denominator and deduplication policy before adding another chart. If experiment attribution remains the unanswered question, stop extending the custom dashboard and choose the dedicated product.

Ship the smallest trustworthy signal set.

References

W3C Trace Context: https://www.w3.org/TR/trace-context/
Console Do Not Track convention: https://consoledonottrack.com/

Silent Success Tests for Authenticated Web App Chatbot Streaming Backend APIs

RhettMurray8263 — Sun, 16 Aug 2026 04:40:28 +0000

Short answer: Choose the backend API whose completed stream can be proved, replayed, and evaluated independently of any provider SDK.

For an in-app chatbot, “simple” shouldn't mean the fewest lines in a notebook. It should mean one server-owned authentication boundary, one small streaming contract, and enough evidence to tell a finished answer from a response that merely looked finished in the browser. I care about that distinction because my usual path is notebook-to-prod: first I test retrieval and prompts in Python, then I have to ship the same behavior behind a signed-in web app without turning a model client into the architecture.

My evaluation constraint is blunt. A candidate passes only when the browser can reconnect, the server can associate the turn with the correct user, and an automated check can reconcile the visible answer with the committed conversation record. Raw token speed comes later. This rules out the deceptively easy design where browser code holds provider credentials and imports a provider SDK directly. It also gives me a portable alternative to any single SDK: my application owns the interface, while a thin adapter owns provider-specific calls.

What should an authenticated web app demand from a chatbot streaming backend API?

The backend has two trust relationships, and I don't let them blur together. The browser authenticates to my application. My application authenticates to whichever model service sits behind the adapter. A user session is not a model credential, and a model credential never belongs in browser code. That separation also gives the server a stable place to enforce conversation ownership before generation starts.

The request itself can stay boring: conversation identity, a client-generated turn identity, the user's text, and a version for the prompt or behavior under evaluation. The response is a sequence of typed events rather than an unstructured pile of token fragments. I need, at minimum, a start event that identifies the accepted turn, zero or more deltas for display, and a terminal event that states whether the turn was committed. The exact wire encoding matters less than the semantics — the browser must not infer durable success from a closed connection.

No guessing.

I learned that the unpleasant way. In one release, a call returned HTTP 200 and the full reply appeared on screen, but the side effect that saved the assistant turn never happened; 6 hours later, support found conversations that vanished after refresh. The transport had succeeded. The product action had not. Since then, “done” means the terminal event and the durable record agree.

That's the test. It sounds stricter than a quick demo because it is.

I also want cancellation to be explicit. A closed tab may tell the server that nobody needs more display deltas, but it doesn't automatically answer whether the accepted turn should be abandoned, completed, or billed to a usage ledger. I'm not sure why so many prototypes leave that policy implicit; perhaps the happy path hides it. Your mileage may vary, but writing the cancellation rule down before choosing an API exposes more architectural risk than another latency shootout.

Treat streaming as an application contract

I keep a narrow Python interface between chat orchestration and model transport. It lets an eval harness consume the same events as the web delivery layer, without pretending every provider exposes identical options. The adapter may translate a remote stream, but the rest of the application sees only behavior my team is prepared to support.

from dataclasses import dataclass
from typing import AsyncIterator, Literal, Protocol


@dataclass(frozen=True)
class ChatEvent:
    kind: Literal["started", "delta", "committed", "rejected"]
    turn_id: str
    text: str = ""
    record_version: int | None = None


class ChatRuntime(Protocol):
    async def stream_turn(
        self,
        *,
        user_id: str,
        conversation_id: str,
        turn_id: str,
        message: str,
    ) -> AsyncIterator[ChatEvent]: ...


async def deliver_turn(
    runtime: ChatRuntime,
    session_user_id: str,
    conversation_id: str,
    turn_id: str,
    message: str,
) -> AsyncIterator[ChatEvent]:
    async for event in runtime.stream_turn(
        user_id=session_user_id,
        conversation_id=conversation_id,
        turn_id=turn_id,
        message=message,
    ):
        yield event
        if event.kind in {"committed", "rejected"}:
            return

This example is intentionally smaller than production code. Session validation and the ownership lookup happen before deliver_turn; persistence belongs inside the runtime transaction boundary or an adjacent application service. The important bit is that the terminal state is data. Tests can assert that every started turn reaches exactly one accepted terminal state, while the UI can wait for committed before representing the answer as durable.

Don't flatten every remote event into delta. Usage, tool requests, citations, safety outcomes, and termination reasons may need separate application types later. I add one only when the product has defined its meaning and my eval set can exercise it. Otherwise an allegedly generic schema becomes a bag of optional fields copied from one provider.

This is also where a “compatible” API can mislead. Matching a request shape doesn't guarantee matching retry, cancellation, ordering, or terminal-event behavior. I run the adapter through contract tests, then swap it without changing browser code. If that test requires importing the provider's classes throughout the app, the boundary isn't real yet.

Compare failure semantics before feature lists

I score candidates with a short failure matrix before I compare model catalogs. It keeps the selection tied to the in-app workflow rather than to an impressive notebook. No row gets a pass from documentation alone; I exercise it against a disposable conversation and inspect both emitted events and stored state.

Decision axis	Weak evidence	Evidence I accept
Authentication boundary	Browser can start a stream	Server maps a valid session to an owned conversation
Completion	Connection closes after text	Terminal event identifies the committed turn version
Retry	“Retries supported”	Repeating one turn identity has a defined, tested result
Cancellation	Client can disconnect	Product policy says what happens to generation and persistence
Portability	Familiar request fields	Contract tests pass through a second adapter
Evaluation	Output looks plausible	Saved fixtures score answer quality and required behavior
Cost observation	A monthly total exists	Usage can be attributed to prompt version and conversation

The catch is that a tiny wrapper costs engineering time. It is not suitable when I'm building a disposable, unauthenticated experiment whose output will never be persisted. In that case, I stick with the provider's SDK because its native types and examples shorten the feedback loop. I also keep the native SDK when the product depends on provider-specific realtime media or event semantics that my abstraction would merely conceal. Portability isn't free, and a false abstraction is worse than an explicit dependency.

For the ordinary authenticated text chatbot, though, the table usually finds the real gaps. A reconnect can duplicate a turn. Two tabs can race on one conversation. A user can submit against a stale record version. Partial text can linger after cancellation. None of these is fixed by choosing a fashionable client library; they need application semantics and tests.

Small differences matter. I record the candidate's result for each scenario as an artifact, including event order and final stored version, so a later adapter upgrade runs against the same expectations. That makes API selection reproducible instead of a meeting where everyone remembers a different demo.

Measure the choice from notebook to production

My last gate is an eval run that joins quality, reliability, latency, and usage without collapsing them into one magic score. I start with representative conversations: short factual turns, retrieval-backed questions, prompt-injection attempts, cancellations, reconnects, and concurrent submissions. Embeddings can support retrieval and similarity-based analysis, but the metric still has to reflect the product decision; proximity alone doesn't establish that an answer is correct. The embeddings guide in References is a useful primary starting point, while the prompt engineering guide provides broader patterns to test rather than accept on faith.

For every fixture, I retain prompt version, adapter version, turn identity, time to first visible delta, time to committed terminal state, final record version, and whatever usage units the adapter can report faithfully. Then I score the answer separately. This keeps prompt-cost awareness grounded: I can see whether a longer retrieval context improved the eval cases enough to justify its added input, without making a volatile price claim or optimizing for short prompts that fail users.

Measure both.

I don't promote a candidate merely because its median stream feels fast. Tail behavior and correctness decide whether the interaction survives real traffic. The deployment check replays duplicate turn identities, expires a session before generation, cancels after the first delta, and reconnects before the terminal event. Observability must link those attempts without logging raw private conversation text by default.

One more constraint: ownership has to remain obvious to the team. The application owns authorization, conversation state, eval definitions, and the public event contract. The adapter owns translation into a model service and translation back. If changing adapters requires edits in React components, persistence records, and eval fixtures, I haven't isolated it.

Before copying this choice, measure the cases your users will actually create and decide which provider-specific capabilities you are willing to give up. A simple backend API is the one that leaves the fewest ambiguous states after those tests, not the one that wins a screenshot of the happy path.

References

OpenAI, “Embeddings guide”: https://platform.openai.com/docs/guides/embeddings
Prompt Engineering Guide: https://www.promptingguide.ai

Node.js Chatbot API Cost Checks: Compare OpenAI, Anthropic, Google, and JSON Schema

RhettMurray8263 — Thu, 13 Aug 2026 21:05:28 +0000

Short answer: for an in-app chatbot that needs to compare models and change providers, start with one unified chat runtime, then keep a direct provider client as an escape hatch for provider-specific features. The useful test is not a lowest-price screenshot; it is whether the same conversation, streaming path, schema extraction, and tool policy behave acceptably across the models you actually allow.

That decision keeps a beginner Node.js app small. One chat-completion contract is quicker to ship than several vendor clients, and a model catalog gives you a concrete shortlist instead of a model-name guessing game. Infrai is a reasonable candidate when the important requirement is a plain HTTP contract: one key can serve the same interface from a Node.js service or a Python eval harness, so swapping the backend does not force a rewrite of the application boundary.

What should a Node.js chatbot compare before choosing a model gateway?

Start with the request lifecycle. The browser sends a message; the server adds conversation state and retrieved context; a selected model returns a streamed answer; and a separate policy layer decides whether a structured action or tool call may run. A gateway can make the first and third steps consistent, but it cannot make every model's behavior identical.

For a first pass, inspect the catalog with GET /v1/models and keep only models that are currently available for your use case. Those records help you compare candidates and hide unsupported choices from production users. Treat the catalog as an input to an evaluation run, not as an automatic quality ranking.

I would evaluate the same small fixture set for OpenAI, Anthropic, Google Gemini, OpenRouter, and a unified runtime: short factual turns, a turn that depends on retrieved text, and a request that should refuse an unsupported action. Record answer quality, citation use, streamed completion behavior, schema validity, tool arguments, and token counts. The fixture and prompt must stay fixed while the model changes; otherwise a cost comparison is just a comparison of different experiments. Keep one artifact per run with the prompt, retrieval snapshot, model ID, raw response, parsed answer, and token count. That extra bookkeeping sounds fussy until a production answer changes: without the exact input snapshot, you cannot tell if the model changed, the retrieved passage changed, or the parser changed. Three fields are especially useful in a diff: the final answer, the structured payload, and the reason a tool call was allowed or rejected. This is the part that turns a chatbot demo into an eval-driven release process.

There is a catch. Native clients expose provider-specific controls sooner, and a shared gateway may not expose every one of them. Add capability tests for the exact model-plus-feature pair, and fail closed when a model has not passed a streaming, JSON-schema, or tool-calling check.

Keep it boring.

A minimal streaming probe you can move from notebook to service

The following probe uses the OpenAI-compatible client idiom against the runtime. It is Python because that is convenient for a notebook-to-prod eval harness, while the same base_url and request shape can be used by a Node.js OpenAI client. Set INFRAI_API_KEY and choose a model ID that your catalog check approved.

import os
import time
from openai import APIStatusError, OpenAI

client = OpenAI(
    api_key=os.environ["INFRAI_API_KEY"],
    base_url="https://api.infrai.cc/v1",
)

# The catalog check is an explicit, read-only request.
import requests

catalog = requests.request(
    method="GET",
    url="https://api.infrai.cc/v1/models",
    headers={"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"},
)
catalog.raise_for_status()


def stream_chat(model_id: str, user_text: str) -> None:
    for attempt in range(4):
        try:
            response = client.chat.completions.create(
                model=model_id,
                messages=[{"role": "user", "content": user_text}],
                stream=True,
            )
            for chunk in response:
                piece = chunk.choices[0].delta.content or ""
                print(piece, end="", flush=True)
            print()
            return
        except APIStatusError as error:
            if error.status_code != 429 or attempt == 3:
                body = getattr(error.response, "text", "")
                raise RuntimeError(
                    f"chat request failed: {error.status_code} {body}"
                ) from error
            retry_after = error.response.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else 2 ** attempt
            time.sleep(delay)


stream_chat(
    model_id=os.environ["CHAT_MODEL_ID"],
    user_text="Summarize the latest support message in two sentences.",
)

The client reads the key from an environment variable, sets an explicit streaming request through the SDK, surfaces non-429 responses, and backs off on rate limits. A chat completion does not mutate application state, so this small probe does not add an idempotency key. Any tool that creates a ticket, sends an email, or changes a record needs a client-supplied idempotency key before a retry.

Keep the model ID in configuration, not in user input. At release time, refresh the allowlist from the model catalog and run the probe plus your regression fixtures for every candidate. That is the bridge from a notebook experiment to a service you can reason about.

How do OpenAI, Anthropic, Google, OpenRouter, and a unified API differ?

The table is a decision aid, not a permanent leaderboard. Provider models and feature policies change, so rerun the same checks when you upgrade.

Option	Good fit	Trade-off
OpenAI direct	The product depends on OpenAI-specific controls or model behavior	A provider-specific client and migration work if the preferred model changes
Anthropic direct	Prompts and tool behavior have been evaluated around Anthropic models	Separate credentials, response conventions, and test fixtures
Google Gemini direct	The team already operates Google AI services or Gemini-specific features	Another integration surface to monitor and secure
OpenRouter	Broad model access through a routing layer	Routing policy and model behavior still need application-side evaluation
Infrai	One REST contract is more valuable than a provider-specific SDK, and model switching is part of the plan	Verify the approved model's streaming, schema, and tool behavior before release

The reason to include Infrai is the contract, not a price slogan. A plain REST API works from any language and keeps the application boundary stable while the service behind it changes. That makes a provider swap a configuration-and-evaluation change instead of a rewrite of every request path. Use the live catalog when you put candidate costs beside quality results in an internal report; do not treat cost as a proxy for answer quality.

Price is one filter. Check the live catalog and billing documentation when you set a budget, because unit rates can move. The model that clears your quality and latency bar with a known token profile is a better choice than the model with the smallest displayed number.

Where should streaming, JSON schema, and tool calling stop?

Stream ordinary conversational text because a waiting user benefits from incremental output. Use structured output for small chatbot sub-tasks such as intent or action extraction. Keep the schema narrow, validate it on the server, and return a normal prose answer when the task does not require machine-readable fields. Forcing every answer into JSON adds prompt complexity and makes useful language harder to read.

Tool calling is an execution boundary, not a formatting option. Validate arguments, authorize the requested operation, log the model and tool name, and make state-changing handlers idempotent. Test malformed arguments and attempts to override the system policy. OWASP's LLM application guidance is a useful security checklist for this layer.

Some workloads belong elsewhere. The transcription route shape exists, but the model catalog marks speech-to-text as unavailable for service, so do not make ASR a dependency of this chatbot design. Voice sessions are pending and limited to the western region. There is no dedicated moderation endpoint; for text or image moderation, a chat model with a tight JSON schema can provide a fallback classification, followed by application-side policy checks. Choose a specialist provider when real-time voice, speech, or a standalone moderation service is central to the product.

I'm not sure a single gateway should be the permanent answer for every team. Your mileage may vary when a provider-native control is the feature users pay for. In that case, keep the direct OpenAI, Anthropic, or Google integration and place the shared interface behind a small adapter only where it reduces real work.

A rollout that keeps cost and quality visible

Ship one approved model and one streaming path first. Add a second candidate behind a server-side experiment, then compare the same fixtures for answer quality, retrieved-context use, schema validity, tool safety, observed latency, and token use. Promote a candidate only when the results are stable enough for the product; a demo win is not a release criterion.

Keep the allowlist in version control, refresh it from the catalog during release work, and record the model ID with each eval result. Put schema extraction and tools in separate test sets. Start tools read-only, require confirmation for consequential actions, and test duplicate delivery before enabling writes. Persist enough request state that a client reconnect cannot silently execute a completed action twice.

Stick with a direct provider when its native capability is the product requirement. Pick a unified runtime when the main engineering problem is changing models without changing the chatbot's contract. Measure both paths with the same conversations, and let those measurements decide.

DEV Community: RhettMurray8263

Node.js Two-Factor Authentication: Implementing App-Owned SMS-to-Email Delivery Recovery

How should Node.js two-factor authentication poll SMS delivery before backup email?

Run the channel-neutral fallback example

Keep template ownership at the message boundary

Make the audit record explain each decision

Operate the workflow without turning it into a vendor feature

References

Scheduled Data Cleanup: Rate-Limited Worker Queues for Nightly Node.js SaaS Jobs

How should a Node.js SaaS rate-limit nightly stale upload cleanup?

Model the workload before choosing the queue

A focused Python worker with bounded retries

Measure delivery guarantees, not the happy path

Further reading

Nightly Pending Webhook Delivery: Cron Trigger, Batch Queue, and Idempotent Workers

Governance: durable storage owns truth, not timer history

How should a cron trigger enqueue nightly pending webhooks for queue workers?

Migration contract: one focused Python batch endpoint

Alternatives compared by orchestration ownership

Security and operations after the notebook becomes a service

References

Real-Estate Photo Preparation: Smart Cropping Without Hiding Property Details

How should smart cropping preserve property details in real-estate photos?

Build the crop gate before tuning composition

Trade-offs that matter in production

Operate it as a publishing control

References

Further reading

Building a 3-Queue Marketplace Notification Center Backend for Event Audit Polling

How should a notification center backend expose email SMS delivery history and audit log polling?

A runnable three-queue example

Template ownership is an architecture decision

Failure modes worth testing before deployment

Operate the polling API as a projection

References

Seller Order Event Notifications — Email/SMS Idempotency Across Rate Limits

Govern the notification intent as an order record

Implement one narrow send adapter

What should email and SMS APIs own under rate-limit pressure?

Rehearse fallback before choosing a provider

References

OTP Login Reliability — 3 Signals Before Adding an Email Backup Channel

When should SMS OTP failure trigger an email fallback for login?

What does the provider comparison actually decide?

Build the fallback as an evaluated state machine

Measure this before copying the choice

References

FastAPI Marketplace Order SMS: US/EU Batches, Suppressions, and Status Polling

What reliability budget should a marketplace set before comparing providers?

Own the crash window before choosing transport

How should a web app handle US/EU SMS batch status without webhooks?

Replay the adapter contract in Python

Release only with reconciliation evidence

References

US/EU Ask-Your-Docs Cost: Comparing Cheap Embeddings and Rerank Alternatives

What must a US/EU tenant ledger capture before semantic search is tuned?

How should an ask-your-docs pipeline compare embeddings, rerank, and cost?

When does selective reranking earn its place?

How do stable adapters make retrieval comparisons fair?

What is the right alternative when embeddings or rerank do not fit?

References

Python Metrics Dashboard APIs Explained (Statsig, PostHog, and Grafana Cloud Compared)

What should a budget metrics dashboard measure around a feature rollout?

How should a US-EU startup compare Statsig, PostHog, and Grafana Cloud?

A focused Python example for nightly structured logs

The limitations that change the choice

What to measure before copying this choice

References

Silent Success Tests for Authenticated Web App Chatbot Streaming Backend APIs

What should an authenticated web app demand from a chatbot streaming backend API?

Treat streaming as an application contract

Compare failure semantics before feature lists

Measure the choice from notebook to production

References

Node.js Chatbot API Cost Checks: Compare OpenAI, Anthropic, Google, and JSON Schema

What should a Node.js chatbot compare before choosing a model gateway?

A minimal streaming probe you can move from notebook to service

How do OpenAI, Anthropic, Google, OpenRouter, and a unified API differ?

Where should streaming, JSON schema, and tool calling stop?

A rollout that keeps cost and quality visible