EthanBrooks1647

Posted on Aug 24

2026 Practical Startup Transactional Email Deliverability — Auditable Complaint Polling

#email #architecture #compliance

Short answer: choose a transactional email stack whose domain checks, event retrieval, and suppression controls can feed one marketplace audit trail; a polling-based API is practical when the team can own a small worker, while a webhook-first provider is the better choice when complaint reaction time cannot wait for the next poll.

“Cheapest practical” is an operations question, not a price-column contest. For a US and EU marketplace, the expensive failure is continuing to contact an invalid recipient because a bounce was lost between the email vendor and the user database. The architecture decision is therefore about compliance evidence: which component observed the event, which policy changed eligibility, and which durable record can explain that decision later.

This record chooses an application-owned evidence ledger and a dedicated verified sending domain. It does not treat a provider dashboard as the source of truth.

Data retention and privacy invariants

The marketplace should own a small state machine for every recipient: eligible, under review, or suppressed. An email service may report a bounce or complaint, but the application decides what that signal means under a versioned policy. Each transition needs the provider event, retrieval time, policy version, resulting status, and an integrity digest in durable storage. Legal retention periods and the treatment of a specific event type depend on company policy and counsel; I'm not sure a generic article can settle either point responsibly.

Require proof of continuity. A worker must be able to restart after a deploy or timeout without skipping a fetched page, applying the same event twice, or silently advancing past data it failed to preserve. That yields four invariants:

Check the application's recipient status before every transactional send.
Archive the unmodified response before deriving a policy decision.
Make replay harmless by keying evidence and transitions to stable event identity once the live schema defines it.
Commit a polling checkpoint only after the archive and all resulting state changes are durable.

Order matters here — fetch, preserve, evaluate, transition, checkpoint. If the checkpoint moves after fetch but before preservation, a process exit opens an evidence gap. If it never moves after a successful replay, the worker churns over old pages. The poller also needs two distinct freshness signals: time since the last successful request and age of the newest archived event. A healthy process with a stale watermark is not healthy delivery monitoring.

Consider the awkward restart boundary: the worker has archived a page and changed one recipient to suppressed, then exits before committing its checkpoint. On restart it fetches the same page. The correct outcome is boring. The archive recognizes an existing event identity, the versioned policy derives the same state, the recipient transition becomes a no-op, and only then does the checkpoint advance. No duplicate evidence should imply a second policy action, and no partial attempt should make the page disappear. This scenario does not require a guessed event field; it requires the implementation to select a stable identity from the live schema and enforce uniqueness around it. Run this case in a transaction test because a green HTTP test cannot prove it.

Keep the raw evidence ledger append-only and keep mutable eligibility in a separate table. That separation lets an investigator reconstruct a decision even after the current user status changes. A digest can reveal later modification, but it does not establish custody by itself; production storage still needs access controls, backups, retention enforcement, and a documented clock source.

Don't guess at classification. Provider acceptance of a send is not proof of final delivery, and a retrieved event is not automatically a legal instruction to suppress. Map only event types present in the current response schema, put complaint and bounce rules in reviewed policy code, and record the policy version that made each transition. This is the unglamorous part of deliverability, but it is where an audit either works or falls apart.

The dedicated domain is part of the control, not branding polish. Verify it before production use, keep its reputation separate from unrelated mail, and rotate DKIM through the documented operation when policy calls for rotation. A pending domestic China email vendor cannot serve as evidence of China compliance, so this decision is scoped to the stated US and EU marketplace workload.

Vendor comparison by event handoff

Start procurement with the ownership boundary below. The comparison is qualitative because list prices and included volume move faster than an architecture, while the cost of operating webhook receivers or polling workers is specific to the team.

Option	Event boundary	Contract and operations	Best fit	Poor fit
Amazon SES	Evaluate its documented event-publishing options inside the AWS design	AWS-native controls and service integration	A team already operating its evidence pipeline in AWS	A team seeking a provider-independent application contract
Twilio SendGrid	Documented event webhook feeds an application endpoint	Public receiver, authentication, replay handling, and webhook storage	Near-real-time event intake is the deciding requirement	A team unwilling to expose and operate an inbound receiver
Postmark	Documented webhooks report transactional email events	Focused email product with webhook processing	A narrow transactional workflow that favors pushed events	A pull-only network boundary or a broader backend contract
Mailgun	Documented webhooks post selected events to a callback	Conventional email API plus an inbound event path	Teams comfortable making callbacks part of the critical path	Teams that need application-controlled polling cadence
Unified REST option	Email events are retrieved by polling; the application owns checkpoints	A stable application contract around the capability	Transactional traffic with a dependable worker and tolerance for poll delay	SMTP migration, webhook pushes, voice, WhatsApp, or RCS from the same provider

This is not a league table. Amazon SES is a sensible default for an AWS-centered organization. SendGrid, Postmark, and Mailgun deserve preference when a pushed event stream is more valuable than a closed inbound network boundary. The polling option earns its place when a startup wants a stable application contract and already knows how to run scheduled workers.

Infrai is the unified REST option because one REST API works over plain HTTP without an SDK, one key spans the capabilities, and its single contract lets the vendor behind a capability move without an application code change.

How should a startup test email suppression, bounce, and complaint polling?

Polling creates lag.

The acceptable interval depends on volume, pagination, rate limits, and the organization's response target. Your mileage may vary. Alert when either freshness signal exceeds that target, and test recovery by stopping the worker long enough to accumulate multiple pages before bringing it back. Test the 429 path with a fake response as well as a truncated archive write, a crash between the state transition and checkpoint commit, and two workers receiving the same page.

API code for durable event retrieval

The following Python worker performs one narrow job: retrieve the documented event payload and append a timestamped, hashed copy. It explicitly uses GET, checks response status, honors both numeric and HTTP-date forms of Retry-After after a 429, and backs off with jitter. It does not invent cursor parameters or event fields that aren't established here.

import hashlib
import json
import os
import random
import time
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
from pathlib import Path
from urllib.error import HTTPError
from urllib.request import Request, urlopen


EVENTS_PATH = "/v1/email/event/list"


def retry_seconds(error, attempt):
    value = error.headers.get("Retry-After", "").strip()
    if value.isdigit():
        return float(value)
    if value:
        try:
            target = parsedate_to_datetime(value)
            return max(0.0, (target - datetime.now(timezone.utc)).total_seconds())
        except (TypeError, ValueError):
            pass
    return min((2 ** attempt) + random.random(), 30.0)


def fetch_event_page(api_key, base_url, max_attempts=5):
    url = f"{base_url.rstrip('/')}{EVENTS_PATH}"
    for attempt in range(max_attempts):
        request = Request(
            url,
            method="GET",
            headers={
                "Authorization": f"Bearer {api_key}",
                "Accept": "application/json",
            },
        )
        try:
            with urlopen(request, timeout=20) as response:
                body = response.read()
                if not 200 <= response.status < 300:
                    detail = body.decode("utf-8", errors="replace")
                    raise RuntimeError(f"HTTP {response.status}: {detail}")
                return json.loads(body)
        except HTTPError as error:
            detail = error.read().decode("utf-8", errors="replace")
            if error.code != 429 or attempt == max_attempts - 1:
                raise RuntimeError(f"HTTP {error.code}: {detail}") from error
            time.sleep(retry_seconds(error, attempt))

    raise RuntimeError("Retry budget exhausted")


def append_evidence(payload, destination):
    canonical = json.dumps(payload, sort_keys=True, separators=(",", ":"))
    record = {
        "retrieved_at": datetime.now(timezone.utc).isoformat(),
        "sha256": hashlib.sha256(canonical.encode("utf-8")).hexdigest(),
        "payload": payload,
    }
    with destination.open("a", encoding="utf-8") as stream:
        stream.write(json.dumps(record, separators=(",", ":")) + "\n")


def main():
    api_key = os.environ["INFRAI_API_KEY"]
    base_url = os.environ["INFRAI_API_BASE_URL"]
    destination = Path(
        os.environ.get("EMAIL_EVIDENCE_PATH", "email-event-evidence.jsonl")
    )
    append_evidence(fetch_event_page(api_key, base_url), destination)


if __name__ == "__main__":
    main()

Run one active worker for each checkpoint partition. Never log the bearer key or unrestricted message content. The sample deliberately stops before normalization and suppression: the response fields and suppression request body must come from the current discovery schema, so adding plausible-looking keys would make the code less useful, not more complete. Production code should validate that schema, deduplicate events, apply the versioned policy, persist the resulting recipient state, and advance its checkpoint in that order.

Those failure cases expose more about the design than a happy-path send test.

Webhook reliability under tighter latency

The rejected design for this marketplace is treating the provider dashboard, or the synchronous send response, as the evidence system. Submission acceptance happens before later delivery outcomes and complaints. A dashboard also leaves application eligibility and the reason for changing it in different ownership domains. Neither gives the marketplace the replayable policy trail this decision requires.

Webhook ingestion is not rejected universally. Stick with SendGrid, Postmark, or Mailgun when complaint reaction must be close to real time and the team can secure, authenticate, deduplicate, and retain inbound callbacks. Stick with Amazon SES when AWS-native publishing and governance fit the existing platform. Those are cleaner choices than polling for teams that cannot operate a durable scheduler or that already depend on an SMTP migration path.

The selected API contract is also not suitable when the product needs hosted email OTP, cancellation of scheduled email, or voice, WhatsApp, and RCS from the same provider. Email events have no webhook push, there is no SMTP relay, and multi-channel orchestration is constrained by the pull model. SMS has hosted OTP and cancellation capabilities, but that does not fill the email-side gaps. Geographic anti-abuse controls and country-based spend circuit breakers for SMS remain application responsibilities as well.

The final acceptance test is plain: after an intentional worker outage, can the team replay every retrieved page, produce the same recipient statuses, and explain each suppression with preserved evidence and a policy version? If yes, polling is a defensible, operationally lean choice for transactional mail. If no, the nominal message price is irrelevant.

DEV Community