BrockFletcher1438

Posted on Oct 10

FastAPI Startup Transactional Email Stack — Practical Deliverability and Suppression Control

#email #architecture #python

Short answer: make a local, append-only recipient-status ledger the authority for every transactional send. Feed authenticated bounce and complaint events into it, suppress before enqueueing, poll only to reconcile missing events, and retain the evidence that explains each decision. This is the cheapest practical architecture in operational terms because it prevents repeated delivery attempts from becoming reputation, support, and compliance work; the transport can change without rewriting the policy boundary.

For a marketplace, the invariant is blunt: an address with an active hard-bounce, complaint, or policy suppression must never reach the outbound transport. A dedicated sending domain and verified identity establish who may send, but neither replaces recipient-level suppression. FastAPI belongs at the decision edge, while workers, event ingestion, and reconciliation sit behind it.

What should a practical startup transactional email deliverability stack own?

Delivery is asynchronous. The HTTP request that accepts an order receipt cannot prove that a mailbox will accept it, and event delivery can be delayed, duplicated, or reordered. Build around those boundaries instead of treating a successful submission as delivery.

Duplicates happen.

Use five invariants:

Normalize an email address once, then derive a stable keyed hash for logs and joins. Do not scatter raw addresses through observability systems.
Authenticate every event before parsing it. Invalid signatures fail closed and do not mutate suppression state.
Make event ingestion idempotent on the transport's event identifier. A duplicate complaint must not create a second state transition.
Apply the most restrictive active state before enqueueing. No race-prone check belongs inside the transport adapter.
Preserve the received event, verification result, source, timestamps, and resulting decision as compliance evidence. Restrict access and retention according to the applicable purpose and policy.

One subtle edge case matters: a later delivery event does not erase an earlier complaint. Events describe observations; policy decides state. Keep both. SMTP enhanced status codes add useful evidence but do not make the policy decision by themselves: RFC 3463 defines a three-digit structure whose first digit distinguishes success, persistent transient failure, and permanent failure, while the remaining digits describe subject and detail. Store the original code and diagnostic, map the observation into policy, and keep that mapping versioned. If an operator eventually clears a suppression after validating a recipient-controlled correction, record that as a new, attributable action rather than editing history.

The failure boundary is narrow. A malformed callback may enter quarantine, and a reconciliation job may lag, but neither condition grants permission to send. Unknown status can be retried through a controlled path; known suppression cannot.

Decision record: where should suppression live?

The relevant cost is not the smallest advertised unit price. It is the engineering and compliance burden of proving why a message was or was not attempted across US and EU operations. Compare ownership models on that axis.

Option	Send-time authority	Evidence quality	Failure behavior	Operational trade-off
Transport-only suppression	External transport	Depends on exported history and retention	Account changes can split policy state	Lowest initial code, highest coupling
Local ledger plus event ingestion	Application database	Connects event, policy, and decision	Sending stops safely when status is uncertain	More schema and worker ownership
Local ledger plus polling only	Application database	Reconciliation snapshots are available	Suppression may lag until the next poll	Useful without signed events, but slower

The decision is the middle row: a local ledger fed by authenticated events, with scheduled polling as a backstop. It gives the marketplace one enforcement point for receipts, seller alerts, password resets, and other transactional mail. It also keeps transport-specific payloads at the perimeter.

This choice has a real price. The team owns migrations, idempotency, access control, retention, and on-call diagnostics. In return, a provider migration does not reset suppression memory, and an audit question does not require correlating several dashboards by hand.

That trade-off is deliberate.

Critical path in FastAPI and Python

The critical path should be boring. The API records intent, checks authoritative state in the same database transaction, and enqueues only an allowed recipient. Transport calls occur after commit so a slow remote service cannot hold database locks.

from dataclasses import dataclass
from enum import StrEnum
from hashlib import sha256
from hmac import new as hmac_new, compare_digest

class RecipientState(StrEnum):
    ALLOWED = "allowed"
    HARD_BOUNCE = "hard_bounce"
    COMPLAINT = "complaint"
    POLICY_BLOCK = "policy_block"

@dataclass(frozen=True)
class DeliveryEvent:
    event_id: str
    recipient_hash: str
    state: RecipientState
    occurred_at: str
    raw_digest: str

def recipient_key(address: str, secret: bytes) -> str:
    normalized = address.strip().casefold().encode()
    return hmac_new(secret, normalized, sha256).hexdigest()

def ingest_event(payload: bytes, signature: str, adapter, ledger) -> bool:
    expected = adapter.signature(payload)
    if not compare_digest(expected, signature):
        raise ValueError("invalid event signature")

    event = adapter.parse(payload)
    # The event ID is a unique idempotency key.
    return ledger.insert_event_and_apply_state(event)

def queue_if_allowed(address: str, message, secret: bytes, ledger, queue) -> str:
    key = recipient_key(address, secret)
    with ledger.transaction() as tx:
        state = tx.locked_recipient_state(key)
        if state is not RecipientState.ALLOWED:
            tx.record_decision(key, "suppressed", state.value)
            return "suppressed"
        job_id = tx.create_outbox_job(key, message)
        tx.record_decision(key, "queued", job_id)
    queue.publish(job_id)
    return "queued"

The adapter is deliberately small: verify, parse, and map. Its parser should map a transport's detailed taxonomy into the marketplace policy enum without discarding the original payload digest. The ledger performs the atomic work. An outbox dispatcher can recover jobs committed before a queue publish failed, closing the awkward gap between database commit and broker acknowledgement.

Do not log the message body or full recipient address to debug this path. Record the keyed recipient identifier, internal message identifier, event identifier, policy version, and transition result. Those fields are more useful during an incident, and they reduce unnecessary exposure.

Less data helps.

Reconciliation compares transport-side suppressed recipients and recent terminal events with local state. Page through bounded time windows, checkpoint the cursor, and make every imported event idempotent. Alert on age of the oldest unreconciled window, rejected signatures, quarantine depth, and attempted sends blocked by policy. Raw counts alone hide a stalled pipeline.

Verification, rollout, and evidence

Domain verification belongs in deployment readiness, not on the request path. Verify the dedicated transactional identity and required DNS records before enabling production traffic, then monitor their continued presence. Keep marketing and transactional policy separate even if they share infrastructure; their consent basis, content, and operator expectations differ.

Test state transitions more aggressively than happy-path delivery. Replay the same hard bounce twice. Deliver a complaint after a newer-looking delivery event. Race two workers against the same recipient. Corrupt a signature. Pause event consumption while reconciliation runs. Each case should end in a deterministic state and leave an evidence trail.

Roll out in observation mode first: calculate the local decision while the existing path remains authoritative, then compare outcomes without sending extra mail. Any disagreement needs classification before enforcement. After cutover, a kill switch should halt outbound jobs without disabling event ingestion; losing feedback during a send freeze makes recovery harder.

For evidence, retain enough to answer four questions: what arrived, how authenticity was checked, which policy version evaluated it, and what action followed. EU data-protection obligations push toward purpose limitation, data minimization, storage limitation, and accountable controls. Retention should be a documented policy, not an arbitrary forever table. US obligations vary with message class and jurisdiction, so legal review must define the policy; the architecture's job is to enforce and evidence it consistently.

Why reject transport-only suppression?

Transport-only suppression is attractive for a small system with one account, one region, and no migration horizon. It can be valid when the transport's retention, export, access controls, and event guarantees satisfy the organization's evidence requirements. That is the rejected option here, not a broken one.

A marketplace has a wider failure surface. Multiple services may send to the same buyer, account boundaries can multiply, and a transport migration can overlap. A local decision point prevents one service from retrying an address another service already learned was invalid. It also makes the conservative behavior explicit during outages: if current recipient status cannot be read, defer the send rather than guessing.

Polling-only is another legitimate fit for low-volume systems whose transport cannot emit authenticated events. Its weakness is the interval: every minute between polls is a window in which another attempt may be queued. Use it knowingly, measure reconciliation lag, and shorten the interval for terminal events without turning the poller into an unbounded scan.

The durable decision is simple. Own suppression state and its audit trail; rent transport. A dedicated domain, authenticated event intake, idempotent transitions, and reconciliation form one control system. Optimize transport fees after those boundaries are correct, because a cheap send that should have been suppressed is still the expensive one.

DEV Community