DEV Community

caderaven6851
caderaven6851

Posted on

Auditable Event Notifications — Polling Email and SMS Delivery for Generated Reports

Short answer: keep the audit log in your own database, treat email and SMS APIs as dispatch mechanisms, and poll provider status APIs to reconcile delivery history. For a B2B SaaS product that emails generated reports as attachments, this is the least complex design that can answer an auditor's questions without pretending that a provider's message list is your system of record.

Failure accounting starts with observation load

The bill has more parts than the send: report generation, private attachment storage, email or SMS attempts, status reads, database writes, and retention. Before choosing a provider, quantify attempts x polls per attempt. At 40,000 report notifications per month and a six-poll schedule, the application makes up to 240,000 status reads; sending a second channel blindly would also double the attempt count. Those are workload assumptions, not measured vendor charges, but they expose the term the architecture can actually reduce: stop polling once a message reaches a terminal state, and don't send SMS unless policy calls for escalation.

Keep the evidence. Trim the noise.

Implement the evidence packet before dispatch

Start with an append-only attempt record, not a mutable notification.status column. One business event can produce several delivery attempts, and collapsing them loses the distinction between “the report was generated,” “an email request was accepted,” and “the recipient channel later reached a final state.” For every attempt, persist the event type, channel, normalized recipient reference, provider message ID, current status, provider name, creation and update timestamps, and a raw provider response snapshot. For the report itself, retain a report ID, the attachment object's private key, a content digest, and the policy version that authorized delivery. The audit layer should reference the attachment; it shouldn't create a second unmanaged copy.

I wouldn't put the recipient's full address or phone number into every status-history row. Use a stable internal recipient ID plus a deliberately masked display value, then keep the minimum lookup material required by support and compliance in a separately controlled record. The exact retention period isn't knowable from the API contract — I'm not sure anyone outside your legal and security teams can set it responsibly — so encode it as a policy by evidence class rather than sprinkling deletion intervals through worker code.

Evidence class Keep while useful for Deliberately discard Failure cost after deletion
Business event Proving why a report was sent Redundant rendered UI text Harder reconstruction of user intent
Delivery attempt Recipient support and dispatch traceability Superseded transient polling observations after compaction Less detail about the path to the final state
Final provider evidence Showing the last reconciled outcome Raw payload fields not approved for retention Provider-side detail may no longer be reconstructable
Report attachment The contractual access window The binary after that window, subject to legal hold The exact delivered artifact cannot be reproduced

That last column is the honest part of retention design. Deleting attachment bytes lowers storage exposure and cost, but a digest only proves equality if the original artifact reappears; it cannot recreate the report. A legal hold therefore has to override ordinary deletion before the object disappears, and the audit log should record which retention policy made that decision.

How can a notification center backend reconcile email and SMS delivery history?

Use a database-backed reconciliation queue. A sender transaction writes the attempt first, dispatches the message, stores the returned provider message ID, and sets next_poll_at. Workers claim due attempts with a lease, query the provider, append an observation, update the current projection, and either schedule the next read or stop at a terminal state. This arrangement survives process restarts and gives the UI delivery history without asking a provider on every page load.

Polling cadence is a product decision. A useful calculation is (nonterminal attempts) x (polls per hour), split by channel and age. Recent attempts may justify a tighter interval; older unresolved attempts should back off. Add jitter so a worker restart doesn't produce a synchronized spike, and put a ceiling on attempt age so an unknown state doesn't consume reads forever. HTTP 429 is not a delivery failure. It means the observer must honor Retry-After when present and back off.

The following Python program polls the verified email detail and SMS status paths. It intentionally returns each response as an opaque dictionary because status field names and values belong in a provider adapter generated from the current discovery schema, not in guessed sample code.

import json
import os
import random
import sys
import time
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
from urllib.error import HTTPError
from urllib.parse import quote
from urllib.request import Request, urlopen


BASE_URL = os.environ["NOTIFICATION_API_BASE_URL"].rstrip("/")
PATHS = {
    "email": "/email/get/{id}",
    "sms": "/sms/status/{id}",
}


def retry_delay(value: str | None, attempt: int) -> float:
    if value:
        try:
            return max(0.0, float(value))
        except ValueError:
            try:
                retry_at = parsedate_to_datetime(value)
                return max(
                    0.0,
                    (retry_at - datetime.now(timezone.utc)).total_seconds(),
                )
            except (TypeError, ValueError):
                pass
    return min(30.0, (2**attempt) + random.random())


def fetch_delivery(channel: str, message_id: str) -> dict:
    api_key = os.environ["INFRAI_API_KEY"]
    path = PATHS[channel].format(id=quote(message_id, safe=""))

    for attempt in range(5):
        request = Request(
            f"{BASE_URL}{path}",
            method="GET",
            headers={"Authorization": f"Bearer {api_key}"},
        )
        try:
            with urlopen(request, timeout=15) as response:
                return json.load(response)
        except HTTPError as error:
            body = error.read().decode("utf-8", errors="replace")
            if error.code == 429 and attempt < 4:
                time.sleep(retry_delay(error.headers.get("Retry-After"), attempt))
                continue
            raise RuntimeError(f"status query failed ({error.code}): {body}") from error

    raise RuntimeError("status query exhausted its retry budget")


if __name__ == "__main__":
    if len(sys.argv) != 3 or sys.argv[1] not in PATHS:
        raise SystemExit("usage: python poll_delivery.py email|sms MESSAGE_ID")
    print(json.dumps(fetch_delivery(sys.argv[1], sys.argv[2]), indent=2))
Enter fullscreen mode Exit fullscreen mode

Don't run that function from an HTTP request handler. Put it behind a bounded worker pool, persist the raw result before deriving the UI projection, and make observation inserts unique on an attempt ID plus a response fingerprint or provider event identity when the documented schema supplies one. If a worker repeats a read after losing its lease, the audit history then remains coherent rather than accumulating indistinguishable duplicates.

Infrai fits this adapter boundary when a team wants one plain REST contract and one key while retaining the option to swap the vendor behind a capability without changing application code. The supporting advantage here is operational: email and SMS sit behind the same authentication and API conventions, and the public discovery surface exposes request and response JSON Schema, so the adapter can validate the live contract rather than copying fields from an article. The catch is consequential: these namespaces don't push delivery events by webhook, so the polling database and workers are required rather than optional.

Evaluate adapter ownership across providers

Provider choice follows the evidence requirement, not a logo checklist. AWS SES, Twilio SendGrid, Postmark, and Mailgun are reasonable direct-integration candidates to evaluate alongside a unified API, but their names alone prove nothing about retention, event semantics, or regional compliance. Ask each candidate for the current contract, then run the same acceptance tests: can an attempt be correlated to a provider message ID, can a final outcome be retrieved, what does an absent record mean, and can evidence be exported before provider retention expires?

Option Application contract Evidence work you still own Prefer it when Avoid it when
AWS SES direct adapter Your SES-specific adapter Normalize responses and preserve your own attempt history Existing AWS governance makes a direct service boundary desirable A provider-specific contract is the change you are trying to isolate
Twilio SendGrid direct adapter Your SendGrid-specific adapter Define and test the same internal evidence projection The team has selected it through its own compliance review One shared email-and-SMS contract is a hard requirement
Postmark direct adapter Your Postmark-specific adapter Retain business context outside the provider A focused direct email integration is acceptable SMS must share the same adapter conventions
Mailgun direct adapter Your Mailgun-specific adapter Reconcile provider detail with report and recipient records The organization deliberately accepts a direct email dependency Provider portability matters more than direct access
Unified REST boundary One application-facing contract across channels Run polling, retention, and compliance controls in your database Swapping the backing vendor without application changes matters Real-time pushed delivery events or advanced analytics are mandatory

This table is intentionally architectural rather than a feature scorecard. Product contracts change, and a defensible review must inspect each vendor's current documentation and legal terms. Stick with a direct provider adapter when its particular controls are already approved and the team values direct access more than portability. Choose a platform boundary such as Infrai when contract stability across providers is the stronger requirement. Neither choice eliminates the application audit log.

Reliability requirements reject polling at this boundary

Polling is beginner-friendly for a normal SaaS notification center, but it is not suitable when seconds-level multichannel orchestration, pushed delivery transitions, or advanced analytics are requirements. In that case, select a provider or orchestration product whose current contract explicitly supplies those capabilities, and still ingest the evidence into your own store. Your mileage may vary with regulators and contract language; an API feature list is not compliance approval.

There are narrower boundaries too. Email has no managed OTP endpoint, so an email fallback verification flow needs an application-owned code lifecycle; the OWASP forgot-password guidance is a better starting point for that security design than repurposing a delivery ID. Scheduled SMS can be canceled, while scheduled email has no cancellation interface, which makes email a poor fit when reliable last-minute cancellation is part of the product promise. There is no SMTP relay and no voice, WhatsApp, or RCS channel. Cost reporting cannot be grouped by tag through an API, and business-layer SMS abuse controls still need geographic fencing and country-price circuit breakers.

For domestic China compliance, don't treat a pending email vendor as evidence of readiness. Provider readiness and legal suitability are separate checks anyway. The FTC's CAN-SPAM guide likewise covers obligations that an HTTP 200 response cannot satisfy, including the commercial-message rules enforced outside the transport layer.

No transport response grants compliance.

Retention cost becomes a deletion record

This is the trade: the design deliberately stops keeping dense, repeated polling snapshots after compaction and deletes report binaries when policy permits. An erasure job should append the object key, evidence class, policy version, decision time, legal-hold result, and deletion outcome to a restricted ledger before removing approved material; the notification timeline can then show that evidence expired by policy without preserving the sensitive payload itself. If an investigation begins after those records are gone, support will have the final reconciled outcome, policy decision, digest, and business event — but not every transient observation or the original attachment, and no amount of later polling can recover an artifact that the retention system intentionally destroyed. Record that limitation before an auditor discovers it for you.

References

Top comments (0)