Short answer: put a durable queue behind the Express.js request, keep the wrapper contract narrower than the provider API, and record one immutable delivery timeline per message. For a B2B SaaS report, store the generated attachment once, enqueue its reference, return an accepted response, and let a worker perform bounded retries without an SMTP relay.
The bill has three parts: provider attempts, attachment retention, and operational evidence. Write it as total = sends + attachment_byte_days + queue_and_log_byte_days before choosing infrastructure. A retry multiplies the send term; copying a report into every job and log multiplies the storage term. The useful change is to retain one report object and pass its identifier through the queue. Deliberately stop keeping duplicate MIME payloads and verbose success responses. The cost is forensic depth: if the retained event fields are too thin, a later compliance review can prove that a job ran but not what policy governed it.
That trade is acceptable only when the evidence record is designed first.
How can an Express.js transactional email API wrapper implement queue logging and retry?
Treat the HTTP handler, queue, worker, and evidence store as separate trust boundaries. The handler authenticates the application request, validates a report identifier and recipient, chooses a template version, and creates a stable message ID. It does not send. A fast provider response still leaves the application exposed to client disconnects, duplicate form submissions, and ambiguous timeouts, so doing network delivery inside the route gives the least reliable component ownership of retry policy.
The queue item should describe intent, not a provider-shaped request. For a welcome email it may contain a user ID and template version. For a generated report it needs a tenant ID, recipient, report object ID, content digest, policy version, and message ID. Keep authorization claims out of the payload when the worker can resolve them from the tenant and policy records. Queue data gets copied into dashboards and dead-letter tools more often than people expect — an email address that doesn't need to be there becomes another disclosure surface.
The worker resolves the attachment, verifies its digest, renders the message, and calls a small transport interface. That interface needs a send operation and a normalized result. It doesn't need every setting exposed by a commercial email API. Provider-specific options belong in the adapter, where they can be reviewed and replaced without teaching the application about a second vocabulary.
This is the whole control flow:
- Accept one logical send and assign one message ID.
- Enqueue by that ID with a deduplication rule.
- Resolve and verify the report in the worker.
- Submit through the current HTTP email adapter.
- Append the attempt outcome to the evidence timeline.
- Retry only outcomes classified as transient; quarantine exhausted or permanent failures.
No SMTP relay is required for this boundary. An HTTP adapter can authenticate to an email delivery API while the domain layer remains unaware of its wire format. The catch is portability: HTTP APIs differ in authentication, attachment encoding, idempotency behavior, and response semantics. If direct SMTP interoperability with existing mail infrastructure is a hard requirement, stick with an SMTP-capable transport behind the same interface.
Make retry a state machine, not a loop
“Retry three times” is not a policy. A defensible policy says which failure classes may repeat, how long each message may remain actionable, what makes two attempts part of the same logical send, and who owns the final disposition. Without those answers, the queue merely moves accidental behavior away from the request thread.
Use a stable message ID across attempts and a distinct attempt ID for each provider call. The stable ID prevents an application retry from becoming a fresh welcome email or a second report. The attempt ID preserves causality. A worker lease should be shorter than the business expiry window, and a lost lease must not silently create a new logical message. Exact durations depend on the queue and delivery API; I'm not sure there is a universal interval that is honest for both a welcome message and a report with a contractual deadline. Production evidence from queue latency and accepted-to-delivered timing should settle it.
The reference logic below is intentionally transport-neutral. The surrounding application may be Express.js, but retry classification belongs in the worker contract, and the mandated code style here makes that contract easy to inspect without implying a real provider route.
from dataclasses import dataclass
from enum import Enum
from typing import Protocol
class Outcome(Enum):
ACCEPTED = "accepted"
TRANSIENT = "transient"
PERMANENT = "permanent"
@dataclass(frozen=True)
class DeliveryIntent:
message_id: str
tenant_id: str
recipient: str
template_version: str
report_object_id: str
report_digest: str
policy_version: str
@dataclass(frozen=True)
class AttemptResult:
outcome: Outcome
provider_message_id: str | None
reason_code: str
class EmailTransport(Protocol):
def send(self, intent: DeliveryIntent, attachment: bytes) -> AttemptResult:
...
def decide_next_step(result: AttemptResult, attempt_number: int) -> str:
if result.outcome is Outcome.ACCEPTED:
return "complete"
if result.outcome is Outcome.PERMANENT:
return "quarantine"
if attempt_number >= 3:
return "quarantine"
return "retry_with_backoff"
Three attempts in this example are a local policy choice, not an industry constant. More attempts may raise the chance of eventual acceptance, but they also extend queue retention, consume provider calls, and increase the window in which a stale report can arrive. For an expired access grant, the correct result is quarantine, not persistence.
Keep a dead-letter or quarantine path, but don't turn it into an automatic replay bucket. Replaying requires a fresh authorization check, the original message ID, and a recorded operator or system decision. Otherwise a well-meaning repair can send a report after a user lost access.
Retention policy starts with the evidence ledger
Compliance evidence is a chain of decisions. For each logical message, record when the request was accepted, which tenant initiated it, which template and policy versions were selected, the attachment digest, the queue transition, every attempt classification, the provider message ID when one exists, and the terminal disposition. Add timestamps and correlation IDs. Keep the event schema append-only even if the storage system offers updates; corrections should be new events that point to the event they supersede.
Do not put the subject, HTML body, attachment bytes, bearer token, or full provider response in ordinary logs. Redact the recipient in broad operational views and restrict the evidence store separately. A digest can show that the same report artifact moved through generation and delivery without making that artifact readable to every person with log access. It cannot prove that the recipient opened or understood the report.
Open tracking is especially weak evidence. Apple's Mail Privacy Protection downloads remote content in the background when enabled, preventing senders from learning Mail activity and masking the recipient's IP address. An “open” event therefore should not authorize an account transition, satisfy a report acknowledgement requirement, or close an audit case. Use an authenticated in-product acknowledgement when acknowledgement is the business event.
A compact evidence model is easier to test than free-form lines:
| Event | Required evidence | Avoid retaining |
|---|---|---|
message.accepted |
message ID, tenant ID, policy version, template version | rendered body |
attachment.bound |
object ID, digest, authorization decision | attachment bytes |
attempt.finished |
attempt ID, class, reason code, provider message ID | raw response and credentials |
message.closed |
terminal state, timestamp, disposition reason | duplicate prior events |
The long paragraph here matters because retention is where apparently small implementation choices turn into governance obligations. Queue payloads, application logs, delivery dashboards, object storage, backups, analytics exports, and support screenshots can each preserve a different slice of one email. Deleting the attachment object while leaving its MIME representation in a dead-letter job doesn't meet the intended minimization rule. Map every copy, assign an owner and retention class, test deletion, and make the evidence event contain only the reference and digest needed for later verification. Your mileage may vary on retention periods because contracts and legal duties differ; the architecture should make those periods configurable by data class rather than burying one duration in worker code.
Keep less.
Delivery status stays outside the business workflow
Provider acceptance means the delivery service accepted a request. It is not proof of inbox placement, human reading, or permission to disclose a report. Model queued, attempting, accepted, failed, quarantined, and expired as delivery states. Model report_available, access_revoked, and acknowledged in the product domain. Joining those state machines is allowed through explicit events, but one must not impersonate the other.
This separation also keeps welcome email behavior sane. A delayed welcome message may still be useful; a delayed attachment may cross an authorization or freshness boundary. Before every report attempt, resolve current access and expiry rather than trusting the snapshot that created the job. If access is gone, close the logical message as expired or quarantined and preserve the policy decision as evidence.
Rate limits need the same boundary. Apply a tenant-level submission budget before enqueueing, a worker-level delivery budget before provider calls, and a recipient-level control for abuse-sensitive flows. SMS OTP systems face a sharper version of this problem: Twilio's guidance on SMS pumping recommends rate limits, geographic permissions, fraud monitoring, and other controls. Email and SMS aren't interchangeable, but the architectural lesson transfers: retry must never bypass the abuse decision that guarded the original request.
Rollout begins with duplicate injection
Unit tests for template rendering are useful, but the expensive failures sit between components. Exercise a client disconnect after acceptance, duplicate queue delivery, worker termination after the provider accepts but before the result is stored, attachment deletion before retry, access revocation while queued, permanent recipient rejection, and quarantine replay. Assert on the final state and evidence timeline, not just the number of mock calls.
Deployment deserves a compatibility rule: workers must read the previous queue schema while old application instances can still enqueue it. Add fields as optional, deploy readers before writers, and version templates and policies explicitly. A rolling deploy should not reinterpret an old report job under a new compliance policy without recording that decision.
Observability should answer four questions quickly: How old is the oldest actionable job? Which tenants are producing unusual retry volume? Which reason codes are filling quarantine? Can an operator reconstruct one message without viewing its content? Queue depth alone misses a single old job, while aggregate error rate hides a tenant-specific abuse burst. Alerts need both age and rate dimensions.
This design is not suitable when the application has no durable store for generated reports, cannot recheck authorization at send time, or must use an existing SMTP-only gateway. In those cases, fix the ownership boundary first or keep the SMTP transport and place the wrapper above it. An API-only adapter is also a poor choice when contractual controls require a capability the chosen delivery API cannot document. The wrapper reduces migration work; it cannot manufacture provider evidence.
References
- Apple, “Use Mail Privacy Protection on iPhone”: https://support.apple.com/guide/iphone/use-mail-privacy-protection-iphf084865c7/ios
- Twilio, “Preventing Toll Fraud (SMS Pumping)”: https://www.twilio.com/docs/verify/preventing-toll-fraud
Further reading
The two primary references above cover the privacy boundary around open tracking and the abuse boundary around retry-sensitive messaging flows:
Top comments (0)