The constraint that changes this design is template ownership: the service that decides what a new-order message means must also retain enough evidence to explain which version it sent. A provider dashboard can report transport events, but it cannot reconstruct an application decision that was never recorded.
Short answer: keep seller-order templates and their immutable version identifiers in the Node.js application boundary, map every provider receipt to one internal message_id, and let the dashboard poll a compact four-state projection: queued, sent, delivered, or bounced.
This is deliberately a projection, not a copy of every webhook payload. Store raw receipts briefly for replay and audit, then retain the normalized state and timestamps according to an explicit support window. That choice makes the dashboard useful without turning each recipient address, template label, and provider response into a permanent high-cardinality observability dimension.
How should a Node.js dashboard poll email events by message ID?
Start with two records that have different owners. The order service owns notification intent: order identifier, seller identifier, template version, locale, and the internal message_id. The delivery adapter owns transport evidence: provider receipt identifier, normalized event, provider event time, ingestion time, and a hash used for deduplication. Do not let the adapter quietly choose copy or substitute a provider-hosted template. If it does, a support agent can see that mail was delivered but cannot prove which order wording the seller received.
A useful message_id is generated before the send attempt and remains stable across a controlled retry. The provider's receipt identifier is an attribute of an attempt, not the primary key of the business notification. This distinction matters when a timeout leaves the caller uncertain about acceptance: generating a fresh business identifier for every retry can make one order appear to have several unrelated notifications. The ingestion path should acknowledge an authenticated event only after durable write, deduplicate repeated receipts, and update the projection transactionally. The polling path then reads the projection through a cursor rather than scanning all messages on every refresh. A local, pseudonymous dashboard contract can look like this:
curl --get 'http://localhost:3000/dashboard/email-events' \
--data-urlencode 'message_id=msg_order_8f31' \
--data-urlencode 'after=evt_0042' \
--data-urlencode 'limit=50' \
--header 'Accept: application/json'
The response should include the current state, its effective time, a monotonically usable cursor, and whether more results remain. It should not expose the recipient address merely because the operator filtered by a message. Polling every two seconds does not improve the truthfulness of a delayed upstream receipt; it only multiplies read load. For a human support screen, begin with a modest interval, pause when the tab is hidden, use conditional requests when available, and back off after unchanged responses.
Keep the transition rule narrow. queued means the application committed the intent. sent means the delivery system accepted an attempt. delivered and bounced are terminal outcomes for this projection. Events can arrive twice or out of order, so compare event time and precedence rather than trusting arrival order. Never allow a late sent receipt to move a delivered message backward.
Order is not truth.
There is still uncertainty. A delivered receipt normally describes acceptance by the destination mail system, not proof that a human read the message or that it landed in the inbox. The dashboard label should say “delivered,” not “seen,” and the support playbook should preserve that distinction.
Template ownership determines what the dashboard can explain
Template ownership is an operational contract, not a preference about where HTML is convenient to edit. Application-owned templates make the exact version part of the same deployment and review trail as the order-notification logic. Provider-owned templates can give a communications team a separate editing workflow, but then the application must capture the provider template identifier and immutable version used for each send. If the provider only exposes a mutable name, the audit story is weak: today's new-order content may not be yesterday's content.
For this marketplace scenario, I would keep the canonical subject, text body, HTML body, and substitution schema with the notification service. The rendered payload can be handed to a delivery adapter, while the database stores a content hash and a non-secret template version. Do not put the rendered body in logs. An order email may contain seller and buyer data, and duplicating it across log storage expands both cost and access scope.
Three commercial choices illustrate why the boundary must be yours: Amazon SES, SendGrid, and Postmark each has its own API and operational surface, while the application still needs one stable definition of message_id, template version, and normalized status. This is not a ranking. A provider migration should require a new adapter and event mapper, not a rewrite of the support dashboard or a reinterpretation of historical rows.
The catch is that application ownership is not suitable when non-engineering editors must publish copy independently, with provider-side approvals and no service deployment. In that case, keep the template in the chosen delivery system, but require a versioned identifier in the send record and test that an old version remains resolvable during the support retention window. Stick with provider ownership when that editorial workflow is the harder constraint. Choose application ownership when reproducible order behavior and provider portability dominate.
The same decision affects testing. A template fixture should fail CI when a required substitution such as seller_display_name, order_number, or order_url is absent. A pre-production send checks MIME rendering and authentication configuration, but it does not replace a deterministic render test. Google's sender guidelines also make domain authentication and transport hygiene part of delivery engineering; a polished template cannot compensate for an unauthenticated sending setup.
Four states are enough; raw telemetry is not
The projection needs four states, but the event store may receive more detail. Resist turning every provider event type into a dashboard state. Operators need a stable answer to “where is this order notice?” Provider-specific distinctions can remain in a short-lived diagnostic field or raw receipt, then expire.
| State | Evidence represented | Allowed next state | Operator interpretation |
|---|---|---|---|
queued |
Notification intent committed |
sent, bounced
|
Waiting for transport evidence |
sent |
An attempt was accepted |
delivered, bounced
|
Destination outcome pending |
delivered |
Destination system accepted it | none | Transport complete, reading unproven |
bounced |
Delivery failed terminally | none | Review address or support path |
The table is intentionally smaller than a typical provider taxonomy. A transient attempt failure can stay inside retry control rather than becoming a durable seller-facing state. If all permitted attempts end without delivery, the projection moves to bounced with a sanitized reason category. Store the original diagnostic only where its sensitivity and retention are controlled.
Cardinality is where a simple dashboard becomes an expensive telemetry system. Suppose the marketplace sends 1,000,000 order notices in 30 days and records four receipts per notice. That is 4,000,000 event rows before retries and duplicates. At an illustrative 600 bytes per normalized row, the base data alone is about 2.4 GB; indexes, replicas, and raw payloads add to it. This arithmetic is a capacity example, not a measured benchmark. Measure actual row and index sizes in the chosen database before setting retention. Do not use message_id, order ID, seller ID, recipient domain, or bounce text as metric labels. Each can create a large or unbounded series set. Metrics should aggregate low-cardinality dimensions such as normalized state, environment, and perhaps a small, governed provider key. Investigation by one message_id belongs in an indexed event table. Logs should carry the identifier as a searchable field under shorter retention, not as a metric label. Sampling needs two policies. Aggregate success traffic may be sampled in diagnostic logs after the durable projection is updated; terminal bounces should be retained unsampled for the support window because they drive action. Never sample the state transition itself. If raw receipts are held for seven days and normalized outcomes for 90, write those periods down, calculate the resulting storage from observed daily volume and bytes per row, and revisit the estimate after a launch spike. I'm not sure what the right retention is for every marketplace because dispute periods and privacy obligations vary; legal and support owners must settle that input.
SMS fallback deserves a separate budget. A message that fits 160 GSM-7 characters may split when a Unicode character forces UCS-2 encoding, whose single-message limit is 70 characters; concatenated segments have lower per-segment limits. That means a seemingly small copy edit can change segment count. Track encoding and segment_count in the send record, but keep phone numbers and full text out of metric labels. Email delivery state and SMS segment accounting answer different questions, even when both notify the same seller.
How can the projection roll out without losing old evidence?
Begin in shadow mode. Generate the internal message_id and template version for every new-order notification, ingest receipts, and compute the four-state projection without replacing the existing support view. Compare aggregate counts by day and state; investigate gaps through sampled identifiers in the event table rather than exporting every identifier to metrics.
Then move the dashboard query to the new projection for a small operator group. Set an explicit polling interval and cursor contract, alert on ingestion age rather than on each individual pending message, and document how long a bounce remains searchable. This phase should also test duplicate delivery receipts, out-of-order sent and delivered events, a controlled retry with two attempt identifiers, and a template rollback that preserves the original version reference.
Only after those checks should the old view be retired. Keep the migration reversible until the new retention window has accumulated enough evidence for support review. The final design is modest: one business identifier, versioned template ownership, a monotonic projection, and intentional retention. That is enough to answer the seller's question without paying to preserve every byte forever.
Top comments (0)