DEV Community

nathanielbrooks0360
nathanielbrooks0360

Posted on

Node.js Email Event Polling Ledgers for Auditable Bounce and Complaint Handling

Use an append-only ledger for Node.js email bounce and complaint handling, committing each event polling checkpoint together with the suppression decisions it supports before allowing another settled-payment receipt to be dispatched. For a marketplace, the deciding constraint is whether an operator can reconstruct why an address was eligible at the moment an order receipt was released. A successful polling job is weak evidence on its own.

Short answer: for a Node.js transactional email application, treat bounce and complaint polling as an evidence ingestion job. Deduplicate events, record the decision, update suppression, and advance the cursor in one durable transaction; retry that transaction after failure, and handle ambiguous send attempts through a separate reconciliation path. The Go example below describes that boundary without depending on a delivery vendor.

This is an auditability runbook. It proposes an internal event model and operating policy, not a provider API contract or a legal determination about which messages a marketplace must send.

How should email event polling handle bounce and complaint suppression?

Start with the payment settlement record. Give the receipt intent a stable identity tied to the marketplace tenant, order, settlement, and receipt purpose, then record which recipient and template revision the intent selected. An example fixture might identify tenant market_a, order order_132, settlement settlement_7, and receipt revision 2. These are illustrative identifiers, not production observations. A repeated settlement notification should find the same intent rather than manufacture another receipt.

Now connect that intent to the eligibility decision, the send attempt, and any later delivery events. Keep event time and ingestion time separately: an event that arrives after dispatch can explain a later suppression without proving that the dispatcher knew about it earlier. Record the policy revision used for each decision as well. Otherwise a future policy change can make a historical decision appear inexplicable even when the application followed its rules at the time.

Evidence comes first.

A green job status doesn't prove eligibility.

Use a stable recipient identifier in ordinary operational records and restrict access to the address mapping and original event payload. Decide retention, deletion, access logging, and export requirements with the people responsible for the marketplace's obligations; an append-only application interface does not settle those questions, and copying every payload into an unrestricted log is a poor substitute for an evidence policy.

The event names in this note are internal normalization choices: accepted, delivered, deferred, hard_bounce, and complaint. Preserve the original source event identifier and classification alongside them. An adapter must establish what the source actually means by each event before production ingestion is enabled. Do not classify an unfamiliar event as successful delivery just to keep a dashboard green.

Detect a broken evidence chain before replaying mail

The signal that warrants this runbook is a receipt whose dispatch decision cannot be joined to the suppression state and event history that existed when it was released. Investigate checkpoint progress without matching ledger records, repeated complaints without matching suppression decisions, and receipt intents with several unresolved send attempts. Each condition points to a different repair; replaying the whole receipt queue can make all three harder to understand.

A checkpoint acknowledges ingestion, not delivery.

Adopt an explicit policy for the normalized event list. In this design, hard_bounce and complaint create an active suppression; deferred records a temporary outcome for investigation or bounded retry, while accepted and delivered add evidence without clearing any suppression. That last rule matters when an older delivery event arrives after a complaint. Clearing suppression because the latest ingested event looks positive would let arrival order override policy. A removal needs its own authorized decision and audit record.

Scope suppression deliberately. This example keys it by tenant and recipient, with receipt eligibility checked against that scope; a marketplace requiring a wider safety boundary must define and enforce it consistently across tenants and message purposes. Document which events block which purposes before rollout. Do not infer that a complaint about one message grants permission to move the next receipt to another channel.

For operations, define separate SLOs for settlement-to-receipt processing and event-to-enforced-suppression delay. Include polling interval, source availability, backlog drain time, and transaction latency in the second budget. A cursor that moves quickly while dispatch reads stale suppression state does not meet that objective. If freshness exceeds the agreed limit, hold affected receipt intents for review or recovery under a documented policy rather than quietly treating an empty poll as proof that sending is safe.

Commit the event page and its decisions together

The following Go code is an application-side contract for a poller serving the Node.js receipt system. It intentionally leaves storage and source adapters behind interfaces: the transactional requirements belong in their implementation and integration tests. A Page must come from a verified source adapter, and Before and After are opaque cursors. Run must serialize updates to the same stream checkpoint, verify its expected value, and commit all callback writes together or roll them all back.

package receiptpoll

import (
    "context"
    "fmt"
)

type Event struct {
    SourceID  string
    Tenant    string
    Recipient string
    Kind      string
}

type Page struct {
    Stream string
    Before string
    After  string
    Events []Event
}

type Tx interface {
    // InsertEvent enforces uniqueness on stream + source ID.
    // A duplicate with different contents must return an error.
    InsertEvent(context.Context, string, Event) (bool, error)
    // Suppress records the cause and applies the receipt policy.
    Suppress(context.Context, string, Event) error
    SaveCursor(context.Context, string, string) error
}

type Store interface {
    // Run checks the expected cursor under a lock and commits atomically.
    // All Tx calls must use the same database transaction.
    Run(context.Context, string, string, func(Tx) error) error
}

func ApplyPage(ctx context.Context, db Store, p Page) error {
    if p.Stream == "" {
        return fmt.Errorf("missing stream identity")
    }
    return db.Run(ctx, p.Stream, p.Before, func(tx Tx) error {
        for _, e := range p.Events {
            if e.SourceID == "" || e.Tenant == "" || e.Recipient == "" {
                return fmt.Errorf("incomplete event identity")
            }
            switch e.Kind {
            case "accepted", "delivered", "deferred", "hard_bounce", "complaint":
            default:
                return fmt.Errorf("unmapped event kind %q", e.Kind)
            }
            fresh, err := tx.InsertEvent(ctx, p.Stream, e)
            if err != nil {
                return err
            }
            if !fresh {
                continue
            }
            if e.Kind == "hard_bounce" || e.Kind == "complaint" {
                if err := tx.Suppress(ctx, p.Stream, e); err != nil {
                    return err
                }
            }
        }
        return tx.SaveCursor(ctx, p.Stream, p.After)
    })
}
Enter fullscreen mode Exit fullscreen mode

There is no network send inside this transaction. The ledger insert, suppression cause, active suppression update, and checkpoint must share a commit boundary, including when the page contains duplicate events. The production evidence schema should also carry the source classification, event and ingestion timestamps, receipt linkage where available, and policy revision described earlier; the example limits its fields to the ones needed to show control flow.

A crash before commit leaves the old checkpoint available for another fetch. A crash after commit but before the worker receives confirmation requires the worker to reload the checkpoint; the expected-cursor check prevents a stale page from moving it backward. An overlapping source page can still contain already recorded events, which the uniqueness constraint handles. Consider a synthetic page containing a complaint followed by a delivery event for the same recipient: if the worker stops after inserting the complaint but before applying suppression, neither write may survive alone. On replay, the transaction must insert that complaint again, apply its suppression, record the delivery event without removing the suppression, and advance the checkpoint only when all those writes can commit. Otherwise the deduplication check could skip a complaint whose safety decision was never applied. These properties depend on the Store contract being implemented, not on the presence of a retry loop.

Unknown classifications deliberately stop the page. Preserve the rejected payload in a restricted quarantine mechanism outside the failed transaction, alert on the stalled checkpoint, and repair the mapping through review. Quarantine without an owner and a recovery deadline just creates a second backlog. If the source lacks stable identifiers, replayable history, or a usable pagination boundary, establish an alternative reconciliation contract before claiming this design is safe.

The dispatcher must read current suppression before releasing a pending receipt and record the decision version it observed. Coordinate that check with the local dispatch state transition; otherwise the sender can select an address, wait in a queue, and send using an eligibility decision that predates a newly committed complaint. Even a coordinated local decision cannot recall an external send already in flight. Keep that limitation visible in the audit record.

Treat an uncertain send result separately from an uncertain ingestion commit. If a send request times out, keep its attempt identity and reconcile its outcome using the delivery system's documented capabilities before issuing another attempt. Do not promise exactly-once external delivery merely because the local event ledger deduplicates correctly.

Size the backlog and choose who owns the evidence

Capacity planning starts with event volume, not just receipt volume. For an explicitly hypothetical workload of 6,000 events per minute and a 60-second polling interval, a 3-minute polling interruption produces 18,000 events before recovery begins. At 300 events per page, that is 60 pages of existing backlog, while new events continue arriving. The worker needs sustained drain capacity above the arrival rate and enough history retention at the source to cover detection, repair, and replay. Those figures are planning inputs, not measured throughput.

Track oldest unprocessed event age, checkpoint age, duplicate count, quarantined events, transaction failures, and receipt intents held for stale suppression. Keep recipient identifiers out of metric labels. Alert on the freshness budget and backlog growth; a high duplicate count can be expected during replay, whereas an unmapped complaint classification needs investigation even at low volume.

A buy-vs-build decision should turn on evidence ownership and on-call obligations. Require an export-and-replay demonstration before choosing an operating model.

Operating model Evidence to verify Responsibility and trade-off
Buy managed ingestion Stable event identity, documented history window, exportability, checkpoint behavior Reduces internal ingestion work if the contract fits; the marketplace still owns receipt policy and must plan around external retention and export boundaries.
Build ingestion workers Atomic ledger and suppression writes, replay procedure, access controls Allows direct control over the evidence schema; the platform team owns backlog recovery, schema changes, and on-call coverage.
Buy transport and build the ledger Source-to-ledger reconciliation and durable receipt linkage Keeps the decision history under application control; requires maintaining the adapter and investigating disagreements across systems.

Count storage retention, adapter maintenance, incident review, and evidence export work in the operating budget. The trade-off is explicit: building the ledger buys control over retention and replay, but adds an on-call responsibility for every schema change and stalled cursor. Unit price can't answer who reconstructs a disputed receipt after source history expires.

Verify the policy and roll back without losing history

Use synthetic receipts and events to test the transaction boundary before exposing it to production dispatch. Deliver the same complaint twice, interrupt the worker before and after commit, submit an older delivered event after suppression, and run two workers against the same checkpoint. Verify that the complaint cause remains traceable, suppression remains active, and a stale worker cannot overwrite cursor progress. Also test a duplicate identifier with changed content: silently accepting that collision would conceal evidence corruption.

Exercise the unpleasant path.

Feed an unknown classification, exhaust the freshness budget, and simulate an ambiguous send response. The expected results are a visible ingestion hold, receipt handling consistent with the freshness policy, and an unresolved send attempt awaiting reconciliation. None of those cases should create an automatic stream of new receipt attempts. For every test receipt, export enough records to reconstruct settlement, intent, eligibility decision, attempt, and subsequent event history without querying an operator's terminal logs.

Roll out the mapper in observation mode first, comparing proposed decisions with the active policy using approved synthetic or appropriately controlled data. Then enable enforcement for a bounded slice of traffic with explicit lag and error thresholds. A rollback should pause the affected dispatcher and poller or restore a compatible implementation while preserving committed ledger records and active suppressions. Never clear the suppression table or rewind a checkpoint as a convenient way to undo a deployment. Replay into a versioned projection when decision logic needs correction, reconcile the result, and resume held intents through the normal eligibility check.

Keep unsubscribe policy separate from delivery classification. RFC 8058 specifies one-click unsubscribe; it does not define this internal bounce taxonomy or determine a marketplace's obligations for payment receipts. The OWASP recovery guidance is relevant if a receipt flow links into account recovery: keep that flow's token and enumeration protections separate from delivery troubleshooting. Neither reference makes an emailed receipt proof that its recipient read it.

The release criterion is a reconstructable decision: a settled order has one traceable receipt intent, an explainable eligibility decision, and a send history that survives event replay and application rollback. Require that evidence before calling the polling job ready.

References

Top comments (0)