Node.js Bulk Transactional Emails: 4 Batch States for Marketplace Onboarding

#node #email #architecture

Send each new-order email through a durable job with an idempotency key, then let a worker claim, send, and record the result. For a small marketplace, that is the least complex design that prevents an API retry from quietly becoming two seller notifications.

TL;DR: Treat a batch as an intake convenience, not as the delivery guarantee. Give every seller-order-template combination one stable key, persist four states (pending, sending, sent, and failed), and retry only records that have not reached sent. Keep email delivery separate from order creation so a slow mail system cannot hold the checkout path open.

The same shape works for welcome email after seller onboarding, but the concrete example below is a new-order alert. That distinction matters: a welcome campaign can often tolerate a delay, while an order alert drives fulfillment and needs a clearer retry policy.

How should Node.js batch bulk transactional onboarding emails?

Return the existing notification record. Do not enqueue a second logical message.

An order service may publish an event again because its acknowledgement was lost. A worker may also finish the network request and stop before it records success. Those are different duplicate paths, but they meet at the same design rule: the application owns a deterministic idempotency key and a durable state record. A useful key is sellerId:orderId:templateVersion; changing the template version is then an explicit decision, not an accidental resend. The plain-language flow is short. The order event creates or finds a notification row, a queue carries that row's ID, and a worker claims the row. The worker renders the message from stored business data, asks an email transport to accept it, and records the provider's message identifier. A batch endpoint can accept many events, but each item advances independently. One bad address must not erase the successful work beside it.

A small Node.js implementation

This runnable TypeScript example keeps storage and transport in memory so the delivery rules stay visible. In production, NotificationStore belongs behind a database transaction and the queue must be durable. The transport remains an interface; swapping providers should not rewrite order logic.

type DeliveryState = "pending" | "sending" | "sent" | "failed";

type OrderAlert = {
  sellerId: string;
  orderId: string;
  sellerEmail: string;
  itemCount: number;
  templateVersion: number;
};

type Notification = OrderAlert & {
  key: string;
  state: DeliveryState;
  attempts: number;
  transportId?: string;
  lastError?: string;
};

interface EmailTransport {
  send(input: {
    to: string;
    subject: string;
    text: string;
    idempotencyKey: string;
  }): Promise<{ id: string }>;
}

class NotificationStore {
  private readonly rows = new Map<string, Notification>();

  createOnce(input: OrderAlert): Notification {
    const key = `${input.sellerId}:${input.orderId}:${input.templateVersion}`;
    const existing = this.rows.get(key);
    if (existing) return existing;

    const row: Notification = {
      ...input,
      key,
      state: "pending",
      attempts: 0,
    };
    this.rows.set(key, row);
    return row;
  }

  get(key: string): Notification | undefined {
    return this.rows.get(key);
  }

  update(key: string, change: Partial<Notification>): Notification {
    const current = this.rows.get(key);
    if (!current) throw new Error(`Unknown notification: ${key}`);
    const next = { ...current, ...change };
    this.rows.set(key, next);
    return next;
  }
}

async function deliver(
  key: string,
  store: NotificationStore,
  transport: EmailTransport,
): Promise<void> {
  const row = store.get(key);
  if (!row || row.state === "sent") return;

  store.update(key, {
    state: "sending",
    attempts: row.attempts + 1,
    lastError: undefined,
  });

  try {
    const result = await transport.send({
      to: row.sellerEmail,
      subject: `New order ${row.orderId}`,
      text: `You have a new order containing ${row.itemCount} item(s).`,
      idempotencyKey: row.key,
    });
    store.update(key, { state: "sent", transportId: result.id });
  } catch (error) {
    const message = error instanceof Error ? error.message : "Unknown error";
    store.update(key, { state: "failed", lastError: message });
    throw error;
  }
}

async function enqueueBatch(
  alerts: OrderAlert[],
  store: NotificationStore,
  enqueue: (key: string) => Promise<void>,
): Promise<void> {
  for (const alert of alerts) {
    const row = store.createOnce(alert);
    if (row.state !== "sent") await enqueue(row.key);
  }
}

The in-memory createOnce operation explains the contract, but it is not concurrency-safe across processes. Back it with a unique constraint on the key. Claiming work also needs an atomic conditional update from pending or retryable failed to sending; otherwise two workers can read the same row and both send it.

There is still an unavoidable ambiguity window. A transport can accept the email and the worker can stop before persisting sent. Passing the same key to a transport that supports idempotent submission narrows that window. If the transport offers no such contract, the application can provide at-least-once attempts, but it cannot honestly promise exactly-once delivery across that network boundary.

That is the hard part.

Batch size is flow control, not correctness

Do not make one request containing 500 recipients the unit of recovery. A large envelope ties unrelated sellers together, makes per-recipient status harder to reason about, and increases the amount of work repeated after an ambiguous timeout. The database row should remain the recovery unit even if intake, queue reads, or transport submissions are grouped for efficiency.

Use bounded concurrency rather than Promise.all() over the entire campaign. The bound should come from observed transport limits, worker memory, database capacity, and the latency your order path can tolerate. No universal number is defensible. Start conservatively, record queue age and send latency, then raise concurrency while those signals remain inside the service objective.

Retries need classification. A timeout or temporary remote rejection may be retryable with exponential backoff and jitter. A malformed address is not repaired by waiting. Cap attempts, move exhausted jobs to a reviewable terminal path, and retain the last error category without storing more message content than operations actually needs.

Retries spend trust.

Four measurements carry most of the operational value: oldest pending job age, accepted-to-sent latency, attempts per notification, and permanent-failure rate. Counts should be segmented by template version and destination domain without putting full email addresses into metric labels. High-cardinality personal data makes both observability and privacy work worse.

Transactional does not mean policy-free

A seller order alert is triggered by an existing marketplace transaction; a welcome sequence that promotes upgrades is a different use. Do not rely on the label in the code. Classify each template by purpose, document the lawful basis used for EU recipients, and have counsel review the actual message and relationship. GDPR Article 6 lists lawful bases for processing, while Article 5 sets principles including purpose limitation and data minimization.

In the United States, the FTC explains that CAN-SPAM applies to commercial email and distinguishes transactional or relationship content. It also warns that mixed-content messages can be treated as commercial based on their primary purpose. Keep promotional modules out of operational order alerts unless the resulting obligations are deliberately handled.

SMS is a separate channel, not a shorter email. Consent, sender identification, opt-out handling, carrier filtering, and country-specific rules affect the design. CTIA publishes messaging principles and best practices for the US ecosystem, but those industry materials do not replace applicable law. Store consent evidence and suppression state outside the send worker so every transport follows the same decision.

The practical data rule is austere: pass the seller address, order reference, template version, locale, and the minimum fields needed to render the alert. Avoid putting full order contents in queue names, idempotency keys, logs, or error strings. A stable opaque order reference is enough for tracing.

Shipping the worker without guessing

Before deployment, exercise duplicate event intake, concurrent claims, a timeout after remote acceptance, a permanent address failure, and a retry that eventually succeeds. Assert state transitions and send counts. Then test rendering separately with representative locales and the longest realistic seller and item names; delivery correctness does not rescue a broken message.

Deploy the schema and unique constraint before enabling producers. Start one worker at low concurrency, watch pending age and failure categories, and confirm that alerts can be traced from order ID to notification key to transport ID. Increase concurrency only after the queue drains predictably. The runbook should say who can pause producers, how failed rows are replayed, how suppression is honored, and how a template version is retired. This is enough process for a small team without building a campaign platform inside the marketplace.

The decision rule is simple: keep order creation fast, make every notification independently recoverable, and treat sent as a recorded transport acceptance rather than proof that a human read the email. Build richer batching only when measurements show that the straightforward worker is the constraint.

DEV Community