A sequential send loop hit a 10 per second rate limit, because it was fast

CogniPrep sends lifecycle emails: a nudge to the reader who practised interview questions but never booked one, a win back for someone who has not opened the app in a while, a note to the reader who has a provider unlocked but has never seen a feedback report. A cron runs twice a day, selects an audience per campaign, and emails them.

The first version lost email to rate limiting, and the reason is the kind that is invisible in code review, because the loop is sequential:

for (const candidate of candidates) {
  await sendEmail({ to: candidate.email, ... });
}

Our email provider allows 10 requests per second per account. send returns in well under 100ms. So that loop ran at roughly 13 per second, and everything past the tenth send in any given second came back rate_limit_exceeded and was gone.

Sequential does not imply slow enough. If your per call latency is under 1000 / limit milliseconds, an await in a for loop breaches the cap on its own, with no concurrency anywhere.

Two fixes, at different layers

The general fix is a shared schedule that every send reserves a slot on:

const SENDS_PER_SECOND = 8;
const SEND_SPACING_MS = Math.ceil(1000 / SENDS_PER_SECOND);

let nextSendSlotAt = 0;

async function awaitSendSlot(): Promise<void> {
  const now = Date.now();
  const slot = Math.max(now, nextSendSlotAt);
  nextSendSlotAt = slot + SEND_SPACING_MS;
  if (slot > now) await sleep(slot - now);
}

The important detail is that the slot is claimed synchronously, before any await. Concurrent callers each advance nextSendSlotAt in turn, so they queue behind one another. If you read the clock after an await, every caller reads the same value and they all collide, which is the bug this shape exists to avoid.

Eight rather than ten, for headroom, and a 429 that still slips through is retried with backoff, because this schedule is per process and serverless runs many processes. It cannot be the only defence.

But for a campaign, pacing is the wrong answer. The right one is to stop making one request per recipient:

export const RESEND_BATCH_LIMIT = 100;

The batch endpoint takes up to 100 fully rendered emails in one request. A 500 recipient campaign costs 5 requests instead of 500. The rate limit stops being the constraint rather than being negotiated with.

The ordering problem underneath

Each nurture email carries a personal single use discount code. That code is a database row, and the row is also the send log: its unique (user_id, campaign) constraint is what stops anyone being emailed the same campaign twice.

One row doing two jobs is good, because it removes the class of bug where "we sent it" and "we minted the code" disagree. But it means the order of operations decides your failure mode:

• Send first, then mint: a crash between the two sends the email again on the next run.
• Mint first, then send: a crash between the two marks the user as emailed when they got nothing.

We mint first, because a missing email is recoverable and a duplicate email is not. Then every path that fails to send explicitly gives the row back:

try {
  const email = await buildNurtureEmail({ ... });
  pending.push({ codeId: code.id, userId: candidate.userId, email });
} catch (err) {
  // Rendering failed, so this user is getting nothing: release the send-log row
  // rather than marking them emailed.
  await discardUnsentNurtureCode(code.id);
}

Batching adds a wrinkle here. If you mint the whole campaign up front and then send it in chunks, everything minted but not yet sent is exposed to a mid run death. So minting and sending are interleaved one chunk at a time, which bounds that exposure to a single batch of 100.

And the batch API validates permissively: one bad recipient does not reject the other 99. Rejects come back indexed by payload position, which is the only thing you can reconcile against:

const rejected = new Map(result.failures.map((f) => [f.index, f.message]));

for (const [index, p] of pending.entries()) {
  const rejection = rejected.get(index);
  if (rejection === undefined) { sent++; continue; }
  await discardUnsentNurtureCode(p.codeId);
}

The accepted ids that come back cannot be indexed against the payload, so the rejects are the authoritative signal and everything unlisted was accepted. A whole request failure rolls back all 100.

The idempotency key is derived from the batch itself:

const idempotencyKey = `nurture-${campaign}-${pending[0].codeId}`;

Those code ids were minted moments ago and are unique to this batch, so a request replayed by the 429 retry is recognised as the same batch rather than delivering a second time.

Stop on your own terms

The route has a 300 second ceiling. The run gives itself 240:

const RUN_BUDGET_MS = 240_000;
const deadline = Date.now() + RUN_BUDGET_MS;

Batching made the API calls cheap, but rendering is still per recipient: every email carries a signed unsubscribe URL, and some carry an employer lookup. A run at the candidate cap is not instant.

The difference between stopping yourself and being killed is precisely the mint-then-send window. Your own deadline is checked between candidates, so the run ends cleanly at a chunk boundary and anyone not reached is simply picked up next time. A platform timeout lands wherever it lands, including between a mint and its send, which is the one state that costs a user their email permanently.

The deadline uses the real clock, not the injected now that anchors the audience windows and the code expiry, because that one is a fixed value in tests.

Ordering the campaigns is the anti spam rule

const CAMPAIGNS: CampaignSpec[] = [
  { campaign: 'assessment_centre', candidates: assessmentCentreCandidates },
  { campaign: 'interview', candidates: interviewCandidates },
  { campaign: 'scores_feedback', candidates: scoresFeedbackCandidates },
  { campaign: 'win_back', candidates: winBackCandidates },
  { campaign: 'referral', candidates: referralCandidates, sendLogOnly: true },
];

Campaigns run sequentially in priority order, and a freshly minted code makes the audience predicate exclude that user from every later campaign in the same run. So "nobody receives two nurture emails at once" is not a rule anywhere in the code. It is a consequence of the ordering plus the send log, which means there is no second rule to forget to update.

The referral email sits last because it sells nothing, so anyone who also qualifies for a conversion email gets that one instead. It is also the one campaign that mints its row purely as a send log, with no spendable discount on it, because the code in that email is the reader's own referral code rather than ours.

Try it

This one is not visible from the outside until you are in it, so the demonstration is to become a recipient: sign up at cogniprep.app, practise something, and see which email arrives and what it offers. Every one of them carries a single use code and an unsubscribe link that works without a login.

If you want the more useful exercise instead, go and open your own campaign sender and answer two questions about it. What happens if the process dies between your write and your send? And how fast is your sequential loop actually going?