Short answer: for a rate limited email sending queue, use a leased worker backed by durable reservation and outbox records when an expired customer-support hold must be recovered after a crash; use one timer per hold only when occasional missed work is acceptable and operational simplicity matters more than replay.
The choice is about delivery guarantees, not timer precision. A support system may reserve a callback slot for 20 minutes, expire the hold at its deadline, and then send an email through Resend, Postmark, or SES. The expiration changes business state. The email reports that change. Treating those as one background job hides the most important failure boundary.
My decision rule: commit the expiration and an email intent together, then let a rate-limited queue deliver that intent separately. This gives the state transition a durable recovery path while admitting that an external email API can't join the database transaction. It also makes the dashboards honest.
Timers and leases produce different failure shapes
The before model is tempting: create a reservation, start a delayed timer, and call the email provider when it fires. It reads like one clean arrow: held -> timer -> expired -> email. In a single process that never restarts, the model is wonderfully easy to explain.
Processes restart.
A memory timer can disappear before it fires. A durable scheduler improves recovery, but the handler still has to answer harder questions: did another worker already expire this hold, did the email call finish before the worker lost its acknowledgement, and can an operator replay the job without changing state twice? Cron is a time-based job scheduler, so it is useful for periodically finding due work; it isn't, by itself, a delivery guarantee for each reservation.
The after model has two explicit lanes. Say it out loud: the expiry worker scans due holds, locks a small batch, changes each eligible hold from held to expired, and inserts an outbox row in the same transaction. Then the email worker leases pending outbox rows, waits for its rate limiter, sends, and records the result. A crash in lane one leaves either both database changes committed or neither. A crash in lane two may cause a retry, which is why the message needs a stable key and the email content must tolerate duplicate attempts.
| Design | Recovery after restart | Duplicate-send exposure | Best fit |
|---|---|---|---|
| One timer per hold | Depends on where timers are persisted | Hidden inside the handler | Low-consequence reminders and small systems |
| Periodic scan plus leased rows | Due records remain queryable | Visible at the external-send boundary | Expiry rules that must survive worker loss |
The catch is extra machinery. Leases need an expiry time, workers need retry policy, and operators need queue-age alerts. A timer is the better choice when losing a notification has little consequence, the reservation state is repaired elsewhere, and the team doesn't want to operate a durable queue. Delivery guarantees have a carrying cost.
How should a SaaS backend rate-limit email queue background jobs?
Rate limiting belongs in the delivery lane, after the reservation transaction. If a provider slows acceptance, held reservations should still expire on time; only the notification backlog should grow. Coupling the provider's pace to the expiry scan turns a messaging constraint into stale business state.
Give the adapter one narrow contract. It should accept a stable message key, return a provider receipt when accepted, and classify a rate-limit response as retryable. The queue owns attempt timing. The provider adapter doesn't own reservation state.
type EmailIntent = {
id: string;
reservationId: string;
recipient: string;
template: "reservation-expired";
attempts: number;
};
type SendResult =
| { kind: "accepted"; receipt: string }
| { kind: "rate-limited"; retryAfterMs: number }
| { kind: "rejected"; reason: string };
interface EmailGateway {
send(intent: EmailIntent): Promise<SendResult>;
}
interface Outbox {
lease(limit: number, leaseMs: number): Promise<EmailIntent[]>;
markSent(id: string, receipt: string): Promise<void>;
release(id: string, nextAttemptAt: Date): Promise<void>;
markRejected(id: string, reason: string): Promise<void>;
}
const addJitter = (baseMs: number, seed: number): number =>
baseMs + (seed % Math.max(1, Math.floor(baseMs / 5)));
async function deliverBatch(
outbox: Outbox,
gateway: EmailGateway,
now: Date,
): Promise<void> {
const intents = await outbox.lease(50, 30_000);
for (const intent of intents) {
const result = await gateway.send(intent);
if (result.kind === "accepted") {
await outbox.markSent(intent.id, result.receipt);
continue;
}
if (result.kind === "rate-limited") {
const delayMs = addJitter(result.retryAfterMs, intent.attempts);
await outbox.release(intent.id, new Date(now.getTime() + delayMs));
continue;
}
await outbox.markRejected(intent.id, result.reason);
}
}
The numbers are example configuration, not universal defaults: a batch of 50 and a 30-second lease make the concurrency contract concrete enough to test. Tune them from measured send latency, granted provider throughput, worker count, and database load. Don't scatter provider-specific limits across job handlers. Put the current policy in one limiter so a quota change has one deployment path.
This design is at-least-once at the delivery boundary. Walk one intent through the awkward sequence. Worker A leases reservation-expired:r-104, waits for the limiter, and submits the email. The provider accepts it, but the process exits before markSent commits. Thirty seconds later, the lease becomes eligible again. Worker B has no durable receipt to inspect, so it submits the same stable intent. A provider-supported idempotency mechanism can narrow that gap when its documented behavior matches the required retention window; otherwise, the product decision is to accept a possible duplicate, add a reconciliation process, or move the message through infrastructure with a stronger contract. The database transaction solved atomic expiry plus enqueue. It did not make an external network effect atomic. Calling this “exactly once” would only erase the riskiest arrow from the diagram, and I wouldn't ship a design review that hides it.
Test that gap.
Resend, Postmark, and SES can sit behind the same local interface, but that doesn't make their quotas, retry semantics, or idempotency behavior interchangeable. Verify those items against the provider documentation and the limits granted to the actual account before deployment. No static comparison can decide that account-specific boundary.
A copyable expiry worker
The expiry transaction needs a conditional state change. Multiple workers may scan the same deadline, but only one should move a reservation out of held. PostgreSQL documents FOR UPDATE locking and SKIP LOCKED; together they let queue-like consumers avoid waiting on rows another worker has already locked. The documentation also warns that SKIP LOCKED gives an inconsistent view, which is appropriate for queue consumers but not for general-purpose reporting.
This TypeScript sketch uses a generic PostgreSQL-shaped client. The query selects only due holds, locks a bounded batch, updates their state, and derives deterministic outbox IDs from reservation IDs. A unique constraint on the outbox ID is the final guard against duplicate intents.
type QueryResult<T> = { rows: T[] };
interface Transaction {
query<T>(sql: string, values?: unknown[]): Promise<QueryResult<T>>;
}
interface Database {
transaction<T>(work: (tx: Transaction) => Promise<T>): Promise<T>;
}
type DueReservation = {
id: string;
customer_email: string;
};
async function expireDueReservations(
db: Database,
now: Date,
): Promise<number> {
return db.transaction(async (tx) => {
const due = await tx.query<DueReservation>(
`SELECT id, customer_email
FROM support_reservations
WHERE status = 'held' AND expires_at <= $1
ORDER BY expires_at
FOR UPDATE SKIP LOCKED
LIMIT 100`,
[now],
);
for (const reservation of due.rows) {
await tx.query(
`UPDATE support_reservations
SET status = 'expired', expired_at = $1
WHERE id = $2 AND status = 'held'`,
[now, reservation.id],
);
await tx.query(
`INSERT INTO email_outbox
(id, reservation_id, recipient, template, state, available_at)
VALUES ($1, $2, $3, 'reservation-expired', 'pending', $4)
ON CONFLICT (id) DO NOTHING`,
[
`reservation-expired:${reservation.id}`,
reservation.id,
reservation.customer_email,
now,
],
);
}
return due.rows.length;
});
}
Run that worker on a short recurring cadence. The cadence controls lateness, not durability: if a run is skipped, the next scan still sees every row whose expires_at is in the past. Keep application clocks consistent, pass one captured now through the transaction, and store enough timestamps to distinguish “became eligible,” “was expired,” “email became available,” and “email was accepted.” Those four moments turn a vague complaint like “the email was late” into a traceable queue story.
There is one subtle product race. A support agent may confirm a reservation while the expiry worker is locking it. The conditional WHERE status = 'held' protects the transition only if every competing transition uses the same transaction discipline. Define the state machine first. Then write a concurrency test that starts confirmation and expiry together and asserts that exactly one terminal state wins.
What should we test and observe before choosing?
Start with invariants, not throughput. One reservation can leave held once. Every committed expiration creates one stable email intent. A rejected address stops retrying. A temporary rate limit changes available_at without losing the row. An expired lease makes work eligible again. These assertions survive a provider swap because they describe the system you own.
Then test the crash windows on purpose. Terminate a worker after it locks rows but before commit; no expiration or outbox write should remain. Terminate it after commit; the pending intent should remain. For delivery, simulate acceptance followed by worker loss before the receipt is saved, and verify the documented duplicate policy. Also run two expiry workers against 101 due reservations. The exact split doesn't matter. No eligible reservation should vanish.
Observe the two lanes separately. For expiry, graph oldest due hold age, expired rows per run, transaction duration, and lock contention. For email, graph oldest pending intent age, accepted and rejected counts, rate-limited attempts, lease expirations, and attempts per intent. Alert on age against the customer promise, not merely on worker CPU. A quiet worker with a growing oldest-item age is failing its job politely.
Keep logs joinable by reservationId, intentId, and attempt number, but don't put the recipient address in routine log fields. One structured event at each state transition is more useful than a stream of “worker running” messages. Crisp signals win.
The “cheapest SaaS backend” part of the decision needs a workload model. I'm not sure a static cheapest label is defensible without send volume, burst shape, retention, database traffic, support requirements, and the account's granted limits. Compare the full monthly path: scheduler invocations, queue operations, storage and lock load, email delivery, observability retention, and engineer time spent on recovery. Stick with a managed scheduler and queue when the team values low operational ownership; keep the database-leased design when transactional coupling and inspectable recovery justify running the worker. Neither choice removes the external-send duplicate window automatically.
Ship with one operational drill: pause delivery, let intents accumulate past one hold window, resume at a controlled rate, and watch oldest-item age return to normal without a burst beyond the configured limit. This proves more than a happy-path demo.
Top comments (0)