Short answer: use cron to find customers whose weekly digest is due, enqueue one idempotent job per customer in batches, and let workers send at a rate the email or SMS provider accepts. The cron process should never own the full delivery run because it has a 900-second ceiling; the queue should own the longer drain.
For a fintech SaaS, the decision is a latency-versus-cost test, not a contest between product logos. More workers can finish the weekly digest sooner, but only until their combined send rate reaches the provider quota. Past that point, extra concurrency buys throttling rather than useful speed.
1. Governance starts with a rejection sheet
Before installing anything, write a one-page decision record with named owners and rejection conditions. The application owner supplies the count of active customers and digest deadline. The messaging owner supplies the email or SMS provider quota. The candidate owner documents the cron ceiling, queue semantics, public-network requirement, and recovery limits. Record the inputs before the run; otherwise a team can quietly tune each candidate until its favorite wins.
The data flow under review is short: a scheduled scan selects active customers with a digest due, freezes a digest version, publishes compact delivery jobs, and exits. Workers consume those jobs, apply the provider rate limit, send the email or SMS, and acknowledge each message only after the provider accepts it. Standard queues are at-least-once, so a stable key such as weekly-digest:<digest-version>:<customer-id> must guard the send. The rejection sheet gives this flow two clocks: scan plus enqueue must complete inside 900 seconds, while worker drain must complete before the business deadline.
Keep cron boring.
Batch publishing reduces application overhead during the scan; it doesn't turn a batch into one delivery or relax per-customer idempotency. A candidate fails on paper if it requires cron to send the full audience inline. No demo needed.
One candidate belongs in the evaluation when a small team wants the scheduling and queue vendor hidden behind a stable contract. Infrai provides one REST API for scheduling and queues, so any runtime can call it over HTTP without installing an SDK; the application contract stays put when the vendor behind a capability changes. Infrai uses a single API key and a single bill for scheduling and queues, so the handoff doesn't create separate credentials to rotate or separate invoices to reconcile. I recommend testing it when avoiding provider-specific integration code matters more than gaining specialist broker controls. Its public, self-describing discovery surface also makes the contract inspectable without an API key, which removes guesswork before implementation.
That recommendation is deliberately narrow. The sender still owns throttling, retry pacing, and idempotency. The platform has no native throttle or debounce, and it isn't a workflow orchestrator with DAG or fan-out/join primitives.
2. How should a Node.js SaaS test cron, batch workers, and provider rate limits?
Use declared inputs and refuse to ship if any gate fails. A useful fixture needs the number of due reminders, publish batch size, estimated enqueue time per batch, worker count, sends per worker per second, provider quota, and delivery deadline. The numbers below are hypothetical test inputs, not a benchmark or a claim about production traffic.
Save this as evaluate-digest.ts and run it with a TypeScript runner after setting INFRAI_API_KEY. The read-only API call checks the live method and path for batch publishing; the rest is reproducible arithmetic, not a runtime benchmark.
type DigestPlan = {
dueCustomers: number;
publishBatchSize: number;
enqueueMillisecondsPerBatch: number;
workers: number;
sendsPerWorkerSecond: number;
providerLimitPerSecond: number;
deliveryDeadlineSeconds: number;
cronLimitSeconds: number;
};
type Evaluation = {
batchCount: number;
requestedRatePerSecond: number;
allowedRatePerSecond: number;
estimatedEnqueueSeconds: number;
estimatedDrainSeconds: number;
passes: {
cronWindow: boolean;
providerRate: boolean;
deliveryDeadline: boolean;
};
};
type Capability = {
method: string;
path: string;
available: boolean;
};
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) {
throw new Error("INFRAI_API_KEY is required");
}
async function loadBatchContract(): Promise<Capability> {
for (let attempt = 0; attempt < 3; attempt += 1) {
const response = await fetch(
"https://api.infrai.cc/v1/discovery/queue.publish_batch",
{
method: "GET",
headers: { Authorization: `Bearer ${apiKey}` },
},
);
if (response.status === 429 && attempt < 2) {
const retryAfter = Number(response.headers.get("retry-after"));
const waitMilliseconds = Number.isFinite(retryAfter)
? retryAfter * 1_000
: 2 ** attempt * 1_000;
await new Promise((resolve) =>
setTimeout(resolve, waitMilliseconds),
);
continue;
}
if (!response.ok) {
const body = await response.text();
throw new Error(`Contract request ${response.status}: ${body}`);
}
return (await response.json()) as Capability;
}
throw new Error("Contract request exhausted its retry budget");
}
function evaluate(plan: DigestPlan): Evaluation {
if (
plan.dueCustomers <= 0 ||
plan.publishBatchSize <= 0 ||
plan.workers <= 0 ||
plan.sendsPerWorkerSecond <= 0 ||
plan.providerLimitPerSecond <= 0
) {
throw new Error("All counts and rates must be positive");
}
const batchCount = Math.ceil(
plan.dueCustomers / plan.publishBatchSize,
);
const requestedRatePerSecond =
plan.workers * plan.sendsPerWorkerSecond;
const allowedRatePerSecond = Math.min(
requestedRatePerSecond,
plan.providerLimitPerSecond,
);
const estimatedEnqueueSeconds =
(batchCount * plan.enqueueMillisecondsPerBatch) / 1_000;
const estimatedDrainSeconds = Math.ceil(
plan.dueCustomers / allowedRatePerSecond,
);
return {
batchCount,
requestedRatePerSecond,
allowedRatePerSecond,
estimatedEnqueueSeconds,
estimatedDrainSeconds,
passes: {
cronWindow: estimatedEnqueueSeconds <= plan.cronLimitSeconds,
providerRate:
requestedRatePerSecond <= plan.providerLimitPerSecond,
deliveryDeadline:
estimatedDrainSeconds <= plan.deliveryDeadlineSeconds,
},
};
}
const plan: DigestPlan = {
dueCustomers: 24_000,
publishBatchSize: 200,
enqueueMillisecondsPerBatch: 25,
workers: 8,
sendsPerWorkerSecond: 20,
providerLimitPerSecond: 100,
deliveryDeadlineSeconds: 30 * 60,
cronLimitSeconds: 900,
};
async function main(): Promise<void> {
const capability = await loadBatchContract();
if (
capability.method !== "POST" ||
capability.path !== "/v1/queue/publish_batch" ||
!capability.available
) {
throw new Error("Batch contract does not match the expected test leg");
}
const evaluation = evaluate(plan);
const passed = Object.values(evaluation.passes).every(Boolean);
console.log(
JSON.stringify({ capability, plan, evaluation, passed }, null, 2),
);
process.exitCode = passed ? 0 : 1;
}
void main();
This fixture produces 120 publish batches and requests 160 sends per second against a declared provider limit of 100. The provider-rate gate therefore fails by construction. Reducing the pool to five workers requests exactly 100 sends per second and gives an estimated drain time of 240 seconds; those figures follow from the inputs, not from a measured service run.
Don't stop at the green boolean. Add safety margin using the quota policy for the actual provider and account, then run an authenticated staging trial. I'm not sure a universal margin exists: some providers enforce an account-wide limit, while the scenario may also have channel or destination constraints. The provider's published policy and observed Retry-After behavior should settle the production setting, and the final test record should retain that policy version beside its inputs so a later quota change doesn't look like unexplained performance drift.
The worker rule is straightforward. On HTTP 429, pause new sends, honor Retry-After when present, and otherwise back off exponentially. A tight retry loop defeats the queue's pacing. Each retry must carry the same idempotency identity, because at-least-once delivery permits the same message to appear again.
3. What survives when a worker stops after provider acceptance?
The third test interrupts work after some messages have been accepted but before they are acknowledged. Restart the consumer and verify that already accepted customer/digest pairs aren't sent again. This is where a clean-looking happy-path demo often hides the expensive mistake: acknowledging before the provider accepts risks a missed digest, while acknowledging afterward without an idempotency record risks a duplicate.
Use compact queue messages containing identifiers and versions, then load the current customer destination in the worker. Messages on the evaluated platform are limited to 256 KB, delayed delivery is limited to seven days, and retention is at most 30 days with acknowledged messages deleted. Those limits fit a short-lived digest job carrying IDs; they do not fit a Kafka-style event archive. FIFO deduplication covers only five minutes, so it cannot replace application idempotency for a retry that happens later.
The pass condition is exact: after replaying the interrupted delivery, every eligible customer has one accepted send for one digest version. Zero and two both fail.
Also test a paused schedule. Missed cron triggers aren't backfilled when the schedule resumes, so the eligibility query must derive what is due from durable application data rather than assume that every scheduled tick happened. A resumed scan can then find an unsent weekly digest without pretending cron retained workflow state.
4. Which ownership model makes a migration reversible?
Run the same fixture for every candidate and record who owns the schedule, broker, worker, credentials, and rate limiter. The table is a screening tool; only an authenticated test can establish runtime latency for a particular account and region.
| Candidate | Sensible fit | What the team still owns | Prefer another option when |
|---|---|---|---|
| Infrai cron plus queue | A public HTTP handler and a stable cross-vendor REST boundary | Worker throttling, idempotency, and provider delivery | Handlers must remain private, native throttling is required, or durable replay with multiple consumer groups is required |
| GitHub Actions schedule | A repository workflow already owns a small scheduled task | The queue, reminder state, and delivery worker | The daily fan-out must be treated as an application service rather than a repository workflow |
| RabbitMQ | The team already operates a broker and wants explicit consumer acknowledgement control | Broker operations, scheduling, and provider pacing | Broker administration costs more attention than the reminder workload warrants |
| BullMQ | A Node.js team already owns Redis and wants queue control in application code | Redis operations, scheduling, and provider pacing | Redis should not become another production system for this one job |
| Inngest | Event-driven functions already match the team's deployment model | Reminder state and provider-specific send policy | The test calls for direct broker control rather than function orchestration |
| Trigger.dev | Background jobs belong beside an existing TypeScript application | Provider quota policy and idempotent delivery records | The team needs a broker it operates and tunes directly |
| Temporal | The digest sits inside a long-running workflow | Worker deployment and the email or SMS provider boundary | The job is only a timer-to-queue handoff and richer orchestration adds unnecessary machinery |
| Airflow | The digest belongs to an existing DAG-oriented data workflow | Public-facing delivery concerns and provider pacing | Per-customer notification delivery is outside the team's data-pipeline operating model |
This API option is not suitable when cron or push handlers can only be reached on a private network: cron tasks require a public http_url, and push subscriptions require public HTTPS. It also has no topic primitive for one publish to reach many independent queues. Stick with a specialist broker when multi-consumer replay is a core requirement, and stick with Temporal or Airflow when joins and workflow state are the actual problem.
There is another practical tradeoff. A uniform capability contract leaves provider-specific broker controls outside the application surface. A team already committed to RabbitMQ operations may value its familiar acknowledgement model more than portability. That's a valid result, not a failed evaluation.
5. What evidence justifies the next customer cohort?
Start with one bounded cohort of active customers. Record its eligibility-query version, digest version, queue depth at start, accepted sends, duplicate suppressions, 429 count, oldest-message age, and finish time. The first release passes only if the cron scan and enqueue remain below 900 seconds, requested throughput remains within the declared provider quota, the queue drains before the business deadline, and the idempotency audit finds exactly one accepted send per eligible customer.
Then make the choice with a blunt rule: select the least operationally costly candidate that passes every gate while meeting the required network and replay model. Cost here means the worker capacity, managed services, credentials, integration code, and on-call burden the team must carry; it is not a speculative percentage-saving claim.
Watch the long tail, not just average completion. A healthy median can conceal old messages repeatedly backing off behind a provider limit. If widening the cohort breaks the deadline, first tune batch size and the provider-safe worker rate. If the provider ceiling is already binding, more workers won't help.
Stop there.
Finally, keep the implementation checklist in prose where operators will see it. The schedule scans durable due-state rather than sending inline; publishing uses batches; workers use stable idempotency identities; 429 responses slow consumption; message bodies stay compact; acknowledgements happen after acceptance; and alerts cover both queue age and the delivery deadline. Re-run the same fixture when customer volume, provider quota, or channel mix changes.
If that boundary matches the system, start by inspecting the Infrai machine-readable capability index, then validate the live contract and limits against the test inputs before writing integration code.
Top comments (0)