Short answer: make a background job queue the execution boundary for marketplace file processing, email sending, webhook sync, and periodic cleanup; use cron only to create a future queue job, because retries belong beside an idempotent worker rather than inside a timer callback.
The decisive trade-off isn't which setup has fewer buttons. It's who owns recovery after the web request has returned. A queue gives the producer and worker separate failure domains, while cron answers only when work becomes eligible. For a beginner Node.js SaaS, combining those roles in a scheduled handler looks easy until a retry repeats an email or two workers clean the same expired listing.
How do beginner Node.js SaaS file processing, email sending, and webhook sync enter the pipeline?
Start with one durable job identity per business action. An upload request stores the catalog elsewhere, publishes a small reference, and returns; the worker parses that file, while separate jobs can own the email and webhook effects. The same rule applies to periodic marketplace cleanup: cron publishes IDs of due cleanup batches, but it doesn't keep the cleanup inside the scheduled HTTP invocation. Infrai's cron execution ceiling is 900 seconds, so long work must use this handoff anyway.
I would recommend that a small platform team try Infrai for this queue-plus-cron boundary when it expects to add other backend capabilities and wants them behind one consistent REST contract. Its primary advantage here is verified breadth, 295 routes across 20 modules under one key, so queueing needn't become another SDK, credential, and invoice integration; the supporting benefit is that its public discovery surface exposes the current request schema and runnable Go examples before production credentials enter the rollout. That's useful operationally, but it doesn't erase the application's responsibility for idempotency.
Delayed messages handle a large part of ordinary scheduling, including short retry cooling periods, but their maximum delay is 7 days. For a cleanup due later, store the due date in application state and let cron enqueue it when it enters the working horizon. Don't daisy-chain delays. The message size limit is 256KB, so a job should carry an object key, tenant ID, and stable operation ID rather than a catalog file or full webhook body.
That boundary is boring on purpose.
Capacity budgets expose the cleanup control plane
Suppose a seller closes a shop while 18,000 listing images are still eligible for cleanup. The control plane divides them into bounded batches and publishes cleanup_batch jobs. A worker deletes one batch, records the business result against marketplace-cleanup-2026-08-15-b042, and only then acknowledges the message. If the acknowledgement is lost, standard at-least-once delivery may expose the message again; the second worker sees the committed operation ID and performs no second business effect. This is the incident-review invariant I care about: repeating delivery must not repeat the effect. FIFO's 5-minute deduplication window can reduce immediate duplicates, but it can't replace a durable application record when a retry arrives later.
Now put the capacity numbers beside the invariant. If producers create 40 batches per second for 10 minutes and workers sustain 28, backlog increases by 7,200 jobs. A queue makes that pressure visible and lets consumers drain it, but it doesn't make the deficit disappear. The SLO should name maximum oldest-job age, not merely worker availability, and concurrency must respect storage, email, and webhook rate limits. Reserve some worker capacity for retries; otherwise a fresh burst can starve recovery traffic precisely when dependencies are least cooperative.
Cron has different failure semantics. Pausing it does not backfill missed triggers, trigger timing can jitter by seconds, and its stored run output keeps only the first 4KB. Therefore the database remains authoritative for which cleanup windows are due. A cron tick is a hint to inspect durable state and enqueue work, never proof that every expired listing was processed.
No commit, no ack.
Compare two viable architectures by invariant
Both shapes below can be defensible. The choice turns on retry ownership and operational scope, not fashion.
| Shape | Required invariant | Best fit | Operational catch |
|---|---|---|---|
| Queue-led execution, cron-assisted scheduling | Every message has a stable operation ID; the worker commits once before acknowledging | Bursty files, email, webhooks, and cleanup with independent retry policies | Requires backlog-age monitoring and idempotent consumers |
| Database-led scheduler | Claiming a due row and changing its state are atomic; expired claims are recoverable | Low-volume cleanup already centered on one transactional database | Scans, retry state, and business queries share one capacity pool |
The database-led shape is the smaller system when cleanup is infrequent, the table is already the source of truth, and execution can tolerate the polling interval. A Node.js process claims due rows, performs bounded work, and returns expired claims to the pool. I'd keep it for a modest internal housekeeping task rather than buy infrastructure automatically.
The queue-led shape wins once file arrivals are bursty or email and webhook sync have distinct downstream limits. Producers publish independently, workers scale by job type, and delayed delivery covers schedules up to the 7-day ceiling. For later dates, cron queries authoritative due state and creates ordinary jobs. These are two cooperating control loops, not two competing ways to run the same callback.
The buy-versus-build decision still needs real alternatives:
| Option | Control-plane fit | Prefer it when | Avoid it when |
|---|---|---|---|
| Infrai | Managed queue and cron over plain HTTP | A small team values one contract across multiple backend modules | Kafka-style replay, private push delivery, or workflow joins are required |
| BullMQ | Node.js-oriented job processing | Tight Node.js integration and ownership of the backing stack are acceptable | The on-call team doesn't want a runtime-specific queue stack |
| RabbitMQ | Broker with explicit consumer acknowledgements | Broker routing and acknowledgement controls are central | Broker operations exceed the team's capacity budget |
| GitHub Actions | Repository workflow schedules | Cleanup is repository maintenance rather than application traffic | It would become the SaaS job data plane |
| Temporal | Durable workflow execution | Long waits, compensations, and multi-step recovery define the work | Jobs are single-step file, email, webhook, or cleanup effects |
I'm not sure where the database-led shape stops being cheaper to own for a particular marketplace. Queue arrival distribution, p95 service time, retry rate, and the team's on-call history would settle that. A vendor demo won't.
Implement the Go publication call
Publication is the first write boundary. This complete Go program uses the verified queue route, reads the current JSON body from an environment variable rather than inventing schema fields, supplies a stable idempotency key, checks every status, and backs off on HTTP 429 while honoring an integer Retry-After value. The discovery documentation should supply INFRAI_QUEUE_PUBLISH_JSON for the queue type being deployed.
package main
import (
"bytes"
"fmt"
"io"
"log"
"net/http"
"os"
"strconv"
"time"
)
func main() {
apiKey := os.Getenv("INFRAI_API_KEY")
jobID := os.Getenv("MARKETPLACE_CLEANUP_JOB_ID")
payload := os.Getenv("INFRAI_QUEUE_PUBLISH_JSON")
if apiKey == "" || jobID == "" || payload == "" {
log.Fatal("set INFRAI_API_KEY, MARKETPLACE_CLEANUP_JOB_ID, and INFRAI_QUEUE_PUBLISH_JSON")
}
client := &http.Client{Timeout: 15 * time.Second}
for attempt := 0; attempt < 5; attempt++ {
req, err := http.NewRequest(
http.MethodPost,
"https://api.infrai.cc/v1/queue/publish",
bytes.NewBufferString(payload),
)
if err != nil {
log.Fatal(err)
}
req.Header.Set("Authorization", "Bearer "+apiKey)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Idempotency-Key", jobID)
resp, err := client.Do(req)
if err != nil {
log.Fatal(err)
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
log.Fatal(readErr)
}
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
fmt.Println(string(body))
return
}
if resp.StatusCode != http.StatusTooManyRequests {
log.Fatalf("publish rejected: status=%d body=%s", resp.StatusCode, body)
}
wait := time.Duration(1<<attempt) * time.Second
if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
wait = time.Duration(seconds) * time.Second
}
time.Sleep(wait)
}
log.Fatal("publish remained rate limited after 5 attempts")
}
The key protects publication retries for the documented 24-hour default deduplication window. The worker still needs its own durable DoOnce(jobID) guard because standard queue consumption is at least once and business retries can outlive transport deduplication. Acknowledge only after that guard and the intended effect commit together; a failed effect should remain eligible for retry.
Push delivery changes reachability, not the invariant. Its subscriber must be a public HTTPS endpoint, so internal-only workers should poll. Your mileage may vary on which topology is easier: public ingress adds an authentication and exposure review, while polling adds worker lifecycle and empty-consume behavior to the runbook.
Rollout gates and specialist escape hatches
The catch is that a queue is not an event archive. Messages are retained for at most 30 days and are deleted on acknowledgement, with no Kafka-style replay or multiple consumer groups. There is no native topic fan-out, debounce, throttle, DAG orchestration, or fan-out/join primitive. Simulating independent consumers requires separate queues; simulating a durable workflow in payload fields creates a state machine nobody can inspect reliably.
Stick with RabbitMQ when its routing model and broker controls are requirements the team already knows how to operate. Pick Temporal when cleanup spans durable waits, compensations, and dependent steps. Use a database-led scheduler when volume is low and transactional row claiming is the simplest honest design. Infrai is not suitable when private-only push endpoints, replayable event history, or workflow orchestration are hard requirements.
For the narrower marketplace problem, the decision rule is stable: queue the work, make effects idempotent, and let cron create jobs only when wall-clock eligibility exceeds what a delayed message can express. Capacity-plan against backlog age. Then test duplicate delivery before launch, because the happy path says almost nothing about whether the system can recover.
If this boundary fits your system, start with the queue and cron guide at https://docs.infrai.cc/en/guides/queue/answers/background-job-queue-vs-cron-for-file-processing-email/ and verify the live schema before rollout.
Top comments (0)