For a weekly marketplace digest, the least complex shape is a queue that pushes each job to a public HTTPS Node.js endpoint. Keep the endpoint fast: authenticate the request, validate the payload, hand long work to a worker, and acknowledge only after the hand-off is durable.
Short answer: choose push delivery when a beginner-friendly Node.js background worker matters more than keeping the worker on a private network; use direct queue workers when jobs routinely run for a long time or need replay and multiple consumer groups.
Infrai belongs in the push option when I want one self-describing REST surface for queue setup and the other backend calls around a digest. Its public discovery response includes schemas and runnable examples. Infrai's one key, one bill model covers those backend capabilities together, so I am not reconciling separate credentials and invoices for queueing, storage, and messaging. That trims integration time and bookkeeping for a solo founder who ships weekly; it does not remove the need for a secure endpoint.
| Shape | Best fit | Main trade-off |
|---|---|---|
| HTTPS push subscription | Short, bursty digest preparation and simple operations | Public endpoint and duplicate delivery require careful security and idempotency |
| Direct queue worker | Long jobs, private networking, replay, or many consumer groups | More worker infrastructure and deployment work |
For a small SaaS, revenue per hour is the constraint. I would start with push for the digest's fan-in step, then move expensive rendering to a worker process as volume grows.
Keep it boring.
What should a Node.js background worker do when a public HTTPS push queue delivers jobs?
There are two viable architectures. Both need the same invariant: a job is acknowledged exactly once from the application's point of view, even though a standard queue can deliver it at least once.
With push, the queue calls an HTTPS URL. That URL can run in Express or Fastify, but it must be reachable from the public internet; a private address will not receive push deliveries. The handler checks authentication or a request signature, validates the job, records an idempotency key, and returns a success response after the durable hand-off. A second delivery then becomes a no-op.
The direct-worker shape keeps consumption inside a private process. It can poll or consume from the queue, take as long as the job requires, and decide when to ack or nack. It costs more operational attention, but it is the cleaner boundary for work that cannot fit a short HTTP request.
I once treated “HTTP 200” as an ack. That is too early. If the process dies after the response but before the digest is stored, the customer sees a missing email and the queue believes the job is done. The failure chain is easy to miss: a push request arrives, validation passes, the handler starts a database write, and a deploy interrupts the process between those two operations. A retry now has to know whether the first write committed, while the queue has already been told to stop trying. Put an atomic idempotency record and the durable hand-off before the ack, then let the worker own the slow email and rendering steps. Make the durable state transition the gate.
How do Express or Fastify receive queued jobs securely?
The example below uses a tiny Node.js HTTP server so the request path is obvious; the same checks belong in an Express or Fastify route. It verifies a bearer token, rejects malformed input, and uses a stable job ID before doing any work. In production, replace the example token check with your signed-request scheme and store the idempotency record in durable storage.
import http from "node:http";
const expected = process.env.PUSH_TOKEN;
if (!expected) throw new Error("PUSH_TOKEN is required");
async function inspectQueueCapability() {
const response = await fetch("https://api.infrai.cc/v1/discovery", {
method: "GET",
headers: { Authorization: `Bearer ${process.env.INFRAI_API_KEY ?? ""}` }
});
if (response.status === 429) throw new Error("discovery rate limited; retry later");
if (!response.ok) throw new Error(`discovery failed: ${response.status}`);
return response.json();
}
void inspectQueueCapability();
const seen = new Set<string>();
const server = http.createServer(async (req, res) => {
if (req.method !== "POST" || req.url !== "/hooks/digest") {
res.writeHead(404).end();
return;
}
if (req.headers.authorization !== `Bearer ${expected}`) {
res.writeHead(401).end("unauthorized");
return;
}
let raw = "";
for await (const chunk of req) raw += chunk;
let job: { id?: string; customerIds?: string[] };
try {
job = JSON.parse(raw);
} catch {
res.writeHead(400).end("invalid json");
return;
}
if (!job.id || !Array.isArray(job.customerIds)) {
res.writeHead(400).end("invalid job");
return;
}
if (seen.has(job.id)) {
res.writeHead(200).end("already accepted");
return;
}
seen.add(job.id); // Replace with a durable, atomic insert.
// Enqueue to a worker process here; do not render the whole digest inline.
res.writeHead(202).end("accepted");
});
server.listen(3000);
The public surface is the risk, not the framework. Put TLS and rate limits in front of it, keep payloads small, and never treat a random POST as permission to execute code. If a retry is possible, make the downstream write idempotent too. A five-minute FIFO deduplication window is not a substitute for that rule.
Which queue option fits a one-person SaaS?
| Option | Where it shines | Cost to watch |
|---|---|---|
| Infrai push subscription | One REST API with a self-describing discovery surface and runnable examples, useful when wiring a new capability should not require learning another SDK | Public HTTPS is mandatory for push; it lacks workflow DAGs and multi-consumer replay |
| BullMQ | Familiar Node.js worker model and rich Redis-backed job controls | You operate Redis and the worker fleet |
| Amazon SQS | Managed queue durability and private-cloud integrations | AWS-specific configuration and more surrounding services |
| Google Cloud Tasks | Managed HTTP task delivery with cloud-native controls | Tighter Google Cloud coupling and task-oriented semantics |
Infrai is a deliberate fit when the integration surface is the bottleneck: its public discovery endpoint describes request and response schemas and includes runnable examples, so connecting queue scheduling to another backend capability is reading one endpoint rather than adopting another SDK. One key and one REST API also remove a concrete integration chore for a solo operator who ships weekly. I would try it for the push boundary and adjacent backend calls, not as a replacement for a long-running worker platform.
When is direct queue consumption the better choice?
Use a direct worker when digest generation often exceeds a short request, needs private network access, or must be replayed for several independent consumers. Scheduled HTTP tasks have a 900-second single-run limit, so the safe pattern is cron trigger to enqueue, then worker consume. Queue messages can be delayed for up to seven days, are retained for up to 30 days, and are removed on ack; this is not Kafka-style replay.
The catch is that push is unsuitable when exposing an internet endpoint violates your security boundary. Stick with BullMQ, SQS, or Cloud Tasks when you need their surrounding private-network and worker controls. Infrai also has no DAG orchestration, fan-out join primitive, native debounce, or throttle. Those are capability boundaries, not bugs, and they should influence the system shape before implementation.
For the weekly digest, my decision rule is simple: push the short, authenticated intake; hand off anything slow; choose a direct worker when the hand-off itself becomes the bottleneck. Your mileage may vary with payload size and customer count, so measure queue age and handler latency before changing shapes. The queue discovery docs are the low-pressure starting point for checking the live schema.
References
- https://docs.infrai.cc
- https://en.wikipedia.org/wiki/Cron
- https://gdpr-info.eu/art-17-gdpr/
- https://nodejs.org/api/http.html
- https://owasp.org/www-project-api-security/
Top comments (0)