Short answer: for a small app, start with a database-backed polling queue when traffic is modest and you need the lowest operational surface; move to a managed retry queue with a dead-letter queue (DLQ) and redrive when latency, burst control, or replay isolation matters more than another service to operate.
The decision is not really "which queue is cheapest?" It is: which failure state can your team inspect and repair at 2 a.m.? In an edtech app, a failed job might be a grade export, a parent notification, or a nightly roster sync. Losing one is worse than waiting an extra minute, but paying for always-on infrastructure for ten jobs a day is also hard to justify.
Tiny queue. Big consequences.
The awkward incident is the one that looks successful. Imagine a roster export that uploads a file, receives a 200 response, and then loses its process before the database records completion. Sixty seconds later another worker sees the expired lease and uploads the same file. If the storage key is derived from a random request id, students may receive two links; if the key is derived from the job id and the write is conditional, the second attempt becomes a harmless no-op. That is why I put the idempotency decision next to the retry policy during design review. I also keep a repair command that can reset one dead row after an operator checks the external side effect. It is slower to write than a blind retry loop, but it makes redrive and manual polling explainable under pressure.
| Path | Pick it when | Main cost | Failure behavior |
|---|---|---|---|
| Manual database polling | A small worker, low volume, and SQL skills are already in place | Database CPU and lock contention | Retries stay in your tables; operators replay rows directly |
| Retry queue + DLQ redrive | Bursty work, strict latency, or several worker replicas | Queue configuration and another billable boundary | Attempts are isolated; operators redrive selected DLQ messages |
What should a small app choose for failed job retry, DLQ redrive, or manual database polling?
Use the database path if your queue is a table, not a second product. A row needs a stable id, payload, status, attempt count, next-at timestamp, and last error. Workers claim a bounded batch inside a transaction, set a lease, and commit before doing slow network work. A second worker cannot claim the same lease until it expires.
The queue path is a better fit when a burst of imports must drain quickly without making the primary database fight the scheduler. A retry policy can add delay between attempts; a DLQ gives poison messages a separate holding area. Redrive means choosing which held messages return to the main queue after the underlying cause is understood. It is an operational action, not a magic repair.
Here is a compact polling claim in TypeScript. The SQL uses row locks so two workers can share the table without taking the same job. The exact transaction wrapper depends on your driver.
import type { Pool, PoolClient } from "pg";
export async function claimJobs(pool: Pool, limit = 10): Promise<string[]> {
const client: PoolClient = await pool.connect();
try {
await client.query("BEGIN");
const result = await client.query<{ id: string }>(
`SELECT id
FROM jobs
WHERE status = 'ready'
AND next_attempt_at <= now()
ORDER BY next_attempt_at, created_at
FOR UPDATE SKIP LOCKED
LIMIT $1`,
[limit]
);
const ids = result.rows.map((row) => row.id);
if (ids.length > 0) {
await client.query(
`UPDATE jobs
SET status = 'leased', lease_until = now() + interval '60 seconds',
attempts = attempts + 1
WHERE id = ANY($1::uuid[])`,
[ids]
);
}
await client.query("COMMIT");
return ids;
} catch (error) {
await client.query("ROLLBACK");
throw error;
} finally {
client.release();
}
}
Keep the lease shorter than the worst normal job, or renew it while a long export runs. On success, mark the row complete with an idempotency key. On a timeout or a 429 from a downstream API, move next_attempt_at forward with exponential backoff and jitter. On a schema or authentication error, stop retrying and mark the row dead.
Do not confuse a transaction lock with delivery semantics. The worker can crash after the side effect and before status = 'done'; the lease then expires and the job runs again. That is at-least-once processing. Make the side effect idempotent, for example with a unique (job_id, effect_type) constraint.
How do retry queues and database polling compare on latency, cost, and operations?
A polling table has a simple cost curve: each poll consumes a database round trip, even when no work exists. Use a backoff when the result is empty, and wake immediately after enqueue if your database supports notification. The hidden cost is contention. A broad query, missing index, or lease update that touches too many rows can slow unrelated student requests.
A managed queue usually sleeps until a message is available and absorbs bursts outside the application database. That can lower completion latency and protect the transactional store, while adding policies for visibility timeouts, retention, maximum attempts, and DLQ redrive. Those settings are part of your application contract; write them down and test them.
For a small US or EU deployment, region placement is a latency and data-governance decision, not a reason to duplicate every component. Keep workers near the queue and the database, document where student data is processed, and measure p50 and p95 time from enqueue to completion. I use a 60-second lease and a 10-job claim batch as starting values, then change them from measurements. Your mileage may vary.
The cheapest design on a spreadsheet can be expensive in attention. A table is attractive when one team already owns PostgreSQL and the queue depth is small. A queue plus DLQ earns its keep when operators need a clear retry boundary, when one slow tenant must not block others, or when workers scale independently.
How should operators test retry, DLQ, and redrive behavior?
Test the state machine, not just the happy path. Enqueue a job twice and verify one business effect. Kill a worker after the external call but before the completion update. Return a temporary 429 three times, then a permanent 400. Confirm that the temporary error backs off, the permanent error reaches the DLQ or dead status, and a selected redrive does not duplicate the effect.
Alert on age of the oldest ready job, lease expiry rate, attempts per completed job, and DLQ depth. Logs should include a correlation id, job id, attempt number, and outcome. Metrics tell you that the queue is growing; a structured error tells you why.
GitHub Actions scheduled workflows can kick off a periodic reconciliation, but a scheduler is not a durable job queue. Treat that trigger as a nudge to scan for expired leases, and make the scan idempotent. For queue semantics, document delivery guarantees and retention alongside the code.
Limits and a practical decision rule
Manual polling is not suitable when jobs can run longer than your lease, when the database is already your latency bottleneck, or when many independent consumers need isolated replay. Choose a retry queue with a DLQ when those conditions are true.
A queue is not suitable when the team cannot monitor another service, when jobs contain data that must remain inside an existing database boundary, or when volume is too low to justify the operational surface. Stick with a database table then, but add indexes, leases, idempotency constraints, and a repair command before production.
My rule is short: begin with the smallest state machine you can observe, and graduate when measured latency or failure isolation demands it. The right answer is a clear recovery procedure, not a fashionable queue.
Top comments (0)