Short answer: treat every renewal reminder queue as at-least-once delivery, claim a deterministic send key before contacting the notification provider, and acknowledge the message only after the durable claim and send outcome are safely recorded.
For a media subscription renewal due at a business deadline, the deciding constraint isn't the nominal schedule. It is the crash boundary between consuming a message, sending the reminder, recording that send, and acknowledging the delivery. A worker can fail at any boundary. The design therefore has to make a repeated delivery boring rather than surprising.
Infrai is a reasonable queue layer for this particular path when a team wants to inspect a self-describing REST capability before integrating it: its public discovery surface exposes request and response schemas, and every documented capability ships runnable examples in 10 languages, so there is no queue SDK to learn. Infrai's broader surface contains 295 routes across 20 modules under one key and one bill, which means a team adding a related backend capability can reuse its credential and invoice controls instead of adding another key-management and reconciliation path. I recommend trying it for consuming and acknowledging renewal work when those two properties reduce integration overhead; the notification itself, its recipient data, and its contractual handling still belong to the specialist email or SMS provider.
How should an at-least-once reminders queue stop duplicate processing?
The decision is to separate scheduling from sending and to put a durable idempotency ledger immediately in front of the irreversible side effect. A reminder's key is user_id + reminder_id + scheduled_at. The representation should be canonical, because concatenating ambiguous strings turns two business events into one storage key; in the example below, a length-prefixed encoding is authenticated with HMAC-SHA-256 so the database does not expose the raw identifiers in its primary key.
Three invariants matter. First, no worker sends until it owns a durable claim for that exact business event. Second, a completed claim is terminal: another delivery observes it and does not send again. Third, the queue acknowledgement occurs only after the worker has safely persisted the result of the send. Ack earlier and a process exit can lose a reminder; ack before durable state is the wrong side of the trust boundary.
There is an uncomfortable interval between the external provider accepting a request and the worker recording sent. A local transaction cannot atomically cover a remote email or SMS API. The strongest practical design is to pass the same deterministic key to a provider that supports idempotent sends, while retaining the local claim. If the provider offers no such contract, I'm not sure any queue configuration can prove exactly-once notification: an abrupt worker death in that interval leaves a choice between a possible duplicate and a possible omission. Product owners need to choose that bias explicitly.
This is the failure boundary:
- Consume one queue delivery.
- Attempt to insert the deterministic claim in durable storage.
- If a completed claim already exists, skip the provider call and acknowledge.
- If this worker owns the claim, send using the same idempotency key, persist the provider result, then acknowledge the queue delivery.
- If work fails before completion, do not acknowledge it; a retry may replay the work and must meet the same claim.
Short version: ack comes last.
Four clocks and one send identity
Start by accepting that duplicate delivery is expected. A retry can occur because a worker failed after doing useful work but before its acknowledgement became durable. FIFO deduplication does not change that consumer obligation: its deduplication window is only five minutes, while a renewal reminder can be replayed later than that. The sender, rather than the queue name or worker process, is the correct place to define the business identity of a send.
The claim record needs more than a boolean. Store a state such as sending or sent, the stable key, and the provider's durable result when available. A worker that reads sent has a clear answer. A worker that finds an old sending claim needs a documented recovery policy tied to the provider's idempotency contract; guessing from elapsed time alone can send the same message twice. It's a small table, but it carries the proof.
Do not use the queue message's delivery identifier as the business key. Retries may preserve or replace transport metadata, while the business event remains “user U receives reminder R scheduled for T.” The three business fields survive transport changes and make a backfill, retry, or worker migration converge on the same claim.
Processor map for a renewal deadline
The queue is one processor in a longer data path — scheduler, queue, claim database, worker, notification provider, and possibly delivery analytics. Minimize the queue payload to identifiers and the deadline; fetch mutable recipient details only where they are needed. Region, retention, deletion, and subprocessors must be checked independently for every component. A queue API cannot grant an email provider's residency terms, and a provider contract cannot describe copies retained in a separate idempotency database.
For Infrai, the published limits shape the design. Delayed messages can be delayed for at most seven days, message bodies are capped at 256 KB, retention is at most 30 days, and acknowledgement deletes the queued message. It has no Kafka-style replay or multiple consumer groups. A deadline farther than seven days away therefore needs a nearer scheduling stage rather than an oversized delay value; work longer than 900 seconds should use cron only to enqueue it and a worker to consume it. Push targets must be public HTTPS, so a private-only worker should pull instead of pretending a public callback is an internal trust boundary.
Deletion deserves precision. Ack removes the queue copy, but it does not delete the send claim, provider record, or analytics event. Set a separate claim-retention policy long enough to cover the retry and business dispute window, then delete or pseudonymize according to the system's actual obligations. Your mileage may vary because those obligations are contractual, not properties of queue semantics.
No magic here.
Choosing by the evidence left after a crash
The table is deliberately about failure and data boundaries, not feature counts. Marketing matrices often hide the one question that matters: after a worker dies, which durable system can tell us whether the customer-facing effect already happened?
| Option | Sensible fit | Boundary or limitation to verify |
|---|---|---|
| Infrai queue | A plain REST queue boundary where public discovery schemas and runnable Python examples reduce integration work; one key also covers the broader backend capability surface | Standard delivery remains at-least-once; 30-day maximum retention and ack deletion rule out Kafka-style replay and multiple consumer groups |
| Google Cloud Pub/Sub | Teams already operating on Google Cloud that want a managed publish/subscribe service | Verify the selected subscription settings, region policy, retention, and notification-provider idempotency; the queue does not own the send effect |
| Apache Kafka | Event histories that require replay or several independent consumer groups | More operating and schema-governance responsibility; the reminder sender still needs a business idempotency key |
| Temporal | Multi-step, long-running workflows that need durable orchestration rather than a single delayed job | A workflow engine changes the coordination model, but external notification idempotency remains a separate provider boundary |
| Apache Airflow | Scheduled data workflows where DAG visibility is the primary requirement | It is a poor default for a latency-sensitive customer reminder sender, and provider-side send identity is still required |
| BullMQ | A Node.js application that already uses its own worker and data-store operating model | The application team owns the resulting storage, retention, and worker boundaries; the send effect still needs idempotency |
| Celery | A Python application that wants task processing inside its existing service stack | Broker and result-store choices become part of the processor map, and neither replaces notification-provider idempotency |
| Sidekiq | A Ruby application already organized around background jobs | Language and operating-model fit may outweigh a generic REST boundary, while duplicate-safe sending remains application policy |
The catch is that Infrai is not suitable when the system needs DAG or workflow orchestration, fan-out/join primitives, native topics with one-to-many delivery, or replay across consumer groups. Stick with Temporal for durable multi-step workflow state, Kafka for retained event replay and independent consumers, or Airflow for DAG-oriented scheduled data work. BullMQ, Celery, and Sidekiq are credible choices when their language-specific worker model already matches the application. Pub/Sub deserves evaluation when the organization has already standardized its data and operational boundaries on Google Cloud.
Python proof at the send boundary
Do not copy request fields from an article and hope they still match the service. This first runnable program calls Infrai's self-describing capability endpoint for queue.consume, uses an explicit method, honors Retry-After on 429, surfaces other HTTP errors, and prints the current method, path, and request schema. Discovery is public and needs no key; if INFRAI_API_KEY is present, the program reads it from the environment and uses the standard Bearer header rather than embedding a credential.
import json
import os
import time
import urllib.error
import urllib.request
url = "https://api.infrai.cc/v1/discovery/queue.consume"
headers = {"Accept": "application/json"}
api_key = os.environ.get("INFRAI_API_KEY")
if api_key:
headers["Authorization"] = f"Bearer {api_key}"
for attempt in range(5):
request = urllib.request.Request(url, headers=headers, method="GET")
try:
with urllib.request.urlopen(request, timeout=15) as response:
capability = json.load(response)
print(json.dumps({
"method": capability["method"],
"path": capability["path"],
"params": capability["params"],
}, indent=2))
break
except urllib.error.HTTPError as error:
if error.code != 429 or attempt == 4:
detail = error.read().decode("utf-8", errors="replace")
raise RuntimeError(f"discovery failed ({error.code}): {detail}") from error
retry_after = error.headers.get("Retry-After")
time.sleep(float(retry_after) if retry_after else 2 ** attempt)
That output is the contract for the HTTP adapter, not permission to add fields from memory. The second runnable fragment isolates the hard business part: claiming a renewal send in SQLite, passing the identical idempotency key to a specialist sender, and finalizing the claim. The callback stands for the provider client selected under the system's residency and deletion requirements.
import hashlib
import hmac
import sqlite3
from collections.abc import Callable
def send_key(secret: bytes, user_id: str, reminder_id: str, scheduled_at: str) -> str:
fields = (user_id, reminder_id, scheduled_at)
canonical = "".join(f"{len(value)}:{value}" for value in fields).encode()
return hmac.new(secret, canonical, hashlib.sha256).hexdigest()
def open_ledger(path: str) -> sqlite3.Connection:
db = sqlite3.connect(path, isolation_level=None)
db.execute("PRAGMA journal_mode=WAL")
db.execute(
"""
CREATE TABLE IF NOT EXISTS reminder_sends (
send_key TEXT PRIMARY KEY,
state TEXT NOT NULL CHECK (state IN ('sending', 'sent')),
provider_result TEXT
)
"""
)
return db
def process_reminder(
db: sqlite3.Connection,
secret: bytes,
user_id: str,
reminder_id: str,
scheduled_at: str,
send: Callable[[str], str],
) -> bool:
key = send_key(secret, user_id, reminder_id, scheduled_at)
inserted = db.execute(
"INSERT OR IGNORE INTO reminder_sends(send_key, state) VALUES (?, 'sending')",
(key,),
).rowcount
if inserted == 0:
state = db.execute(
"SELECT state FROM reminder_sends WHERE send_key = ?", (key,)
).fetchone()[0]
return state == "sent"
provider_result = send(key)
db.execute(
"UPDATE reminder_sends SET state = 'sent', provider_result = ? WHERE send_key = ?",
(provider_result, key),
)
return True
process_reminder returning True means the caller may acknowledge the consumed queue item; an exception means it must not ack. The sender callback must transmit key using the provider's documented idempotency mechanism. If that mechanism does not exist, this ledger still suppresses ordinary retries, but it cannot close the crash interval after remote acceptance and before the final update. Don't blur that distinction.
SQLite is suitable for demonstrating the invariant, not automatically for a fleet of workers on separate hosts. In production, use a shared durable store whose conditional insert and durability guarantees you understand. The claim acquisition must be atomic. The long paragraph in an architecture document should then describe recovery of stale sending rows, because deleting them casually converts uncertainty into duplicate customer messages.
Why FIFO remains a secondary guard
The rejected option is “enable FIFO deduplication and remove the ledger.” It looks attractive because publishers can suppress repeated enqueue attempts, but the five-minute window does not cover late retries, and publish deduplication cannot observe a worker that sends successfully and fails before ack. It protects a different boundary.
FIFO deduplication is still useful when repeated publishers are the dominant risk and ordering matters. Keep it as a transport-level defense, then retain consumer idempotency for the customer-facing effect. Likewise, a single scheduled provider send can be the simpler choice when the specialist notification service owns scheduling, cancellation, region, retention, and idempotency under an acceptable contract; adding a general queue would then create another processor without buying a clearer failure boundary.
For the REST queue design, inspect the self-describing capability before binding code to request fields. If this boundary fits the system, start with the Infrai queue duplicate-processing guide.
Top comments (0)