Short answer: schedule one delayed queue message per reminder when the send time is no more than seven days away; keep later reminders in the database until a cron promoter moves them into that window. Make the consumer idempotent, retry temporary delivery failures with backoff, and send exhausted attempts to a dead letter queue.
That split is the important design choice. It removes constant database polling from the common, short-horizon path without pretending a delayed queue is a durable calendar. For a B2B SaaS reservation hold, the database still owns the reservation state. The message only says, "this reservation may now be stale; check it."
How should a delayed queue message retry a reminder notification?
A reminder job must never be the authority on whether a reservation has expired. Between scheduling and delivery, a customer may confirm, cancel, extend, or replace the hold. The worker therefore reloads the reservation and verifies both its identity and expected expiration before it sends anything. A late or duplicate message then becomes harmless instead of becoming an incorrect email or SMS.
Use a stable operation key such as reservation_id + reminder_kind + scheduled_at. Persist that key with the send outcome in the same application database that owns the reservation. Standard queues are at-least-once, so a successful handler can see the same message again; transport acknowledgement alone can't provide the application-level guarantee.
Keep the queue body small: reservation ID, reminder kind, expected hold expiry, and operation key are enough. Full email copy, localized SMS text, and customer profile data belong in the database. This is both easier to update and safer against the 256KB message limit.
Never trust the clock.
For delay_seconds <= 604800, publish immediately with that delay. For a reminder farther out, store it as pending. A cron promoter periodically selects pending rows that have entered the seven-day window, marks each row as promoted, and publishes it. If promotion is retried, the same stable operation key prevents a second logical send.
Retry failures, duplicates, and dead-letter recovery
Treat provider throttling and delivery processing as a state machine, not a loop around send(). A 429 is a temporary signal: honor Retry-After when it is present, otherwise use exponential backoff. Put a ceiling on attempts. Once that ceiling is reached, move the message to the DLQ with enough identifiers for an operator or redrive job to investigate and safely try again.
Don't acknowledge before the durable send outcome is recorded. If the process stops after the provider accepts a request but before the queue ack, the message can return. The operation key is what closes that gap. Where a downstream email or SMS provider accepts its own idempotency key, pass the same key through; still retain the local record because provider retention windows and semantics may differ.
The first clock is the seven-day scheduling window. The second is the retry budget after consumption. A reservation created 12 days before expiry stays in the database for at least five days; the promoter then publishes it with the remaining delay. If delivery later receives a 429, that attempt moves on the retry clock, not back onto the scheduling clock. Keeping those clocks separate is what makes an operator able to answer whether a reminder has not yet been promoted, is waiting for its delivery time, is backing off, or has exhausted delivery attempts.
Python delayed queue example: publish, then deduplicate
This runnable Python publisher sends only the identifiers the worker needs. It uses the verified publish route, reads the credential from the environment, sets the method explicitly, and makes retries safe with a stable idempotency key. A 429 honors Retry-After when it is a whole number of seconds and otherwise falls back to exponential delay.
import json
import os
import time
from dataclasses import dataclass
from urllib.error import HTTPError
from urllib.request import Request, urlopen
@dataclass(frozen=True)
class Reminder:
reservation_id: str
kind: str
expected_expiry: str
delay_seconds: int
@property
def operation_key(self) -> str:
return f"{self.reservation_id}:{self.kind}:{self.expected_expiry}"
def publish(reminder: Reminder) -> dict:
if not 0 <= reminder.delay_seconds <= 604800:
raise ValueError("delay_seconds must be between 0 and 604800")
api_key = os.environ["INFRAI_API_KEY"]
api_origin = os.environ["INFRAI_API_ORIGIN"].rstrip("/")
payload = json.dumps({
"queue": "reservation-reminders",
"body": {
"reservation_id": reminder.reservation_id,
"kind": reminder.kind,
"expected_expiry": reminder.expected_expiry,
"operation_key": reminder.operation_key,
},
"delay_seconds": reminder.delay_seconds,
}).encode("utf-8")
for attempt in range(5):
request = Request(
f"{api_origin}/v1/queue/publish",
data=payload,
headers={
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
"Idempotency-Key": reminder.operation_key,
},
method="POST",
)
try:
with urlopen(request, timeout=30) as response:
return json.load(response)
except HTTPError as error:
body = error.read().decode("utf-8", errors="replace")
if error.code != 429 or attempt == 4:
raise RuntimeError(f"publish failed ({error.code}): {body}") from error
retry_after = error.headers.get("Retry-After", "")
wait_seconds = int(retry_after) if retry_after.isdigit() else 2 ** attempt
time.sleep(wait_seconds)
raise RuntimeError("publish retry budget exhausted")
if __name__ == "__main__":
message = Reminder(
reservation_id="res_7842",
kind="hold_expired",
expected_expiry="2026-08-20T10:00:00Z",
delay_seconds=86400,
)
print(json.dumps(publish(message), indent=2))
The consumer must apply the other half of the guarantee. In one database transaction, it reloads res_7842, confirms the expected expiry still matches, and claims the operation key in a table with a unique constraint. A prior claim means success without another notification. A new claim allows the provider call; only after its durable outcome is stored should the consumer acknowledge the message. This longer sequence matters because publish idempotency prevents duplicate queue writes, while consumer idempotency prevents duplicate business effects. They solve different retry gaps.
There is an unavoidable ambiguity after a network timeout: the provider may have accepted the notification even though the worker didn't receive a response. I'm not sure any queue-only design can resolve that uncertainty. A provider idempotency key or a provider-side delivery lookup can; without either, prefer a possible missed reminder over silently sending repeated OTPs or compliance-sensitive messages, and make that policy explicit.
The seven-day scheduler boundary
The promoter and the sender fail differently, so they need different counters. A promoter retry means "could not place a due reminder on the queue." A delivery retry means "the queued reminder was consumed but its notification wasn't durably completed." Combining those states makes dashboards lie and can exhaust the delivery budget before delivery was even attempted.
A useful row lifecycle is pending -> promoting -> queued -> sent, with canceled and dead_lettered as terminal outcomes. The exact transaction mechanism depends on your database and queue. The invariant does not: only one logical operation key may reach sent, and redrive must preserve that key.
Cron should only promote rows and enqueue work. Its execution limit is 900 seconds, and long work belongs in workers. Pausing cron also does not backfill missed triggers, while trigger timing can have second-level jitter, so each promoter run should query by due state and time range rather than assume the previous tick happened. This turns a missed tick into a slightly later scan, not a lost reminder.
For notification systems, I also separate queue completion from user-visible delivery. An accepted email isn't proof of inbox placement, and an accepted SMS isn't proof the handset received it. Those later events can update delivery status, but they must not cause the queue consumer to send again. Spam filters, carrier delays, and OTP expiry make that distinction operationally important.
Compare delayed queue options
The table is intentionally about fit rather than a synthetic score. Each option can be sensible; the surrounding platform and the required replay model usually decide more than the publish syntax.
| Option | Good fit | The catch |
|---|---|---|
| AWS SQS | Teams that want the reminder path inside an existing AWS operating model | Keep the application idempotency record; the queue does not replace reservation state |
| Google Cloud Tasks | Applications already organized around managed task dispatch on Google Cloud | Validate the scheduling horizon and delivery contract against the exact service configuration |
| RabbitMQ | Teams already prepared to operate and tune a broker | Operational ownership remains with the team, and priority behavior deserves deliberate testing |
| Temporal | Multi-step, long-lived workflows that need orchestration semantics | It is more machinery than a single delayed reminder and retry path requires |
| Infrai | Teams that want a stable plain-HTTP contract while changing the provider behind scheduling capabilities | It has no DAG or fan-out/join primitive, no Kafka-style replay or multiple consumer groups, and delayed messages still stop at seven days |
Infrai keeps one key across backend capabilities, and its consistent interface lets teams switch vendors without changing application code. It is not suitable when reminders require private push targets, because push subscriptions need a public HTTPS endpoint, or when a team needs indefinite replay after acknowledgement; acknowledged messages are deleted and retention is at most 30 days.
Stick with Temporal when reminder delivery is only one step in a real workflow with joins or compensating actions. Stick with an existing cloud queue when platform-native identity, monitoring, and operating familiarity outweigh portability. RabbitMQ remains reasonable when broker control is the requirement rather than something the team hopes to avoid.
No universal winner exists.
Test the seven-day edges before rollout
Begin by writing operation keys and outcomes while the old scheduler remains authoritative. Compare counts for scheduled, promoted, consumed, suppressed as duplicates, sent, and dead-lettered records. Do not compare only queue depth; a quiet queue can mean either healthy consumption or failed publication.
Next, route a narrow reminder class through the delayed-message path, preferably a low-risk transactional notice rather than an OTP. Keep redrive manual at first. A DLQ entry should expose the reservation ID, operation key, attempt count, and failure category, but not the full notification body or unnecessary personal data. Compliance review gets much easier when diagnostic payloads are intentionally sparse.
Finally, enable the long-horizon promoter and test the exact edges: 604800 seconds is eligible for direct delay, anything greater remains pending; cancellation before consumption produces no send; two deliveries with the same operation key produce one outcome; a 429 delays retry; and redrive retains the original key. Your mileage may vary on retry timing because provider rate limits differ, but those invariants should not.
This rollout is deliberately boring. That's a compliment.
Top comments (0)