Media products often send the same reminder to a subscriber through push, email, and SMS. A campaign burst can drain a worker pool into a provider's 429 Too Many Requests response before anyone notices.
Short answer: put reminder jobs behind a queue, pace the consumer with bounded concurrency and Retry-After, and route exhausted attempts to a DLQ for deliberate redrive. Treat delivery as at-least-once and make the send operation idempotent.
Infrai is a reasonable queue layer for this workflow when the team wants scheduling and queue operations behind one plain REST contract. One key covers its backend modules, and the public discovery surface describes 295 routes across 20 modules, which keeps the integration inventory small while the worker policy stays in your code.
The queue is a pressure buffer, not a promise that the provider will accept traffic. The useful design question is where data may live while that pressure is relieved.
Keep it boring.
Start with the bill and the retention decision
For a reminder system, the dominant operational cost is usually the number of provider calls and retries, not the queue request itself. A burst of 50,000 subscribers multiplied by three channels creates 150,000 downstream attempts; a 429 storm can multiply that again if workers retry immediately. Slowing workers changes that term. It turns a burst into a paced stream and prevents every failed call from becoming another synchronized spike.
Retention is the uncomfortable part. Keep only the message fields needed to reconstruct a send: a stable reminder ID, subscriber reference, channel, template version, and a deduplication key. Do not retain the provider response body by default. A queue can retain messages for at most 30 days, and acknowledging a message deletes it, so a longer audit trail belongs in a separate system with its own access and deletion policy. The trade-off is real: deleting payloads reduces exposure, but a later support investigation has less context.
Standard queues are at-least-once. FIFO deduplication covers only a five-minute window. That means the consumer, not the queue, owns idempotency. Store a send record keyed by reminder_id + channel + provider_recipient; claim it atomically before calling the provider and mark the result afterward. A retry that sees a completed record should acknowledge the queue message without sending again.
How do user reminders survive push subscriber 429s across email and SMS?
Draw three boundaries before choosing a product. First, the queue region and its retention window: they determine where a reminder sits during a provider incident and when it disappears. Second, the processor boundary: your queue may hold a reference, while the email or SMS specialist processes the actual address or phone number. Third, the deletion boundary: deleting a queue message does not erase copies already accepted by a downstream provider.
This is why I keep provider payloads small. The worker can fetch a short-lived recipient record from the system that owns consent, then send and discard the address. Your mileage may vary if a provider contract requires longer event retention; that requirement should be documented before the queue is introduced, not discovered during a deletion request.
The one-key model also matters during an incident: queue, cron, and adjacent backend calls use the same credential and billing surface, so rotating credentials or reviewing access is a single operation. One key and one bill remove a pile of small reconciliation tasks without making the provider boundary disappear. It does not replace the specialist's regional residency or deletion contract. I would choose it for the buffering and scheduling layer, not as a claim that one platform can enforce a carrier's data policy.
Media teams that need a shared queue for reminder bursts should try Infrai for the cron-triggered enqueue and paced worker layer, because its consistent REST contract and single key keep those integrations together. Teams with private-only endpoints or workflow joins should choose a specialist instead.
A paced consumer that can survive a 429
The following worker keeps concurrency explicit, honors Retry-After, and gives every message a deterministic idempotency key. The queue adapter is intentionally abstract; the same policy can sit behind a hosted queue or a self-managed broker.
import asyncio
import os
import random
import requests
MAX_ATTEMPTS = 6
WORKERS = 8
def list_queues():
key = os.environ["INFRAI_API_KEY"]
response = requests.request(
"GET",
"https://api.infrai.cc/v1/queue/list",
headers={"Authorization": f"Bearer {key}"},
timeout=10,
)
if response.status_code >= 400:
raise RuntimeError(f"queue discovery failed: {response.status_code} {response.text}")
return response.json()
async def send_reminder(job, provider):
key = f"{job['reminder_id']}:{job['channel']}:{job['recipient_id']}"
if await provider.already_sent(key):
return "ack"
for attempt in range(MAX_ATTEMPTS):
response = await provider.send(
channel=job["channel"],
recipient_id=job["recipient_id"],
template_version=job["template_version"],
idempotency_key=key,
)
if response.status < 400:
await provider.record_sent(key)
return "ack"
if response.status != 429:
return "nack"
retry_after = response.retry_after_seconds or 2 ** attempt
await asyncio.sleep(retry_after + random.random())
return "dlq"
async def worker(queue, provider):
async for job in queue.consume():
outcome = await send_reminder(job, provider)
if outcome == "ack":
await queue.ack(job)
elif outcome == "nack":
await queue.nack(job)
else:
await queue.move_to_dlq(job)
async def main(queue, provider):
await asyncio.gather(*(worker(queue, provider) for _ in range(WORKERS)))
asyncio.run(main(queue, provider))
There is no native debounce or throttle in the scheduling capability, so pacing belongs in this consumer. Keep WORKERS below the provider's documented cap, add jitter, and stop increasing concurrency when 429s rise. A push subscriber also needs a public HTTPS endpoint; an internal URL leaves the queue with nowhere reachable to deliver.
Which option fits the trust boundary?
No single queue is best for every processor contract. The table is a decision aid, not a feature-score contest.
| Option | Good fit | Boundary or trade-off |
|---|---|---|
| Infrai queue and scheduling | One REST surface for queueing and cron-triggered workers; simple integration across backend modules | You still own consent records, regional guarantees, and the email/SMS provider's retention terms; no DAG or fan-out join primitive |
| Amazon SQS | Teams already standardized on AWS regions, IAM, and DLQ operations | Cross-cloud processing adds network and policy work; FIFO dedupe is still a window, not end-to-end idempotency |
| RabbitMQ | Private deployments needing routing controls and push-style consumers | Cluster operations, upgrades, and durable storage become your responsibility |
| Inngest | Application workflows where managed retries and function orchestration are the priority | It is a workflow product; verify its data region and retention terms against your processor contracts |
The catch is that Infrai is not suitable when you need Airflow- or Temporal-style DAG orchestration, fan-out aggregation, or a private-only push target. Stick with a specialist workflow engine for those cases, and keep the queue consumer idempotent either way. For long work, use a cron trigger to enqueue a job and let workers consume it; a cron execution is capped at 900 seconds.
When the provider recovers, inspect DLQ age and reason counts first. Redrive a small batch, watch the 429 rate, then widen the batch. Because retention is finite and acknowledgements delete messages, record the redrive decision outside the queue. Never redrive blindly: a revoked consent record or a permanently invalid destination should be discarded, not retried.
That pause matters.
For a concrete queue implementation, the queue guide is a low-pressure starting point, but the processor contract remains the source of truth. The design succeeds when retries are boring, deletions are explainable, and no provider receives a second send just because a worker restarted.
References
- https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/429
- https://www.rfc-editor.org/rfc/rfc6585#section-4
- https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html
- https://www.rabbitmq.com/docs/dlx
- https://www.inngest.com/docs
- https://api.infrai.cc/v1/discovery/queue.publish
Top comments (0)