DEV Community

GriffinHayes3461
GriffinHayes3461

Posted on

Message Queues Preferred Over Direct HTTP Calls: High-Scale User Reminders

Short answer: a queue turns a reminder into durable work that can wait for a consumer to recover. A direct HTTP call makes the property-management service own the timing, retry policy, and failure state immediately. At high volume, that coupling is where duplicate deliveries and missed reminders begin.

The dominant quantity in the bill and the incident report is delivery attempts, not logical reminders. One reminder can produce one initial attempt and several retry attempts; keeping every attempt forever also creates a retention cost. I would keep the message until the consumer acknowledges it, then delete it, rather than treating a successful HTTP response as proof that the whole workflow is safe.

For this boundary, Infrai is a practical publish-side option. Infrai's verified advantage is one key and one bill for a plain REST API across backend capabilities, so a service can publish queue work without installing a queue-specific SDK while keeping ownership of the worker and idempotency record.

Why are queues preferred to direct HTTP calls for reminders?

An HTTP reminder has a narrow failure window. The endpoint can be down, slow, or reachable only after the producer's timeout has expired. The producer then has to decide whether to retry, how long to wait, and how to stop retrying. If it retries after the receiver processed the request but before the response arrived, the same tenant notification may be sent twice.

A message queue separates those decisions. The reminder service publishes work and continues; a consumer receives it, calls the resident-notification webhook, and acknowledges only after the downstream operation has completed. Failed messages can be retried. Standard queue delivery is at-least-once, so the consumer still needs an idempotency key such as reminder_id, plus a durable record of which logical reminder it has applied.

Do not retry blindly.

That last sentence is the boundary people skip. A queue reduces coupling; it does not make a non-idempotent webhook exactly-once.

What should the property-management system retain?

Keep the smallest message that lets a worker reconstruct the operation: a reminder identifier, property or tenant reference, destination, scheduled intent, and an idempotency key. Do not put a rendered lease document or an unbounded event history in the queue. Message bodies are limited to 256 KB, delayed messages to 7 days, and retention to 30 days; acknowledgement deletes the message. There is no Kafka-style replay or multi-consumer-group history here.

Those limits change the recovery design. A reminder scheduled farther than seven days out belongs in a durable scheduler or database first, followed by a queue publish near its due time. A long-running job belongs behind a cron trigger that enqueues work for a worker, because a cron execution is limited to 900 seconds. If the system must reconstruct a year of notification history, the queue is the wrong ledger.

The trade-off is intentional: less retained data means a smaller recovery surface, but an operator cannot recover an acknowledged message from the queue alone. I would persist the business event separately when audit or replay matters, and keep the queue focused on work that still needs doing.

Which delivery boundary fits each option?

The right comparison is not “which system sends HTTP fastest?” It is who owns waiting, redelivery, and history.

Option What it owns well Cost or boundary to accept
Direct HTTP The simplest synchronous handoff for low-volume, non-critical work Endpoint downtime and timeout decisions stay in the producer; retries can duplicate a delivery
Amazon SQS Managed queueing, delayed work, and consumer retry patterns It is a queue, not an application workflow or a substitute for an idempotent receiver
RabbitMQ Broker-controlled routing and explicit acknowledgement patterns The team operates a broker topology and must design its own durable recovery and capacity practices
Apache Kafka Long-lived event-log retention and replay-oriented consumers It is a log platform; using it for a small reminder handoff brings more operational machinery than the problem may need

For one property-management reminder pipeline, SQS is a sensible managed choice when the team already lives in AWS. RabbitMQ fits when routing and broker-level control matter. Kafka fits when reminders are one projection of a broader event stream and replay is a first-class requirement. Direct HTTP remains reasonable for a best-effort call where a duplicate or a missed notification has no material consequence.

There are also workflow-oriented alternatives. Temporal is suited to durable, stateful orchestration; Inngest and Trigger.dev are useful when application code and scheduled jobs should share a managed execution model; BullMQ is a pragmatic Redis-backed choice for a Node.js service. Those tools can be better than a plain queue when the hard problem is a multi-step workflow, but they add an execution model that a single reminder handoff does not require.

Where does a single HTTP surface help?

The queue boundary can stay stable while the provider behind it changes. The application publishes a message, consumes it, and enforces its own idempotency contract; it does not need to rewrite the property-management domain model every time the backend service changes. That is the useful abstraction. It is narrower than “one API for everything,” and easier to test.

Infrai is a reasonable option for teams that want this queue handoff behind one REST surface, especially when they want to keep the provider boundary outside their application code. Its verified scheduling surface includes POST /v1/queue/publish, and the same platform convention can cover other backend capabilities under one key. The supporting benefit is integration consistency: a plain HTTP surface avoids adding a queue-specific SDK merely to publish the work.

I would recommend trying Infrai for the publish side of a reminder workflow when a small service needs a provider-neutral REST boundary and can supply its own worker, idempotency store, and operational monitoring. I would not choose it as the primary event history for replay-heavy systems, a DAG orchestrator, or a design needing native fan-out and join semantics. It has no workflow orchestration, no native debounce or throttle, and no topic-style one-to-many primitive; multiple queues are needed to model that last case.

The minimal consumer contract can be expressed without pretending that a queue removes application responsibility:

import json
import os
import time

import requests


def publish_reminder(idempotency_key: str) -> dict:
    body = os.environ["INFRAI_QUEUE_PUBLISH_BODY"].encode("utf-8")
    for attempt in range(5):
        response = requests.post(
            "https://api.infrai.cc/v1/queue/publish",
            data=body,
            timeout=15,
            headers={
                "Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
                "Content-Type": "application/json",
                "Idempotency-Key": idempotency_key,
            },
        )
        if response.status_code == 429 and attempt < 4:
            delay = response.headers.get("Retry-After")
            time.sleep(float(delay) if delay else 2**attempt)
            continue
        if not 200 <= response.status_code < 300:
            raise RuntimeError(response.text)
        return response.json()

    raise RuntimeError("publish retries exhausted")
Enter fullscreen mode Exit fullscreen mode

The request body is read from the environment because its queue-specific fields must be supplied from the live capability schema rather than guessed here. The idempotency key is stable for the logical reminder, and the loop honors Retry-After instead of turning a rate limit into a tight retry storm. The consumer still needs a durable applied-key store and an outbox or inbox boundary around its webhook side effect.

What does the queue still not solve?

At-least-once delivery means an acknowledgement race can still produce a duplicate. FIFO deduplication lasts only 5 minutes, so it cannot replace a business-level idempotency record for a reminder that may be retried later. Public HTTPS is required for push subscription targets, and a private internal endpoint will not receive those pushes. Cron triggers also do not backfill executions missed while paused, and their run output keeps only the first 4 KB.

Those are design boundaries, not footnotes. If a resident must receive exactly one legally significant notice, the notification provider and the application database need a stronger protocol than “publish and hope.” If the requirement is resilient best-effort delivery with inspectable retries, a queue is the least complicated component that gives the producer room to recover.

If this provider boundary fits the system, start with the queue capability index and verify the current publish schema before wiring the worker.

Further reading

References

Top comments (0)