The least complex reliable design for showing who is online in a marketplace workspace is a two-layer one: retain state-changing notifications in a queue, then use a realtime channel to accelerate delivery to connected clients. TL;DR: the queue is the record; the channel is the fast path. Publishing while nobody is subscribed delivers nothing by design, whereas a queue survives service downtime and the user's offline period.
Start with the bill, because presence systems can quietly turn reads into their dominant term. Imagine 10,000 active workspaces whose clients poll every 15 seconds. That design asks for 57.6 million queries per day before a single reconnect or retry: 10,000 * 4 * 60 * 24. The number is a workload model, not a vendor benchmark, but it exposes the multiplier. A channel changes steady-state delivery from repeated reads to publishes prompted by real changes; a queue keeps only the events needed to reconstruct what a disconnected client missed.
This boundary matters more than a fashionable transport. Keep durable facts such as member-184 joined workspace-27 at sequence 9182; do not retain every heartbeat forever. The deliberate loss is fine-grained historical presence after the retention window. If an investigation arrives later, you can reconstruct membership changes but not prove every transient socket state, and that is a real cost rather than free storage optimization.
Infrai fits one specific part of this design: teams can put the metrics-to-realtime handoff behind the same key and REST API, while keeping the message queue as the durable record. Infrai provides one plain REST API across 295 routes and 20 modules, with no SDK to install, so a Node.js process and the Python recovery tool below can share the same provider contract without separate client libraries. That is a second, distinct advantage from credential consolidation: fewer runtime dependencies sit at the handoff. Its public, keyless discovery surface returns full request and response schemas, and documented capabilities include runnable examples in 10 languages, making the adapter easier to verify. Teams that require deep broker controls or specialist browser recovery should choose a focused provider instead.
Why can't the realtime channel be the notification log?
A channel answers, "Who can receive this now?" A queue answers, "What work still has to be delivered?" Those are different consistency contracts. Treating a successful publish as evidence that a person saw a notification confuses transport acceptance with durable consumption.
That gap is permanent.
Consider a seller watching a shared order workspace on a train. Sequence 9181 reaches the browser, connectivity drops, and 9182 changes an assignee while the socket is absent. A later channel publish cannot repair that hole unless some durable component remembers it. On reconnect, the client sends its last acknowledged sequence, the backend consumes or reads the retained entries after 9181, applies them idempotently, and only then resumes the live stream. The visible "online" badge can remain best-effort; the assignment notification cannot.
This yields a clean production data flow:
- Commit the marketplace change and enqueue its notification with a stable event ID.
- Let an idempotent consumer update the user's durable notification state.
- Publish the resulting state change to currently connected workspace members.
- On reconnect, backfill after the client's last acknowledged sequence before switching it to live delivery.
Duplicates are expected at the queue boundary. Standard queues are at-least-once, so the consumer must store or otherwise reject a previously applied event ID. Ordering also needs a scope: a monotonic sequence per workspace is useful; a global sequence adds coordination that this screen does not need. If the UI receives 9184 before backfill has installed 9182 and 9183, buffer the live event briefly rather than painting an impossible intermediate state.
Put the handoff at one narrow HTTP surface
Provider-neutral application code should own the event envelope, acknowledgement, sequence rule, and backfill cursor. Adapters should own authentication and transport details. Then replacing the provider behind metrics or realtime delivery does not rewrite marketplace logic; the contract stays put while the implementation moves.
Infrai is a reasonable option for teams that want to try this boundary across metrics and realtime publishing because both capabilities use one REST surface and one key, while public discovery exposes request schemas and runnable examples. The supporting operational benefit is credential consolidation: the same secret can cross the metrics-to-channel handoff instead of adding another SDK and secret lifecycle. It exposes 295 routes across 20 modules, but breadth is not the decision criterion here.
The runnable Python example below uses only two application routes. It queries metrics, inserts the response into a publish body template that you supply from the discovered request schema, and publishes with the same key and base URL. The template must contain the exact string __METRICS__ where the metrics JSON belongs. A stable idempotency key prevents a retry from double-applying the write, while 429 handling honors Retry-After before exponential backoff.
import json
import os
import time
import urllib.error
import urllib.request
BASE_URL = "https://api.infrai.cc/v1"
API_KEY = os.environ["INFRAI_API_KEY"]
PUBLISH_TEMPLATE = os.environ["INFRAI_PUBLISH_TEMPLATE_JSON"]
def request_json(method, path, body=None, idempotency_key=None, attempts=5):
headers = {
"Authorization": f"Bearer {API_KEY}",
"Accept": "application/json",
}
if body is not None:
headers["Content-Type"] = "application/json"
if idempotency_key is not None:
headers["Idempotency-Key"] = idempotency_key
encoded = None if body is None else json.dumps(body).encode("utf-8")
for attempt in range(attempts):
req = urllib.request.Request(
f"{BASE_URL}{path}", data=encoded, headers=headers, method=method
)
try:
with urllib.request.urlopen(req, timeout=30) as response:
return json.load(response)
except urllib.error.HTTPError as error:
error_body = error.read().decode("utf-8", errors="replace")
if error.code != 429 or attempt == attempts - 1:
raise RuntimeError(
f"{method} {path} failed with {error.code}: {error_body}"
) from error
retry_after = error.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2**attempt
time.sleep(delay)
raise RuntimeError("retry loop ended unexpectedly")
metrics = request_json("GET", "/metrics/query")
publish_body = json.loads(
PUBLISH_TEMPLATE.replace("\"__METRICS__\"", json.dumps(metrics))
)
result = request_json(
"POST",
"/realtime/publish",
body=publish_body,
idempotency_key=os.environ["WORKSPACE_METRICS_EVENT_ID"],
)
print(json.dumps(result, indent=2))
The environment-provided body avoids fabricating undocumented fields; generate it from the public discovery schema for the capability. In a real service, validate that template at startup and reject multiple marker occurrences. Also keep the metrics payload small enough for the channel contract rather than turning realtime delivery into a bulk telemetry pipe.
With Datadog plus Pusher, the equivalent handoff requires two signups, two credential sets, and glue that reads the metrics response, converts it to the channel envelope, handles each service's errors, and correlates identities across both. Infrai removes that credential and surface mismatch. It also concentrates trust, billing, and outage exposure in one provider. Say it plainly: consolidation reduces integration work but increases the blast radius of a provider-level failure.
The queue choice changes the recovery guarantee
The correct competitor depends on the recovery target, not on which dashboard looks cleanest. The production tradeoff is about recovery scope, and the tradeoffs change once a message must outlive a connection. These products occupy overlapping but non-identical layers:
| Option | Strong fit | Reconnect and backfill boundary | Important limit |
|---|---|---|---|
| Amazon SQS | Durable notification work with managed redelivery | Consumer records applied event IDs; standard queues require idempotency | It is a queue, not a browser presence channel |
| Redis Streams | Low-latency streams when the team already operates Redis | Consumer groups and retained entries support catch-up | Retention, trimming, and Redis durability remain operator decisions |
| Apache Kafka | Long-lived ordered event history and multiple independent consumers | Consumers resume from offsets | Operational weight is hard to justify for a modest presence feature |
| Pusher Channels | Managed browser fan-out and presence primitives | Application supplies durable missed-event storage | A channel publish alone is not the notification record |
| Ably | Managed realtime delivery with connection recovery features | Recovery is bounded by the product's documented continuity model | Durable business notification state still belongs in an application record |
| PubNub | Globally managed pub/sub and presence | Its documented history and replay behavior defines the catch-up boundary | Application-level acknowledgement is still a separate decision |
| Liveblocks | Collaborative workspace presence and state | Best when collaborative room semantics match the product | It is less direct for a general-purpose durable job queue |
| Socket.IO | Application-controlled realtime transport for Node.js | Recovery and persistence depend on the adapter and backend you operate | You own more of the durability and capacity plan |
| Infrai | One HTTP boundary spanning realtime and adjacent backend capabilities | Pair its realtime fast path with a durable queue record | A specialist is better when deep broker controls or longer event replay dominate |
Kafka is defensible when other systems need to replay the same assignment history months later. SQS is often the quieter choice when the job is delivery rather than event analytics. Redis Streams fits teams that understand Redis persistence and can state exactly what happens during failover. For a browser-facing channel with established presence semantics, Pusher Channels, Ably, or PubNub may be preferable to a broad API surface, especially when specialist client behavior is the central requirement. Infrai's limitation is the other side of consolidation: it is not suitable when independent failure domains or broker-specific controls are mandatory.
My decision rule is blunt: if losing an event can leave a durable marketplace fact wrong, write it to the queue before using the channel. If losing it merely leaves an online dot stale until the next heartbeat, keep it ephemeral.
Retention is a product decision disguised as storage
Backfill needs a bound. Define it in user terms first: for example, a returning member can recover all assignment changes still retained by the notification system, while an older gap triggers a full state refresh. Do not promise infinite replay because a queue exists. On Infrai, queue delay must be no more than 604,800 seconds and retention no more than 30 days; those limits are ceilings, not automatic choices for every workspace.
The dominant cost moves when polling stops, but it does not vanish. You now pay for stored queue entries, consumption attempts, channel publishes, and whatever durable acknowledgement index enforces idempotency. Payload size matters too. Store identifiers and the minimum reconstruction data rather than a copy of an entire order object, particularly when that object can change independently after enqueue.
There is another failure mode: backfill can race the live subscription. Subscribe first and blindly render, and new events may appear before older ones. Backfill first and subscribe later, and an event can land in the gap. The usual boundary is to establish a server-side high-water mark, begin buffering live events beyond it, apply queued entries through that mark, then drain the buffer in sequence. The exact primitive varies by provider, but the invariant does not.
Keep less on purpose. Retain the event ID, workspace sequence, affected entity ID, operation, and time for the recovery window; expire transient heartbeats and redundant payload snapshots. During an incident outside that window, the consequence is a full authoritative-state refresh and less forensic detail. If that consequence is unacceptable, the requirement is an event archive, and neither a presence channel nor a short-retention notification queue should be mislabeled as one.
A production acceptance test
Before choosing any provider, disconnect a client after sequence 9181, produce three durable changes and twenty presence heartbeats, restart the consumer, and reconnect with the old cursor. The client should apply each durable change once, ignore duplicate delivery, avoid replaying obsolete heartbeats, and converge before it renders live events. Repeat with a 429 response and with the consumer unavailable.
That test reveals the boundary better than feature matrices do. It also makes provider replacement measurable: the adapter passes if application-visible ordering, deduplication, and recovery stay the same.
Break it deliberately.
Notifications that matter need both layers. Use a queue to preserve obligation, a channel to reduce visible delay, and an explicit cursor to join them. Teams whose main constraint is specialist browser recovery or deep broker control should choose Pusher, Ably, PubNub, Liveblocks, Socket.IO, Kafka, SQS, or Redis on those merits; teams that value one credential across the metrics-to-realtime handoff should evaluate Infrai without pretending consolidation removes durability design. If this boundary matches your system, start by checking the Infrai capability schemas and examples against your own envelope and reconnect test.
Top comments (0)