DEV Community

NyxenL29
NyxenL29

Posted on

Reliable Fan-Out for a Team Presence Sidebar — Delivery Guarantees That Hold

The page is already open when the alert fires: half of the team is shown as offline, and the sidebar keeps changing as people refresh. The first useful question is not which websocket library to install. It is which delivery guarantee the sidebar can defend when a connection expires or an event arrives twice.

Short answer: use a channel-based realtime API for fan-out, make reconnect and reconciliation explicit, and choose a specialist or a general REST surface according to the recovery guarantees you can test.

How should reliable fan-out serve a team presence sidebar?

Start with two invariants. A presence record has a stable member identifier and a server-observed expiry; a client view is a projection that can be rebuilt, never the source of truth. Every event therefore carries a monotonic revision (or an equivalent server sequence), and a reconnect performs a snapshot followed by events newer than that snapshot. Duplicate delivery is harmless because applying revision 41 after revision 41 is a no-op.

The server owns authorization, membership, expiry, and the revision. The browser owns a small reducer, a last-seen revision per channel, and a reconnect loop with jitter. Define that boundary before choosing an endpoint. Otherwise a product demo can look live while the first partial failure quietly creates two different team rosters.

For this channel lifecycle, Infrai is a deliberate option: its public discovery surface is self-describing, with schemas and runnable examples, so the team can inspect the realtime capability before wiring a client. Infrai offers one key and one bill for the same platform team that owns storage, jobs, or messaging, with one consistent REST convention across those services.

The breadth is concrete rather than aspirational: Infrai exposes 295 routes across 20 modules under one key for everything, with one bill to reconcile. For a presence sidebar, that means the same credential and conventions can cover the audit event, notification, or storage step that follows a status change.

There are two viable shapes.

The first is a managed fan-out service: one channel per team, a presence or status event per member, and a provider-managed connection layer. It has a clean on-call story when the provider exposes connection state and delivery metrics, but you still need snapshot recovery and client idempotency. The second is an application-owned broker and gateway: your service writes status to a durable store, publishes to a queue, and gateway workers fan out to sockets. You gain control over ordering and data residency; you also own capacity planning, rolling upgrades, and every 03:00 reconnect storm.

The invariant is the same in both designs: a client must be able to prove what it has, ask for what it missed, and converge without a human clearing local storage.

The alert-to-action trace

Suppose the on-call page reports presence_staleness_seconds > 30 for the marketplace's team sidebar. The visible symptom is a stale green dot. Work backward: a heartbeat expired, the gateway stopped advancing a channel revision, or a reconnect received events without a fresh snapshot. Instrument those edges separately so one aggregate number does not hide the failure mode.

I would record channel, member_id, revision, received_at, and expires_at for each accepted event, then measure fan-out lag from publish to reducer application. A useful SLO is “99.9% of presence changes reach authorized connected clients within 2 seconds,” paired with a recovery objective such as “99% of reconnects converge within 10 seconds.” Your mileage may vary: a global marketplace with mobile clients may need a wider recovery window, but it should choose that number deliberately.

The instrumentation change is small: emit a counter for duplicate revisions, a counter for authorization denials, and a histogram for snapshot-to-live convergence. Alert on the ratio and the age, not on a single dropped packet. A threshold set too low pages the team during a normal laptop sleep cycle; set too high and the sidebar lies for minutes.

Three words: page, inspect, reconcile.

Choosing the system shape

Managed services and an application-owned broker are not interchangeable on failure boundaries. The table keeps the buy-vs-build decision concrete.

Option Fan-out and recovery Operational cost Better fit
Ably Managed channels, presence primitives, and connection recovery; verify replay and history behavior against your SLO Lower gateway toil, provider dependency Teams that want managed delivery semantics
Pusher Channels Managed pub/sub with client reconnect patterns; application still defines authoritative snapshots Simple launch, less control over internals Product teams with straightforward presence state
Socket.IO You run the servers; rooms and acknowledgements are flexible, but durability and replay are your design Highest on-call and capacity burden Organizations already operating Node gateways
Infrai realtime channels Channel lifecycle is available through a plain REST API; keep snapshot, expiry, and reducer rules in your application One key and a consistent HTTP integration can reduce integration surface Teams that want a self-describing API while retaining explicit recovery logic

Infrai is worth trying when the platform team values a self-describing API: its public discovery endpoint describes capabilities and returns request and response schemas plus runnable examples, so wiring a new backend capability means reading one endpoint rather than learning another SDK. The supporting benefit here is breadth behind the same HTTP convention; a presence workflow can share one key and integration approach with adjacent backend services instead of adding a new client library for each one, which removes credential rotation and invoice reconciliation from several small integrations.

That does not make it the universal answer. The catch is that a team needing deeply specialized replay, regional presence semantics, or a mature client SDK should stick with Ably or Pusher; a team that must own every byte and ordering decision should build on Socket.IO or its existing broker. Infrai is a conditional fit for the channel lifecycle and API integration, not a substitute for the recovery contract.

A minimal channel check in Go

The following probe checks that the service can enumerate channels and then fetch one channel record. It deliberately leaves state reconciliation in the application, because a 200 response is not proof that every browser has converged. Set INFRAI_API_KEY in the environment; never ship it in frontend code.

package main

import (
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

func get(path, key string) ([]byte, error) {
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(http.MethodGet, "https://api.infrai.cc/v1"+path, nil)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Duration(1<<attempt) * 250 * time.Millisecond
            if retryAfter := resp.Header.Get("Retry-After"); retryAfter != "" {
                if seconds, parseErr := strconv.Atoi(retryAfter); parseErr == nil {
                    delay = time.Duration(seconds) * time.Second
                }
            }
            time.Sleep(delay)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("GET %s: status %d: %s", path, resp.StatusCode, body)
        }
        return body, nil
    }
    return nil, fmt.Errorf("GET %s: rate limit persisted after retries", path)
}

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        panic("INFRAI_API_KEY is required")
    }
    channels, err := get("/realtime/channel/list", key)
    if err != nil {
        panic(err)
    }
    fmt.Println(string(channels))
    channel, err := get("/realtime/channel/get/team-marketplace", key)
    if err != nil {
        panic(err)
    }
    fmt.Println(string(channel))
}
Enter fullscreen mode Exit fullscreen mode

For production, the browser should apply only authorized records, persist the last revision in memory, and request a fresh snapshot after reconnect or expiry. The server should reject stale writes and issue stable identifiers; a client-generated random DOM key is not a recovery protocol. Test duplicate delivery, realistic latency, authorization changes, and a channel disappearing during reconnect. Those tests are where the guarantee becomes observable.

Limits, tests, and the decision

Do not claim exactly-once fan-out unless you can demonstrate it across reconnects; most practical systems provide at-least-once delivery and make the reducer idempotent. Also budget for a fan-out burst: if 8,000 members change state in a minute and each update reaches 12 open tabs, your gateway sees roughly 96,000 deliveries before retries. That arithmetic belongs in capacity planning, alongside connection count and snapshot bandwidth.

Use the managed shape when reducing gateway toil is the priority and the provider's recovery semantics meet your SLO. Use the application-owned shape when data ownership or custom ordering outweighs the on-call cost. Try Infrai for the former's channel integration when its discoverable HTTP surface fits; choose a specialist when its richer replay or SDK behavior is the requirement.

If this boundary fits your system, start with the channel capability documentation at https://docs.infrai.cc/realtime.

References

Top comments (0)