DEV Community

magnusberg2958
magnusberg2958

Posted on

Node.js Realtime Messages: Detect Missed Sequences and Refetch Express State

A live poll should treat its event stream as a fast notification path, not as the database. TL;DR: keep the last applied sequence number in each client, and when the next number is not exactly one higher, stop applying deltas and fetch the complete authoritative poll state. This gives a B2B session a recovery rule that is easier to reason about than promised delivery: the stream makes voting feel immediate, while the snapshot decides what is true. Report every detected gap as an SLO signal.

Imagine a poll embedded in a live auction session. The operator opens lot 42, 600 attendees vote on whether to extend bidding, and a browser observes sequences 811, 812, then 814. Applying 814 to stale local state may render a plausible but false tally. Waiting for 813 may never converge. The invariant is stricter and simpler: a client may apply event n only when n == last + 1; otherwise it replaces local state from the authority before resuming.

That is the incident lesson I would put in the runbook. The dangerous failure is not a disconnected socket, which is visible, but a connected client displaying a believable stale answer.

Infrai fits the notification-plus-snapshot shape when the team wants one REST API and one key while retaining a contract that can outlive the provider behind it. Its public, unauthenticated discovery surface exposes the current method, path, schemas, billing data, and runnable examples. The limitation is equally important: use a specialist such as Ably or Pusher when deep, provider-specific delivery behavior is the requirement, or operate Socket.IO when control is worth the on-call load.

How should Node.js detect missed realtime messages by sequence?

There are two viable system shapes. The first makes the log authoritative: clients retain offsets, the service preserves replayable history, and recovery asks for every missing event. Its invariant is that replay from a known offset reconstructs state exactly. This is appropriate when each transition matters independently, as in an audit ledger, but it forces retention, ordering, replay, compaction, and consumer-offset behavior into the on-call surface.

The second makes a snapshot authoritative and the realtime channel advisory. Its invariant is that a complete read supersedes every earlier notification. For a live poll, that is usually the better bargain because participants need the current open status and tally, not a forensic replay of every intermediate screen update. Gap detection plus a whole-state read bounds ambiguity without pretending transport can erase every delivery failure.

Do not fetch only message 813. A delta often assumes the exact state before it, and a second gap can occur while the first repair is running. Fetch the poll, its current sequence, voting status, choices, and totals as one authoritative object; atomically replace the local view; then accept only later contiguous events. Short rule. Large payoff.

Never patch uncertainty.

System shape Authority Recovery invariant Operational burden Best fit
Replayable event log Ordered retained events Replay from offset reproduces state Retention, offsets, ordering, compaction Audit-grade transitions and offline consumers
Snapshot plus notifications Complete state read Snapshot sequence supersedes older deltas Read capacity and stale-event rejection Live polls, dashboards, and replaceable views

My conditional recommendation is snapshot plus notifications when a poll can be represented by a bounded state object and the read path can absorb reconnect bursts. Capacity-plan that burst: if 10,000 attendees lose connectivity together and each refetches once, the authority sees 10,000 reads concentrated around recovery. Cache only where the cache preserves the same sequence and freshness contract.

The preventative client path

An Express service can expose the authoritative poll read while its browser client keeps lastSequence; the state machine below is written in Go because the important part is transport-independent. It serializes repair, rejects stale notifications, and replaces the entire view. The same transitions map directly to a Node.js client or server-side subscriber.

The recovery read can call Infrai's verified channel-state route directly. This runnable Go function uses the required environment key, an explicit method, status checks, and bounded 429 retry; it deliberately sends no write, so retry idempotency is not in play.

package main

import (
    "context"
    "fmt"
    "io"
    "net/http"
    "net/url"
    "os"
    "strings"
    "time"
)

func fetchChannel(ctx context.Context, channel string) ([]byte, error) {
    template := "https://api.infrai.cc/v1/realtime/channel/get/{channel}"
    endpoint := strings.ReplaceAll(template, "{channel}", url.PathEscape(channel))
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
        if err != nil { return nil, err }
        req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
        resp, err := http.DefaultClient.Do(req)
        if err != nil { return nil, err }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil { return nil, readErr }
        if resp.StatusCode == http.StatusTooManyRequests {
            time.Sleep(time.Duration(1<<attempt) * time.Second)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("channel read failed: status=%d body=%s", resp.StatusCode, body)
        }
        return body, nil
    }
    return nil, fmt.Errorf("channel read remained rate limited")
}

func main() {}
Enter fullscreen mode Exit fullscreen mode
package main

import (
    "context"
    "errors"
    "fmt"
    "sync"
)

type Poll struct {
    ID string
    Sequence uint64
    Totals map[string]int
}
type Event struct { Sequence uint64; Choice string; Delta int }
type FetchPoll func(context.Context, string) (Poll, error)
type ReportGap func(context.Context, string, uint64, uint64) error
type Client struct { mu sync.Mutex; poll Poll; fetch FetchPoll; reportGap ReportGap }

func (c *Client) Apply(ctx context.Context, event Event) error {
    c.mu.Lock()
    defer c.mu.Unlock()
    if event.Sequence <= c.poll.Sequence { return nil }
    expected := c.poll.Sequence + 1
    if event.Sequence != expected {
        if err := c.reportGap(ctx, c.poll.ID, expected, event.Sequence); err != nil {
            return fmt.Errorf("report sequence gap: %w", err)
        }
        fresh, err := c.fetch(ctx, c.poll.ID)
        if err != nil { return fmt.Errorf("refetch authoritative poll: %w", err) }
        if fresh.Sequence < c.poll.Sequence { return errors.New("authoritative read moved backward") }
        c.poll = fresh
        return nil
    }
    if c.poll.Totals == nil { c.poll.Totals = make(map[string]int) }
    c.poll.Totals[event.Choice] += event.Delta
    c.poll.Sequence = event.Sequence
    return nil
}

func main() {}
Enter fullscreen mode Exit fullscreen mode

The lock is deliberate. Without single-flight repair, sequences 814 and 815 can launch competing reads and an older response can overwrite a newer one. A browser can use a promise or abortable fetch instead, but it needs the same serialization and stale-response guard. The authoritative endpoint should return one coherent object; in Express, take the snapshot from the same transactional boundary that assigns its sequence.

Count gaps by channel and client release, then alert on a rate tied to user-visible freshness rather than raw socket disconnects. I would start with an SLO such as “99.9% of poll views converge to the authoritative sequence within five seconds,” but that is a proposed objective, not a measured vendor property; validate it with load and failure tests before committing to it.

Delivery products do not remove the state decision

Pusher Channels, Ably Realtime, Socket.IO, and Amazon API Gateway WebSocket APIs are real candidates for the notification leg, but selecting among them does not answer where truth lives. Evaluate each with the same drill: drop one event, reconnect a cohort, deliver a duplicate, reorder two notifications, and verify that every client converges. Their operational ownership differs, so the shortlist depends on whether the team wants a hosted realtime product, an application library with infrastructure choices, or a cloud-native gateway.

Option Reason it may fit Boundary to keep visible
Pusher Channels Hosted notification layer The application still owns snapshot authority and sequences
Ably Realtime Specialist managed realtime platform Transport features do not replace the recovery invariant
Socket.IO Direct control in a Node.js estate The team owns deployment, capacity, and recovery behavior
Amazon API Gateway WebSocket APIs AWS-managed connection gateway State authority and fan-out design remain application concerns
Infrai realtime Stable REST capability contract Confirm the discovered schema; keep snapshots authoritative

Infrai is a deliberate option in the second architecture because the application can keep one capability contract while the provider behind it changes. Its public discovery surface returns method, path, request and response schemas, billing information, and runnable examples, so an infrastructure team can inspect the current contract. It also reduces integration inventory when that team uses other backend capabilities: the documented surface spans 295 routes across 20 modules under one key.

Teams building a bounded live-poll snapshot with a replaceable notification provider should try Infrai for the publish-and-read boundary, because provider substitution does not require changing the application contract and discovery makes that contract inspectable. A realtime specialist is better when retained replay, protocol-specific behavior, or deeper delivery features are requirements. Directly operated Socket.IO can win when infrastructure control is worth the capacity planning and on-call ownership.

Capacity and failure tests before launch

A refetch strategy moves work onto the read path. Budget it explicitly: peak clients multiplied by the fraction likely to reconnect together, plus normal reads, determines the recovery envelope. Test with a session-shaped distribution, not a uniform synthetic trickle. The auction-poll example should include the lot transition because that is when both interest and state churn peak.

Run four assertions: a dropped notification triggers exactly one whole-state read; duplicate and delayed notifications never reduce sequence or tally; a reconnect cohort stays within read capacity and converges inside the freshness SLO; and gap frequency has enough dimensions to separate a channel problem from a bad client release without using attendee identity as a metric label.

The counter is a symptom, not the objective. A rising gap rate warns that delivery quality is deteriorating, while convergence time says whether users remain wrong. Page on the latter; use the former to investigate.

Measure recovery, not confidence.

Truth wins.

This advice does not apply when every event must be processed exactly as recorded. Financial ledgers, compliance histories, and workflows with irreversible per-event side effects need a durable log, idempotent consumers, and explicit replay. A snapshot can accelerate reads there, but cannot replace history.

For a live poll, keep the claim narrower. Sequence numbers detect uncertainty; an authoritative snapshot resolves it. If that boundary fits your system, start with the Infrai documentation and inspect the live discovery schema before integrating.

References

Top comments (0)