DEV Community

MirageB18
MirageB18

Posted on

Team Presence Sidebar State Machines (3 Clocks for Trustworthy Realtime Updates)

#realtime #webhooks #architecture

Designing reliable webhook-to-realtime bridges starts with reconnect recovery: a realtime connection does not prove that a team presence sidebar is current.

Short answer: Put an ordered, replayable bridge between webhook ingestion and the realtime channel; make the browser resume from a cursor, and derive each sidebar row from versioned events plus explicit freshness deadlines. A socket alone cannot tell a game studio whether an on-call engineer is present or whether the test device beside that engineer merely stopped reporting.

The simple design is tempting: receive a webhook, broadcast its JSON, paint the newest icon. It also confuses transport activity with truth. Duplicate delivery, event reordering, a sleeping browser tab, and a reconnect during deployment can each leave a convincing but stale green dot. The better evaluation constraint is blunt: after any disconnect, the client must reach the same state it would have reached if it had stayed connected.

Why connection state is not presence state

There are three different claims hiding behind one green dot. The browser may be connected to the bridge. The bridge may have recently heard from the presence source. The person or device may still be inside the application's chosen presence window. Those claims expire at different times, so one Boolean cannot represent them honestly.

For a gaming live-ops team, this distinction is practical. Imagine a sidebar row for member ops-17, paired with test device kit-204. The bridge receives member version 841 saying online, then the network drops before version 842 saying away reaches one browser. When that browser reconnects, another online broadcast does not repair the missed transition; it only repeats a state that may already be obsolete. The UI needs either the missing event or a snapshot whose version is at least 842.

Webhooks also do not create ordering by themselves. HTTP defines request and response semantics, while the application's delivery contract must define identifiers, retries, ordering scope, and authentication. CloudEvents provides a common event envelope, including an event id, source, type, and time, but its specification does not turn an arbitrary transport into exactly-once delivery. Treat event identity as the deduplication key and treat a source-controlled version as the ordering key. If the source supplies no monotonic version, an ingestion sequence can order arrival at the bridge, but it cannot prove which of two source changes happened first.

That boundary matters.

How should a webhook bridge update a realtime team presence sidebar?

The bridge should acknowledge ingestion only after the event is durably recorded, then publish the record with a monotonically increasing bridge cursor. On first load, the client receives a snapshot and its high-water cursor. On reconnect, it asks for records after its last applied cursor. If that cursor has fallen outside retention, the server returns a fresh snapshot rather than pretending a partial replay is complete.

Snapshot creation and replay need one shared boundary. A useful sequence is:

  1. Read the current high-water cursor H.
  2. Build a snapshot that includes all accepted source versions through H.
  3. Send that snapshot tagged with H.
  4. Stream records whose bridge cursor is greater than H.

Without that boundary, an event can land between the snapshot query and the subscription, producing a quiet gap that no reconnect logic notices. The bridge can implement the boundary with a transaction, an append log plus materialized view, or another mechanism that gives the same consistency property. The storage choice is secondary; the observable contract is not.

For the browser channel, WebSocket supplies a two-way connection, Server-Sent Events supplies a one-way event stream with browser reconnection behavior, and WebRTC can establish peer connections and data channels. None determines presence semantics. Choose the transport after deciding whether the client must send messages on the same channel, what intermediaries permit, and how resume tokens are carried. Don't let a transport demo choose the data model.

Model three clocks, then reduce events deterministically

Use separate timestamps for occurrence, observation, and expiry:

  • occurredAt is when the source says the change happened.
  • receivedAt is when the bridge durably accepted it.
  • expiresAt is the application deadline after which the claim becomes stale unless renewed.

The ordering decision should use a monotonic version, not wall-clock time. Machine clocks drift, timestamps can have coarse precision, and delayed delivery can make an older event arrive last. Timestamps remain useful for debugging and freshness, but version wins the state transition.

Here is the focused part of the client reducer. It deliberately ignores a duplicate or older source version, advances the replay cursor independently, and turns an expired claim into unknown instead of guessing offline.

type Presence = "online" | "away" | "offline" | "unknown";

type PresenceEvent = {
  eventId: string;
  memberId: string;
  deviceId?: string;
  sourceVersion: number;
  bridgeCursor: number;
  state: Exclude<Presence, "unknown">;
  occurredAt: string;
  receivedAt: string;
  expiresAt: string;
};

type SidebarRow = {
  memberId: string;
  deviceId?: string;
  sourceVersion: number;
  lastCursor: number;
  state: Presence;
  expiresAt: string;
};

function applyPresenceEvent(
  current: SidebarRow | undefined,
  event: PresenceEvent,
): SidebarRow {
  if (current && event.sourceVersion <= current.sourceVersion) {
    return {
      ...current,
      lastCursor: Math.max(current.lastCursor, event.bridgeCursor),
    };
  }

  return {
    memberId: event.memberId,
    deviceId: event.deviceId,
    sourceVersion: event.sourceVersion,
    lastCursor: event.bridgeCursor,
    state: event.state,
    expiresAt: event.expiresAt,
  };
}

function visibleState(row: SidebarRow, nowMs: number): Presence {
  return nowMs >= Date.parse(row.expiresAt) ? "unknown" : row.state;
}
Enter fullscreen mode Exit fullscreen mode

Advancing lastCursor for an obsolete member update is subtle but necessary. The bridge cursor describes progress through the shared log, while sourceVersion describes progress for one member. Conflating them can make the client request an already processed range forever, or accept a late member event because its global cursor is larger.

Keep deduplication server-side too. A bounded table keyed by event ID prevents a webhook retry from creating another log entry, while a uniqueness constraint makes concurrent duplicate deliveries safe. Return success for an event already accepted; the producer is retrying an outcome, not asking for a second state change. Authentication should cover the raw request body, and replay defense should include a signed timestamp with a documented acceptance window. The exact signature format belongs to the webhook contract.

Reconnect and backfill are one protocol

The browser should persist its last fully applied cursor, not merely the last cursor received by the socket callback. Applying a batch and saving its cursor must behave as one local operation. Otherwise a crash between those steps either repeats harmless idempotent work or, in the worse ordering, skips work that never reached the visible store.

Keep it boring.

On reconnect, the server has two valid responses: replay everything after the cursor, or declare the cursor too old and replace state with a snapshot. 410 Gone is a reasonable HTTP status for an expired backfill cursor when the resume request uses HTTP, provided the response contract tells the client to fetch or accept a snapshot. Do not silently start at the oldest retained event. Silence turns a known gap into incorrect presence.

Backfill also needs flow control. A live-ops dashboard may receive a burst when a lab rack wakes, yet rendering 600 intermediate device changes is wasted work if only the latest state per device will be visible. Preserve the durable event log and its cursors, but allow the delivery layer to coalesce updates by entity within a batch. The client must still learn the highest covered cursor. This reduces rendering pressure without changing recovery truth.

The browser's Page Visibility state is another input, not a presence signal. A hidden tab can defer work, and the user can return after the freshness deadline. Recompute expiry when the document becomes visible, then reconnect and backfill. A locally ticking countdown may improve display, but it cannot substitute for synchronization.

What should you measure before copying this design?

Test the invariants with generated sequences rather than only clicking through a happy path. Deliver versions 841, 843, 842, repeat 843, disconnect before 844, then reconnect from the last committed cursor. The final row must match a clean ordered replay. Add a cursor older than retention and verify that the client replaces its full snapshot. Also test two events with the same event ID arriving concurrently, a tab resuming after expiry, and a slow consumer crossing the coalescing boundary.

Operationally, measure webhook acceptance latency, duplicate rate, out-of-order rate by source version, publish lag, reconnect count, replay batch size, expired-cursor count, and time from reconnect to a current sidebar. Split transport health from semantic freshness: a dashboard can have a connected socket while its newest durable cursor is minutes behind. Alert on the latter.

The catch is storage and protocol complexity. A durable log, snapshot boundary, retention policy, and idempotency index cost more to operate than periodic reads. This design is not suitable when presence is low-value, updates are rare, and a refresh every 30 seconds meets the product requirement; stick with polling plus explicit last updated text in that case. Peer-to-peer WebRTC is also a poor default for an authoritative team roster because peers joining late still need a trusted state and recovery path, though it can fit direct device-to-device data where the application already has signaling and peer lifecycle machinery.

I'm not sure there is a universal retention window. Your mileage may vary because the right duration depends on observed offline time, event volume, recovery objectives, and storage limits. Measure reconnect age percentiles first, choose a window that covers the product's target, and make snapshot fallback a normal tested path rather than an emergency branch.

The decision rule is compact: use a replayable bridge when missing one transition can mislead an operator; use polling when bounded staleness is acceptable. In both cases, display unknown when evidence expires. An honest gray dot beats a fictional green one.

References

Top comments (0)