Short answer: replay connection and delivery metadata, not recorded voice, and stop replay at the point where a player could mistake an old event for the current lobby state. In a gaming voice lobby, this boundary is more important than collecting every possible metric: a reconnecting client needs a trustworthy state transition, while an operator needs enough evidence to explain why audio or membership lagged.
The WebRTC Recommendation defines the browser-side statistics surface for peer connections, transports, candidate pairs, codecs, inbound and outbound RTP streams, and related objects. Those stats are snapshots, though. They are not an event log and they do not promise that a later read can reconstruct every earlier moment. That distinction drives the storage design here.
What should offline replay reveal about observability signals in a gaming voice lobby?
Start with two timelines. The live timeline is the lobby as players experience it: joined, muted, speaking, disconnected, and joined again. The diagnostic timeline is what the service can prove later: signaling messages, membership changes, selected candidate pair, packets received, jitter, and the reason a peer connection ended. They overlap, but they are not interchangeable.
For each timeline, define a replay horizon and a trust level. A membership event can be replayed as an authoritative fact if it was committed by the lobby service. A client-side audioLevel sample is a measurement with a clock and a reporting interval; it is useful evidence, not a command to restore state. Treating both as durable truth is how a reconnect UI ends up showing a player as speaking after they have already left.
I use a small envelope for every stored signal. It carries a monotonic sequence within the lobby, a server receive time, the producer clock when available, and a schema version. The sequence gives a deterministic order for state changes. The two clocks let an investigator separate network delay from a bad client clock. The version prevents a decoder from silently assigning new meaning to old fields.
Keep it boring.
from dataclasses import dataclass
from typing import Any
@dataclass(frozen=True)
class Signal:
lobby_id: str
seq: int
kind: str
received_at_ms: int
producer_at_ms: int | None
schema: int
payload: dict[str, Any]
def replayable(signal: Signal) -> bool:
"""Only committed state transitions are safe to hydrate into a client."""
return signal.kind in {"member_joined", "member_left", "member_muted"}
The replayable decision is deliberately narrower than the set of signals retained for diagnosis. A reconnect can consume the committed membership prefix, then start a fresh WebRTC negotiation. It should not blindly replay old RTP counters or emit historical speaking indicators as if they were live. WebRTC statistics such as packets lost, jitter, and round-trip time help explain the old connection; they do not recreate it.
Which signals belong in storage, and which belong only in live paths?
The useful split is semantic, not merely based on payload size. Store enough to answer “what did the lobby believe?” and “what did the transport report?” with separate retention and access controls.
| Signal family | Replay to reconnecting client | Keep for operations | Main boundary |
|---|---|---|---|
| Membership and mute state | Yes, through the last committed sequence | Yes | Must be ordered and idempotent |
| Signaling offers, answers, candidates | No; issue a new negotiation | Short retention | Old candidates can be stale |
WebRTC inbound-rtp / outbound-rtp counters |
No | Yes, sampled | Counters describe a past transport |
| Candidate-pair and transport stats | No | Yes, sampled | A new pair may be selected after reconnect |
| Speaking or level indications | Only if explicitly marked historical | Yes, coarse and bounded | A past level is not current presence |
| Raw audio | No by default | Rarely, with explicit consent | Privacy and retention cost dominate |
The table encodes a hard rule: replay is for rebuilding state, while observability is for explaining behavior. A storage layer can keep both, but the consumer must not confuse their contracts. I would rather lose a noisy level sample than let a stale sample mutate a player's visible status.
There is also a privacy boundary. Voice payloads and detailed per-player traces can identify people or reveal play patterns. If an incident can be explained with counters and state changes, retaining audio is not justified. Your mileage may vary for regulated environments, but that uncertainty belongs in a documented retention decision, not in an implicit default.
How do failure modes change the replay boundary?
Most reconnect bugs are ordering bugs wearing a network costume. Consider a client that receives member_left at sequence 108, loses connectivity, and later receives a snapshot ending at 106. If the client applies both without a sequence check, the departed member reappears. The fix is not a larger replay buffer; it is a monotonic cursor and an explicit rule for gaps.
I model the rules this way:
def apply_membership(state: dict[str, str], signal: Signal, last_seq: int) -> int:
if signal.seq <= last_seq:
return last_seq # duplicate delivery is harmless
if signal.seq != last_seq + 1:
raise ValueError("membership gap requires a fresh snapshot")
member_id = str(signal.payload["member_id"])
if signal.kind == "member_joined":
state[member_id] = "joined"
elif signal.kind == "member_left":
state.pop(member_id, None)
elif signal.kind == "member_muted":
state[member_id] = "muted"
return signal.seq
This is an at-least-once consumer: duplicates are expected, and a gap triggers resynchronization. Exactly-once delivery is not required to make the UI correct; idempotent application and an authoritative snapshot are enough. A late transport report can remain in the diagnostic stream with its original receive time, even when the membership stream has moved on.
Other failure modes deserve distinct labels. A selected candidate pair changing can mean a network path changed, not that the service dropped a player. A rising packetsLost count can indicate congestion, a radio handoff, or a remote sender stopping; it is evidence to correlate with timestamps, not a single-cause alert. A missing stats sample is a collection gap, not proof of silence. Naming those limits keeps incident reviews honest.
What trade-offs should an edtech-style lobby team make at fan-out?
Even in a gaming lobby, fan-out is a delivery contract. A small room may use a central signaling service and peer connections; a larger room may route media through an SFU. The replay boundary stays the same: membership facts are durable, negotiation artifacts expire, and media observations are sampled.
Choose the smallest contract that matches the room's risk:
- For a six-player party, retain ordered membership events and one-minute transport summaries. Reconnect from the last acknowledged sequence.
- For a tournament room, retain shorter-interval summaries around join, leave, and renegotiation, because a disputed disconnect needs a tighter timeline.
- For moderation-sensitive sessions, add an explicit consent and retention workflow before considering any audio capture. Do not smuggle it into “debug logging.”
The catch is operational complexity. A design that stores every browser stats object at high frequency will create a lot of cardinality and clock-normalization work, while still failing to prove what the user saw. It is not suitable when the product needs legal-grade recording or deterministic media reconstruction; use a purpose-built recording and audit system then. Stick with a lean event log when the goal is reconnect correctness and post-incident explanation.
Rolling this out without turning replay into a second realtime system
Ship the schema and cursor before changing the reconnect UI. During a shadow period, write envelopes and compare the derived membership state with the current session state. Measure sequence gaps, duplicate rates, snapshot frequency, and the delay between producer and server clocks. Set alerts on those measurements, not on raw event volume.
At rollout, gate hydration behind a schema version and reject a snapshot whose sequence is older than the client's acknowledged cursor. Keep a bounded dead-letter path for malformed envelopes; it should preserve the original identifier and reason, then move on. Never retry an unbounded queue in the request path.
The final test is adversarial: kill the network during a mute, reorder two membership events, change the candidate pair, and reconnect after the retention horizon. The expected result is a fresh negotiation plus a correct membership snapshot, with diagnostic evidence that explains the interruption. If a stale speaking indicator survives that test, the replay boundary is still too wide.
Top comments (0)