DEV Community

SyltharWave2946
SyltharWave2946

Posted on

A Practical FastAPI Guide to High Availability WebRTC Video Consultation Rooms

#webrtc #highavailability #video

Short answer: A high-availability video consultation room should treat media, signaling, and durable room history as three different systems: let WebRTC carry live media, make signaling replaceable, and keep a short replayable event log so a reconnecting participant can recover without pretending the network delivered every message exactly once.

The bill is usually a traffic equation before it is a database equation. A 30-minute stream averaging 2 Mbit/s contains 450 MB of payload before protocol overhead: 2,000,000 * 1,800 / 8. Direct peer-to-peer media avoids server fan-out when connectivity permits it; relayed or server-routed media makes transferred bytes, fan-out count, and retention the terms to watch. Measure those terms separately. A room record with ten kilobytes of events is rarely the interesting cost beside hundreds of megabytes of media, although local traffic patterns and provider contracts can reverse that conclusion.

That distinction also gives the simplest useful design: don't persist media just to make reconnect work. Persist the control-plane facts needed to rebuild the room, expire them deliberately, and renegotiate a fresh media path.

Start there.

What actually makes a room highly available?

Availability here means a user can reach another signaling instance, prove which room session they belong to, learn what changed while they were absent, and establish media again. It does not mean a TCP, WebSocket, ICE, DTLS, or SRTP session migrates intact between processes. The W3C WebRTC API exposes peer-connection and ICE state, including ICE restart behavior, but the application still owns its signaling transport and message contract.

For a logistics video consultation between a remote driver and an operations specialist, use a stable room_id, a new connection_id on every socket, and a participant identity that survives sockets. Keep membership and authorization in a shared transactional store. Put immutable, monotonically sequenced room events in a bounded log. Any healthy signaling process can then authenticate a reconnect, read the participant's last acknowledged sequence, replay the missing interval, and continue live delivery. Presence is softer state: derive it from leases and heartbeats instead of treating a dropped socket as an authoritative departure.

The boundary matters. Media routing may be peer-to-peer, relayed through TURN, or handled by a selective forwarding unit, while the room log remains the source for control events such as participant_joined, consultation_started, and consultation_ended. Don't put SDP offers or ICE candidates into a long-lived audit archive by default; they are transient negotiation material, may contain network information, and become useless after their negotiation window.

How should high availability WebRTC rooms handle video consultation reconnects?

Make reconnect an explicit protocol, not a side effect of reopening a socket. The client sends the room session identifier and its last contiguous acknowledged sequence. The server answers with one of three outcomes: replay is available, a snapshot is required because the replay window has expired, or the room has ended. Those outcomes should have stable application-level codes; for example, REPLAY, SNAPSHOT_REQUIRED, and ROOM_CLOSED are clearer than asking clients to infer state from a generic disconnect.

Consider the awkward boundary in detail. The driver's browser has applied room event 416, sends its acknowledgement, and loses connectivity before it can know whether the signaling process committed that cursor; meanwhile, the operations specialist ends and reopens the consultation, producing events 417 and 418. On reconnect, the driver reports 416. The new signaling process reads the durable log and sends both later events even if another process had already attempted delivery. If event 418 arrives first because concurrent work completed out of order, the reducer must hold it instead of advancing the cursor, request the interval beginning at 417, apply 417, and only then apply 418. If the retention boundary has already removed 417, the server returns SNAPSHOT_REQUIRED and the client replaces its derived view. This is why a reconnect token alone is insufficient: identity answers who may enter, while the acknowledged sequence and room epoch answer which history is safe to apply.

Duplicates are normal.

Order is local.

At-least-once fan-out plus idempotent application is the practical contract. A process can publish an event and lose the acknowledgement, or a client can apply an event and disconnect before advancing its cursor; retrying then produces a duplicate. Exactly-once delivery across the broker, socket, client memory, and user-visible effect would require one atomic commit spanning systems that do not share a transaction. Sequence every event within a room, give every command an idempotency key, and make the reducer reject an already applied sequence.

from dataclasses import dataclass, field
from typing import Any


@dataclass
class RoomView:
    room_id: str
    last_sequence: int = 0
    participants: set[str] = field(default_factory=set)
    closed: bool = False


def apply_event(view: RoomView, event: dict[str, Any]) -> RoomView:
    sequence = int(event["sequence"])
    if sequence <= view.last_sequence:
        return view
    if sequence != view.last_sequence + 1:
        raise ValueError("event gap: request replay or a fresh snapshot")

    if event["type"] == "participant_joined":
        view.participants.add(str(event["participant_id"]))
    elif event["type"] == "participant_left":
        view.participants.discard(str(event["participant_id"]))
    elif event["type"] == "consultation_ended":
        view.closed = True
    else:
        raise ValueError(f"unknown room event: {event['type']}")

    view.last_sequence = sequence
    return view
Enter fullscreen mode Exit fullscreen mode

There is a trap in the gap check. Delivery can be concurrent, so event 42 may reach a browser before event 41 even though the durable log is correctly ordered. Buffer a small out-of-order window, request replay after a short deadline, and replace the local view with a signed or authenticated snapshot if the requested sequence has aged out. The exact window and deadline depend on measured reconnect duration and event rate; I'm not sure a fixed value is defensible without those two distributions.

Delivery guarantees are a product decision

The strongest guarantee belongs to the effect, not the transport label. A repeated typing indicator can be ignored. A repeated consultation_ended command must be idempotent. A missed authorization revocation must force a snapshot or terminate access rather than wait for another event. Classify messages before choosing retention and acknowledgement rules.

Message class Delivery target Durable retention Duplicate handling Failure response
ICE candidate or SDP negotiation Best effort within the current negotiation Short expiry Scope to a negotiation generation Restart negotiation
Presence or typing state Latest value Lease or none Replace older value Recompute from current leases
Room lifecycle event At least once, ordered per room Bounded event log Ignore an applied sequence Replay, then snapshot
Authorization change At least once with immediate enforcement Audit policy dependent Idempotency key Deny until state is current

This is also where multi-region designs become uncomfortable. A single writer per room gives a clean order but introduces ownership transfer during regional failure. Active writers in multiple regions improve write locality but require a conflict rule, and wall-clock timestamps are not an ordering rule. For two-party consultations, assigning each room to one region and failing over through an epoch-fenced lease is often easier to reason about: every accepted event carries the current room epoch, and an old owner cannot continue writing after a newer epoch is committed.

Don't hide the choice behind “eventual consistency.” State the invariant. No two participants should observe different consultation-ending decisions; presence may lag; media quality can degrade without rewriting room history. Those are testable promises.

Name the failure.

Prove recovery before tuning cost

Test the protocol at the boundaries that usually get skipped: disconnect immediately before an acknowledgement, disconnect immediately after one, deliver duplicate events, reverse two adjacent events, expire the replay window, revoke access during a reconnect, and move room ownership while an old process is still alive. A load test that only opens stable rooms says little about availability. Fault injection should verify both the final client view and the durable cursor, because a screen that looks right once can still store the wrong recovery point.

For observability, record room epoch, event sequence, connection generation, replay count, replay age, snapshot count, ICE state transitions, and time from reconnect start to usable media. Avoid putting SDP bodies, candidate addresses, tokens, or consultation content in ordinary logs. A trace should answer “why did this participant request sequence 417 twice?” without becoming a second, poorly governed copy of sensitive session data.

Then tune the dominant term. Suppose a room emits 30 durable control events averaging 700 bytes; the raw event payload is about 21 KB, before indexes and storage overhead, versus 450 MB for one 30-minute 2 Mbit/s media stream before overhead. This illustrative calculation doesn't predict a production bill, but it tells you where measurement effort belongs. Track relay ratio, routed media egress, average bitrate, participants per room, retained recording bytes, event-log growth, and cross-region replication separately rather than reducing them to “cost per room.”

The material change is retention policy. Keep immutable control events only through the maximum supported reconnect window, compact older events into a room snapshot, and retain audit records or recordings under their own legal and operational schedules. Do not use one indefinite retention setting for all three. Compaction reduces storage, index size, replay reads, and replicated bytes; it does not reduce live media transfer.

The deliberate limit

This design is not suitable when every transient signal must remain available for forensic reconstruction, when a consultation must continue with no perceptible media interruption during regional loss, or when regulations require full recording retention. In those cases, keep a longer append-only audit stream, use redundant media paths or a media topology designed for regional failover, and budget for recording durability independently. The catch is greater storage, replication traffic, operational complexity, and a larger body of sensitive data to govern.

For the bounded design, stop keeping expired negotiation messages, superseded presence updates, and per-event history after compaction has produced a verified snapshot. If an incident appears after that boundary, operators can establish the final room state and inspect metrics, but they cannot reconstruct every candidate exchange or every intermediate presence transition. That loss is intentional. Write it into the service objective and the incident-response playbook before deleting the data.

References

Further reading

Top comments (0)