Short answer: model moderation as an ordered state transition, drive it with a virtual clock, and assert the state visible to every participant rather than waiting for messages to arrive. For a B2B SaaS session that combines a shared stock trading watchlist with a live poll, this gives the test a precise pass condition: a removed participant cannot mutate either feature after the moderation event, while authorized viewers converge on the same ordered result.
Don't make wall-clock speed part of correctness. A 200 ms sleep can pass on a laptop and lose on a loaded runner; a sequence boundary expresses the guarantee the product actually needs.
How should realtime participant moderation tests cover a stock trading watchlist?
Treat each client as a projection of one authoritative event log. Commands such as adding NVDA to a watchlist or voting in a session poll enter through the same authorization boundary. Accepted commands receive an increasing sequence number. A moderation command that removes participant p-17 also receives a sequence number, and any later command from that participant is rejected before fan-out. The test should control command order directly, then ask every connected view to catch up to a named sequence.
This distinction matters because delivery and authorization answer different questions. A transport may duplicate, delay, or reorder observations while the server still enforces the right access rule. The projection therefore needs idempotency and an ordering policy; the command handler needs to check membership at the point where an action becomes authoritative. Testing only the WebSocket callback mixes those responsibilities and makes a failed assertion hard to interpret.
The key invariant is small: after participant_removed at sequence 42, no command authored by that participant may become an accepted event with a sequence greater than 42. Other participants can continue adding symbols and voting. Their views may briefly sit at different offsets, but once each reaches the test's barrier, they must agree.
A runnable state-machine test
The following Python example keeps transport concerns outside the domain model. It uses a tiny deterministic scheduler to deliver accepted events in an intentionally awkward order. There is no sleep, retry loop, or dependence on CPU speed.
from dataclasses import dataclass, field
from typing import Callable
@dataclass(frozen=True)
class Event:
sequence: int
kind: str
actor: str
value: str = ""
@dataclass
class Session:
members: set[str]
events: list[Event] = field(default_factory=list)
def record(self, kind: str, actor: str, value: str = "") -> Event:
if kind not in {"participant_removed", "participant_added"}:
if actor not in self.members:
raise PermissionError(f"participant {actor} is not active")
event = Event(len(self.events) + 1, kind, actor, value)
self.events.append(event)
if kind == "participant_removed":
self.members.discard(value)
elif kind == "participant_added":
self.members.add(value)
return event
@dataclass
class View:
watchlist: list[str] = field(default_factory=list)
votes: dict[str, str] = field(default_factory=dict)
applied: set[int] = field(default_factory=set)
next_sequence: int = 1
pending: dict[int, Event] = field(default_factory=dict)
def receive(self, event: Event) -> None:
if event.sequence in self.applied:
return
self.pending[event.sequence] = event
while self.next_sequence in self.pending:
current = self.pending.pop(self.next_sequence)
if current.kind == "watchlist_added":
self.watchlist.append(current.value)
elif current.kind == "poll_voted":
self.votes[current.actor] = current.value
self.applied.add(current.sequence)
self.next_sequence += 1
class Scheduler:
def __init__(self) -> None:
self.tasks: list[Callable[[], None]] = []
def later(self, task: Callable[[], None]) -> None:
self.tasks.append(task)
def run(self, order: list[int]) -> None:
for index in order:
self.tasks[index]()
def test_removed_participant_cannot_write_after_barrier() -> None:
session = Session({"moderator", "p-17", "p-23"})
first = session.record("watchlist_added", "p-17", "NVDA")
vote = session.record("poll_voted", "p-23", "hold")
removed = session.record("participant_removed", "moderator", "p-17")
last = session.record("watchlist_added", "p-23", "MSFT")
try:
session.record("watchlist_added", "p-17", "TSLA")
except PermissionError as error:
assert str(error) == "participant p-17 is not active"
else:
raise AssertionError("removed participant command was accepted")
views = [View(), View()]
scheduler = Scheduler()
accepted = [first, vote, removed, last]
for view in views:
for event in accepted:
scheduler.later(lambda view=view, event=event: view.receive(event))
# Each view gets duplicates and gaps, but both cross the same barrier.
scheduler.later(lambda: views[0].receive(first))
scheduler.later(lambda: views[1].receive(vote))
scheduler.run([2, 0, 1, 3, 8, 6, 5, 4, 9, 7])
assert all(view.next_sequence == last.sequence + 1 for view in views)
assert all(view.watchlist == ["NVDA", "MSFT"] for view in views)
assert all(view.votes == {"p-23": "hold"} for view in views)
test_removed_participant_cannot_write_after_barrier()
Notice what the example does not claim: it doesn't prove that a network delivers every frame exactly once. It proves that duplicate and out-of-order delivery cannot corrupt the projection, and that the moderation boundary prevents a later unauthorized write from entering the accepted log. Those are observable properties. Fast tests follow.
The scheduler order is ugly on purpose — indices 2, 0, 1, 3 give the first view a gap before its prefix is complete, while the second view receives a different order. If the projection applies event 3 immediately instead of buffering it, the final assertion exposes the defect without asking a runner to be fast or slow at just the right moment.
Delivery guarantees that deserve separate assertions
“Realtime” hides several contracts. Keep them separate in the eval harness so a transport failure does not look like an authorization failure. For this watchlist, the useful contracts are acceptance, ordered projection, duplicate handling, reconnect recovery, and revocation. The live poll shares the same contracts even though its merge rule differs: a watchlist may preserve ordered additions, while a poll normally keeps one current choice per active participant.
A compact test matrix makes the distinction visible:
| Case | Injected condition | Assertion at the barrier |
|---|---|---|
| Duplicate | Deliver sequence 41 twice | Symbol and vote appear once |
| Reorder | Deliver 43 before 42 | 43 waits; final projection is ordered |
| Disconnect | Stop a view at 40, then replay 41–44 | Reconnected view matches the log |
| Remove | Remove p-17 at 42, then submit its command |
Command never receives sequence 43 |
| Concurrent valid write | Let p-23 act after 42 |
Its accepted event reaches every view |
Exactly-once language is often too coarse for this design review. The application can tolerate at-least-once delivery if events have stable identities and projections are idempotent. It cannot tolerate an authorization decision made only at connection time, because an already connected participant could retain write access after removal. That is the catch: a simpler connection-level check is suitable only when membership cannot change during a session.
For peer-to-peer delivery, the topology changes but the questions do not. The W3C WebRTC Recommendation defines peer connections and data channels; it does not choose an application-level moderation authority or projection rule for a stock watchlist. A test still needs to identify which component accepts commands, how revocation reaches peers, and what barrier proves convergence. I'm not sure a single topology is best for every session size; queue depth, reconnect frequency, and moderation latency from production traces would resolve that choice.
From notebook assertions to a production eval harness
Start with the pure state-machine test because it runs on every change and produces a useful counterexample. Then run the same event traces against the deployed adapter. The adapter test should replace direct record calls with the system's public command interface and replace scheduler delivery with a subscriber that reports its last applied sequence. Keep the trace and assertions unchanged. This notebook-to-prod path stops the fast suite and integration suite from quietly testing different semantics.
Seeded property tests are a good next layer. Generate joins, removals, watchlist edits, votes, duplicates, gaps, and reconnects; save the seed whenever an invariant fails. Do not use random sleeps as the generator. Generate event order explicitly, because a replayable sequence such as [join, add, remove, duplicate, reconnect] is far more useful than “failed after 7.8 seconds.”
Prompt-cost awareness matters if an AI moderator proposes removals or labels abusive content. Put that model call before a human or policy decision, but keep the accepted moderation event deterministic and auditable. Record the model input version, decision category, and token usage alongside evaluation metadata; don't let generated prose become the authorization primitive. A replay test can reuse the recorded decision when it is exercising fan-out, while a separate model eval measures decision quality and cost. This separation keeps an expensive, variable inference call from making every delivery test variable too.
Observe offsets, not vibes. Useful signals include the accepted sequence, each subscriber's applied sequence, duplicate count, buffered-gap depth, reconnect replay count, and the time from an accepted removal to each view crossing that sequence. Percentile latency can be an operational objective, but a test should wait on the explicit barrier with a bounded deadline and print the missing sequence if the deadline expires. The deadline protects the suite; it is not the condition that makes the assertion true.
Where this approach does not fit
An ordered log plus per-view buffering costs memory and operational attention. It is not suitable when updates are disposable presence hints, cursor positions, or market-price ticks where only the newest value matters. For those streams, use a latest-value snapshot with monotonic versions and allow intermediate updates to disappear. Stick with the log for moderation, membership, watchlist mutations, and poll votes because losing one of those changes alters permission or durable user intent.
There is another boundary. A central acceptance point is a clean fit when the SaaS operator owns session policy. Fully decentralized collaboration needs a conflict-resolution and trust model that this example deliberately does not provide. Moving to peer delivery without naming that authority merely relocates the ambiguity.
Before deployment, run the deterministic trace in the fast suite, replay it through the actual subscriber adapter, and verify that reconnect begins after the last durable offset. In staging, pause one subscriber, remove a participant, allow another participant to add a symbol and vote, then resume the paused subscriber and wait for the named sequence barrier. Emit enough context to explain a failure in one log line: session, actor, command ID, accepted sequence, applied sequence, and moderation state. Finally, load-test fan-out independently from correctness tests; throughput pressure should reveal lag and queue growth, while the same invariants continue to decide whether the result is valid.
Further reading
The transport standard is useful background for the peer-delivery branch of this design.
References
- W3C, WebRTC Recommendation: https://www.w3.org/TR/webrtc/
Top comments (0)