The page says in-app notification delivery has fallen behind across the realtime channel and polling fallback, so shoppers cannot enter a seller's live video room. The room itself is healthy; the missing step is the invitation that tells an eligible shopper to join and gives the client enough context to request a scoped room token. On-call sees a growing gap between notifications created and notifications acknowledged, while application request latency still looks ordinary.
Short answer: for a small SaaS, start with polling when a notification can arrive one polling interval late and the read load is acceptable; choose a realtime channel when delivery latency is part of the product contract, but keep the notification log authoritative and use the channel as a wake-up signal. The token must still be minted by a trusted server after authorization. A hybrid design usually has the cleanest failure semantics because reconnecting clients can recover by cursor instead of trusting an ephemeral push.
This is a control-plane problem before it is a transport problem. WebRTC covers realtime media and data between peers, but its specification leaves signaling to the application. The notification path that announces a room, the authorization decision, and the token-minting step therefore need their own observable contract.
What does the page actually tell us?
A page on "notification delivery failed" is too late and too vague. Work backward from the user-visible action. A useful chain is: room becomes joinable, an audience record becomes eligible, a notification is appended, a connected client is signaled or a polling client reads it, the client acknowledges its cursor, and the trusted backend evaluates current access before issuing a scoped token. Each transition needs an identifier and a timestamp. Otherwise the dashboard can show a healthy websocket fleet while one partition of the notification log is no longer advancing.
I've been paged by both missed jobs and duplicate deliveries. The reflex those incidents teach is boring but valuable: assume delivery is at least once, make the consumer idempotent, and measure age rather than raw event count. A duplicate room invitation is annoying; issuing broader credentials because a retry bypassed authorization is a security failure. The client may present a room identifier, but it doesn't get to choose token claims.
For the page, alert on the age of the oldest eligible notification that has not crossed the acknowledgement boundary, split by delivery mode. Queue depth alone is weak evidence because a busy but advancing queue may be fine, while a nearly empty queue containing one stuck high-value notification is not. Also record the age of the newest acknowledged cursor. That separates "nothing was created" from "created but not observed" without guessing from websocket connection counts. The first response is then mechanical: identify the oldest affected cursor, confirm that authorization input exists, compare created and acknowledged timestamps, and replay the read from the authoritative log. If the shopper reconnects with an older cursor, the server returns the same durable notification and the consumer acknowledges it idempotently; it does not invent a second invitation. Don't mint a replacement token from cached client state, because the delivery record proves only that an invitation existed, not that access still exists. Reload the current user-to-room grant, record the authorization decision separately, and issue credentials only after that check. Tokens should be short-lived and narrowly scoped according to the application's policy, but the exact lifetime depends on room duration, reconnect behavior, and the cost of reauthorization; I'm not sure there is one useful universal value.
Transport is not authority.
How should a small SaaS compare realtime channels and polling for notification delivery?
Compare the two options against latency, concurrent clients, request cost, operational ownership, and recovery semantics. Polling repeats reads even when nothing changes. Its approximate request rate is active clients divided by the polling interval, so halving the interval roughly doubles those reads. That model is easy to inspect, cache, rate-limit, and test with ordinary request tooling. It also puts a hard lower bound on freshness: an event created just after a poll normally waits nearly one full interval.
A realtime channel changes the bill of materials. Idle clients no longer need periodic application reads, but the service now owns connection admission, heartbeats, reconnect storms, fan-out pressure, backpressure, and cursor resumption. The relevant cost is not "one push versus many polls." It is the sum of open-connection capacity, transferred messages, state lookups, and engineering time needed to make recovery predictable. Your mileage may vary because traffic shape matters more than the label on the transport. A small SaaS with brief, sparse sessions can reasonably prefer polling even if a realtime architecture looks more sophisticated on paper.
The practical middle ground is a durable notification log plus a transient signal. When a channel says new data exists, the client reads notifications after its last acknowledged cursor. After disconnect, it performs the same read. If the channel is unavailable to that client, bounded polling uses that identical path. This keeps one source of truth and one authorization boundary; the transports differ only in how quickly they prompt a read.
| Decision pressure | Polling tends to fit | Realtime channel tends to fit |
|---|---|---|
| Delivery expectation | Delay up to the chosen interval is acceptable | Users expect an invitation while the room is actionable |
| Traffic shape | Few active clients or short sessions | Many connected clients with infrequent but urgent updates |
| Team ownership | The team wants ordinary request and cache operations | The team can own connection lifecycle and fan-out behavior |
| Recovery | Re-read after a cursor on every poll | Resume after a cursor on reconnect |
| Failure containment | Rate limits cap repeated reads | Backpressure and reconnect controls cap fan-out |
This table is a decision aid, not a benchmark. Put observed active-client counts, event frequency, transfer volume, and operator time into both models. Don't substitute a vendor's per-message headline for a workload model.
Instrument the transition, not just the transport
The earlier signal should fire when delivery progress stops, before shoppers report that the join prompt never appeared. That requires a stable notification ID, a per-recipient cursor, creation time, acknowledgement time, room ID, selected transport, and an authorization decision ID. Avoid logging the token itself. The token is a credential; observability needs its scope metadata and issuance outcome, not the secret.
A small Go state machine makes the invariant explicit. It accepts duplicate acknowledgements, rejects cursor regression, and leaves token issuance outside the client-controlled transition.
package delivery
import (
"errors"
"time"
)
type Notification struct {
ID string
RoomID string
UserID string
Cursor uint64
CreatedAt time.Time
}
type DeliveryState struct {
LastAcknowledged uint64
}
func (s *DeliveryState) Acknowledge(n Notification) error {
if n.Cursor < s.LastAcknowledged {
return errors.New("cursor regression")
}
if n.Cursor == s.LastAcknowledged {
return nil // Duplicate delivery is already accounted for.
}
s.LastAcknowledged = n.Cursor
return nil
}
The production write for the cursor must be conditional or transactional. Two consumers can observe the same notification; both should converge on the same acknowledged cursor. A separate trusted operation then reloads the user-to-room grant and mints a token limited to the permitted room and actions. This separation matters during retries — transport success does not prove authorization, and authorization performed when the notification was created may be stale when the shopper finally taps it.
Measure the handoff.
Instrument four durations: eligibility to append, append to signal, signal to read, and read to acknowledgement. These are application-defined measurements, not universal service targets. Histograms expose a slow tail that an average hides, while counters for reconnects, duplicate acknowledgements, cursor regressions, and authorization denials explain why that tail moved. Correlate by notification ID and decision ID, with bounded-cardinality dimensions for transport and outcome. Keep user and room identifiers in trace context or controlled logs rather than metric labels.
Then test the ugly sequence: create a notification, deliver the signal, disconnect before acknowledgement, reconnect with the old cursor, read the same notification, acknowledge it twice, and attempt a token request after access has been revoked. The expected result is one durable notification state, harmless duplicate processing, and no token after revocation.
Fast happy-path delivery proves very little.
Where does each design stop fitting?
Polling is not suitable when the polling interval itself would consume a meaningful share of the allowed join window, or when empty reads dominate the measured request load. Tightening the interval can postpone the decision, but it also increases repeated authorization, cache, and database work. Stick with polling when the product tolerates its bounded delay and the team benefits from its simpler operational surface.
A realtime channel is not suitable merely because the feature is called "realtime." If the team cannot test reconnect floods, enforce backpressure, resume from a durable cursor, and operate long-lived connections, the faster nominal path creates a larger failure surface. In that case, polling is the more honest contract.
The hybrid has a catch too: two wake-up mechanisms mean more test cases, even though they share one read path. It earns its keep only when graceful degradation and prompt normal delivery both matter. Otherwise, choose one transport and spend the saved complexity on idempotency, authorization, and observability.
For video rooms, keep media concerns separate. The WebRTC specification defines peer connections and related APIs, while signaling remains an application responsibility. A notification can tell the app that a room is available; it should not smuggle a reusable room credential through the delivery channel. The backend should evaluate client trust at the moment of token issuance, bind the result to the authorized room and actions, and let expiry limit exposure according to local policy.
False positives have an operating cost
The new progress alert will be noisy if it ignores traffic shape. During a quiet period there may be no acknowledgement movement because there is no work. During a burst, the oldest age can rise while the system continues draining. Gate the alert on eligible work being present, require the condition to persist for a deliberate window, and pair age with a progress signal. Tune that window from observed room invitation behavior and the response time your product promises, not from a copied threshold.
Too sensitive, and on-call learns to distrust the page. Too loose, and the first reliable detector is a shopper who never sees the room invitation.
That is the final decision rule: choose polling or a realtime channel based on the delay contract and measured workload, but page on end-to-end notification progress. Keep a durable cursor, treat delivery as repeatable, and authorize the scoped video-room token on the server at action time. The transport can change later. Those invariants should not.
Top comments (0)