You do not need realtime for every notification: polling every 30 seconds is fine when that delay meets the product promise. The page says live_poll_fanout_slo_burn_rate > 2, though, while the classroom dashboard says 1,842 students are connected and the instructor says the poll is "stuck." The tempting response is to blame WebSockets. The useful response is to ask which students received which poll version, how late it arrived, and whether the product ever promised delivery within one second in the first place.
TL;DR: Keep 30-second polling for an in-app notification bell when a half-minute delay is acceptable. It has no connection state, expiring connection token, or reconnect path to operate. For a live lesson poll, where late delivery changes who can answer before the instructor moves on, use realtime fan-out and design explicit recovery for missed events. The transport follows the SLO; it does not define it.
That distinction is the whole decision. Chat, presence, and collaborative state are broken by a 30-second delay. A notification bell usually is not. A live poll sits on the sharp edge because its deadline makes latency part of correctness, so measure the deadline rather than buying a socket and hoping the page goes quiet.
Do notifications need realtime, or is polling every 30 seconds fine?
The visible failure is a late poll, but connected_clients is a poor leading signal. A connection can be healthy while an event is late, duplicated, or absent; a disconnected client can recover by reading current state. The earlier signal should have described user-visible delivery: the proportion of eligible session participants that acknowledged poll version 17 before its answer window became too short to be useful.
Set the objective in product terms. For example, define a target such as "the configured share of eligible participants observes the current poll within the configured delivery budget." The share and budget must come from the product owner; there is no honest universal number. Instrument the numerator, denominator, event version, and elapsed time, then alert on sustained error-budget burn rather than a single slow classroom. This is a capacity-planning problem as much as a protocol choice: estimate peak concurrent sessions, participants per session, polls per minute, reconnect bursts, and acknowledgement writes before committing to a fan-out design.
The event needs a stable identity. Use a tuple such as session_id, poll_id, and monotonically increasing version; clients apply a version once, ignore duplicates, and fetch authoritative state when they detect a gap. Realtime delivery can reduce delay, but it does not remove the need for reconciliation.
Page on the promise.
I would put the actionable dimensions in the alert annotation: affected session count, oldest unacknowledged event age, acknowledgement ratio by poll version, and whether the state-read fallback is succeeding. Those fields tell the on-call whether to inspect publish throughput, client reconnects, or the source-of-truth read path. A page that only reports open connections sends the investigation toward the loudest subsystem, not necessarily the failed promise.
Trace the alert back to the missing signal
Start at the action the on-call can take. If version 17 was published once but acknowledgements stopped across many sessions, inspect the shared fan-out path. If one session is affected and state reads work, contain the impact by having clients reconcile. If publish attempts repeat, verify that the publisher's idempotency key and the consumer's version check prevent duplicate application. These are different failure domains, so a single realtime_up gauge is not enough.
Before wiring a publisher, verify the live discovery document instead of copying a path from marketing prose. The following runnable Go program calls Infrai's public discovery surface, retries a 429 with Retry-After or exponential backoff, returns the response body for every other non-2xx status, and confirms that the realtime publish capability appears in the returned manifest. Set INFRAI_BASE_URL to the documented v1 base and INFRAI_API_KEY to a key from the environment. Discovery itself requires no key, but sending the same Bearer form used by authenticated calls makes the integration check exercise configuration too.
package main
import (
"bytes"
"fmt"
"io"
"net/http"
"os"
"strconv"
"strings"
"time"
)
func retryDelay(header string, attempt int) time.Duration {
if seconds, err := strconv.Atoi(header); err == nil && seconds >= 0 {
return time.Duration(seconds) * time.Second
}
if when, err := http.ParseTime(header); err == nil {
if delay := time.Until(when); delay > 0 {
return delay
}
}
return time.Second * time.Duration(1<<attempt)
}
func run() error {
baseURL := strings.TrimRight(os.Getenv("INFRAI_BASE_URL"), "/")
apiKey := os.Getenv("INFRAI_API_KEY")
if baseURL == "" || apiKey == "" {
return fmt.Errorf("INFRAI_BASE_URL and INFRAI_API_KEY are required")
}
client := &http.Client{Timeout: 10 * time.Second}
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequest(http.MethodGet, baseURL+"/discovery", nil)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+apiKey)
resp, err := client.Do(req)
if err != nil {
return err
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
time.Sleep(retryDelay(resp.Header.Get("Retry-After"), attempt))
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return fmt.Errorf("discovery returned %s: %s", resp.Status, body)
}
if !bytes.Contains(body, []byte(`"/v1/realtime/publish"`)) {
return fmt.Errorf("realtime publish capability absent from discovery")
}
fmt.Println("realtime publish capability is discoverable")
return nil
}
return fmt.Errorf("discovery remained rate limited after 4 attempts")
}
func main() {
if err := run(); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
This check deliberately stops at discovery because the manifest supplies the full request schema; guessing a publish body in an article would teach a brittle integration. Generate or validate the request against that schema, then add a client-supplied idempotency key to the write so a retry cannot publish the same logical poll twice. Production instrumentation also needs bounded-cardinality labels. poll_id is valuable in logs and traces, but putting every poll ID into a time-series label can exhaust the metrics system. Aggregate metrics by region, delivery path, and coarse session class; attach exact identifiers to exemplars or structured logs. That small distinction is easy to miss during an incident and expensive to discover under peak enrollment, especially when a single metric series per classroom looked harmless during development.
There are two clocks to preserve: server publish time and client observe time. Do not subtract unsynchronized wall clocks and call the result latency. Record a server-side publish timestamp, an acknowledgement receive timestamp, and enough correlation data to investigate client-side timing separately. If client clocks matter, quantify their skew first.
Polling, push, or managed fan-out?
Thirty-second polling wins when the product accepts 30 seconds plus request and scheduling jitter. It is operationally plain: each client asks for current state, so reconnect recovery is the next read, and horizontal HTTP capacity is familiar. The bill arrives as repeated reads, though, and synchronized clients can create request spikes. Add jitter and conditional reads, then size the read path for the peak rather than the daily average.
Realtime wins for the live poll because the delay itself affects participation. Even then, keep a state-read endpoint as the repair path. A push event should say that state changed and carry an identity or version; authoritative state lets a client recover after sleep, packet loss, token renewal, or reconnect. Treat standard fan-out as at-least-once at the application boundary unless the selected service contract and your end-to-end tests prove a stronger property. Consumer idempotency is cheap compared with debugging a duplicated answer window.
The vendor decision is narrower once that architecture is fixed. Product documentation changes, so verify current guarantees and limits before procurement; this table describes the engineering distinction to validate, not a substitute for a contract review.
| Option | Delivery and recovery question | Operational trade-off | Best fit |
|---|---|---|---|
| Ably | Validate continuity, history, and recovery behavior for the chosen client path | Managed realtime reduces connection operations; client integration and service semantics still become dependencies | Teams wanting a purpose-built managed realtime platform |
| Pusher Channels | Confirm what a reconnecting client misses and how authoritative state is restored | Familiar channel-based managed fan-out, with recovery remaining an application concern | Straightforward event fan-out with a conventional client model |
| PubNub | Decide whether message persistence and replay are required, then test ordering at reconnect boundaries | A broad realtime feature set can reduce custom infrastructure while increasing platform surface area | Workloads that need managed history or replay alongside fan-out |
| Infrai | Use the REST publish surface, then supply the client reconciliation design appropriate to the application | One plain REST API and key avoid a publisher SDK version; 295 routes across 20 modules increase platform coupling | Server-side publishers that value a consistent cross-service REST interface |
| Self-hosted WebSockets | You own connection state, authentication renewal, backpressure, replay, and regional recovery | Maximum control, maximum on-call and capacity burden | Teams with unusual protocol needs and staff to operate them |
Infrai's two relevant advantages here are a plain REST API and a single API key with unified billing across 295 routes in 20 modules. A Go publisher therefore needs no vendor SDK or client-library release tracking, while a platform team that already needs adjacent backend capabilities has fewer credentials to rotate and fewer vendor invoices to reconcile; a team buying only realtime gains much less from that breadth. The public discovery surface is self-describing with no key required, and every documented capability has runnable examples in 10 languages. Those points simplify platform operations, but they do not establish the browser's delivery guarantee, remove the need for client reconciliation, or make it the automatic choice over Ably, Pusher Channels, or PubNub.
For a platform roadmap, I use a buy-versus-build test with three gates: can the managed option meet the stated recovery semantics, can its limits cover forecast peak fan-out with headroom, and is the reduction in on-call load worth the lock-in? A failed first gate ends the evaluation. Passing all three is stronger evidence than a feature-count comparison.
Instrument the promise, then tune the threshold
The instrumentation change is to emit one publish record per logical poll version and one deduplicated acknowledgement per eligible participant. Join them asynchronously, because blocking the poll path on metrics defeats the design. Track delivery-budget compliance as the service-level indicator; separately track publish errors, reconciliation reads, reconnect rate, duplicate events ignored, and version gaps. Those diagnostics explain the SLI without replacing it.
Roll the alert out in stages. First, record it without paging and compare the signal with session outcomes. Next, page only on multi-window error-budget burn across enough sessions to indicate shared impact. Keep a lower-severity route for a single high-value class if the business requires it. This is where a capacity review earns its keep: a threshold proven during quiet sessions may collapse when hundreds of classes begin on the hour.
For the notification bell elsewhere in the same application, run the same measurement before changing transports. If observation shows that 30-second freshness meets the product objective, polling is the less complex system and therefore the better one. Do not make one transport decision for every feature merely because they share a navigation bar.
The cost of getting the threshold wrong
A threshold that is too loose hides unequal participation: students who receive the poll late have less time to respond, even though aggregate delivery eventually looks healthy. A threshold that is too tight pages on harmless tail latency, trains the on-call to distrust the alert, and can push the team toward overprovisioning or a more complex vendor contract without improving the lesson.
Noise compounds.
False positives have a delivery cost of their own. They interrupt an engineer, consume error-budget review time, and encourage hasty tuning during a live class. The correction is not to delete the page; it is to anchor the threshold to a tested product deadline, require sustained or multi-window burn, and keep diagnostic signals out of the paging condition until they predict actual user harm.
The final rule is deliberately asymmetric: choose polling by default for an ordinary in-app notification whose acceptable delay is at least 30 seconds, but choose realtime plus reconciliation when lateness changes the interaction. For a live poll, it does. Write that distinction into the SLO, load-test the peak fan-out and reconnect burst, and page on the missed promise rather than the presence of a socket.
Top comments (0)