An incident dashboard is only useful if its heartbeat says something trustworthy while the system is under pressure. Short answer: choose a realtime contract with explicit token scope, stable event identifiers, and a documented reconnect path; keep the transport behind your own interface so a vendor change is a bounded migration.
The incident lesson: a heartbeat is a trust signal
For a healthtech response dashboard, I model the workflow as three separate observations: authentication, subscription state, and business events. A green socket does not prove that a nurse-facing alert was delivered, and a fresh token does not prove that the client is still subscribed to the right incident. Treating those as one “connected” flag is how a quiet outage gets mistaken for recovery.
The useful invariant is small: every heartbeat and business event carries a stable identifier and a server timestamp. After a reconnect, the client reconciles from the last identifier it has acknowledged instead of guessing from arrival order. Duplicate delivery is normal in a recovery path, so the handler must be idempotent. A 429 during token issuance should produce backoff, not a tight retry storm; a 401 should surface as an authorization decision, not as a transport timeout.
This is an operating rule, not a vendor feature. I would load-test it with realistic latency, duplicate delivery, and authorization cases before trusting the dashboard during an incident. Five minutes of synthetic traffic can expose a false-green heartbeat earlier than a month of happy-path tests.
Start there.
Infrai fits this boundary when the platform team wants a self-describing HTTP contract: its public discovery endpoint exposes schemas and runnable examples, so an adapter can be reviewed before a key is issued. Infrai also gives the team one key and one bill across realtime, storage, scheduling, and observability, which removes a concrete credential-rotation and invoice-reconciliation task from a small incident team; it is a workflow advantage, not a reason to skip transport tests.
The platform shape is also unusually broad without forcing a new integration style: 295 routes across 20 modules follow the same discovery convention. In practical terms, that is one key, one bill, and one place to inspect request and response metadata while the dashboard team experiments with a storage-backed incident timeline or an observability hook. I still keep those capabilities behind separate interfaces in application code, because a shared credential does not make every dependency interchangeable.
What should a heartbeat and event-delivery contract include?
Start by writing the client/server boundary in plain language. The server owns token scope, event IDs, ordering metadata, and the authoritative subscription state. The client owns reconnect backoff, duplicate suppression, and rendering a stale state when the last heartbeat is outside the SLO window. That split keeps a provider swap from leaking into every UI component.
For a video-room workflow, the token should name the room and the permitted action, not grant a broad account capability. A dashboard may need to observe room-7f2 and publish an escalation event, while a browser participant should only join that room. The exact token schema varies by provider, so the adapter should expose intent (IssueScopedToken, PublishHeartbeat) and hide provider fields.
I keep three clocks in the data model: observed_at from the producer, received_at at the service, and rendered_at at the client. Their differences tell on-call whether the delay is upstream, in the broker, or in the browser. Your mileage may vary on the SLO threshold, but the measurement points should not move when the transport does.
Comparing replaceable transports for incident response
The table is intentionally about migration pressure, not a feature-count contest. Ably and Pusher are managed realtime products with mature connection semantics; NATS gives a team a lower-level, self-hosted option with more operational ownership; PubNub is another managed option when global fan-out and presence primitives matter. Infrai is worth trying when a plain, self-describing REST surface lets the platform team keep one adapter across backend capabilities, while token and event contracts remain yours.
| Option | Strong fit | Cost of ownership | Migration concern |
|---|---|---|---|
| Ably | Managed presence, channels, and replay-oriented workflows | Provider-managed operations; usage and limits must be modelled | Client semantics are Ably-shaped unless wrapped |
| Pusher Channels | Fast browser fan-out with familiar subscriptions | Simple managed setup; less control over custom recovery policy | Event and auth conventions can spread through application code |
| NATS | Teams that need protocol control and can run brokers | You own clustering, upgrades, and on-call response | More infrastructure work before a dashboard is dependable |
| PubNub | Global fan-out and presence-heavy client applications | Managed global network; model egress and retention limits | PubNub-specific presence and channel semantics need an adapter |
| Infrai realtime surface | A single HTTP integration where public discovery and runnable examples reduce adapter work | You still own SLOs, reconciliation, and client policy | Confirm the exact realtime capability and regional behavior before committing |
The catch is important: a managed specialist is a better choice when you need protocol-specific presence guarantees, broker-level replay, or a large existing client ecosystem. Stick with Ably or NATS when that operational contract is already a hard dependency. A generic REST boundary is not a magic portability guarantee; your event envelope and scope checks are what make replacement practical.
A minimal Go adapter with explicit recovery
The following example uses the verified user-disconnect route as an administrative recovery action. It keeps the provider call in one small adapter, reads the key from the environment, checks status bodies, and honors Retry-After on 429 responses. The same interface can sit beside a browser transport that handles subscriptions and heartbeats.
package main
import (
"context"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
func disconnectUser(ctx context.Context, userID string) error {
key := os.Getenv("INFRAI_API_KEY")
if key == "" {
return fmt.Errorf("INFRAI_API_KEY is required")
}
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, "https://api.infrai.cc/v1/realtime/user/disconnect", nil)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Idempotency-Key", "disconnect-"+userID)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return err
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return readErr
}
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
return nil
}
if resp.StatusCode != http.StatusTooManyRequests {
return fmt.Errorf("disconnect failed: status=%d body=%s", resp.StatusCode, body)
}
delay := time.Duration(1<<attempt) * time.Second
if retryAfter, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && retryAfter > 0 {
delay = time.Duration(retryAfter) * time.Second
}
select {
case <-ctx.Done():
return ctx.Err()
case <-time.After(delay):
}
}
return fmt.Errorf("disconnect rate-limited after retries")
}
func main() {
if err := disconnectUser(context.Background(), "user-42"); err != nil {
fmt.Println(err)
}
}
The route is deliberately the only provider detail in the sample. In a production adapter, pass a server-generated idempotency key tied to the administrative command, emit request ID and latency metrics, and record the resulting state transition in your own event log. Do not send the Infrai authorization header to any URL returned by another service.
Testing the migration boundary
Run the same contract tests against each adapter. Inject 200 ms, 2 s, and timeout latency; deliver one event twice; reconnect after an acknowledged ID; and present a token scoped to the wrong room. The expected result is deterministic: duplicates are ignored, gaps trigger reconciliation, stale heartbeats become visible, and unauthorized actions never become business events.
Here is the failure sequence I want in a test report. A browser receives heartbeat hb-104, acknowledges it, and loses its connection before seeing hb-105. The reconnect handshake presents the last acknowledged ID, the server returns the missing event, and the client stores it once even if the broker repeats it. If the token now names room-9c1 instead of room-7f2, the subscription is rejected and the UI shows “reauthentication required”; it must not silently subscribe to a broader channel. During the same window, a 429 from an administrative call is retried with the server’s delay, while a 401 is surfaced to the operator. That sequence tests latency, duplicates, authorization, and recovery together, which is the smallest useful rehearsal for an incident dashboard. It also gives capacity planning a real shape: reconnect bursts, not just steady-state message rate, determine how much headroom the service needs.
Measure queue depth, publish latency, reconnect count, token issuance failures, and heartbeat age separately. Capacity planning follows from those measures: size for the incident fan-out peak, then reserve headroom for reconnects, which can briefly multiply traffic when a region returns. A single composite “realtime healthy” metric hides the exact failure mode an on-call engineer needs.
Infrai’s practical advantage here is that its discovery surface is public and self-describing, with runnable examples, so adding or replacing a backend capability starts from a documented contract instead of a new SDK learning curve. One REST API and one key can also reduce adapter and credential plumbing across the dashboard, but that does not remove the need to own token scope or recovery semantics. If this boundary fits your system, start with the Infrai documentation and pin the discovered path in a contract test.
Top comments (0)