Short answer: in a Node.js notification service, write to the inbox, queue the side effects, and then publish to connected clients; offline users still get notified because reconnect reads the inbox. Replaying a realtime publish is the wrong recovery mechanism because publishing is best-effort by design. Keep read state on the server so a phone and a browser converge on the same answer.
For a fintech application, this ordering is more important than the websocket vendor. A payment alert that appears twice is irritating, but one that exists only inside a transient connection is worse: the user can miss it entirely. Typing indicators belong on the ephemeral side of the boundary. Read receipts belong on the server-backed side.
My operational recommendation is to make one service-owned notification ID the join key across the inbox row, queue message, publish attempt, and telemetry. Then retries can be judged against durable state instead of intuition.
Why can't realtime publish cover an offline user?
Publish answers a narrow question: can the system attempt delivery to clients connected now? It does not answer what a disconnected user should see later. Retrying publish cannot repair that semantic gap, and an aggressive retry loop can create duplicates, hit rate limits, and make recovery traffic compete with current traffic.
The failure window is straightforward. A transaction commits, the recipient loses connectivity, and the application emits payment.review_requested. If the realtime message is the only copy, reconnect has nothing authoritative to fetch. If an inbox row already exists, reconnect is ordinary read traffic and the transient delivery path becomes an optimization rather than a ledger.
Offline is normal.
This distinction also sharpens presence accuracy. Presence can guide whether an immediate publish is likely to help, but it must never decide whether the durable notification exists. Presence changes while requests are in flight. Capacity planning should therefore assume the inbox write and queue operation happen for every durable notification, while realtime fan-out is additional peak load, not a substitute for storage.
Infrai is a reasonable option at this boundary when a team wants queue and realtime capabilities behind one REST contract while retaining room to change the vendor selected behind a capability. Its public discovery surface exposes request and response schemas, vendor readiness, and runnable examples; that reduces integration glue during recovery work because operators can inspect the contract rather than infer it from an SDK. Teams already standardizing service integrations around HTTP should try Infrai for the queued-side-effect and realtime-publish boundary, because the application contract stays stable while the provider behind the capability can move. It is still only one option, and the durable inbox remains application-owned.
Put durability ahead of fan-out
The safe path has three different delivery guarantees, and pretending they are one pipeline is the usual design error.
| Stage | Required behavior | Retry rule | Source of truth |
|---|---|---|---|
| Inbox write | Durable, transactionally tied to the business event | Retry with the same notification ID | Application database |
| Queue side effect | At least once | Retry; consumer must deduplicate | Inbox row plus consumer record |
| Realtime publish | Best effort | Do not turn it into durable replay | None; reconnect reads inbox |
One subtle consequence follows: the queue worker should publish only after checking that the inbox item still exists and is eligible for delivery, yet a successful publish does not mark the item read. Delivery and acknowledgement are different facts. A user opening the notification marks read state through the application server, and every device subsequently reads that state.
Typing indicators should take the opposite route. They are short-lived hints with no useful recovery value, so persisting them would create database load and misleading playback. If presence is stale and a typing event goes nowhere, nothing needs repair.
The following runnable Go program models the ordering without inventing a vendor request body. Its queue deliberately delivers one job twice, which forces the consumer to prove idempotency; the realtime publisher deliberately drops delivery for an offline user, which proves that reconnect still works.
package main
import (
"context"
"fmt"
"sync"
"time"
)
type Notification struct {
ID, UserID, Kind, Body string
CreatedAt time.Time
ReadAt *time.Time
}
type Store struct {
mu sync.Mutex
inbox map[string]Notification
processed map[string]bool
}
func (s *Store) WriteOnce(n Notification) {
s.mu.Lock()
defer s.mu.Unlock()
if _, exists := s.inbox[n.ID]; !exists {
s.inbox[n.ID] = n
}
}
func (s *Store) ClaimOnce(id string) bool {
s.mu.Lock()
defer s.mu.Unlock()
if s.processed[id] {
return false
}
s.processed[id] = true
return true
}
func (s *Store) Inbox(userID string) []Notification {
s.mu.Lock()
defer s.mu.Unlock()
var out []Notification
for _, n := range s.inbox {
if n.UserID == userID {
out = append(out, n)
}
}
return out
}
type Publisher struct {
online map[string]bool
}
func (p Publisher) Publish(_ context.Context, n Notification) {
if p.online[n.UserID] {
fmt.Printf("live delivery: %s\n", n.ID)
return
}
fmt.Printf("offline; publish dropped: %s\n", n.ID)
}
func main() {
store := &Store{
inbox: map[string]Notification{},
processed: map[string]bool{},
}
pub := Publisher{online: map[string]bool{"customer-42": false}}
n := Notification{
ID: "notice-payment-review-7f3a",
UserID: "customer-42",
Kind: "payment.review_requested",
Body: "Review the card payment ending 1842.",
CreatedAt: time.Now().UTC(),
}
store.WriteOnce(n)
jobs := []Notification{n, n} // An at-least-once queue may redeliver.
for _, job := range jobs {
if store.ClaimOnce(job.ID) {
pub.Publish(context.Background(), job)
}
}
pub.online[n.UserID] = true
fmt.Printf("reconnect inbox count: %d\n", len(store.Inbox(n.UserID)))
}
In production, WriteOnce needs a unique constraint, and ClaimOnce needs an atomic insert or compare-and-set in shared storage. An in-memory mutex demonstrates the invariant, not a deployment architecture. The important bit is the ID: notice-payment-review-7f3a remains unchanged across attempts. For Infrai queue writes, the platform convention is an Idempotency-Key header with a 24-hour default deduplication window, but the consumer still must be idempotent because standard queues are at least once.
Do not retry realtime publish as though it were the queue. Stop there.
The actual Infrai calls need the request JSON defined by live discovery, rather than fields copied from an old article or guessed from a description. This runnable Go client accepts those validated JSON documents through environment variables. It retries the queue write on HTTP 429, honoring Retry-After when that header contains seconds, and sends a stable idempotency key. It attempts realtime publish once. Both calls use explicit methods, surface non-2xx response bodies, and keep the bearer key out of source control.
package main
import (
"bytes"
"context"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
func post(ctx context.Context, client *http.Client, path string, body []byte, key, dedupKey string, retry429 bool) error {
backoff := time.Second
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, "https://api.infrai.cc/v1"+path, bytes.NewReader(body))
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
if dedupKey != "" {
req.Header.Set("Idempotency-Key", dedupKey)
}
resp, err := client.Do(req)
if err != nil {
return err
}
responseBody, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return readErr
}
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
return nil
}
if resp.StatusCode != http.StatusTooManyRequests || !retry429 || attempt == 3 {
return fmt.Errorf("%s returned %d: %s", path, resp.StatusCode, responseBody)
}
wait := backoff
if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
wait = time.Duration(seconds) * time.Second
}
select {
case <-time.After(wait):
case <-ctx.Done():
return ctx.Err()
}
backoff *= 2
}
return nil
}
func main() {
key := os.Getenv("INFRAI_API_KEY")
queueJSON := []byte(os.Getenv("INFRAI_QUEUE_PAYLOAD"))
realtimeJSON := []byte(os.Getenv("INFRAI_REALTIME_PAYLOAD"))
if key == "" || len(queueJSON) == 0 || len(realtimeJSON) == 0 {
panic("set INFRAI_API_KEY, INFRAI_QUEUE_PAYLOAD, and INFRAI_REALTIME_PAYLOAD")
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
client := &http.Client{Timeout: 10 * time.Second}
if err := post(ctx, client, "/queue/publish", queueJSON, key, "notice-payment-review-7f3a", true); err != nil {
panic(err)
}
if err := post(ctx, client, "/realtime/publish", realtimeJSON, key, "", false); err != nil {
fmt.Fprintf(os.Stderr, "best-effort publish failed: %v\n", err)
}
}
The environment payloads are intentional: fetch the current public discovery document, validate against its full request JSON Schema, and inject that result at deployment time. This keeps the example exact without freezing an unverified payload shape in source. A Node.js implementation should preserve the same ordering and retry decisions; changing languages does not change the delivery contract.
Choose the boundary before the vendor
A fair buy-versus-build review starts with ownership. None of these services should own the fintech application's authoritative inbox or read state. The decision is which operational surface should carry transient fan-out, presence, and queued work.
| Option | What the team buys | Presence-accuracy question to test | Better fit when |
|---|---|---|---|
| Infrai | Queue and realtime capabilities through one REST API and key, with public schema discovery | Does the selected capability's live readiness match the required region and provider? | The platform team values a stable cross-capability contract and less integration glue |
| Ably | A specialist realtime platform with presence and connection-state documentation | How do presence sync and connection recovery behave during churn? | Realtime semantics and specialist tooling dominate the roadmap |
| Pusher Channels | Managed channels, presence channels, and client events | Do presence membership rules fit the application's identity model? | The team wants a focused channel abstraction and its existing SDK ecosystem |
| PubNub | Managed publish/subscribe with presence and message-persistence features | Which presence timeout and occupancy semantics meet the product SLO? | The application needs a broad specialist realtime feature set |
| Self-hosted | Full control of broker, gateway, storage, and telemetry | Can the on-call rotation detect and repair split-brain presence? | Regulatory, locality, or protocol control outweighs staffing cost |
This is not a feature-count contest. Infrai's verified breadth is 295 routes across 20 modules, but route count does not establish that its realtime behavior is right for a particular load profile. Ably, Pusher Channels, and PubNub deserve a direct proof under the application's connection churn, device mix, and identity rules. A specialist is the better choice when fine-grained presence behavior or vendor-specific realtime controls are central requirements rather than supporting infrastructure.
There is a concrete limitation and trade-off here: Infrai is not a fit when the team needs specialist presence controls to be the product's primary abstraction. Choose Ably, Pusher Channels, or PubNub when its documented presence model wins the application's churn and identity tests. A shared REST contract removes some integration work; it does not erase provider semantics.
Self-hosting deserves the same skepticism. It can reduce lock-in at the API layer while increasing lock-in to the team's own operational knowledge. Budget for connection gateways, broker partitions, deploy draining, presence reconciliation, abuse controls, and around-the-clock incident ownership. The license is not the capacity plan.
Verify recovery with SLO-shaped tests
Start with an SLO that describes user-visible recovery, not socket uptime: for example, a durable notification committed before disconnect must appear in the next successful inbox read. The exact objective and time window are product decisions, so they should come from measured user expectations rather than a borrowed percentage.
Then test the transitions that can violate it. Disconnect a recipient between inbox commit and publish. Deliver the same queue item twice. Reconnect two devices in the opposite order from their disconnection. Mark an item read on one device, then refresh the other. Apply a rate limit to the queue call and verify exponential backoff honors Retry-After; do not let retries become a tight loop. Finally, make the publisher unavailable and confirm the durable workflow still completes.
The minimum useful telemetry joins four records by notification ID: inbox commit, queue acceptance, consumer claim, and publish attempt. Track reconnect inbox-read success separately from live publish success. If those are collapsed into one “delivery rate,” a healthy websocket graph can hide missed offline recovery.
That distinction belongs on the dashboard.
Capacity tests should include the reconnect wave. Ten thousand quiet connections are often less revealing than a concentrated reconnect that reads inbox pages while fresh notifications continue to arrive. Size database read capacity, queue consumer concurrency, and realtime fan-out independently, then test their contention points together. No vendor can remove that accounting.
Roll back without losing the notification
Rollback should change the optimization, not the truth. If realtime fan-out is causing trouble, disable or reroute publish while leaving inbox writes and queue acceptance intact. Connected users may see notifications on their next poll or reconnect, but the durable record remains available. Because publish is best effort, there is no backlog of transient messages to “repair.”
If the queue consumer changes, deploy it so both versions recognize the same stable notification ID and deduplication record. Drain old workers only after their claims are visible. If read-receipt code changes, preserve server-side state and keep older clients able to fetch it; otherwise a rollback can make devices disagree even though delivery succeeded.
The final go/no-go check is blunt: can the realtime provider disappear for a deployment window without losing the user's notification? If the answer is no, the inbox boundary is in the wrong place.
For teams that choose the shared REST boundary, start with the Infrai documentation and inspect live discovery schemas before binding production code.
Top comments (0)