Short answer: when notifications are lost while a user is offline, debug the durable record and cursor before the live connection. For a financial chat room that must survive reconnects, commit each notification to an ordered room log before attempting best-effort publish. Give the client a room-scoped read token, require it to acknowledge a monotonic cursor, and replay everything after that cursor when it reconnects. This is the least complex design that makes an offline gap explainable.
Start with the delivery contract.
| Contract | Pick this when | Offline behavior | Client trust boundary |
|---|---|---|---|
| Ephemeral publish | Typing or presence can disappear | Nothing is recovered | Client observes only the live room |
| Durable room log plus cursor | Every chat notification must appear after reconnect | Replay after the acknowledged cursor | Token grants read access to one room |
| Durable per-user inbox | Private notices span rooms | Replay from the user's inbox cursor | Token binds user, audience, and operations |
For reconnect-safe fintech chat, pick the durable room log by default. Add a per-user inbox only when notifications must outlive room membership or span rooms. Do not ask a live transport to manufacture durability it never promised.
Why did the notification vanish while the user was offline?
A best-effort publish path has no durable state to query after the connection closes. The server can accept a publish, attempt fan-out, and still have no record for a subscriber who was absent at that instant. Reconnecting repairs the connection. It does not recreate history.
That distinction gives us a sharp root-cause checklist. First, prove that the business event was committed. Next, prove that a notification record was appended with a stable event ID and room sequence. Then inspect the subscriber's last acknowledged cursor, current room membership, and token claims. Only after those checks should you inspect the live socket.
The order matters. A green connection metric says almost nothing about the missing interval. A successful publish counter says even less unless it is downstream of the durable commit.
Start there.
Picture the system as a line: payment event, transaction commit, room-log append, live nudge, client fetch, client render, cursor acknowledgement. The durable edge is the append. The live nudge only reduces latency. If it disappears, fetch-after-cursor still closes the gap.
Consider one concrete failure timeline. A room member applies sequence 417, loses connectivity, and misses the live nudges for 418 and 419. The server appends both records anyway. On reconnect, the client presents a valid room-scoped token; the server reads its saved cursor, returns up to 100 events after 417, and waits for a contiguous acknowledgement. If the response contains 419 but not 418, the client stops rather than advancing. This separates five possible faults that a generic "publish failed" label would blur together: append failure, incomplete read, authorization rejection, client-side gap handling, and acknowledgement failure.
Pick the contract before debugging the transport
Use ephemeral publish for transient state. Replaying a ten-second-old "is typing" event would make the interface worse. Presence has similar semantics. Loss is acceptable because freshness is the value.
Use a durable room log for messages, moderation decisions, payment-status notices, and other records whose absence would produce a false account of the conversation. Give each committed entry a server-assigned sequence within the room. A client stores the highest contiguous sequence it has applied, not merely the largest sequence it has seen. That one word, contiguous, prevents a client from acknowledging past a hole.
Use a per-user inbox when room history and notification history have different retention or authorization rules. An account-level security notice may need to remain available after a person leaves a room. This design adds another projection, cursor, retention policy, and reconciliation path. Choose it for semantics, not as a reflex.
Token scope is part of the contract. A reconnect token should identify the subject, audience, expiry, permitted operation, and room or inbox scope. The server derives authorization from verified claims and current membership; it must not trust a room ID, user ID, or cursor merely because the client sent it. A cursor controls where reading resumes. It never grants permission to read.
Implement one durable path end to end
The useful invariant is short: an acknowledged business operation has either produced its notification in the same transaction, or produced a durable outbox record that will do so. Publishing directly between two commits creates a dual-write gap.
Here is a compact TypeScript shape. The interfaces are generic on purpose. Watch the order of operations and the scope checks.
type RoomId = string;
type UserId = string;
type Claims = {
subject: UserId;
audience: "chat-notifications";
roomId: RoomId;
operations: readonly ("read" | "ack")[];
expiresAtMs: number;
};
type RoomEvent = {
roomId: RoomId;
sequence: number;
eventId: string;
kind: "message-posted" | "payment-status-changed";
payload: unknown;
};
interface RoomLog {
readAfter(roomId: RoomId, sequence: number, limit: number): Promise<RoomEvent[]>;
}
interface Memberships {
canRead(userId: UserId, roomId: RoomId): Promise<boolean>;
}
interface CursorStore {
get(userId: UserId, roomId: RoomId): Promise<number>;
}
async function resumeRoom(
claims: Claims,
requestedRoomId: RoomId,
log: RoomLog,
memberships: Memberships,
cursors: CursorStore,
): Promise<RoomEvent[]> {
if (claims.audience !== "chat-notifications") throw new Error("wrong audience");
if (claims.expiresAtMs <= Date.now()) throw new Error("expired token");
if (!claims.operations.includes("read")) throw new Error("read not allowed");
if (claims.roomId !== requestedRoomId) throw new Error("room scope mismatch");
if (!(await memberships.canRead(claims.subject, requestedRoomId))) {
throw new Error("membership no longer active");
}
const after = await cursors.get(claims.subject, requestedRoomId);
return log.readAfter(requestedRoomId, after, 100);
}
The server owns the saved cursor. The client may report, "I applied sequence 418." The server still validates that acknowledgement against the delivered range, current authorization, and contiguity before advancing stored state.
Delivery is at least once across reconnects, so application must be idempotent. Keep eventId as the deduplication key and sequence as the ordering signal. They solve different problems. An event ID stops a repeated side effect; a sequence exposes a gap.
type ClientState = {
nextSequence: number;
appliedEventIds: Set<string>;
};
function applyBatch(state: ClientState, batch: readonly RoomEvent[]): ClientState {
const ordered = [...batch].sort((a, b) => a.sequence - b.sequence);
for (const event of ordered) {
if (event.sequence < state.nextSequence) continue;
if (event.sequence > state.nextSequence) break;
if (!state.appliedEventIds.has(event.eventId)) {
renderNotification(event);
state.appliedEventIds.add(event.eventId);
}
state.nextSequence += 1;
}
return state;
}
function renderNotification(event: RoomEvent): void {
// Update a deterministic local projection keyed by event.eventId.
void event;
}
Stop on a gap. Fetch again. Do not sort a sparse batch and pretend continuity has been restored.
A live channel may carry the event or a small "room changed" nudge. On open, reconnect, and every nudge, the client requests the durable suffix. WebRTC data channels can configure ordering and retransmission behavior, but those properties apply to the channel; they do not create application history for an absent peer.
Observe the gap, not just the connection
A single "notifications sent" counter hides every stage that matters. Emit structured records around stable identifiers: eventId, roomId, sequence, an opaque connection ID, token audience, token-scope result, and saved cursor. Do not log token contents or sensitive payloads.
Track the delay from business commit to log append. Count live delivery attempts separately from durable reads. Measure replay batch size and age after reconnect. Alert on cursor stalls, sequence gaps, repeated authorization failures, and outbox age.
A committed event with no room-log sequence points to the transaction or outbox boundary. A present log entry with a cursor frozen below it points to resume, acknowledgement, or authorization. A cursor that advanced while the UI lacks the event points to client application logic. Reconnect churn with successful replay is a transport symptom, not notification loss.
Test the timeline. Disconnect after the durable append but before the live write. Disconnect after delivery but before acknowledgement. Expire the token during an outage. Remove membership before replay. Deliver the same event twice. Return sequences 417 and 419 while withholding 418. Each test needs an explicit expected cursor and visible notification set.
That suite beats repeatedly toggling airplane mode and checking for a green badge.
Limits and decision rules
This design cannot promise that every device displays every notification simultaneously. Devices sleep, networks partition, tokens expire, and users revoke access. It promises something narrower and testable: while authorization and retention permit, a reconnecting client can request the durable suffix after its acknowledged cursor.
The trade-off is extra storage, read traffic, cursor state, and reconciliation work. A durable log is not suitable for typing indicators, disposable presence updates, or any signal whose stale replay is worse than loss. A per-user inbox is also the wrong choice when room history already provides the required retention and access boundary; duplicating every event would create a second projection to secure and repair without changing the user-visible contract.
Retention sets the maximum recoverable outage. If events expire before a client returns, report that the cursor is outside the retained range and start a documented snapshot or resynchronization flow. Silent truncation makes an incomplete timeline look complete. Pick retention from product and regulatory requirements, then load-test the largest permitted replay.
Bound batch size, page through the log, and keep live nudges secondary while replay runs. A slow client should not force unbounded memory growth, and reconnect storms should not turn one missed notification into a database incident.
There is a hard limit.
The decision rule is plain. If losing an item changes the user's understanding of a financial conversation, persist it before live delivery and recover by cursor. If old information becomes misleading, keep it ephemeral. Scope each token to the smallest resource and operation the client needs, and re-check present authorization during replay.
Top comments (0)