Short answer: keep a short, lossless stream for reconnects, a compact snapshot for backfill, and discard movement noise once the map has caught up. That split scales realtime event delivery without pretending a tracking map needs an eternal event log.
The awkward part is choosing the boundary. A courier's typing indicator (the dispatcher is composing a note) wants seconds of retention. A read receipt wants a durable state transition. A GPS marker wants the newest position, not every point recorded while a phone was underground. Treating all three as “events” is how a small map turns into a replay bill and a slow reconnect path.
| Signal | Retention tier | Reconnect behavior | Why |
|---|---|---|---|
| Typing indicator | 30–60 seconds | Drop if stale | It is presence, not history. |
| Read receipt | 7–30 days | Replay until acknowledged | It changes workflow state. |
| Courier location | 2–10 minutes of deltas, plus latest snapshot | Send snapshot, then deltas | The map needs continuity, not archaeology. |
Those windows are starting points, not laws. Measure payload size, reconnect frequency, and the oldest useful update in your own fleet before changing them.
How should realtime event retention shape delivery tracking recovery?
Start with a recovery contract. Every message needs a monotonically increasing sequence within a delivery, an event type, and a server timestamp. A client reconnects with its last applied sequence. The server either replays the missing range or says, plainly, “snapshot required.” There is no third state where the UI quietly fills gaps with guesses.
The sequence is scoped. A global counter creates needless contention and makes a busy city affect a quiet one. A per-delivery counter is easier to reason about, while a per-channel counter can be cheaper if the channel is already the unit of subscription. Whichever scope you pick, document it next to the schema and test a reconnect that crosses a partition boundary.
Use a bounded stream for the first tier. A ring buffer or a broker stream with a maximum age gives you a predictable replay cost. Expiry should be based on server time; device clocks are useful for display, not for retention decisions. On reconnect, reject a cursor older than the bound with a typed result such as snapshot_required, then send the current delivery state followed by new events.
The snapshot is not a dump of every coordinate. It is the authoritative state needed to render: marker position, status, last receipt state, and the version of each field. This is a useful place for an explicit schema version. A client can refuse an unknown version and ask for a full refresh instead of applying half a payload.
Here is the decision logic I use in a TypeScript client. It has no vendor assumptions and makes the gap visible in logs.
type Recovery =
| { kind: "replay"; from: number; to: number }
| { kind: "snapshot_required"; reason: "cursor_expired" | "unknown_channel" };
interface Head {
oldest: number;
newest: number;
}
export function chooseRecovery(lastApplied: number, head: Head): Recovery {
if (lastApplied < head.oldest - 1) {
return { kind: "snapshot_required", reason: "cursor_expired" };
}
if (lastApplied >= head.newest) {
return { kind: "replay", from: head.newest + 1, to: head.newest };
}
return { kind: "replay", from: lastApplied + 1, to: head.newest };
}
That empty replay range is intentional. It keeps one code path for “already current” and “has a gap,” which is less glue when the connection flaps three times in a minute. I log the cursor, the selected branch, and the snapshot version, but never the driver's exact note text or a token.
The failure modes that make a retention policy lie
The first trap is acknowledging before applying. If a client advances its cursor when a frame arrives, then crashes before its reducer commits, the next reconnect skips data. Advance the cursor after an idempotent apply. Read receipts should carry an event id so a duplicate replay is harmless; map updates can use last-write-wins on (deliveryId, fieldVersion).
The second trap is conflating transport delivery with business freshness. A ten-minute-old location may be valid history but useless to a customer watching a van. Put freshness in the payload and let the UI mark a stale marker. Do not extend retention just to hide a stale device.
Here is the outage-shaped case worth simulating. At 09:12, a courier's handset loses service under a concrete overpass. The dispatcher keeps editing a note, so typing events expire; the read receipt remains in the durable state store; the location stream accumulates five minutes of deltas and then reaches its age limit. At 09:18, the handset reconnects with sequence 418 while the channel head is 463 and the oldest retained event is 451. The client must render the latest snapshot, apply 452 through 463 in order, and show the receipt state from the snapshot even though the typing indicator is gone. If your protocol instead replays every signal from 418, you have coupled a transient hint to a business transition and made the recovery path both slower and semantically wrong. This is the kind of test I run before tuning buffer size, because a green throughput chart cannot tell you that a customer saw an old marker.
Measure it.
The third trap is fan-out during backfill. One driver reconnecting is cheap. A depot reopening after a network outage can request thousands of cursors at once. Rate-limit replay per channel, cap events per response, and add jitter to automatic retries. A snapshot endpoint or store read should be cheaper than replaying a day's worth of GPS deltas.
Finally, test the policy with time. Freeze the server clock, publish exactly at the expiry boundary, reconnect with oldest - 1, and verify the client asks for a snapshot. I also inject duplicate receipts and out-of-order location updates. These tests catch more than a happy-path WebSocket test because the real failure is state, not the socket handshake.
Choosing the runner-up when replay is the wrong tool
An append-only log is a better fit when analysts need a complete audit trail or when you must reconstruct a historical route. It is a poor fit for a customer map that only needs current state and a small recovery window. Keep that archive separate from the delivery channel; different retention and access controls are the point.
A snapshot-only design can win for very high-volume markers. It is easier to operate, but a reconnect can jump visibly, and read receipts need another durable path. Stick with snapshots when location precision is approximate and the product can tolerate a refresh boundary. Keep replay when an operator must see every status transition in order.
Your mileage may vary. I am not sure a single universal window exists, because depot topology, handset sleep behavior, and legal retention requirements move the boundary. The decision rule is stable, though: retain what a disconnected client can meaningfully apply, snapshot what it cannot, and instrument the handoff.
Top comments (0)