WebSockets drop. Phones go to sleep. A train goes into a tunnel. Somewhere in the middle of that, your server sends an event, and nobody receives it.
Update (Nov 2026): thanks to Sunnatulla in the comments for catching a bug in the catch-up endpoint when the change log is trimmed. Steps 3 and 4 are fixed below.
Most real-time apps quietly assume every event arrives. When one doesn't, nothing crashes. The screen just shows something slightly wrong, and it keeps showing it until the user pulls to refresh. These bugs are the worst kind: rare, silent and impossible to reproduce on a fast office network.
This post walks through the pattern I use to avoid them. It fits in one sentence:
Number every event, let the client notice a gap, and treat a plain REST read as the source of truth.
The examples use NestJS, Socket.IO, PostgreSQL and a TypeScript client, but the idea works with any stack.
Why "just reconnect" is not enough
Socket.IO reconnects automatically, so it is tempting to think the problem is solved. It is not, for three reasons:
- Events sent while you were offline are gone. Socket.IO delivers at most once. Reconnecting gives you a new connection, not the messages you missed.
- You cannot tell that you missed anything. A client that receives event A and then event C has no way of knowing B existed.
- Timestamps don't help. Ordering by time looks reasonable until two servers with slightly different clocks write to the same stream, or two changes land in the same millisecond.
What the client needs is not a better connection. It needs a way to detect that it fell behind, and a reliable way to catch up.
The example: a shared task board
Picture a board that several people edit at once. Each board is a stream: an ordered list of changes such as "task created", "task moved" or "task renamed". Every person looking at the board should end up seeing the same thing.
Step 1: give every stream its own counter
Each board gets a sequence number. Every change increments it inside the same transaction as the change itself, so the number and the data can never disagree.
CREATE TABLE board (
id uuid PRIMARY KEY,
seq bigint NOT NULL DEFAULT 0
);
CREATE TABLE board_change (
board_id uuid NOT NULL REFERENCES board(id),
seq bigint NOT NULL,
kind text NOT NULL,
payload jsonb NOT NULL,
PRIMARY KEY (board_id, seq)
);
With Prisma, mapped onto those two tables:
async moveTask(boardId: string, taskId: string, column: string) {
return this.prisma.$transaction(async (tx) => {
await tx.task.update({ where: { id: taskId }, data: { column } });
const [{ seq }] = await tx.$queryRaw<{ seq: bigint }[]>`
UPDATE board SET seq = seq + 1
WHERE id = ${boardId}::uuid
RETURNING seq`;
const change = { kind: 'task.moved', payload: { taskId, column } };
await tx.boardChange.create({ data: { boardId, seq, ...change } });
return { seq: Number(seq), ...change };
});
}
Two details matter here:
- The counter is per stream, not global. One global counter would make every write in the system wait on the same row.
- The
UPDATE ... RETURNINGtakes a row lock, so two concurrent changes on the same board get 1 and 2, never 1 and 1.
Step 2: emit after the commit, with the number
Only tell clients about a change once it is committed. Otherwise a client might act on something that gets rolled back.
const change = await this.boards.moveTask(boardId, taskId, column);
this.server
.to(`board:${boardId}`)
.emit('board.change', { boardId, ...change });
The event says what happened and which number it is. It does not have to be perfect, and it is allowed to get lost. That is the point.
Step 3: let the client notice a gap
The client remembers the last number it applied, and checks every incoming event against it:
let lastSeq = 0; // last change applied to the screen
let latestSeen = 0; // highest number we have heard of
let syncing: Promise<void> | null = null;
socket.on('board.change', (e: BoardChange) => {
latestSeen = Math.max(latestSeen, e.seq);
// duplicate or old: ignore
if (e.seq <= lastSeq) return;
// the next one, and no catch-up running: apply it
if (e.seq === lastSeq + 1 && !syncing) {
apply(e);
lastSeq = e.seq;
return;
}
// a gap: we missed something
void catchUp();
});
// every connect, including reconnects
socket.on('connect', () => void catchUp());
// only one catch-up at a time
function catchUp() {
syncing ??= sync().finally(() => (syncing = null));
return syncing;
}
async function sync() {
do {
const before = lastSeq;
const url = `/boards/${boardId}/changes?after=${lastSeq}`;
const { changes, snapshot } = await (await fetch(url)).json();
if (snapshot) {
// too far behind: start over from a full copy
replaceBoard(snapshot);
lastSeq = snapshot.seq;
} else {
for (const c of changes) {
// never apply anything twice
if (c.seq !== lastSeq + 1) continue;
apply(c);
lastSeq = c.seq;
}
}
// no progress: stop, rather than ask the same question forever
if (lastSeq === before) break;
// read again if something newer arrived meanwhile
} while (latestSeen > lastSeq);
}
Three details keep this honest. Only one catch-up runs at a time, so two quick gaps can't apply the same changes twice. The client remembers the highest number it has heard of: if a newer event arrived while it was catching up, it simply reads again. And a round that moves nothing forward ends the loop, so a server answer the client can't use never turns into a request storm.
There are only three cases, and each one is boring:
| What arrives | What the client does |
|---|---|
| The next number | Applies it |
| A number it already has | Ignores it |
| A number further ahead | Asks the server what it missed |
Step 4: the read endpoint is the truth
The catch-up endpoint is an ordinary REST route. It returns the changes after a given number, or a full snapshot when replaying them is not possible or not worth it.
@Get(':id/changes')
async changes(@Param('id') id: string, @Query('after') after = '0') {
const since = Number(after);
// the board's current seq and the rows after `since`, read together
const { seq, changes } = await this.boards.changesAfter(id, since, {
limit: 200,
});
// the log no longer starts right after `since`: it was trimmed
const trimmed = changes.length > 0
? changes[0].seq !== since + 1
: seq > since;
if (trimmed || changes.length === 200) {
return { snapshot: await this.boards.snapshot(id) };
}
return { changes };
}
The trimmed check matters. Once you trim the log, a client returning from far behind would otherwise get rows that start after a hole it can never fill. Answer that with a snapshot, every time.
Build the snapshot and its seq in the same transaction, so the number always matches the data it came with.
This is the part that makes the whole design calm. The socket is no longer responsible for correctness. It is just a fast way of saying "something changed, read again if you need to". You need this endpoint anyway for the first page load, so the reliable path costs almost nothing extra.
What about more than one server?
Add the Socket.IO Redis adapter and every server can emit to every client, wherever they are connected. The ordering does not depend on which server handled the write, because the number comes from the database, not from the process. Two servers, ten servers: the client logic stays exactly the same.
What about when the app is asleep?
A phone in a pocket has no open socket. For the moments that need a human's attention, send a push notification as well. Keep it short ("Task moved to Done") and never treat it as data. When the user opens the app, the socket reconnects, catchUp() runs, and the screen is correct again.
Why not acknowledgements and guaranteed delivery?
You can build acks, retries and per-client queues on top of WebSockets. I avoid it for most product features because:
- It moves a lot of state into the server, per client, per connection.
- It still needs a fallback for clients that were offline for an hour.
- That fallback is a REST read, which you need to build anyway.
So I build the fallback first and make it the main path. Delivery over the socket becomes a performance feature, not a correctness feature.
The trade-offs
Nothing here is free:
- A reconnect costs one extra request. In practice it is tiny, and it only happens when something went wrong.
- A very busy stream contends on its counter row. If one stream takes hundreds of writes per second, split it into smaller streams.
- You keep a change log. Trim it on a schedule, and make sure the endpoint notices when a client asks for rows that were trimmed away. Those clients get a snapshot.
Checklist
- Every stream has its own sequence number, incremented in the same transaction as the change
- Events are emitted after commit and carry their number
- The client ignores old numbers, applies the next one and catches up on a gap
- The client catches up on every connect, not just the first
- One REST endpoint returns "changes after N", or a snapshot when N is too old or was trimmed away
- A catch-up round that makes no progress stops instead of retrying
- Ordering never depends on timestamps
The socket makes your app feel live. The numbers and the read make it correct. Keep those two jobs separate, and the "it only happens sometimes" bugs mostly stop happening.
I'm a software engineer building multi-platform SaaS products with Flutter, NestJS and Next.js. More on GitHub.
Top comments (2)
The trim step and the 200-row check don't quite meet. Once the log is trimmed to seq > 300, a client coming back with after=10 gets 49 rows, so no snapshot, and every row fails
c.seq !== lastSeq + 1. I ran your loop against that: with nothing heard yet it silently stays at 10, and if a seq 349 event arrived first,latestSeen > lastSeqnever clears and it refetched 1000 times before I stopped it. Returning the snapshot whenchanges[0].seq !== after + 1fixed both.Great catch and thank you for actually running it, you're right on both counts , once the log is trimmed, a client returning from far behind gets rows that start after a gap it can never fill, with nothing heard it silently stays stale, and with a newer event heard it loops forever.
I've updated the post with your fix , the endpoint now returns a snapshot when
changes[0].seq !== after + 1, and also when there are no rows but the board's seq has moved pastafter, I also added a no-progress guard on the client, so a bad answer can't turn into a request storm again.