When a node crashes, how does it catch up with the cluster? That’s where Raft shines.

Understanding Raft: How Nodes Stay in Sync After Crashes

In leader election systems, when one node goes down but the majority remain up, the leader can continue accepting and contributing new data.

Example setup:

node0 (leader) --- node1 (follower) --- node2 (follower) --- node3 (follower)

Now imagine node3 crashes:

node0 (leader) --- node1 (follower) --- node2 (follower) --- node3 (crashed)

We see an error:

err: node3 is down

But that’s fine. As long as a majority of nodes (2 out of 3 followers) are still alive, the cluster continues contributing new data.

Recovery scenario

Let’s assume node3 comes back after five minutes.
During its downtime, three new contributions happened.

So what happens to node3 when it rejoins?

Is it up to date?

Obviously not, it’s missing those 3 contributions.

The question is: how does node3 catch up with the rest of the cluster?

The answer: Raft’s log replication

Each contribution in Raft is stored in a log entry.

i = log index

t = term

Before the crash
i = 1, t = 1 | i = 1, t = 1 | i = 1, t = 1 | i = 1, t = 1
i = 2, t = 1 | i = 2, t = 1 | i = 2, t = 1 | i = 2, t = 1

After 3 new contributions
i = 1, t = 1 | i = 1, t = 1 | i = 1, t = 1 | i = 1, t = 1
i = 2, t = 1 | i = 2, t = 1 | i = 2, t = 1 | i = 2, t = 1
i = 3, t = 1 | i = 3, t = 1 | i = 3, t = 1
i = 4, t = 1 | i = 4, t = 1 | i = 4, t = 1
i = 5, t = 1 | i = 5, t = 1 | i = 5, t = 1

Now the logs are inconsistent. That’s not what we want.

How Raft fixes it

When the leader sends the next log entry (say, i = 6), node3 pushes back:

“Hey! My last index is 2, but you’re sending me index 6. Not acceptable!”

The leader responds:

“Ok, here’s the previous entry (i = 5).”

But node3 complains again:

“Wait… I only have index 2!”

This back-and-forth continues until node3 receives a log entry with a matching previous index (i = 2).

At that point, node3 accepts the next entries (i = 3, 4, 5, 6…) and catches up with the cluster.

That’s Raft in action

This process of syncing logs is how Raft ensures consistency across nodes, even when one crashes and comes back later.

So in short:

Raft uses logs to track contributions.

When a node rejoins, the leader resends missing entries.

The follower rejects mismatched logs until it finds the last known consistent entry.

From there, it accepts new data and gets fully up to date.

Hope you found this insightful!
If so, leave a like and comment your thoughts below.