Building an Embedded Raft SDK for Existing Node.js Services

Sometimes an application needs Raft semantics, but it does not need another general-purpose
distributed system.

An existing scheduler, control plane, metadata service, or coordination service may already have its
own domain model and API. What it lacks is a safe way for several instances to elect a leader, agree
on an ordered command stream, and apply that stream to local state.

The usual choices are not always comfortable:

• implement Raft inside the application;
• redesign the application around an external system such as etcd;
• or operate a larger platform whose data model and purpose do not match the application.

The goal of node-raft-rsm is to offer another
option: an embedded TypeScript SDK that lets an existing Node.js service use Raft primitives through
a focused API.

The application should keep ownership of its commands, state machine, storage strategy, transport,
and public API. The SDK should own the difficult consensus mechanics: terms, elections, log
replication, quorum commitment, recovery ordering, and serialized application.

Raft itself is often introduced with three reassuring words: leader, log, majority. That sounds
simple until you try to answer the questions that matter in a real implementation:

• When is a log entry safe to apply?
• What must be durable before a node sends an acknowledgement?
• What happens when a response is duplicated, delayed, or dropped?
• Can a client treat a timeout as failure?
• How do you test all of this without waiting for real clocks and real network failures?

I built node-raft-rsm to put those details behind a small set of typed contracts. It is accompanied
by a deterministic test harness and an interactive React visualizer, but the central product is the
SDK and its integration boundary.

It is not production-ready. That is worth saying at the beginning, not hiding at the end. The
durable adapter, complete snapshot lifecycle, authenticated multi-process transport, dynamic
membership, linearizable reads, and a larger fault matrix are still work in progress.

Project status: node-raft-rsm is under active development. The current release is for
evaluation, deterministic testing, API design, and continued implementation—not production data.

What exists today is a useful implementation slice for integrating and evaluating the API, learning
Raft mechanics, and discussing the boundaries between consensus, durability, and application state.

The integration experience I want

Adding Raft to an existing system should not require rewriting that system around a generic
key-value store. The intended integration flow is:

1. Define the application's commands and typed results.
2. Implement a deterministic state machine for those commands.
3. Provide storage and transport adapters appropriate for the deployment.
4. Create a RaftNode with stable cluster membership and identity.
5. Route existing mutation paths through node.propose().
6. Keep the application's existing HTTP, RPC, or message-based API in front of the SDK.

The public surface is deliberately centered on a few operations:

const node = await RaftNode.create(options);

await node.start();

const result = await node.propose(command, {
  commandId,
  timeoutMs: 5_000,
});

const value = await node.read(reader, { consistency: 'local' });

await node.stop();

The SDK does not decide what an account, job, lock, configuration update, or scheduling decision
looks like. It provides the consensus mechanism that orders those domain operations.

The current implementation is a Node.js library, not a standalone cross-language Raft daemon. A
non-Node program would access consensus through the host service's existing API. A dedicated network
API could be built later, but it should remain a thin boundary over the same SDK contracts.

A concrete SDK integration

Imagine an existing scheduling service. It already has HTTP endpoints, validation, domain types,
metrics, and a local scheduler state. We want every replica to apply the same job mutations in the
same order.

First, create one RaftNode in each service process. Identity and membership must come from stable
configuration rather than being generated at startup:

import { clusterId, nodeId } from '@node-raft-rsm/core';
import { JsonCommandCodec, RaftNode } from '@node-raft-rsm/node';
import { SchedulerStateMachine, validateSchedulerCommand } from './scheduler-state-machine.js';
import { storage } from './raft-storage.js';
import { transport } from './raft-transport.js';

const members = config.raftMembers.map(nodeId);
const machine = new SchedulerStateMachine();
const codec = new JsonCommandCodec(validateSchedulerCommand);

const raft = await RaftNode.create({
  nodeId: nodeId(config.nodeId),
  clusterId: clusterId('scheduler-cluster'),
  members,
  heartbeatInterval: 250,
  electionTimeoutMinMs: 1_500,
  electionTimeoutMaxMs: 3_000,
  storage,
  transport,
  stateMachine: machine,
  codec,
  onEvent: (event) => metrics.recordRaftEvent(event),
});

await raft.start();

storage and transport are application-provided adapters. The repository currently ships
deterministic in-memory implementations for development and tests; production-grade durable storage
and authenticated transport are still under development.

Then route an existing mutation endpoint through propose() instead of mutating scheduler state
directly:

import { randomUUID } from 'node:crypto';
import { NotLeaderError, ProposalTimeoutError } from '@node-raft-rsm/core';

app.post('/jobs', async (request, response) => {
  const commandId = request.headers['idempotency-key'] ?? randomUUID();
  const command = {
    type: 'schedule-job' as const,
    jobId: request.body.jobId,
    runAt: request.body.runAt,
    payload: request.body.payload,
  };

  try {
    const result = await raft.propose(command, { commandId, timeoutMs: 5_000 });
    response.status(201).send(result);
  } catch (error) {
    if (error instanceof NotLeaderError) {
      response.status(503).send({
        code: 'NOT_LEADER',
        leaderHint: error.leaderHint,
      });
      return;
    }

    if (error instanceof ProposalTimeoutError) {
      response.status(504).send({
        code: 'OUTCOME_UNKNOWN',
        commandId,
        retry: 'Retry identical command bytes with the same command ID',
      });
      return;
    }

    throw error;
  }
});

The service keeps its /jobs API and domain model. Raft is an internal consistency mechanism rather
than a replacement public API.

Reads must also communicate their guarantee. Only explicitly local reads exist today:

const job = await raft.read(() => machine.getJob(request.params.jobId), {
  consistency: 'local',
});

That result may be stale, including when read from a former leader isolated by a partition. The SDK
does not yet expose a linearizable ReadIndex-style operation.

Why not just use etcd?

etcd is a mature distributed key-value store with a broad operational ecosystem. If its data model
and API fit the problem, using it is usually safer than adopting an experimental consensus library.

This project is aimed at a different integration shape. It is useful when Raft should be part of the
application runtime and the replicated state machine is application-specific. The host service may
need typed commands, domain-level conflict results, custom snapshots, or state that should not be
modeled as an external generic key-value store.

The goal is not to recreate every etcd feature. It is to provide a smaller, composable Raft layer for
systems that already know what their data and APIs should look like.

When do you actually need embedded Raft?

Most web applications do not.

If several stateless API processes use an authoritative shared database, the database already
provides the consistency boundary. Adding Raft to the application tier would usually create more
operational complexity without improving the system.

Embedded Raft makes sense when every process owns a local copy of state and the processes must agree
on one ordered mutation stream. Examples include:

• replicated schedulers;
• metadata and configuration services;
• control planes;
• coordination systems;
• research and simulation tools.

The mental model is not “make my service highly available with one library call.” It is “give every
replica of my existing service the same committed command sequence.”

The command lifecycle

The most important line in the project README is this one:

propose → append → persist → replicate → quorum → commit → apply → return result

Those steps are deliberately separate.

Append means the leader added a command to its local log. The command is not committed yet.

Commit means a quorum has stored the entry and the leader has advanced its commit index according
to Raft's current-term rule.

Apply means the committed command has been executed by the local replicated state machine.

In this SDK, propose() resolves after the leader has committed and applied the entry locally. It
does not wait for every follower to apply it. A slow or partitioned follower can catch up later.

This distinction is much easier to understand when it is visible:

A pure core and an impure runtime

The implementation separates consensus decisions from I/O.

raft-core is a deterministic transition engine. It owns terms, roles, votes, log matching, peer
progress, elections, and commitment. It does not import timers, sockets, filesystems, or databases.
It receives an event and produces state changes plus effects.

raft-node is the runtime around that core. It owns lifecycle, timers, transport, storage, command
encoding, proposal completion, and serialized state-machine application.

The boundary between them is a Ready/Advance cycle:

event
  ↓
RaftCore.step()
  ↓
Ready { hard state, log changes, messages, committed entries }
  ↓
persist → send → apply
  ↓
advance()

The runtime processes each Ready batch conservatively:

1. Persist hard state, truncations, entries, and snapshot metadata atomically.
2. Wait for durability.
3. Send outbound messages.
4. Apply committed entries in index order.
5. Persist the applied position.
6. Advance the core.

That order matters. A node must not grant a vote or report a successful append if the corresponding
state exists only in volatile memory. A crash between “send success” and “persist” can break the
assumptions Raft relies on.

The application is a deterministic state machine

The SDK does not replicate arbitrary JavaScript memory. It replicates commands.

Here is the shape of the included key-value example:

type KvCommand =
  | { readonly type: 'put'; readonly key: string; readonly value: string }
  | { readonly type: 'delete'; readonly key: string }
  | {
      readonly type: 'compare-and-set';
      readonly key: string;
      readonly expected: string | null;
      readonly value: string;
    };

class KvStateMachine implements ReplicatedStateMachine<KvCommand, KvResult> {
  readonly #values = new Map<string, string>();

  apply(command: Readonly<KvCommand>): KvResult {
    switch (command.type) {
      case 'put':
        this.#values.set(command.key, command.value);
        return { status: 'applied', value: command.value };

      case 'delete': {
        const previous = this.#values.get(command.key) ?? null;
        this.#values.delete(command.key);
        return { status: 'applied', value: previous };
      }

      case 'compare-and-set': {
        const currentValue = this.#values.get(command.key) ?? null;
        if (currentValue !== command.expected) {
          return { status: 'conflict', currentValue };
        }
        this.#values.set(command.key, command.value);
        return { status: 'applied', value: command.value };
      }
    }
  }
}

The apply method must not use Date.now(), randomness, local files, environment variables, or
remote calls to decide replicated state. Any nondeterministic input must be captured in the command
before it is proposed.

Business rejection is also data, not an exception. A failed compare-and-set still occupies a
committed log position so that every replica reaches the same decision in the same order.

Timeouts are ambiguous

One of the least intuitive distributed-systems lessons is that a client timeout does not prove an
operation failed.

The leader may have committed a command just before the response was lost. Retrying it with a new ID
could apply the mutation twice. node-raft-rsm therefore accepts a command ID:

const result = await node.propose(command, {
  commandId: request.headers['idempotency-key'],
  timeoutMs: 5_000,
});

After a timeout, the safe retry is the identical command bytes with the same command ID. The current
deduplication support is bounded and in-process; durable deduplication remains a release blocker.

Node.js pauses are part of the failure model

Running Raft inside Node.js introduces an important operational constraint: heartbeats and election
timeouts are scheduled on the event loop.

A long synchronous computation, synchronous I/O, a large serialization operation, or a garbage
collection pause can prevent timers and message handlers from running on time. A follower may start
an unnecessary election because it did not process the leader's heartbeat. A leader may fail to send
heartbeats quickly enough, creating term churn and temporary loss of availability.

A pause should not by itself violate Raft's safety rules in a correct implementation, but this
project does not yet have enough multi-process and long-pause evidence to present that as a production
guarantee. Event-loop stalls are one of the explicit areas still under development and testing.

An eventual production deployment should:

• choose election timeouts comfortably above normal network, disk, event-loop, and GC latency tails;
• monitor event-loop delay alongside role, term, proposal latency, peer lag, and storage latency;
• avoid synchronous I/O and CPU-heavy work on the Raft process's main thread;
• move expensive application work to worker threads or separate processes;
• bound command sizes and state-machine apply time;
• test deliberate process pauses, not only message loss and network partitions.

This is also why timeout values copied from a LAN example are not universal defaults. A deployment
must tune them using its own worst-case latency measurements.

Making the network controllable

Testing Raft against a normal in-memory event emitter is not enough. A useful simulator must let us
control the uncomfortable cases:

• delay a message;
• deliver one message at a time;
• duplicate or drop a message;
• reverse the pending queue;
• partition two groups of nodes;
• stop and restart a node while retaining its durable state.

The in-memory transport exposes those controls, while seeded randomness and virtual time make
scenarios repeatable.

The React visualizer uses the real RaftNode runtime and that deterministic transport. It is not a
separate toy implementation.

From the UI, you can start an election, inspect RequestVote and AppendEntries messages, manually
deliver or drop them, partition the cluster, disable a node, restart it, and propose key-value
commands. The timeline separates network events, election events, and command lifecycle events.

You can also click a node to select it or press and hold to reposition it. That small interaction
turned out to be surprisingly useful when explaining a partition to someone else.

Try it locally

The current development baseline is Node.js 24 and pnpm 9:

git clone https://github.com/dmitry-bardonov/node-raft-rsm.git
cd node-raft-rsm
nvm use
pnpm install
pnpm check

Run the deterministic three-node key-value example:

pnpm example:kv

Or start the visualizer:

pnpm visualizer

Then open http://127.0.0.1:3000.

A useful first scenario is:

1. Start an election on node-1 and drain the vote messages.
2. Propose PUT counter = 2 and drain replication messages.
3. Partition node-1 away from the other two nodes.
4. Elect a new leader on the majority side.
5. Heal the partition and watch the old leader step down and repair its log.

What is intentionally missing

The project currently lacks several things I would require before trusting it with production data:

• a durable SQLite storage adapter;
• a real authenticated transport;
• a complete snapshot and recovery pipeline;
• durable command deduplication;
• dynamic membership and joint consensus;
• linearizable ReadIndex-style reads;
• broad property, restart, corruption, and multi-process fault testing;
• operational evidence under event-loop stalls and GC pauses.

Raft also does not solve every distributed-systems problem. It does not provide Byzantine fault
tolerance, exactly-once external side effects, transparent distributed transactions, or automatic
consistency for arbitrary process memory.

What I learned

The algorithm is only part of a consensus implementation. The difficult boundaries are where the
algorithm meets durability, transport, recovery, application code, and client expectations.

Three lessons have shaped this project so far:

1. Make ordering constraints explicit. “Persist before send” should be visible in the architecture, not buried in a callback.
2. Make nondeterminism injectable. Clocks, randomness, and delivery order must be controllable if failures are going to be reproducible.
3. Make incomplete guarantees obvious. A local read is not a linearizable read, and a timeout is not proof of failure.
4. Protect the host application's architecture. An embedded consensus SDK should adapt to the application's domain model instead of forcing the application to become a client of a different general-purpose system.

The visualizer helped too. Watching terms, message queues, commit indexes, and applied indexes change
made several design mistakes more obvious than another page of logs would have.

The long-term question behind the project is: how small and predictable can the integration surface
be while still giving an existing service correct Raft elections, replication, commitment, recovery,
and state-machine application?

If you maintain a system that could benefit from embedded consensus, I would appreciate feedback on
the SDK boundaries and integration experience:

👉 github.com/dmitry-bardonov/node-raft-rsm

What would an embedded Raft API need before you would integrate it into an existing service?