DEV Community: Abhinov007

Understanding RESP: The Protocol Behind Redis

Abhinov007 — Thu, 18 Jun 2026 08:53:24 +0000

In the first post, I shared the overview of my Redis clone built from scratch in Node.js.

In this post, I want to go one layer deeper into one of the most important parts of the project:

RESP — the Redis Serialization Protocol.

Before my Redis clone could support commands like:

SET name Alice
GET name
LPUSH queue task1

it first needed to understand what the client was actually sending over the network.
That is where RESP comes in.

Why Redis Needs a Protocol

When we use Redis from a CLI or client library, we usually think in commands:

SET name Alice

But Redis does not internally receive a nice JavaScript array like:

["SET", "name", "Alice"]

It receives raw bytes over a TCP socket.

Those bytes need structure.

The server must know:

where the command starts
where the command ends
how many arguments exist
how long each argument is
whether the value is a string, integer, error, array, or bulk string

That structure is provided by RESP.

RESP is the language Redis clients and Redis servers use to talk to each other.

What RESP Looks Like

A command like:

GET name

is represented in RESP as:

*2\r\n$3\r\nGET\r\n$4\r\nname\r\n

At first, this looks strange.

But once broken down, it is simple.

*2

means this is an array with 2 elements.

$3
GET

means the first bulk string has length 3, and its value is GET.

$4 name

means the second bulk string has length 4, and its value is name.

So the server parses this:

*2\r\n$3\r\nGET\r\n$4\r\nname\r\n

into this:

["GET", "name"]

Now the command engine can execute it.

Example: SET Command in RESP

A command like:


SET name Alice

becomes:


*3\r\n$3\r\nSET\r\n$4\r\nname\r\n$5\r\nAlice\r\n

Breakdown:

*3

There are 3 parts in the command.


$3

SET

First argument is SET.


$4

name

Second argument is name.


$5

Alice

Third argument is Alice.

The parser converts this into:


["SET", "name", "Alice"]

Then the server can route it to the SET command handler.

RESP Data Types

RESP is not only used for commands.

It is also used for server responses.

Some common RESP types are:


+ Simple Strings

- Errors

: Integers

$ Bulk Strings

* Arrays

For example, when Redis replies with OK:


+OK\r\n

When a key does not exist:


$-1\r\n

When returning an integer:


:1\r\n

When returning an error:


-ERR unknown command\r\n

When returning an array:


*2\r\n$5\r\nhello\r\n$5\r\nworld\r\n

This is why a Redis clone needs both:


RESP parser

RESP encoder

The parser reads client commands.

The encoder sends Redis-compatible responses back.

Why Not Just Split by Spaces?

At the beginning, it is tempting to parse commands like this:


const parts = input.trim().split(" ");

For very simple commands, this might work.

Example:


GET name

But it breaks quickly.

What about values with spaces?


SET message "hello world"

What about binary-safe strings?

What about multiple commands arriving together?

What about half a command arriving in one TCP chunk and the rest arriving later?

Redis needs a protocol that can handle all of this reliably.

RESP solves this by including length information.

Instead of guessing where a value ends, the server reads exactly the number of bytes specified.

For example:


$11

hello world

The server knows the value is exactly 11 bytes long.

That makes RESP binary-safe and much more reliable than splitting strings by spaces.

The Hard Part: TCP Is a Stream

The biggest lesson while implementing RESP was this:

TCP is stream-based, not message-based.

That means the server does not always receive one complete command at a time.

A client might send:


*2\r\n$3\r\nGET\r\n$4\r\nname\r\n

But the server may receive it in pieces.

For example, first chunk:


*2\r\n$3\r\nGE

Second chunk:


T\r\n$4\r\nname\r\n

Or the opposite can happen.

Multiple commands may arrive in a single chunk:


*1\r\n$4\r\nPING\r\n*2\r\n$3\r\nGET\r\n$4\r\nname\r\n

So the parser cannot assume:


one socket data event = one command

That assumption is wrong.

Instead, the parser needs to maintain an internal buffer.

How Incremental Parsing Works

In my Redis clone, the RESP parser works incrementally.

The idea is:


1. Receive a chunk from the socket

2. Add it to a buffer

3. Try to parse a complete command

4. If the command is incomplete, wait for more data

5. If one command is complete, return it

6. If more data remains, keep parsing

The parser has to be patient.

If the buffer does not contain enough data yet, it should not throw an error immediately.

It should simply wait for the next chunk.

For example:


*2\r\n$3\r\nGET\r\n$4\r\nna

This is not invalid.

It is just incomplete.

The parser should remember it and continue when the next chunk arrives:


me\r\n

Then it can produce:


["GET", "name"]

This was one of the first moments where the project felt like real systems engineering.

The problem was not just parsing strings.

The problem was parsing an ongoing network stream safely.

Parser Flow

At a high level, the parser does this:


Socket receives bytes

        ↓

Append bytes to parser buffer

        ↓

Check first RESP marker

        ↓

Parse array length

        ↓

Parse each bulk string

        ↓

If complete, emit command

        ↓

Remove parsed bytes from buffer

        ↓

Keep remaining bytes for next parse

For a command like:


*3\r\n$3\r\nSET\r\n$4\r\nname\r\n$5\r\nAlice\r\n

the parser needs to read:


array length = 3

bulk string length = 3

value = SET

bulk string length = 4

value = name

bulk string length = 5

value = Alice

Only then can it emit:


["SET", "name", "Alice"]

How This Fits Into the Redis Clone

Once the RESP parser produces a command array, the rest of the server can work with normal JavaScript data.

The flow becomes:


Raw TCP bytes

    ↓

RESP parser

    ↓

Command array

    ↓

Command router

    ↓

Command handler

    ↓

Storage / persistence / replication

    ↓

RESP response

    ↓

Socket write

So when the client sends:


SET name Alice

the internal flow is:


*3\r\n$3\r\nSET\r\n$4\r\nname\r\n$5\r\nAlice\r\n

        ↓

["SET", "name", "Alice"]

        ↓

SET handler

        ↓

store name = Alice

        ↓

append to AOF

        ↓

update snapshot

        ↓

propagate to replicas

        ↓

+OK\r\n

This is what made me appreciate Redis more.

A simple command goes through many layers before the user sees OK.

RESP Responses

Parsing client commands is only half of the protocol.

The server also needs to send valid RESP responses.

For example:

For PING:


+PONG\r\n

For successful SET:


+OK\r\n

For GET name when the value is Alice:


$5\r\nAlice\r\n

For a missing key:


$-1\r\n

For DEL name returning one deleted key:


:1\r\n

For errors:


-ERR wrong number of arguments\r\n

This matters because Redis clients expect protocol-compatible responses.

If the response format is wrong, the client cannot understand the server.

So the clone needed RESP encoding helpers, not just command logic.

Things I Learned While Building the Parser

1. Protocol design removes ambiguity

Without RESP, parsing commands with spaces, quotes, or binary data becomes messy.

With RESP, the server knows exactly how many bytes to read.

2. TCP does not care about your command boundaries

TCP gives you a stream of bytes.

It does not promise that each command arrives separately.

The application protocol has to define message boundaries.

3. A parser needs state

The parser must remember incomplete data between chunks.

That buffer is what makes incremental parsing possible.

4. Good parsing keeps the rest of the system clean

Once the parser returns:


["SET", "name", "Alice"]

the command handlers do not need to care about raw TCP bytes.

That separation makes the server easier to build.

5. RESP is simple but powerful

RESP is readable enough to debug manually, but structured enough to support real clients and binary-safe values.

That is a great balance.

Why This Part Matters

It is easy to think the database starts at the storage layer.

But actually, the database starts at the protocol layer.

Before Redis can store a key, it has to understand the command.

Before it can return a value, it has to encode the response.

Before it can support clients reliably, it has to handle partial messages, multiple messages, and malformed input.

That is why implementing RESP was one of the most important parts of this project.

It turned a simple TCP server into something Redis-compatible.

Final Thought

Before this project, I saw Redis commands as text commands:


SET key value

GET key

After implementing RESP, I started seeing them as structured messages flowing over TCP.

That shift changed how I think about backend systems.

A protocol is not just a format.

It is the contract between two systems.

And in Redis, RESP is that contract.

In the next post, I’ll go deeper into the in-memory storage layer: how strings, lists, hashes, and expiry are represented internally.

Repo:https://github.com/Abhinov007/redis_clone

Live sandbox:https://abhinov007.github.io/Redis_Clone/

I Built a Redis Clone from Scratch in Node.js — RESP, Persistence, Replication, Pub/Sub and Transactions

Abhinov007 — Wed, 17 Jun 2026 08:40:00 +0000

Most of us use Redis like this:

SET name Alice
GET name

It feels simple.

You send a command.
Redis stores something.
You get the value back.

But I wanted to understand what actually happens behind that simplicity.
So I built a Redis-compatible server from scratch in Node.js.

Not a wrapper around Redis.
Not a client library.
Not a toy object in memory.

A real TCP server that can understand Redis-style commands, parse the RESP protocol, store data in memory, persist writes, support transactions, handle Pub/Sub, and replicate data from master to replica.

The repo includes a Node.js Redis server, a CLI client, a RESP parser, storage modules, persistence modules, replication logic, Pub/Sub, tests, and a React sandbox for visualizing internals.
Repo

Live Sandbox

Why I Built This
I have used Redis in multiple projects for caching, queues, sessions, rate limiting, and real-time features.

But I realized something:
I knew how to use Redis, but I did not deeply understand how Redis works.

SET session:123 abc EX 60

This one command looks tiny.
But internally, a Redis-like server has to do a lot:

Accept a TCP connection
Read raw bytes from the socket
Parse the command format
Validate the command
Store the key-value pair
Track expiry metadata
Append the write to persistence
Return a protocol-compatible response
Potentially propagate the write to replicas

That is a lot of systems engineering hidden behind one line.
So the goal of this project was simple:
Rebuild enough of Redis from scratch to understand the core ideas behind it.

*What I Built
*
The project is a Redis-compatible server built in Node.js. The README describes it as implementing the RESP protocol, dual-layer persistence with AOF and RDB, master/replica replication, Pub/Sub messaging, transactions, and an interactive React sandbox.

The server supports:

Strings
Lists
Hashes
Key expiry
PING
DEL
FLUSHALL
Transactions
Pub/Sub
Replication
Persistence

Some example commands:

SET name Alice EX 60
GET name

LPUSH queue task1 task2 task3
LRANGE queue 0 -1

HSET user:1 name Bob age 30
HGET user:1 name

MULTI
SET a 1
SET b 2
EXEC

SUBSCRIBE news
PUBLISH news "Hello World"

The README lists support for strings, lists, hashes, key expiry, PING, DEL, and FLUSHALL as part of the core command set.

The Architecture

At a high level, the server looks like this:

`TCP Client
   ↓
Node.js TCP Server
   ↓
RESP Parser
   ↓
Command Router
   ↓
Command Handler
   ↓
In-Memory Store
   ↓
Persistence / Replication / Response`

When a client sends:
SET name Alice

the server does roughly this:

Receive bytes over TCP
Parse the bytes into a command
Identify SET as the command
Validate the arguments
Store name = Alice in memory
Append the write to AOF
Update the RDB-style snapshot
Propagate the command to replicas
Send +OK back to the client

That flow made me realize that a database is not just “storage”.

Networking
Protocol parsing
Command execution
Memory management
Persistence
Replication
Client state
Failure handling

Layer 1: TCP Server

The first layer was the TCP server.
Redis does not work like a normal REST API.

There is no:

POST /set
GET /get

Instead, clients connect to a TCP port and send protocol-formatted commands.

In this project, the server runs on port 6379 by default, similar to Redis.

Example:

node server.js

or:

node server.js --port 6380

The server listens for socket connections, receives chunks of data, parses them, executes commands, and writes responses back to the client.

This was the first important lesson:

A database server is also a network server.

Layer 2: RESP Protocol

Redis clients communicate using RESP, the Redis Serialization Protocol.

That means the server cannot just read plain strings and split them by spaces forever.

It needs to understand structured protocol messages.

For example, a Redis command can be represented like this:

*2
$3
GET
$4
name

That means:

["GET", "name"]

The parser has to understand arrays, bulk strings, simple strings, integers, and errors.

The interesting part is TCP chunking.

A command may arrive like this:

*2\r\n$3\r\nGET\r\n

and then later:

$4\r\nname\r\n

Or multiple commands may arrive together in a single chunk.

So the parser cannot assume that every socket event equals one full command.

It needs to buffer incomplete data and continue parsing once more bytes arrive.

That was one of the first places where this stopped feeling like a normal backend project and started feeling like a systems project.

Layer 3: In-Memory Store

Once the command is parsed, the server needs somewhere to store data.

At the core, Redis is an in-memory data store.

So I built a storage layer around an in-memory Map.

But the store cannot treat every value the same way.

A string is different from a list.
A list is different from a hash.
A key with expiry is different from a normal key.

So the storage layer had to track:

key
value
type
expiry metadata

Supported data types include:

String List Hash

The README describes the server as supporting three data types: strings, lists, and hashes.

Example string command:

SET name Alice
GET name

Example list command:

LPUSH queue task1 task2
RPOP queue
LRANGE queue 0 -1

Example hash command:

HSET user:1 name Bob age 30
HGET user:1 name
HGETALL user:1

This layer taught me why Redis has strict type behavior.

For example, if a key is storing a string, you should not be able to run a list command on it.

That kind of validation is what turns a basic key-value map into a real database-like system.

Layer 4: Expiry

Redis is heavily used for temporary data:

sessions
OTP codes
cache entries
rate limits
temporary locks

So expiry support was important.

The clone supports commands like:

SET name Alice EX 60

That means the key should automatically expire after 60 seconds.

There are two ways to think about expiry:

Active expiry: background cleanup
Lazy expiry: delete only when accessed

In this project, expiry metadata is tracked separately, and keys can be checked when accessed.

So when a user runs:

GET name

the server checks:

Does this key exist?
Does it have expiry metadata?
Has the expiry time passed?
If yes, delete and return null.
If no, return the value.

This sounds small, but expiry affects many parts of the system:

GET should respect expiry
DEL should remove expiry metadata
Persistence should preserve expiry information
Replication should handle expiring keys correctly
The sandbox should show TTL countdowns

Layer 5: Persistence

An in-memory store is fast, but if the process dies, memory disappears.

So I added persistence.

The project implements two persistence styles:

AOF
RDB

AOF means Append Only File.

Every write command gets appended to a log file.

Example:

SET name Alice
LPUSH queue task1
HSET user:1 name Bob

On restart, the server can replay the commands and rebuild the database.

RDB is snapshot-style persistence.

Instead of replaying every command, the server stores a full snapshot of the current database.

The README describes this as dual-layer persistence: AOF writes every command to database.aof, while RDB writes a JSON snapshot to dump.rdb.

This helped me understand the tradeoff:

AOF gives a detailed write history.
RDB gives a compact snapshot.
AOF can be more durable.
RDB can be faster to load.

In real Redis, these persistence modes have many more details.

But even implementing a simplified version made the core design much clearer.

Layer 6: Transactions

Redis supports transactions using:

MULTI
EXEC
DISCARD

So I implemented that too.

The idea is:

MULTI starts a transaction block.
Commands are queued instead of executed immediately.
EXEC runs the queued commands.
DISCARD cancels them.

Example:

MULTI
SET name Alice
GET name
EXEC

The important thing here is that transaction state is per client.

One client may be inside a transaction.
Another client may not be.

That means the server needs to maintain client-specific state, not just global database state.

The README lists MULTI, EXEC, and DISCARD as supported transaction commands.

This changed how I thought about the server.

Earlier, I saw it as:

command comes in → execute command

After transactions, it became:

command comes in
    ↓
is this client inside MULTI?
    ↓
if yes, queue command
if no, execute command

That is a much more realistic server model.

Layer 7: Pub/Sub

The next feature was Pub/Sub.

Redis Pub/Sub lets clients subscribe to channels and receive messages when someone publishes to that channel.

Example:

SUBSCRIBE news

Then another client can run:

PUBLISH news "Hello World"

Internally, this means the server needs a mapping like:

news -> [socket1, socket2, socket3]
alerts -> [socket4, socket5]

When a message is published, the server finds all subscribers and pushes the message to their sockets.

The README lists SUBSCRIBE, UNSUBSCRIBE, and PUBLISH, with channel-to-subscriber socket mapping and cleanup on disconnect.

This was one of the most fun parts of the project because it moved the server beyond request-response.

Normal commands are like:

client asks → server replies

Pub/Sub is different:

client subscribes
server remembers the socket
later, server pushes messages to it

That makes the server feel alive.

Layer 8: Replication

Replication was the hardest and most interesting part.

The clone supports master-replica replication.

You can run one server as master:

node server.js --port 6379

And another as replica:

node server.js --port 6380 --replicaof localhost 6379

Writes on the master are propagated to the replica. The README also describes full sync with an RDB snapshot and partial resync on reconnect.

At a high level:

Replica connects to master
Replica performs handshake
Master sends snapshot
Master streams future write commands
Replica applies writes
Replica stays read-only for normal clients

The project includes:

master replication ID
1 MB circular backlog
full resync
partial resync
replica handshake state machine
auto-reconnect
REPLCONF ACK heartbeats
read-only replica mode

Those replication details are listed in the README’s replication section.

This was where I realized that replication is not just “copy data to another server”.

It involves:

connection state
offset tracking
backlogs
snapshots
handshakes
heartbeats
reconnect behavior
read-only enforcement
command propagation

Replication turned this from a key-value store into a distributed systems project.

Layer 9: React Sandbox

After building the server, I wanted a way to make the internals easier to understand.

So I built a React + Vite sandbox that simulates the server in the browser.

The sandbox lets you type Redis commands and watch internal state update live.

It shows:

Database
Expiry store
AOF log
RDB snapshot
Pub/Sub channels

The README says the sandbox simulates the server entirely in the browser and includes tabs for database, expiry, AOF log, RDB snapshot, and Pub/Sub.

This is useful because most backend internals are invisible.

You run:

SET name Alice EX 60

and normally you only see:

But in the sandbox, you can see:

The key appears in the database
The TTL countdown starts
The AOF log updates
The RDB snapshot changes

That made the project not just functional, but explainable.