DEV Community: Abhinov007

Building the In-Memory Store: Strings, Lists, Hashes and Expiry

Abhinov007 — Fri, 19 Jun 2026 17:20:10 +0000

In the previous post, I wrote about RESP — the protocol layer that lets a Redis server understand commands coming over TCP.

But parsing a command is only the first step.

Once the server receives something like:

SET name Alice

and the RESP parser converts it into:

["SET", "name", "Alice"]

the next question is:

Where does this data actually live?

That is what this post is about.

In this part of my Redis clone, I built the in-memory storage layer that supports:

Strings
Lists
Hashes
Key expiry
Type validation
Lazy expiry on access

This is the layer where the server starts feeling like an actual database.

Why the Storage Layer Matters

At first, a Redis clone sounds like it could just be:

const store = {};

or:

const store = new Map();

And for the most basic version, that is true.

If all you want is:

SET name Alice
GET name

then a simple key-value map works.

But Redis is not just a plain key-value object.

Redis has data types.

A key can store a string.

Another key can store a list.

Another key can store a hash.

Some keys may expire after 10 seconds.

Some commands should only work on specific data types.

So the storage layer needs to track more than just:

key -> value

It needs to track:

key -> value
key -> type
key -> expiry time

That is what turns a simple JavaScript object into a Redis-like in-memory database.

The Basic Data Model

The simplest mental model for the store is:

database:
  name      -> Alice
  queue     -> [task1, task2, task3]
  user:1    -> { name: Bob, age: 30 }

expiry:
  session   -> expires at timestamp

So internally, the system has two major responsibilities:

Store the actual data.
Track when keys should expire.

I kept these concerns separate.

The database module focuses on storing and retrieving values.

The expiry module focuses on TTL metadata.

This separation makes the design easier to reason about.

Strings

Strings are the simplest Redis data type.

Example:

SET name Alice
GET name

The server stores:

name -> Alice

When the client runs:

GET name

the server checks:

Does the key exist?
Has the key expired?
Is the key storing a string?
If yes, return the value.

The response is encoded back in RESP format.

For example, if the value is Alice, the server returns:

$5\r\nAlice\r\n

If the key does not exist, it returns a null bulk string:

$-1\r\n

The string data type is simple, but it becomes important because many other commands depend on the same storage rules:

type checking
expiry checking
persistence
replication
response encoding

A simple GET command still passes through multiple layers of the system.

SET with Expiry

Redis allows setting keys with expiry:

SET session abc EX 60

This means:

Store session = abc
Expire it after 60 seconds

In my Redis clone, the value is stored in the main database, while the expiry timestamp is stored separately.

So conceptually:

database:
  session -> abc

expiry:
  session -> currentTime + 60 seconds

This separation makes expiry easier to manage.

The value does not need to know about its own expiry.

The expiry system handles that concern.

Lazy Expiry

There are two common ways to expire keys:

Active expiry
Lazy expiry

Active expiry means a background process keeps scanning for expired keys and deletes them.

Lazy expiry means the server checks whether a key has expired only when someone tries to access it.

For example:

SET token abc EX 10

After 10 seconds, the key is expired.

But instead of deleting it immediately in the background, the server can wait until a client runs:

GET token

At that moment, the server checks:

Does token have an expiry timestamp?
Is the current time greater than the expiry timestamp?
If yes, delete token and return null.

So the flow becomes:

GET token
   ↓
check expiry
   ↓
expired?
   ↓
delete key + expiry metadata
   ↓
return null

This approach keeps the system simpler.

It also taught me an important database design idea:

Sometimes cleanup does not have to happen immediately. It just has to happen before the data is observed.

Why Expiry Affects Everything

Expiry sounds like a small feature, but it touches many parts of the database.

For example:

GET

Before returning a key, the server must check whether it is expired.

DEL

When a key is deleted, its expiry metadata should also be removed.

FLUSHALL

When the database is cleared, the expiry store should also be cleared.

Persistence

If a key has expiry metadata, that information needs to be preserved or handled correctly during save/load.

Replication

If the master writes a key with expiry, the replica needs to receive the same write behavior.

Sandbox

If a key has TTL, the UI should show the countdown clearly.

So expiry is not just an extra field.

It becomes a cross-cutting concern.

Lists

The next data type I implemented was Lists.

Redis lists are useful for queues, stacks, timelines, and task buffers.

Supported commands include:

LPUSH queue a b c
RPUSH queue d e
LPOP queue
RPOP queue
LLEN queue
LRANGE queue 0 -1

A list is stored internally as an ordered array-like structure.

Example:

LPUSH queue task1
LPUSH queue task2

The list becomes:

queue -> [task2, task1]

Because LPUSH inserts on the left.

If we run:

RPUSH queue task3

the list becomes:

queue -> [task2, task1, task3]

This was useful because it forced me to think about command semantics.

LPUSH and RPUSH sound similar, but they mutate different ends of the list.

LPOP and RPOP also remove from different ends.

LRANGE and Index Handling

One of the more interesting list commands is:

LRANGE queue 0 -1

This means:

Return all elements from index 0 to the last element.

Redis supports negative indexes.

So:

LRANGE queue -2 -1

means:

Return the last two elements.

That means the clone needs to normalize indexes.

The server has to convert:

start = -2
stop = -1

into real array indexes based on the list length.

This small feature makes the implementation more realistic.

It is not just pushing and popping.

It is matching Redis-like behavior.

Hashes

Hashes are another important Redis data type.

They let you store field-value pairs under a single key.

Example:

HSET user:1 name Bob age 30
HGET user:1 name
HGETALL user:1

Conceptually:

user:1 -> {
  name: Bob,
  age: 30
}

This is useful for storing object-like data.

In the clone, hashes support commands like:

HSET user:1 name Bob age 30
HGET user:1 name
HDEL user:1 age
HGETALL user:1
HLEN user:1
HEXISTS user:1 name

The interesting part here was handling multiple fields in one command.

For example:

HSET user:1 name Bob age 30 city Delhi

This command contains multiple field-value pairs.

The command handler needs to validate that fields and values are paired correctly.

If there is a missing value, the command should return an error.

That kind of validation is what makes the command engine feel closer to a real Redis server.

Type Checking

One of the most important parts of the storage layer is type checking.

Imagine this:

SET name Alice
LPUSH name Bob

This should not work.

The key name already stores a string.

A list command should not be allowed on it.

So the server must return a wrong type error instead of silently converting the value.

This means each key needs an associated type.

Conceptually:

name:
  type: string
  value: Alice

queue:
  type: list
  value: [task1, task2]

user:1:
  type: hash
  value: { name: Bob }

Before executing a command, the handler checks whether the key has the expected type.

For example:

GET expects string
LPUSH expects list
HGET expects hash

If the key does not exist, some commands create it.

If the key exists with the wrong type, the command returns an error.

This was one of the key differences between a simple map and a Redis-like store.

Command Flow Example: SET

Let’s walk through what happens when a client sends:

SET name Alice EX 60

The flow looks like this:

RESP parser receives raw bytes
        ↓
Parser emits ["SET", "name", "Alice", "EX", "60"]
        ↓
Command router identifies SET
        ↓
SET handler validates arguments
        ↓
Database stores name = Alice
        ↓
Type is marked as string
        ↓
Expiry store records TTL
        ↓
AOF persistence records the write
        ↓
RDB snapshot is updated
        ↓
Replication layer can propagate the write
        ↓
Server returns +OK

So one command touches:

protocol parsing
command validation
storage
expiry
persistence
replication
response encoding

That is why even “simple” Redis commands are great for learning systems design.

Command Flow Example: LPUSH

Now take:

LPUSH queue task1 task2

The server does:

Parse command
        ↓
Check whether queue exists
        ↓
If it does not exist, create a new list
        ↓
If it exists, verify it is a list
        ↓
Push values to the left
        ↓
Persist the write
        ↓
Return the new list length

If queue does not exist, it is created.

If queue already stores a list, the command mutates it.

If queue stores a string or hash, the command returns a type error.

This type behavior keeps the database predictable.

Command Flow Example: HSET

For:

HSET user:1 name Bob age 30

the server does:

Parse command
        ↓
Check whether user:1 exists
        ↓
If not, create a hash
        ↓
Validate field-value pairs
        ↓
Set name = Bob
        ↓
Set age = 30
        ↓
Return number of new fields added

Hashes were interesting because they introduced nested storage.

A string stores one value.

A list stores an ordered sequence.

A hash stores a map inside the main map.

That means the storage layer becomes:

database map
   ↓
key
   ↓
hash map
   ↓
field -> value

This helped me understand why Redis data types are powerful.

They let you model different kinds of data without leaving the key-value model.

Storage and Persistence

The storage layer does not live alone.

Every successful write needs to be persisted.

For example:

SET name Alice
LPUSH queue task1
HSET user:1 name Bob

These commands should update memory, but they should also be recorded by the persistence system.

That way, if the server restarts, the database can be rebuilt.

This is where AOF and RDB connect to storage.

The storage layer owns the current state.

The persistence layer records or snapshots that state.

The relationship looks like:

Command handler
   ↓
Storage update
   ↓
AOF append
   ↓
RDB snapshot

This made the architecture cleaner because storage does not need to know how commands arrived.

It only needs to expose operations that command handlers can use.

Storage and Replication

Replication also depends on successful writes.

When a write happens on the master, it needs to be propagated to replicas.

So after a command updates storage, the server can send the same write command to replica connections.

The flow becomes:

Client writes to master
        ↓
Master updates in-memory store
        ↓
Master persists write
        ↓
Master sends command to replicas
        ↓
Replicas apply the same write

This is where command design matters.

If the command is represented clearly, it can be reused for:

local execution
persistence replay
replication propagation

That was an important architectural learning.

Storage and the React Sandbox

The React sandbox made the storage layer much easier to understand visually.

Instead of only seeing:

+OK

after running a command, the sandbox shows the internal changes.

For example:

SET name Alice EX 60

updates:

Database tab
Expiry tab
AOF log
RDB snapshot

For a list command:

LPUSH queue task1 task2

the database view shows the list changing.

For a hash command:

HSET user:1 name Bob age 30

the database view shows the hash fields.

This made the project more explainable.

It is one thing to say “the database changed.”

It is much better to see how it changed.

What I Learned

1. A key-value store is not always simple

At the beginning, I thought the storage layer would be the easiest part.

But once data types, expiry, type checking, persistence, and replication are added, it becomes much more interesting.

2. Types matter

Redis commands are simple because Redis is strict about what each key contains.

A list command should not work on a string.

A hash command should not work on a list.

That strictness makes the system predictable.

3. Expiry is a system-wide concern

TTL is not just attached to SET.

It affects reads, deletes, persistence, replication, and UI visualization.

4. Command semantics are important

Commands like LPUSH, RPUSH, LRANGE, HSET, and HGETALL all have small behavior details that need to be handled carefully.

5. Separating storage and expiry keeps the design clean

The database stores values.

The expiry module stores TTL metadata.

This makes both parts easier to test and reason about.

6. A database is a collection of layers

The in-memory store is only one layer.

It sits between protocol parsing, command execution, persistence, replication, and client responses.

Final Thought

Before building this, I thought of Redis mainly as:

key -> value

After building the storage layer, I started seeing it more like:

key -> typed value
key -> expiry metadata
commands -> validated mutations
writes -> persistence + replication

That shift matters.

A Redis-like database is not just a map.

It is a carefully designed system where every command has rules, every key has type behavior, and every write can affect persistence, replication, and expiry.

That is what made this part of the project so valuable.

In the next post, I’ll go deeper into persistence: how AOF and RDB work, why both exist, and what I learned while implementing them.

Repo:

https://github.com/Abhinov007/redis_clone

Live sandbox:

https:https://redis-clone.vercel.app/

Understanding RESP: The Protocol Behind Redis

Abhinov007 — Thu, 18 Jun 2026 08:53:24 +0000

In the first post, I shared the overview of my Redis clone built from scratch in Node.js.

In this post, I want to go one layer deeper into one of the most important parts of the project:

RESP — the Redis Serialization Protocol.

Before my Redis clone could support commands like:

SET name Alice
GET name
LPUSH queue task1

it first needed to understand what the client was actually sending over the network.
That is where RESP comes in.

Why Redis Needs a Protocol

When we use Redis from a CLI or client library, we usually think in commands:

SET name Alice

But Redis does not internally receive a nice JavaScript array like:

["SET", "name", "Alice"]

It receives raw bytes over a TCP socket.

Those bytes need structure.

The server must know:

where the command starts
where the command ends
how many arguments exist
how long each argument is
whether the value is a string, integer, error, array, or bulk string

That structure is provided by RESP.

RESP is the language Redis clients and Redis servers use to talk to each other.

What RESP Looks Like

A command like:

GET name

is represented in RESP as:

*2\r\n$3\r\nGET\r\n$4\r\nname\r\n

At first, this looks strange.

But once broken down, it is simple.

*2

means this is an array with 2 elements.

$3
GET

means the first bulk string has length 3, and its value is GET.

$4 name

means the second bulk string has length 4, and its value is name.

So the server parses this:

*2\r\n$3\r\nGET\r\n$4\r\nname\r\n

into this:

["GET", "name"]

Now the command engine can execute it.

Example: SET Command in RESP

A command like:


SET name Alice

becomes:


*3\r\n$3\r\nSET\r\n$4\r\nname\r\n$5\r\nAlice\r\n

Breakdown:

*3

There are 3 parts in the command.


$3

SET

First argument is SET.


$4

name

Second argument is name.


$5

Alice

Third argument is Alice.

The parser converts this into:


["SET", "name", "Alice"]

Then the server can route it to the SET command handler.

RESP Data Types

RESP is not only used for commands.

It is also used for server responses.

Some common RESP types are:


+ Simple Strings

- Errors

: Integers

$ Bulk Strings

* Arrays

For example, when Redis replies with OK:


+OK\r\n

When a key does not exist:


$-1\r\n

When returning an integer:


:1\r\n

When returning an error:


-ERR unknown command\r\n

When returning an array:


*2\r\n$5\r\nhello\r\n$5\r\nworld\r\n

This is why a Redis clone needs both:


RESP parser

RESP encoder

The parser reads client commands.

The encoder sends Redis-compatible responses back.

Why Not Just Split by Spaces?

At the beginning, it is tempting to parse commands like this:


const parts = input.trim().split(" ");

For very simple commands, this might work.

Example:


GET name

But it breaks quickly.

What about values with spaces?


SET message "hello world"

What about binary-safe strings?

What about multiple commands arriving together?

What about half a command arriving in one TCP chunk and the rest arriving later?

Redis needs a protocol that can handle all of this reliably.

RESP solves this by including length information.

Instead of guessing where a value ends, the server reads exactly the number of bytes specified.

For example:


$11

hello world

The server knows the value is exactly 11 bytes long.

That makes RESP binary-safe and much more reliable than splitting strings by spaces.

The Hard Part: TCP Is a Stream

The biggest lesson while implementing RESP was this:

TCP is stream-based, not message-based.

That means the server does not always receive one complete command at a time.

A client might send:


*2\r\n$3\r\nGET\r\n$4\r\nname\r\n

But the server may receive it in pieces.

For example, first chunk:


*2\r\n$3\r\nGE

Second chunk:


T\r\n$4\r\nname\r\n

Or the opposite can happen.

Multiple commands may arrive in a single chunk:


*1\r\n$4\r\nPING\r\n*2\r\n$3\r\nGET\r\n$4\r\nname\r\n

So the parser cannot assume:


one socket data event = one command

That assumption is wrong.

Instead, the parser needs to maintain an internal buffer.

How Incremental Parsing Works

In my Redis clone, the RESP parser works incrementally.

The idea is:


1. Receive a chunk from the socket

2. Add it to a buffer

3. Try to parse a complete command

4. If the command is incomplete, wait for more data

5. If one command is complete, return it

6. If more data remains, keep parsing

The parser has to be patient.

If the buffer does not contain enough data yet, it should not throw an error immediately.

It should simply wait for the next chunk.

For example:


*2\r\n$3\r\nGET\r\n$4\r\nna

This is not invalid.

It is just incomplete.

The parser should remember it and continue when the next chunk arrives:


me\r\n

Then it can produce:


["GET", "name"]

This was one of the first moments where the project felt like real systems engineering.

The problem was not just parsing strings.

The problem was parsing an ongoing network stream safely.

Parser Flow

At a high level, the parser does this:


Socket receives bytes

        ↓

Append bytes to parser buffer

        ↓

Check first RESP marker

        ↓

Parse array length

        ↓

Parse each bulk string

        ↓

If complete, emit command

        ↓

Remove parsed bytes from buffer

        ↓

Keep remaining bytes for next parse

For a command like:


*3\r\n$3\r\nSET\r\n$4\r\nname\r\n$5\r\nAlice\r\n

the parser needs to read:


array length = 3

bulk string length = 3

value = SET

bulk string length = 4

value = name

bulk string length = 5

value = Alice

Only then can it emit:


["SET", "name", "Alice"]

How This Fits Into the Redis Clone

Once the RESP parser produces a command array, the rest of the server can work with normal JavaScript data.

The flow becomes:


Raw TCP bytes

    ↓

RESP parser

    ↓

Command array

    ↓

Command router

    ↓

Command handler

    ↓

Storage / persistence / replication

    ↓

RESP response

    ↓

Socket write

So when the client sends:


SET name Alice

the internal flow is:


*3\r\n$3\r\nSET\r\n$4\r\nname\r\n$5\r\nAlice\r\n

        ↓

["SET", "name", "Alice"]

        ↓

SET handler

        ↓

store name = Alice

        ↓

append to AOF

        ↓

update snapshot

        ↓

propagate to replicas

        ↓

+OK\r\n

This is what made me appreciate Redis more.

A simple command goes through many layers before the user sees OK.

RESP Responses

Parsing client commands is only half of the protocol.

The server also needs to send valid RESP responses.

For example:

For PING:


+PONG\r\n

For successful SET:


+OK\r\n

For GET name when the value is Alice:


$5\r\nAlice\r\n

For a missing key:


$-1\r\n

For DEL name returning one deleted key:


:1\r\n

For errors:


-ERR wrong number of arguments\r\n

This matters because Redis clients expect protocol-compatible responses.

If the response format is wrong, the client cannot understand the server.

So the clone needed RESP encoding helpers, not just command logic.

Things I Learned While Building the Parser

1. Protocol design removes ambiguity

Without RESP, parsing commands with spaces, quotes, or binary data becomes messy.

With RESP, the server knows exactly how many bytes to read.

2. TCP does not care about your command boundaries

TCP gives you a stream of bytes.

It does not promise that each command arrives separately.

The application protocol has to define message boundaries.

3. A parser needs state

The parser must remember incomplete data between chunks.

That buffer is what makes incremental parsing possible.

4. Good parsing keeps the rest of the system clean

Once the parser returns:


["SET", "name", "Alice"]

the command handlers do not need to care about raw TCP bytes.

That separation makes the server easier to build.

5. RESP is simple but powerful

RESP is readable enough to debug manually, but structured enough to support real clients and binary-safe values.

That is a great balance.

Why This Part Matters

It is easy to think the database starts at the storage layer.

But actually, the database starts at the protocol layer.

Before Redis can store a key, it has to understand the command.

Before it can return a value, it has to encode the response.

Before it can support clients reliably, it has to handle partial messages, multiple messages, and malformed input.

That is why implementing RESP was one of the most important parts of this project.

It turned a simple TCP server into something Redis-compatible.

Final Thought

Before this project, I saw Redis commands as text commands:


SET key value

GET key

After implementing RESP, I started seeing them as structured messages flowing over TCP.

That shift changed how I think about backend systems.

A protocol is not just a format.

It is the contract between two systems.

And in Redis, RESP is that contract.

In the next post, I’ll go deeper into the in-memory storage layer: how strings, lists, hashes, and expiry are represented internally.

Repo:https://github.com/Abhinov007/redis_clone

Live sandbox:https://abhinov007.github.io/Redis_Clone/

I Built a Redis Clone from Scratch in Node.js — RESP, Persistence, Replication, Pub/Sub and Transactions

Abhinov007 — Wed, 17 Jun 2026 08:40:00 +0000

Most of us use Redis like this:

SET name Alice
GET name

It feels simple.

You send a command.
Redis stores something.
You get the value back.

But I wanted to understand what actually happens behind that simplicity.
So I built a Redis-compatible server from scratch in Node.js.

Not a wrapper around Redis.
Not a client library.
Not a toy object in memory.

A real TCP server that can understand Redis-style commands, parse the RESP protocol, store data in memory, persist writes, support transactions, handle Pub/Sub, and replicate data from master to replica.

The repo includes a Node.js Redis server, a CLI client, a RESP parser, storage modules, persistence modules, replication logic, Pub/Sub, tests, and a React sandbox for visualizing internals.
Repo

Live Sandbox

Why I Built This
I have used Redis in multiple projects for caching, queues, sessions, rate limiting, and real-time features.

But I realized something:
I knew how to use Redis, but I did not deeply understand how Redis works.

SET session:123 abc EX 60

This one command looks tiny.
But internally, a Redis-like server has to do a lot:

Accept a TCP connection
Read raw bytes from the socket
Parse the command format
Validate the command
Store the key-value pair
Track expiry metadata
Append the write to persistence
Return a protocol-compatible response
Potentially propagate the write to replicas

That is a lot of systems engineering hidden behind one line.
So the goal of this project was simple:
Rebuild enough of Redis from scratch to understand the core ideas behind it.

*What I Built
*
The project is a Redis-compatible server built in Node.js. The README describes it as implementing the RESP protocol, dual-layer persistence with AOF and RDB, master/replica replication, Pub/Sub messaging, transactions, and an interactive React sandbox.

The server supports:

Strings
Lists
Hashes
Key expiry
PING
DEL
FLUSHALL
Transactions
Pub/Sub
Replication
Persistence

Some example commands:

SET name Alice EX 60
GET name

LPUSH queue task1 task2 task3
LRANGE queue 0 -1

HSET user:1 name Bob age 30
HGET user:1 name

MULTI
SET a 1
SET b 2
EXEC

SUBSCRIBE news
PUBLISH news "Hello World"

The README lists support for strings, lists, hashes, key expiry, PING, DEL, and FLUSHALL as part of the core command set.

The Architecture

At a high level, the server looks like this:

`TCP Client
   ↓
Node.js TCP Server
   ↓
RESP Parser
   ↓
Command Router
   ↓
Command Handler
   ↓
In-Memory Store
   ↓
Persistence / Replication / Response`

When a client sends:
SET name Alice

the server does roughly this:

Receive bytes over TCP
Parse the bytes into a command
Identify SET as the command
Validate the arguments
Store name = Alice in memory
Append the write to AOF
Update the RDB-style snapshot
Propagate the command to replicas
Send +OK back to the client

That flow made me realize that a database is not just “storage”.

Networking
Protocol parsing
Command execution
Memory management
Persistence
Replication
Client state
Failure handling

Layer 1: TCP Server

The first layer was the TCP server.
Redis does not work like a normal REST API.

There is no:

POST /set
GET /get

Instead, clients connect to a TCP port and send protocol-formatted commands.

In this project, the server runs on port 6379 by default, similar to Redis.

Example:

node server.js

or:

node server.js --port 6380

The server listens for socket connections, receives chunks of data, parses them, executes commands, and writes responses back to the client.

This was the first important lesson:

A database server is also a network server.

Layer 2: RESP Protocol

Redis clients communicate using RESP, the Redis Serialization Protocol.

That means the server cannot just read plain strings and split them by spaces forever.

It needs to understand structured protocol messages.

For example, a Redis command can be represented like this:

*2
$3
GET
$4
name

That means:

["GET", "name"]

The parser has to understand arrays, bulk strings, simple strings, integers, and errors.

The interesting part is TCP chunking.

A command may arrive like this:

*2\r\n$3\r\nGET\r\n

and then later:

$4\r\nname\r\n

Or multiple commands may arrive together in a single chunk.

So the parser cannot assume that every socket event equals one full command.

It needs to buffer incomplete data and continue parsing once more bytes arrive.

That was one of the first places where this stopped feeling like a normal backend project and started feeling like a systems project.

Layer 3: In-Memory Store

Once the command is parsed, the server needs somewhere to store data.

At the core, Redis is an in-memory data store.

So I built a storage layer around an in-memory Map.

But the store cannot treat every value the same way.

A string is different from a list.
A list is different from a hash.
A key with expiry is different from a normal key.

So the storage layer had to track:

key
value
type
expiry metadata

Supported data types include:

String List Hash

The README describes the server as supporting three data types: strings, lists, and hashes.

Example string command:

SET name Alice
GET name

Example list command:

LPUSH queue task1 task2
RPOP queue
LRANGE queue 0 -1

Example hash command:

HSET user:1 name Bob age 30
HGET user:1 name
HGETALL user:1

This layer taught me why Redis has strict type behavior.

For example, if a key is storing a string, you should not be able to run a list command on it.

That kind of validation is what turns a basic key-value map into a real database-like system.

Layer 4: Expiry

Redis is heavily used for temporary data:

sessions
OTP codes
cache entries
rate limits
temporary locks

So expiry support was important.

The clone supports commands like:

SET name Alice EX 60

That means the key should automatically expire after 60 seconds.

There are two ways to think about expiry:

Active expiry: background cleanup
Lazy expiry: delete only when accessed

In this project, expiry metadata is tracked separately, and keys can be checked when accessed.

So when a user runs:

GET name

the server checks:

Does this key exist?
Does it have expiry metadata?
Has the expiry time passed?
If yes, delete and return null.
If no, return the value.

This sounds small, but expiry affects many parts of the system:

GET should respect expiry
DEL should remove expiry metadata
Persistence should preserve expiry information
Replication should handle expiring keys correctly
The sandbox should show TTL countdowns

Layer 5: Persistence

An in-memory store is fast, but if the process dies, memory disappears.

So I added persistence.

The project implements two persistence styles:

AOF
RDB

AOF means Append Only File.

Every write command gets appended to a log file.

Example:

SET name Alice
LPUSH queue task1
HSET user:1 name Bob

On restart, the server can replay the commands and rebuild the database.

RDB is snapshot-style persistence.

Instead of replaying every command, the server stores a full snapshot of the current database.

The README describes this as dual-layer persistence: AOF writes every command to database.aof, while RDB writes a JSON snapshot to dump.rdb.

This helped me understand the tradeoff:

AOF gives a detailed write history.
RDB gives a compact snapshot.
AOF can be more durable.
RDB can be faster to load.

In real Redis, these persistence modes have many more details.

But even implementing a simplified version made the core design much clearer.

Layer 6: Transactions

Redis supports transactions using:

MULTI
EXEC
DISCARD

So I implemented that too.

The idea is:

MULTI starts a transaction block.
Commands are queued instead of executed immediately.
EXEC runs the queued commands.
DISCARD cancels them.

Example:

MULTI
SET name Alice
GET name
EXEC

The important thing here is that transaction state is per client.

One client may be inside a transaction.
Another client may not be.

That means the server needs to maintain client-specific state, not just global database state.

The README lists MULTI, EXEC, and DISCARD as supported transaction commands.

This changed how I thought about the server.

Earlier, I saw it as:

command comes in → execute command

After transactions, it became:

command comes in
    ↓
is this client inside MULTI?
    ↓
if yes, queue command
if no, execute command

That is a much more realistic server model.

Layer 7: Pub/Sub

The next feature was Pub/Sub.

Redis Pub/Sub lets clients subscribe to channels and receive messages when someone publishes to that channel.

Example:

SUBSCRIBE news

Then another client can run:

PUBLISH news "Hello World"

Internally, this means the server needs a mapping like:

news -> [socket1, socket2, socket3]
alerts -> [socket4, socket5]

When a message is published, the server finds all subscribers and pushes the message to their sockets.

The README lists SUBSCRIBE, UNSUBSCRIBE, and PUBLISH, with channel-to-subscriber socket mapping and cleanup on disconnect.

This was one of the most fun parts of the project because it moved the server beyond request-response.

Normal commands are like:

client asks → server replies

Pub/Sub is different:

client subscribes
server remembers the socket
later, server pushes messages to it

That makes the server feel alive.

Layer 8: Replication

Replication was the hardest and most interesting part.

The clone supports master-replica replication.

You can run one server as master:

node server.js --port 6379

And another as replica:

node server.js --port 6380 --replicaof localhost 6379

Writes on the master are propagated to the replica. The README also describes full sync with an RDB snapshot and partial resync on reconnect.

At a high level:

Replica connects to master
Replica performs handshake
Master sends snapshot
Master streams future write commands
Replica applies writes
Replica stays read-only for normal clients

The project includes:

master replication ID
1 MB circular backlog
full resync
partial resync
replica handshake state machine
auto-reconnect
REPLCONF ACK heartbeats
read-only replica mode

Those replication details are listed in the README’s replication section.

This was where I realized that replication is not just “copy data to another server”.

It involves:

connection state
offset tracking
backlogs
snapshots
handshakes
heartbeats
reconnect behavior
read-only enforcement
command propagation

Replication turned this from a key-value store into a distributed systems project.

Layer 9: React Sandbox

After building the server, I wanted a way to make the internals easier to understand.

So I built a React + Vite sandbox that simulates the server in the browser.

The sandbox lets you type Redis commands and watch internal state update live.

It shows:

Database
Expiry store
AOF log
RDB snapshot
Pub/Sub channels

The README says the sandbox simulates the server entirely in the browser and includes tabs for database, expiry, AOF log, RDB snapshot, and Pub/Sub.

This is useful because most backend internals are invisible.

You run:

SET name Alice EX 60

and normally you only see:

But in the sandbox, you can see:

The key appears in the database
The TTL countdown starts
The AOF log updates
The RDB snapshot changes

That made the project not just functional, but explainable.