Saleh Enab

Posted on May 23

Building a Redis Clone in Go

#redis #go

1. Introduction

To better understand how databases and network servers work internally, I built a Redis-inspired in-memory database in Go.

This project implements several core Redis concepts, including the Redis Serialization Protocol (RESP), concurrent TCP client handling, key expiration, persistence mechanisms, transactions, and basic replication support.

Rather than focusing only on using Redis as a developer, the goal of this project was to explore how systems like Redis work under the hood: how commands are parsed, how data is stored in memory, how persistence is handled, and how multiple clients communicate with the server concurrently.

You can find the full project source code here and follow along with the implementation details throughout the article:

GoRedis GitHub Repository

2. High-Level Architecture

This section provides a high-level overview of the project architecture. The implementation details and code explanations will be covered later in the article.

Server Layer

TCP Server

At its core, Redis is simply a TCP server that communicates with clients over persistent TCP connections.
The first step in building the project was creating a TCP server capable of accepting and handling client requests.

Client Handling

Multiple clients can connect to the server simultaneously.
Each client connection maintains its own isolated state, including:

TCP connection reader/writer
Transaction state (MULTI/EXEC)
Authentication state
Connection-specific metadata

This separation ensures that commands executed by one client do not interfere with another.

Goroutines Per Client

To support concurrent clients efficiently, the server uses goroutines to handle each connection independently.
This allows multiple clients to execute commands at the same time without blocking the entire server.

Since all clients share the same in-memory database, synchronization primitives such as mutexes are required to ensure thread-safe access to shared data structures.

RESP Parser

Parsing Requests

Redis communicates using a custom protocol called the Redis Serialization Protocol (RESP).

RESP defines how commands and responses are encoded between the client and the server, making communication predictable and easy to parse.

For example, a command like:

SET name saleh

is sent over the network as a RESP array containing bulk strings like:

*3
$3
SET
$4
name
$5
saleh

You can read the official RESP specification here:

Redis RESP Protocol Specification

Encoding Responses

Server responses must also be encoded using RESP before being written back through the TCP connection.

Depending on the command result, responses may be encoded as:

Simple strings
Bulk strings
Arrays
Integers
Errors
Null values

Command Router

Command Dispatching

Each supported Redis command is mapped to a dedicated handler function.

After a request is parsed, the command router identifies the command name and dispatches execution to the corresponding handler.

This design keeps the codebase modular and makes adding new commands straightforward.

In-Memory Store

Key-Value Storage

The database itself is implemented using Go maps for simplicity and fast lookups.

In real Redis, the internal storage engine is significantly more sophisticated and optimized for memory efficiency, advanced data structures, eviction policies, and persistence integration. However, using Go maps provides a simple and effective way to understand the core concepts behind an in-memory key-value store.

Expiration Handling

One of Redis’s most useful features is key expiration (TTL support).

Keys can be configured to expire automatically after a specified duration, allowing temporary data to be removed without manual cleanup.

A common real-world use case is storing OTP codes or session data that should automatically become invalid after a short period of time.

The project includes expiration handling to automatically evict expired keys from memory.

Persistence Layer

Since Redis stores data primarily in memory, persistence mechanisms are required to avoid losing data after server restarts.

Redis mainly supports two persistence strategies:

AOF (Append Only File)

AOF persistence logs every write operation that modifies the database state.

When the server restarts, the database can be reconstructed by replaying the logged commands sequentially.

In this project, write commands such as SET and DEL are appended to the AOF file because they modify the database state.

The implementation also includes AOF replaying during startup and log rewriting support.

RDB (Redis Database Snapshot)

RDB persistence works by periodically creating snapshots of the database and saving them to disk.

Unlike AOF, which stores every operation, RDB stores the entire dataset at specific points in time.

As described in the official Redis documentation, RDB performs:

“Point-in-time snapshots of your dataset at specified intervals.”

Redis Persistence Documentation

Transactions

MULTI / EXEC

The project also implements basic Redis-style transactions.

Using MULTI, a client can start a transaction and queue multiple commands without executing them immediately.

Once the client sends EXEC, all queued commands are executed sequentially as a single transaction unit.

This mechanism allows grouped operations to be executed together while preserving command order.

For more information about how Redis transactions work:

Redis Transactions Documentation

TCP Server and Client Connections

Everything starts from the main.go file, which acts as the entry point of the application.

The server listens on TCP port 6379, which is the default Redis port, and starts the server by calling server.Start.

package main

import (
    "log"

    "redis/internal/server"
)

func main() {

    const addr = ":6379"

    if err := server.Start(addr); err != nil {
        log.Fatalf("cannot listen on port %s: %v", addr, err)
    }
}

At this point, the project behaves like any other TCP-based network server: it opens a socket, starts listening for incoming client connections, and waits for requests.

Server Startup Flow

The Start function is responsible for initializing the entire server lifecycle.

It performs several important steps before accepting client connections:

Reads and parses the redis.conf configuration file
Initializes a shared application state
Loads persistence data (AOF/RDB) if enabled
Starts the TCP listener
Accepts incoming client connections
Spawns a new goroutine for every connected client

package server

import (
    "log/slog"
    "net"

    "redis/internal/app"
    "redis/internal/config"
)

func Start(addr string) error {
    slog.Info("Reading the config file...")
    conf := config.ReadConf("./redis.conf")
    state := app.NewAppState(conf)

    l, err := net.Listen("tcp", addr)
    if err != nil {
        return err
    }
    defer l.Close()

    slog.Info("server is listening", "addr", addr)

    for {
        conn, err := l.Accept()
        if err != nil {
            slog.Error("accept failed", "err", err)
            continue
        }

        go HandleConnection(conn, state)
    }
}

Shared Application State

One important concept in the project is the AppState.

Instead of every client having its own isolated database instance, all clients share a single application state that contains:

Server configuration
Persistence state
AOF handler
RDB tracking state
Server statistics and metrics

This mimics how real Redis works internally, where all connected clients interact with the same in-memory database.

package app

import (
    "time"

    "redis/internal/config"
    "redis/internal/persistence"
)

type AppState struct {
    Conf         *config.Config
    Aof          *persistence.Aof
    RDB          *persistence.RDBState
    StartTime    time.Time
    ClientsCount int
    PeakMem      int64
}

Loading Persistence on Startup

Before the server starts accepting clients, it attempts to restore previously persisted data.

If AOF persistence is enabled, the server replays the AOF file to reconstruct the database state.

If RDB snapshots are configured, the latest snapshot is loaded into memory.

if conf.AofEnabled {
    persistence.ReplayAOF(conf)
}

if len(conf.Rdb) > 0 {
    persistence.SyncRDB(conf)
}

This is similar to how Redis restores its dataset after a restart.

Initializing Persistence Background Tasks

After the state is initialized, additional background persistence tasks may start running.

For example, when the AOF fsync policy is set to everysec, a goroutine periodically flushes buffered writes to disk every second.

if conf.AofFsync == "everysec" {
    go func() {
        t := time.NewTicker(time.Second)
        defer t.Stop()

        for range t.C {
            state.Aof.W.Flush()
        }
    }()
}

This behavior is inspired by Redis’s own AOF synchronization strategies, where durability and performance are balanced depending on the selected configuration.

The server also initializes background RDB tracking if snapshotting is enabled.

Accepting Client Connections

Once initialization is complete, the server starts listening for incoming TCP connections using Go’s net.Listen.

l, err := net.Listen("tcp", addr)

The server then enters an infinite loop waiting for clients:

for {
    conn, err := l.Accept()
    ...
}

Every time a new client connects, the server accepts the connection and creates a dedicated goroutine to handle it:

go HandleConnection(conn, state)

This is one of the most important scalability concepts in Go networking.

Instead of blocking the entire server while serving one client, every connection runs independently in its own lightweight goroutine, allowing multiple clients to communicate with the server concurrently.

Handling Client Connections

After a client connects, the connection is passed to HandleConnection.

This function is responsible for:

Creating a client session
Initializing buffered readers/writers
Reading RESP requests
Dispatching commands
Sending encoded RESP responses back to the client

package server

import (
    "bufio"
    "fmt"
    "io"
    "log/slog"
    "net"
    "strings"

    "redis/internal/app"
    "redis/internal/client"
    "redis/internal/commands"
    "redis/internal/protocol"
    "redis/internal/utils"
)

func HandleConnection(conn net.Conn, state *app.AppState) {
    defer conn.Close()

    c := client.NewClient(conn)

    state.ClientsCount++
    defer func() {
        state.ClientsCount--
    }()

    c.Reader = bufio.NewReader(c.Conn)
    c.Writer = bufio.NewWriter(c.Conn)

    for {
        v := protocol.Value{Type: protocol.Array}

        if err := v.ReadArray(c.Reader); err != nil {
            if err == io.EOF || strings.Contains(err.Error(), "forcibly closed by the remote host") {
                slog.Info("client disconnected")
                return
            }
            slog.Error("read error", "err", err)
            return
        }

        fmt.Println("received: ", v.Array)

        res := commands.HandleCommand(c, &v, state)

        if res != nil {
            utils.SendResponse(c.Writer, res)
        }
    }
}

Client State Management

Every connected client has its own isolated connection state represented by the Client struct.

package client

import (
    "bufio"
    "net"
    "redis/internal/db"
)

type Client struct {
    Conn          net.Conn
    Authenticated bool
    Reader        *bufio.Reader
    Writer        *bufio.Writer
    Tx            *db.Transaction
    ExecutingTx   bool
}

This structure stores connection-specific information such as:

TCP connection object
Authentication status
Buffered reader/writer
Active transaction state
Transaction execution status

This design is important because some Redis features are connection-specific.

For example:

Transactions started with MULTI belong only to that client
Authentication state must be isolated per connection
Buffered network I/O should not be shared across clients

Request Processing Flow

After initialization, the connection handler enters an infinite loop:

Read a RESP array from the client
Parse the command
Route the command to its handler
Execute the command
Encode the response
Send the response back to the client

res := commands.HandleCommand(c, &v, state)

if res != nil {
    utils.SendResponse(c.Writer, res)
}

This request-response cycle continues until the client disconnects or an error occurs.

Persistence Layer (AOF & RDB)

Before diving into command handlers and the full request-response lifecycle, it is important to understand the persistence layer, which is one of the most complex and interesting parts of the project.

As mentioned earlier, this project implements two Redis-inspired persistence strategies:

AOF (Append Only File)
RDB (Redis Database Snapshot)

Both approaches solve the same problem — preserving data after server restarts — but they work in very different ways.

Append Only File (AOF)

The AOF strategy works by logging every command that modifies the database state.

Whenever a write operation such as SET, DEL, or FLUSHDB is executed, the command is appended to a file on disk.

When the server restarts, the database state can be reconstructed simply by replaying those commands in order.

This approach is heavily inspired by how real Redis implements AOF persistence.

Initializing AOF During Server Startup

When creating the shared application state, the server first checks whether AOF persistence is enabled in the configuration file.

If it is enabled:

Existing AOF records are replayed to restore the database state
A new AOF handler is initialized for future writes

if conf.AofEnabled {
    persistence.ReplayAOF(conf)
}

Later, a new AOF instance is created:

if conf.AofEnabled {
    state.Aof = persistence.NewAof(conf)
}

AOF Configuration

The following configuration options control AOF behavior:

# AOF
appendonly yes
appendfilename backup.aof
appendfsync always

`appendonly`

Enables or disables AOF persistence.

`appendfilename`

Specifies the filename used to store AOF records.

`appendfsync`

Controls when buffered writes are flushed to disk.

Supported modes in the project:

always → flush after every write
everysec → flush once every second
no → rely on the operating system buffering

This configuration is important because it directly affects the tradeoff between durability and performance.

For example:

always is safer but slower
everysec is faster but may lose up to one second of data during crashes

This is very similar to how Redis itself handles AOF synchronization policies.

Creating the AOF File

The NewAof function initializes the AOF file handler.

func NewAof(conf *config.Config) *Aof {
    aof := Aof{Conf: conf}

    filePath := path.Join(conf.Dir, conf.AofFilename)

    f, err := os.OpenFile(filePath, os.O_CREATE|os.O_APPEND|os.O_RDWR, 0644)
    if err != nil {
        slog.Error(fmt.Sprintf("cannot open %s", filePath), "filepath", filePath)
        return &aof
    }

    aof.W = bufio.NewWriter(f)
    aof.F = f

    return &aof
}

The file is opened using:

O_CREATE → create the file if it does not exist
O_APPEND → append new commands at the end
O_RDWR → allow reading and writing

The implementation also uses a buffered writer (bufio.Writer) to reduce expensive disk I/O operations.

Appending Commands to the AOF

One important design decision in the project is that commands are stored in the exact same RESP format used by Redis clients.

For example, instead of storing some custom internal structure, I write commands exactly as they are received from redis-cli.

This makes replaying much simpler because replayed commands can be treated exactly like normal client commands.

func (aof *Aof) Append(v *protocol.Value) error {
    if aof == nil || aof.W == nil {
        return nil
    }

    aof.Mu.Lock()
    defer aof.Mu.Unlock()

    _, err := aof.W.Write(protocol.Deserialize(v))
    if err != nil {
        return err
    }

    if aof.Conf.AofFsync == "always" {
        return aof.W.Flush()
    }

    return nil
}

A mutex is used here because multiple clients may attempt to append to the AOF concurrently.

Without synchronization, writes from different goroutines could interleave and corrupt the file.

Replaying the AOF File

When the server starts, the AOF file is replayed to restore the previous database state.

func ReplayAOF(conf *config.Config) {
    ...
}

The replay process works almost exactly like handling external client requests:

Read RESP arrays from the file
Parse them into commands
Execute the corresponding database operations

err := v.ReadArray(r)

This reuse of the RESP parser is an important architectural advantage because it avoids creating a completely separate parsing system just for persistence.

The replay implementation currently handles only the commands that are appended to the AOF:

SET
DEL
FLUSHDB

switch cmd {

case "SET":
    ...

case "DEL":
    ...

case "FLUSHDB":
    ...
}

Since only state-changing commands are persisted, replaying them is sufficient to reconstruct the database.

AOF Rewrite

One of the most interesting persistence features implemented in the project is AOF rewriting.

Over time, the AOF file can grow very large because it stores every write operation ever executed.

For example:

SET name saleh
SET name ahmed
SET name mohamed

Only the final value actually matters, yet all previous operations still exist in the log.

To solve this problem, Redis periodically rewrites the AOF file into a minimal version that represents only the current database state.

Thia project implements a simplified version of this idea.

Rewrite Flow

The rewrite process works in three major stages:

1. Redirect Incoming Writes to a Temporary Buffer

aof.Mu.Lock()
aof.W.Flush()
aof.W = bufio.NewWriter(&buff)
aof.Mu.Unlock()

Instead of blocking clients during the rewrite process, new incoming commands are temporarily written into an in-memory buffer.

This allows the server to continue serving clients while the rewrite is happening in the background.

2. Generate a Minimal Snapshot

The current database state is iterated and rewritten as a compact sequence of SET commands.

for k, v := range cp {
    arr := protocol.Value{
        Type: protocol.Array,
        Array: []protocol.Value{
            {Type: protocol.Bulk, Bulk: "SET"},
            {Type: protocol.Bulk, Bulk: k},
            {Type: protocol.Bulk, Bulk: v.V},
        },
    }

    fWriter.Write(protocol.Deserialize(&arr))
}

This produces a clean AOF file containing only the latest state of each key.

3. Merge Buffered Commands

After the snapshot finishes writing, buffered commands that arrived during the rewrite are appended to the end of the new file.

if _, err := newF.Write(buff.Bytes()); err != nil {
    ...
}

Finally, the old file handle is replaced with the new rewritten file.

This design minimizes client blocking time while still preserving consistency.

Redis Database Snapshots (RDB)

Unlike AOF, which logs every write operation, RDB persistence works by periodically taking full snapshots of the database state and saving them to disk.

This approach is more focused on point-in-time backups rather than operation logging.

In Redis itself, RDB snapshots are commonly used because they are compact, fast to load, and efficient for backups.

RDB Configuration

RDB persistence is configured using snapshot rules inside the redis.conf file.

A simple configuration example looks like this:

# RDB
save 10 3
dbfilename backup.rdb

The line:

save 10 3

means:

If at least 3 keys are modified within 10 seconds, create a new RDB snapshot.

This creates a configurable persistence policy based on:

Time intervals
Number of write operations

Tracking Database Changes

To track database modifications over time, I used Go tickers.

Each snapshot rule creates its own SnapshotTracker:

type SnapshotTracker struct {
    keys   int
    ticker time.Ticker
    rdb    *config.RDBSnapshot
}

The tracker stores:

The number of modified keys
A periodic ticker
The snapshot configuration rule

Initializing Snapshot Trackers

When the server starts, all configured snapshot rules are initialized.

func InitRDBTracker(conf *config.Config, state *RDBState) {
    for _, rdb := range conf.Rdb {
        tracker := newSnapshotTracker(&rdb)
        trackers = append(trackers, tracker)

        go func() {
            defer tracker.ticker.Stop()

            for range tracker.ticker.C {
                if tracker.keys >= tracker.rdb.KeysChanged {
                    SaveRDB(conf, state)
                }
                tracker.keys = 0
            }
        }()
    }
}

Each tracker runs inside its own goroutine and periodically checks whether enough keys have changed during the configured time window.

If the threshold is reached, a new snapshot is created automatically.

Tracking Write Operations

Whenever the database state changes, all active snapshot trackers are updated:

func IncrRDBTickers() {
    for _, t := range trackers {
        t.keys++
    }
}

This function is called after write operations such as:

SET
DEL
FLUSHDB

This allows the persistence layer to monitor database activity without tightly coupling snapshot logic to command handlers.

Saving an RDB Snapshot

The SaveRDB function is responsible for serializing the current database state and writing it to disk.

func SaveRDB(conf *config.Config, state *RDBState) {
    ...
}

The overall snapshot flow is:

Serialize the database
Store the serialized data in memory
Compute a checksum
Write the snapshot to disk
Sync the file to storage
Verify integrity using checksum validation

Serializing the Database

I used Go’s built-in gob package to serialize the database map into binary data.

err = gob.NewEncoder(&buf).Encode(&db.Data.M)

This converts the in-memory Go map into a byte stream that can later be restored using gob decoding.

Using gob made the implementation much simpler because it automatically handles encoding and decoding complex Go data structures.

Handling Concurrent Writes During Snapshots

One important challenge during snapshot creation is consistency.

The database may still be receiving writes while the snapshot is being generated.

To handle this safely, the implementation uses two different approaches:

Normal SAVE

For synchronous saves, the database is temporarily locked for reading during serialization:

db.Data.Mu.RLock()
err = gob.NewEncoder(&buf).Encode(&db.Data.M)
db.Data.Mu.RUnlock()

This guarantees a consistent snapshot.

Background BGSAVE

For background saves, a copied snapshot of the database is serialized instead:

err = gob.NewEncoder(&buf).Encode(&state.DBCopy)

This avoids blocking the main database while writes are still happening.

This behavior is inspired by Redis’s own BGSAVE mechanism.

Data Integrity Using Checksums

One interesting feature implemented in the project is checksum verification.

Before writing the snapshot to disk, a SHA-256 checksum is calculated for the serialized buffer:

bsum, err := Hash(bytes.NewReader(data))

After writing and syncing the file, the snapshot is read again and another checksum is calculated:

fusm, err := Hash(f)

Finally, both hashes are compared:

if bsum != fusm {
    ...
}

This ensures that the persisted file exactly matches the original in-memory snapshot.

Although simplified compared to Redis internals, this demonstrates an important real-world systems concept: persistence verification and data integrity validation.

Flushing Data to Stable Storage

After writing the snapshot, the implementation explicitly calls:

err = f.Sync()

This forces the operating system to flush buffered writes to stable storage.

Without this step, data could still remain only in the OS page cache and might be lost during crashes or sudden shutdowns.

Restoring the Database From RDB

Restoring the database from an RDB file is relatively straightforward.

The snapshot file is opened and decoded directly back into the in-memory database map:

err = gob.NewDecoder(f).Decode(&db.Data.M)

Since the snapshot already contains the full database state, reconstruction is much simpler than replaying AOF commands sequentially.

Manual Snapshot Commands

In addition to automatic snapshot tracking, the project also supports manual persistence commands inspired by Redis:

SAVE
BGSAVE

These commands allow clients to trigger snapshot creation manually regardless of the configured snapshot intervals.

The detailed implementation of these commands will be covered later in the article.

Core Database Commands (`GET`, `SET`, `DEL`)

After building the networking layer, RESP parser, persistence mechanisms, and command router, the next step was implementing the actual database commands.

The most important commands in any key-value database are:

GET
SET
DEL

Although these commands look simple from the outside, implementing them properly requires handling:

Concurrency
Expiration
Memory tracking
Persistence
Eviction policies
RESP encoding
Transactions

This section explains how these commands work internally and how the related database functions interact together.

GET Command

The GET command is responsible for retrieving a value from the in-memory database.

From the client side, the command is very simple:

GET username

But internally, several operations happen before the response is returned.

GET Handler

The command handler starts by extracting command arguments from the RESP array.

func Get(c *client.Client, v *protocol.Value, state *app.AppState) *protocol.Value {
    args := v.Array[1:]

The first element in the RESP array is always the command name itself (GET), so the actual arguments start from index 1.

Arguments Validation

The handler first validates the number of arguments:

if len(args) != 1 {
    return &protocol.Value{
        Type:  protocol.Error,
        Error: "ERR Invalid number of arguments for 'GET' command",
    }
}

Redis commands are strict about argument counts, so validating them early prevents invalid requests from reaching the database layer.

Extracting the Key

The requested key is extracted from the RESP bulk string:

key := args[0].Bulk

Then the handler calls the database layer:

item, ok := db.Data.Get(key)

At this point, the actual database retrieval logic starts.

Database GET Logic

Inside the database layer, the Get function performs much more than simply reading from a Go map.

func (db *Database) Get(key string) (*Item, bool)

Thread Safety Using Mutexes

Because multiple goroutines may access the database concurrently, synchronization is required.

I used a mutex to protect the shared map:

db.Mu.Lock()
defer db.Mu.Unlock()

Although GET is technically a read operation, I intentionally used a full write lock instead of RLock.

This is because the operation may modify internal metadata such as:

Access counters
Last access timestamps
Expiration cleanup

Since the database state may change during reads, a write lock becomes necessary.

Reading the Key

The actual lookup is straightforward:

item, ok := db.M[key]

If the key does not exist:

if !ok {
    return nil, false
}

The handler later converts this into a RESP null response.

Lazy Expiration

One important feature implemented inside GET is lazy expiration.

Before returning the value, the database checks whether the key has expired:

if item.IsExpired() {
    db.deleteLocked(key)
    return nil, false
}

Instead of running a dedicated cleanup thread continuously scanning the database, expired keys are removed only when accessed.

This approach is called lazy expiration, and Redis itself uses a similar technique alongside active expiration.

Expiration Check

The expiration logic itself is implemented in the IsExpired method:

func (item *Item) IsExpired() bool {
    return item.Exp.Unix() != utils.UNIX_TS_EPOCH &&
        int(time.Until(item.Exp).Seconds()) <= 0
}

This checks two conditions:

The item actually has an expiration time
The expiration timestamp has already passed

If both conditions are true, the key is considered expired.

Access Tracking

If the key is valid, additional metadata is updated before returning it:

item.Accesses++
item.LastAccess = time.Now()

This information is later used by the eviction system.

For example:

LFU eviction depends on Accesses
LRU eviction depends on LastAccess

This means every GET request also contributes to memory management statistics.

Returning the Response

Finally, the handler converts the database item into a RESP bulk string:

return &protocol.Value{
    Type: protocol.Bulk,
    Bulk: item.V,
}

If the key does not exist, the handler returns:

return &protocol.Value{
    Type: protocol.Null,
}

The RESP encoder later serializes this into a valid Redis response before sending it back to the client.

SET Command

The SET command is one of the most important commands in the entire database because it touches almost every subsystem:

Memory allocation
Eviction
Persistence
RDB tracking
Peak memory tracking
AOF logging

From the client side:

SET name saleh

But internally, the execution flow is much more involved.

SET Handler

The handler begins by validating arguments:

if len(args) != 2 {
    return &protocol.Value{
        Type:  protocol.Error,
        Error: "ERR Invalid number of arguments for 'SET' command",
    }
}

Then the key and value are extracted:

key := args[0].Bulk
val := args[1].Bulk

The handler then delegates the actual storage logic to the database layer:

evicted, err := db.Data.Set(key, val, state.Conf)

Database SET Logic

Inside the database layer:

func (db *Database) Set(
    key string,
    val string,
    conf *config.Config,
)

This function is responsible for safely storing the new value while respecting memory limits and eviction policies.

Acquiring the Lock

Because SET modifies shared state, the database acquires a write lock:

db.Mu.Lock()
defer db.Mu.Unlock()

This prevents concurrent writes from corrupting the database map.

Updating Existing Keys

If the key already exists, its previous memory usage is removed first:

if old, ok := db.M[key]; ok {
    db.Mem -= old.ApproxMemUsage(key)
}

This ensures memory tracking remains accurate after updates.

Creating the Item

A new database item is created:

item := &Item{
    V:          val,
    LastAccess: time.Now(),
}

Notice that:

Access time starts immediately
Expiration is empty by default
Access counter starts at zero

Memory Usage Estimation

One interesting part of the implementation is memory tracking.

Each item estimates its approximate memory usage:

requiredMem := item.ApproxMemUsage(key)

The calculation is implemented here:

func (item *Item) ApproxMemUsage(name string) int64

I used approximate calculations based on:

String sizes
Struct overhead
Map entry overhead

Although this is not perfectly accurate like Redis internals, it is enough to simulate memory-aware eviction behavior.

Memory Limit Checking

Before storing the item, the database checks whether memory limits would be exceeded:

outOfMem := conf.MaxMemSize > 0 &&
    db.Mem+requiredMem > conf.MaxMemSize

If memory is insufficient, eviction starts automatically.

Eviction System

When memory usage exceeds the configured limit:

evicted, err = db.evictKeysLocked(
    conf,
    requiredMem,
)

The eviction system attempts to free enough memory before allowing the new key to be inserted.

Supported Eviction Policies

I implemented two Redis-inspired policies:

LRU (Least Recently Used)

sort.Slice(samples, func(i, j int) bool {
    return samples[i].v.LastAccess.After(samples[j].v.LastAccess)
})

This removes keys that were not accessed recently.

LFU (Least Frequently Used)

sort.Slice(samples, func(i, j int) bool {
    return samples[i].v.Accesses < samples[j].v.Accesses
})

This removes keys that are accessed less frequently.

Sampling Instead of Full Scanning

Instead of sorting the entire database, the eviction system samples a subset of keys:

samples := db.getSampleKeysLocked(conf)

This is inspired by Redis itself, which uses approximate eviction algorithms for performance reasons.

Scanning the entire database on every write would become extremely expensive for large datasets.

Storing the Item

Once enough memory is available, the key is inserted:

db.M[key] = item
db.Mem += requiredMem

And the evicted keys list is returned to the handler.

Updating RDB Persistence Trackers

After successfully storing the value, the handler updates the RDB snapshot trackers:

if len(state.Conf.Rdb) > 0 {
    persistence.IncrRDBTickers()
}

This tells the persistence layer that the database state has changed.

Peak Memory Tracking

The server also tracks the highest memory usage reached:

if db.Data.Mem > state.PeakMem {
    state.PeakMem = db.Data.Mem
}

This information is later exposed through the INFO command.

AOF Persistence Integration

If AOF persistence is enabled, the command is appended to the AOF file:

if err := state.Aof.Append(v); err != nil {
    slog.Error("AOF append failed", "err", err)
}

Logging Evicted Keys

One subtle but important detail is eviction persistence.

If keys are evicted because of memory pressure, corresponding DEL commands are also appended to the AOF:

delCmd := &protocol.Value{
    Type: protocol.Array,
    Array: []protocol.Value{
        {
            Type: protocol.Bulk,
            Bulk: "DEL",
        },
        {
            Type: protocol.Bulk,
            Bulk: key,
        },
    },
}

Without this step, replaying the AOF after restart would restore keys that were previously evicted from memory.

This ensures persistence stays consistent with the actual runtime database state.

Returning the Response

Finally, the handler returns:

return &protocol.Value{
    Type:   protocol.String,
    String: "OK",
}

Which becomes the familiar Redis response:

+OK

DEL Command

The DEL command removes one or more keys from the database.

Example:

DEL name email session

Unlike GET and SET, this command supports multiple keys in a single request.

DEL Handler

The handler first extracts all keys:

var keys []string

for _, arg := range args {
    keys = append(keys, arg.Bulk)
}

Then deletion is delegated to the database layer:

n := db.Data.Delete(keys)

The returned value represents how many keys were actually deleted.

Database DELETE Logic

The database deletion function acquires a write lock:

db.Mu.Lock()
defer db.Mu.Unlock()

Then iterates over all provided keys:

for _, key := range keys {
    if db.deleteLocked(key) {
        deleted++
    }
}

Internal deleteLocked Function

The actual deletion logic is centralized inside:

func (db *Database) deleteLocked(key string) bool

This function:

Finds the item
Updates memory tracking
Removes the key from the map

db.Mem -= item.ApproxMemUsage(key)

delete(db.M, key)

Centralizing deletion logic avoids code duplication and guarantees memory accounting always stays correct.

Persistence Integration

Just like SET, delete operations also interact with persistence systems.

RDB Tracking

persistence.IncrRDBTickers()

AOF Logging

state.Aof.Append(v)

This ensures deletions are properly persisted and replayed after restarts.

RESP Integer Response

Finally, the handler returns the number of deleted keys:

return &protocol.Value{
    Type:    protocol.Integer,
    Integer: n,
}

Which Redis clients interpret as:

(integer) 3

if three keys were successfully deleted.

Persistence and Server Management Commands

After implementing the core database operations (GET, SET, and DEL), the next important step was adding commands responsible for:

Persistence management
Database administration
Authentication
Background saving

These commands interact heavily with the persistence layer and server state, making them more interesting internally than they may initially appear from the client side.

This section covers:

SAVE
BGSAVE
FLUSHDB
AUTH

SAVE Command

The SAVE command forces the server to create an RDB snapshot immediately.

From the client side:

SAVE

Unlike automatic snapshots triggered by configured save intervals, this command allows the client to manually force persistence at any moment.

SAVE Handler

The implementation is intentionally simple:

func Save(c *client.Client, v *protocol.Value, state *app.AppState) *protocol.Value {
    persistence.SaveRDB(state.Conf, state.RDB)

    return &protocol.Value{
        Type:   protocol.String,
        String: "OK",
    }
}

The handler directly calls the persistence layer responsible for serializing the database and writing the snapshot to disk.

Why SAVE Is Blocking

One important detail about SAVE is that it is a blocking operation.

The client waits until the entire snapshot process finishes before receiving the response.

During snapshot creation, several expensive operations occur:

Serializing the database
Writing data to disk
Flushing filesystem buffers
Calculating checksums
Verifying persistence integrity

For small datasets this is acceptable, but for larger databases it may freeze the server temporarily.

This is actually one of the reasons Redis introduced BGSAVE.

SAVE and Database Consistency

During normal synchronous saving, I used a read lock while encoding the database:

db.Data.Mu.RLock()
err = gob.NewEncoder(&buf).Encode(&db.Data.M)
db.Data.Mu.RUnlock()

This guarantees that the snapshot sees a consistent database state while preventing concurrent writes from modifying the map during serialization.

Without synchronization, concurrent writes could corrupt the snapshot or even crash the encoder.

Snapshot Serialization

The database is serialized using Go’s gob package:

err = gob.NewEncoder(&buf).Encode(&db.Data.M)

I used gob because it provides a simple way to convert complex Go structures into binary data that can later be restored easily.

The encoded bytes are then written directly into the RDB file.

Integrity Verification

One important addition I implemented is checksum verification.

Before writing the snapshot to disk, the serialized buffer checksum is calculated:

bsum, err := Hash(bytes.NewReader(data))

After writing and syncing the file, the snapshot file itself is hashed again:

fusm, err := Hash(f)

Finally, both hashes are compared:

if bsum != fusm {
    ...
}

This ensures the persisted file exactly matches the original in-memory serialized snapshot.

Although simplified compared to Redis internals, this demonstrates an important database concept: persistence validation and corruption detection.

Flushing to Stable Storage

After writing the snapshot, the implementation explicitly calls:

err = f.Sync()

This forces the operating system to flush pending writes from memory buffers to stable disk storage.

Without this step, data might still exist only inside the OS page cache and could be lost during crashes or power failures.

BGSAVE Command

Although SAVE works correctly, blocking the server during persistence is not ideal.

To solve this problem, Redis provides BGSAVE, which performs snapshotting in the background.

From the client side:

BGSAVE

The goal is allowing persistence while the server continues serving requests normally.

BGSAVE Handler

The implementation starts by checking whether another background save is already running:

if state.RDB.BGSaveRunning {
    return &protocol.Value{
        Type:  protocol.Error,
        Error: "ERR background saving already in progress",
    }
}

This prevents multiple concurrent snapshot operations from interfering with each other.

Creating a Safe Database Copy

One major challenge with background saving is consistency.

While snapshotting is happening, clients may still execute writes such as:

SET
DEL
FLUSHDB

If the snapshot directly serialized the live database map while writes were occurring, the snapshot could become inconsistent.

To solve this, I first create a copy of the database:

cp := make(map[string]*db.Item, len(db.Data.M))

Then I safely copy the database contents under a read lock:

db.Data.Mu.RLock()
maps.Copy(cp, db.Data.M)
db.Data.Mu.RUnlock()

This copied map becomes the immutable snapshot source for the background save operation.

Running the Save in a Goroutine

After preparing the snapshot copy, the actual save process runs in a separate goroutine:

go func() {
    persistence.SaveRDB(state.Conf, state.RDB)
}()

This allows the main server thread to continue accepting and processing client requests while persistence happens asynchronously.

This concurrency model is one of the biggest advantages of Go for server-side systems programming.

Tracking Background Save State

Before starting the goroutine:

state.RDB.BGSaveRunning = true
state.RDB.DBCopy = cp

And after completion:

defer func() {
    state.RDB.BGSaveRunning = false
    state.RDB.DBCopy = nil
}()

This state tracking is important because:

The server must know whether a save is already running
SaveRDB needs access to the copied snapshot
Resources should be released after completion

Using the Snapshot Copy During Serialization

Inside SaveRDB, background saves serialize the copied snapshot instead of the live database:

if state.BGSaveRunning {
    err = gob.NewEncoder(&buf).Encode(&state.DBCopy)
}

This avoids locking the main database during potentially slow disk operations.

That design is heavily inspired by Redis’s own background persistence mechanisms.

FLUSHDB Command

The FLUSHDB command completely removes all keys from the database.

Example:

FLUSHDB

Although this command looks simple externally, internally it interacts with:

Memory tracking
Persistence
AOF logging
Snapshot generation

Database Flush Logic

The database reset itself is straightforward:

func (db *Database) Flush() {
    db.Mu.Lock()
    defer db.Mu.Unlock()

    db.M = make(map[string]*Item)
    db.Mem = 0
}

I replace the entire map with a new empty map and reset memory tracking back to zero.

This approach is much faster than iterating and deleting keys individually.

FLUSHDB Handler

The command handler first clears the database:

db.Data.Flush()

Then persistence systems are updated.

AOF Integration

If AOF persistence is enabled, the FLUSHDB command itself is appended to the AOF file:

state.Aof.W.Write(protocol.Deserialize(v))

This is important because replaying the AOF after restart must reproduce the exact same database state.

Without logging FLUSHDB, old keys would incorrectly reappear after replaying the append-only log.

Triggering a New Snapshot

After flushing the database, a new RDB snapshot is triggered asynchronously:

go persistence.SaveRDB(state.Conf, state.RDB)

This ensures the persisted RDB snapshot also reflects the newly emptied database state.

AUTH Command

Redis supports optional password-based authentication, and I implemented a simplified authentication system inspired by Redis.

If password protection is enabled in the configuration file, clients must authenticate before executing most commands.

Authentication Flow

The router validates authentication before dispatching commands:

if state.Conf.RequirePass &&
    !c.Authenticated &&
    !slices.Contains(SafeCMDs, cmd) {

If the client is not authenticated, the server responds with:

return &protocol.Value{
    Type:  protocol.Error,
    Error: "NOAUTH authentication required",
}

Only a few safe commands remain accessible before authentication:

var SafeCMDs = []string{
    "COMMAND",
    "AUTH",
}

This behavior closely follows Redis itself.

AUTH Handler

The authentication handler validates the provided password:

func Auth(c *client.Client, v *protocol.Value, state *app.AppState) *protocol.Value

First, argument validation occurs:

if len(args) != 1 {
    return &protocol.Value{
        Type:  protocol.Error,
        Error: "ERR Invalid number of arguments for 'AUTH' command",
    }
}

Then the password is extracted:

pass := args[0].Bulk

And compared against the configured server password:

if pass == state.Conf.Password {
    c.Authenticated = true
    ...
}

If authentication succeeds:

return &protocol.Value{
    Type:   protocol.String,
    String: "OK",
}

Otherwise:

return &protocol.Value{
    Type:  protocol.Error,
    Error: "ERR invalid password",
}

Per-Client Authentication State

Authentication state is stored directly inside the client structure:

type Client struct {
    ...
    Authenticated bool
    ...
}

This means every TCP connection maintains its own isolated authentication session.

One authenticated client does not automatically authenticate other connected clients.

This mirrors how Redis handles authentication internally per connection rather than globally across the server.

Expiration and Transaction Commands

One of the most powerful features in Redis is the ability to work with temporary data and atomic command execution.

To support these concepts, I implemented:

EXPIRE
TTL
MULTI
EXEC
DISCARD

Although these commands seem unrelated at first, they both introduce an important concept: stateful behavior per client connection.

For expiration commands, the database needs to track time-based metadata for every key.

For transactions, the server needs to track queued commands for every connected client independently.

EXPIRE Command

The EXPIRE command assigns a time-to-live (TTL) to a key.

Example:

EXPIRE session 60

This means the key session should automatically expire after 60 seconds.

This feature is extremely useful for temporary data such as:

Sessions
OTP codes
Cache entries
Verification tokens
Rate limiting data

Database Expiration Model

To support expiration, every database item stores an expiration timestamp:

type Item struct {
    V          string
    Exp        time.Time
    LastAccess time.Time
    Accesses   int
}

The Exp field represents the exact time when the key becomes invalid.

If no expiration exists, the timestamp remains equal to a special default epoch value.

EXPIRE Handler

The command handler starts by validating arguments:

if len(args) != 2 {
    return &protocol.Value{
        Type:  protocol.Error,
        Error: "ERR Invalid number of arguments for 'EXPIRE' command",
    }
}

Unlike GET, this command requires:

The key name
Expiration time in seconds

Parsing the Expiration Time

The expiration value arrives as a RESP bulk string:

secs := args[1].Bulk

Before using it, the value must be converted into an integer:

expSecs, err := strconv.Atoi(secs)

If the client provides an invalid value:

return &protocol.Value{
    Type:  protocol.Error,
    Error: "ERR Invalid expiry value",
}

This prevents invalid timestamps from entering the database layer.

Setting the Expiration

After validation, the database layer is called:

res := db.Data.Expire(key, expSecs)

Inside the database:

func (db *Database) Expire(key string, secs int) int

The database first acquires a write lock:

db.Mu.Lock()
defer db.Mu.Unlock()

Since expiration modifies the item state, synchronization is required.

Updating the Expiration Timestamp

If the key exists:

item, ok := db.M[key]

The expiration timestamp is updated:

item.Exp = time.Now().Add(
    time.Second * time.Duration(secs),
)

This calculates the exact future expiration time relative to the current moment.

If the key does not exist, the command returns 0.

Otherwise, it returns 1.

This behavior matches Redis semantics.

Lazy Expiration

One important design decision in the project is using lazy expiration.

Expired keys are not removed immediately by a dedicated cleanup thread.

Instead, expiration is checked whenever the key is accessed.

Inside the database Get function:

if item.IsExpired() {
    db.deleteLocked(key)
    return nil, false
}

This means expired keys are automatically deleted during reads.

This approach is simple, memory efficient, and inspired by part of Redis’s own expiration strategy.

Expiration Check Logic

The expiration check itself is implemented in:

func (item *Item) IsExpired() bool

return item.Exp.Unix() != utils.UNIX_TS_EPOCH &&
    int(time.Until(item.Exp).Seconds()) <= 0

This verifies two conditions:

The key actually has an expiration
The expiration timestamp has passed

Only then is the key considered expired.

TTL Command

The TTL command returns the remaining lifetime of a key.

Example:

TTL session

This allows clients to know how many seconds remain before expiration.

TTL Handler

The handler first validates the arguments:

if len(args) != 1 {
    return &protocol.Value{
        Type:  protocol.Error,
        Error: "ERR Invalid number of arguments for 'TTL' command",
    }
}

Then the requested key is extracted:

key := args[0].Bulk

Reading the Key Safely

Unlike EXPIRE, the TTL command only reads metadata, so I used a read lock:

db.Data.Mu.RLock()

The handler retrieves the item directly from the database map:

val, ok := db.Data.M[key]

If the key does not exist:

return &protocol.Value{
    Type:    protocol.Integer,
    Integer: -2,
}

This follows Redis behavior exactly.

TTL Return Semantics

Redis uses special integer return values for TTL responses.

I implemented the same behavior:

`-2`

The key does not exist.

`-1`

The key exists but has no expiration.

if exp.Unix() == utils.UNIX_TS_EPOCH {
    return &protocol.Value{
        Type:    protocol.Integer,
        Integer: -1,
    }
}

Positive Integer

The remaining number of seconds before expiration.

Calculating Remaining Time

The remaining lifetime is calculated using:

rem := int(time.Until(exp).Seconds())

If the remaining time is already below zero:

if rem <= 0 {
    db.Data.Delete([]string{key})

    return &protocol.Value{
        Type:    protocol.Integer,
        Integer: -2,
    }
}

The key is immediately removed and treated as expired.

This guarantees expired keys do not survive after TTL checks.

Transactions (`MULTI`, `EXEC`, `DISCARD`)

One of the most interesting Redis features to implement was transactions.

Redis transactions allow clients to queue multiple commands and execute them together later.

Example:

MULTI
SET name saleh
SET age 22
EXEC

Instead of executing commands immediately, the server temporarily stores them until EXEC is called.

Implementing this required adding per-client transaction state.

Transaction State

Every client contains its own transaction object:

type Client struct {
    ...
    Tx          *db.Transaction
    ExecutingTx bool
}

This means every TCP connection can have an independent transaction queue.

One client entering transaction mode does not affect other connected clients.

Transaction Structure

The transaction itself is simple:

type Transaction struct {
    CMDs []*TxCommand
}

Each queued command stores the original parsed RESP value:

type TxCommand struct {
    V *protocol.Value
}

This design allows queued commands to be executed later exactly as if they were newly received client requests.

MULTI Command

The MULTI command starts transaction mode.

Example:

MULTI

After this point, commands are no longer executed immediately.

MULTI Handler

The handler first checks whether the client is already inside a transaction:

if c.Tx != nil || c.ExecutingTx {
    return &protocol.Value{
        Type:  protocol.Error,
        Error: "ERR MULTI calls cannot be nested",
    }
}

Nested transactions are not allowed.

If everything is valid, a new transaction object is created:

c.Tx = db.NewTransaction()

And the server responds with:

return &protocol.Value{
    Type:   protocol.String,
    String: "OK",
}

Command Queueing

The most important transaction logic actually happens inside the router.

Before dispatching commands normally, the router checks whether the client is currently inside a transaction:

if c.Tx != nil &&
    cmd != "EXEC" &&
    cmd != "DISCARD" &&
    cmd != "MULTI" {

If transaction mode is active, commands are queued instead of executed:

txCmd := db.TxCommand{V: v}
c.Tx.CMDs = append(c.Tx.CMDs, &txCmd)

And the client receives:

return &protocol.Value{
    Type:   protocol.String,
    String: "QUEUED",
}

This behavior matches how Redis handles transactional commands internally.

EXEC Command

The EXEC command executes all queued commands sequentially.

Example:

EXEC

EXEC Validation

First, the handler ensures a transaction actually exists:

if c.Tx == nil {
    return &protocol.Value{
        Type:  protocol.Error,
        Error: "ERR EXEC without MULTI",
    }
}

This prevents invalid transaction execution.

Preventing Recursive Queueing

One subtle issue appears during execution.

Since the router automatically queues commands while c.Tx != nil, executing queued commands directly would accidentally queue them again forever.

To avoid this, I temporarily disable the transaction before execution:

tx := c.Tx
c.Tx = nil

This is a very important implementation detail.

Without it, EXEC would recursively re-queue commands instead of executing them.

Executing Commands Sequentially

Queued commands are executed one by one:

for i, cmd := range tx.CMDs {
    response := HandleCommand(c, cmd.V, state)
    ...
}

Notice something interesting here:

I reused the exact same command execution pipeline used for normal client requests.

This means transactional commands automatically reuse:

Validation logic
Persistence integration
RESP responses
Database locking
Authentication checks

without needing separate implementations.

Returning Transaction Responses

Redis transactions return an array of command responses.

I implemented the same behavior:

return &protocol.Value{
    Type:  protocol.Array,
    Array: responses,
}

Each executed command contributes one response entry to the final RESP array.

DISCARD Command

The DISCARD command cancels the current transaction without executing queued commands.

Example:

DISCARD

DISCARD Handler

The handler first validates that the client is actually inside a transaction:

if c.Tx == nil {
    return &protocol.Value{
        Type:  protocol.Error,
        Error: "ERR DISCARD without MULTI",
    }
}

Then the queued transaction is simply removed:

c.Tx = nil

Since commands were only stored in memory and never executed, discarding becomes very cheap and straightforward.

Finally, the server responds with:

return &protocol.Value{
    Type:   protocol.String,
    String: "OK",
}

INFO Command

One of the most useful Redis commands for monitoring and debugging is the INFO command.

It provides runtime information about the server such as:

Server uptime
Connected clients
Memory usage
Process information
Eviction configuration

Example:

INFO

Instead of returning a standard RESP array, Redis returns a formatted text response grouped into sections.

I implemented a simplified version of this behavior inspired by real Redis.

INFO Handler

The command handler itself is very small:

func Info(c *client.Client, v *protocol.Value, state *app.AppState) *protocol.Value {
    msg := info.NewInfo().Print(state)

    return &protocol.Value{
        Type: protocol.Bulk,
        Bulk: msg,
    }
}

The handler simply creates a new Info object, generates the formatted server report, and returns it as a RESP bulk string.

INFO Structure

The Info structure stores information grouped by categories:

type Info struct {
    server map[string]string
    client map[string]string
    memory map[string]string
}

I divided the output into three sections:

Server
Client
Memory

This structure makes it easy to extend later with additional categories similar to real Redis.

Building the INFO Output

The actual data collection happens inside the build function:

func (info *Info) build(state *app.AppState)

This function gathers runtime information from different parts of the server and database.

Server Information

The first section contains general server information.

Process ID

The current server process ID is retrieved using:

fmt.Sprint(os.Getpid())

This can be useful for debugging or monitoring server processes.

Executable Path

The current executable path is also collected:

execPath, err := os.Executable()

This returns the location of the running binary on the operating system.

Uptime Tracking

One interesting metric is server uptime.

When the server starts, the startup time is stored inside the shared application state:

state.StartTime = time.Now()

Later, the INFO command calculates uptime dynamically:

fmt.Sprint(int(time.Since(state.StartTime).Seconds()))

This returns the number of seconds the server has been running since startup.

Server Time

The current server timestamp is also included:

fmt.Sprint(time.Now().UnixMicro())

This returns the current Unix timestamp in microseconds.

Server Metadata

The final server section looks like this internally:

info.server = map[string]string{
    "redis_version":      "1.0.0",
    "process_id":         fmt.Sprint(os.Getpid()),
    "tcp_port":           "6379",
    "server_time_usec":   fmt.Sprint(time.Now().UnixMicro()),
    "uptime_int_seconds": fmt.Sprint(int(time.Since(state.StartTime).Seconds())),
    "executable":         execPath,
}

Client Information

The second section tracks connected clients.

info.client = map[string]string{
    "connected_clients": fmt.Sprint(state.ClientsCount),
}

The server increments this counter whenever a new client connects:

state.ClientsCount++

And decrements it after disconnection:

defer func() {
    state.ClientsCount--
}()

This allows the INFO command to expose real-time connection statistics.

Memory Information

The memory section exposes statistics related to database memory usage and eviction configuration.

Current Memory Usage

The database continuously tracks approximate memory usage:

"used_memory": fmt.Sprint(db.Data.Mem),

This value is updated whenever keys are:

Added
Updated
Deleted
Evicted

Peak Memory Usage

I also tracked the highest memory usage reached during server lifetime:

"used_memory_peak": fmt.Sprint(state.PeakMem),

This is updated after successful SET operations.

System Memory Information

To retrieve total system memory, I used the gopsutil package:

memory, err := mem.VirtualMemory()

Then extracted:

memory.Total

This gives the total RAM available on the host machine.

Max Memory Configuration

The INFO output also exposes the configured memory limit:

"max_memory": fmt.Sprint(state.Conf.MaxMemSize),

This helps monitor eviction behavior and memory pressure.

Eviction Policy

The currently active eviction policy is also included:

"evection_policy": string(state.Conf.EvictionPolicy),

This shows whether the server is configured to use:

LRU eviction
LFU eviction
No eviction

Formatting the INFO Response

After gathering all information, the final response is formatted into a human-readable string.

I used a helper function to print each category:

printCategory := func(header string, m map[string]string) string {
    s := fmt.Sprintf("# %s\n", header)

    for k, v := range m {
        s += fmt.Sprintf("%s : %s\n", k, v)
    }

    return s + "\n"
}

The final output becomes something similar to:

# Server
redis_version : 1.0.0
process_id : 1234
tcp_port : 6379
uptime_int_seconds : 152

# Client
connected_clients : 3

# Memory
used_memory : 4096
used_memory_peak : 8192
max_memory : 1048576
eviction_policy : allkeys-lru

This structure closely resembles the real Redis INFO command output style.

Conclusion

Building this Redis-inspired database was one of the most educational projects I have worked on.

At the beginning, Redis looked like a very simple key-value store from the outside, but while implementing it, I realized how many systems concepts are involved behind the scenes:

TCP networking
Concurrent client handling
Custom protocol parsing
Thread-safe shared memory
Persistence and crash recovery
Background tasks
Transactions
Memory management
Eviction strategies

What I enjoyed most about this project was not just recreating Redis commands, but understanding why systems like Redis are designed the way they are.

Of course, this implementation is still much simpler than the real Redis internals, but building it gave me a much deeper understanding of databases, and Go concurrency patterns.

If you reached this point, thank you for reading the article. I hope this walkthrough helped make some of Redis’s internal ideas easier to understand and maybe even inspired you to build something similar yourself.

1. Introduction

2. High-Level Architecture

Server Layer

TCP Server

Client Handling

Goroutines Per Client

RESP Parser

Parsing Requests

Encoding Responses

Command Router

Command Dispatching

In-Memory Store

Key-Value Storage

Expiration Handling

Persistence Layer

AOF (Append Only File)

RDB (Redis Database Snapshot)

Transactions

MULTI / EXEC

TCP Server and Client Connections

Server Startup Flow

Shared Application State

Loading Persistence on Startup

Initializing Persistence Background Tasks

Accepting Client Connections

Handling Client Connections

Client State Management

Request Processing Flow

Persistence Layer (AOF & RDB)

Append Only File (AOF)

Initializing AOF During Server Startup

AOF Configuration

appendonly

appendfilename

appendfsync

Creating the AOF File

Appending Commands to the AOF

Replaying the AOF File

AOF Rewrite

Rewrite Flow

1. Redirect Incoming Writes to a Temporary Buffer

2. Generate a Minimal Snapshot

3. Merge Buffered Commands

Redis Database Snapshots (RDB)

RDB Configuration

Tracking Database Changes

Initializing Snapshot Trackers

Tracking Write Operations

Saving an RDB Snapshot

Serializing the Database

Handling Concurrent Writes During Snapshots

Normal SAVE

Background BGSAVE

Data Integrity Using Checksums

Flushing Data to Stable Storage

Restoring the Database From RDB

Manual Snapshot Commands

Core Database Commands (GET, SET, DEL)

GET Command

GET Handler

Arguments Validation

Extracting the Key

Database GET Logic

Thread Safety Using Mutexes

Reading the Key

Lazy Expiration

Expiration Check

Access Tracking

Returning the Response

SET Command

SET Handler

Database SET Logic

Acquiring the Lock

Updating Existing Keys

Creating the Item

Memory Usage Estimation

Memory Limit Checking

Eviction System

Supported Eviction Policies

LRU (Least Recently Used)

`appendonly`

`appendfilename`

`appendfsync`

Core Database Commands (`GET`, `SET`, `DEL`)

`-2`

`-1`

Transactions (`MULTI`, `EXEC`, `DISCARD`)