I'm building a Redis Clone in Zig: A Deep Dive into Pub/Sub, and Memory Allocation

It's been a while since I started a project called Zedis (Redis written in Zig). It's been a great way to learn low-level programming.

I want to take you on a tour of three core features I've implemented: Pub/Sub and the memory allocation strategy.

Why Build a Redis Clone in Zig?

I have a personal goal of mastering Zig this year; this was the perfect way to accomplish that. Zig offers a number of intriguing features for high-performance systems, such as comptime, which allows code to run at compile time, and explicit memory management. Zedis has been my playground for exploring everything from network programming to custom allocators.

Feature Deep Dive: Pub/Sub

Implementing the Publish/Subscribe mechanism was a fun challenge. At its core, it's a messaging system that decouples senders (publishers) from receivers (subscribers).

Here's how it works in Zedis:

• Channels and Subscribers: When a client subscribes to a channel, they are added to a list of subscribers for that channel. I used a std.StringHashMap([]u64) to map channel names to a list of client IDs.
• Publishing a Message: When a message is published to a channel, the server iterates through the list of subscribers and writes the message to each of their connections.
• Entering Pub/Sub Mode: Once a client subscribes to a channel, they enter a special “Pub/Sub mode” where they can only receive messages and can't execute other commands.

Here is the subscribe function:

pub fn subscribe(client: *Client, args: []const Value) !void {
    var pubsub_context = client.pubsub_context;
    // Enter pubsub mode on first subscription
    if (!client.is_in_pubsub_mode) {
        client.enterPubSubMode();
    }

    var i: i64 = 0;
    for (args[1..]) |item| {
        const channel_name = item.asSlice();
        // Ensure channel exists
        pubsub_context.ensureChannelExists(channel_name) catch {
            try client.writeError("ERR failed to create channel");
            continue;
        };

        // Subscribe client to channel
        pubsub_context.subscribeToChannel(channel_name, client.client_id) catch |err| switch (err) {
            error.ChannelFull => {
                try client.writeError("ERR maximum subscribers per channel reached");
                continue;
            },
            else => {
                try client.writeError("ERR failed to subscribe to channel");
                continue;
            },
        };

        const subscription_count = i + 1;
        const response_tuple = .{
            "subscribe",
            channel_name,
            subscription_count,
        };
        // Use a generic writer to send the tuple as a RESP array.
        try client.writeTupleAsArray(response_tuple);
        i += 1;
    }
}

Memory Allocation Strategy

One of the most interesting aspects of this project has been designing the memory allocation strategy. I opted for a hybrid approach to balance performance and memory usage:

• KeyValueAllocator: This is a custom allocator I built for the main key-value store. It uses a fixed-size memory pool and has a simple eviction policy to stay within its budget. When the allocator runs out of memory, it can evict all keys to make space for new ones.
• Arena Allocator: For temporary, short-lived allocations (like parsing commands), I use an arena allocator. This is incredibly fast because it simply bumps a pointer for new allocations and frees all the memory at once when it's no longer needed.
• Fixed Pools: For objects that are frequently allocated and deallocated, like client connections, I use a fixed-size pool. This avoids the overhead of dynamic allocation and deallocation.

This approach allows Zedis to be memory-efficient while still being performant.

What's Next?

I'm excited to keep building on this foundation. Here's what's on the roadmap:

• Implement AOF (Append Only File) logging
• Add support for more data structures like lists and sets
• Implement key expiration
• Add clustering support

Check it out

I'd love for you to check out Zedis on GitHub, try it out, and let me know what you think. Contributions are always welcome!

Thanks for reading!