DEV Community: Daniel Keya

Understanding Stacks and Queues in Go

Daniel Keya — Sat, 03 Oct 2026 17:29:28 +0000

Stacks and queues are two of the simplest data structures, and two of the most useful. They show up everywhere: undo history, parsing, graph traversal, job processing, and more. In this post we'll build both from scratch in Go, use them to solve real problems, and look at the idiomatic Go alternatives.

The examples use generics, so you'll need Go 1.18 or newer.

The core idea

Both structures are collections where you can only add and remove items in a specific order.

	Stack	Queue
Order	LIFO: Last In, First Out	FIFO: First In, First Out
Add	`Push`	`Enqueue`
Remove	`Pop`	`Dequeue`
Analogy	A stack of plates	A line at a coffee shop

With a stack, the most recently added item comes out first. With a queue, the item that has waited longest comes out first.

Building a stack

Go doesn't ship a built-in stack type, but a slice is a perfect fit. Appending to the end and removing from the end are both cheap.

package main

import "fmt"

type Stack[T any] struct {
    items []T
}

// Push adds an item to the top of the stack.
func (s *Stack[T]) Push(item T) {
    s.items = append(s.items, item)
}

// Pop removes and returns the top item.
// The bool is false if the stack is empty.
func (s *Stack[T]) Pop() (T, bool) {
    var zero T
    if len(s.items) == 0 {
        return zero, false
    }
    last := len(s.items) - 1
    item := s.items[last]
    s.items[last] = zero // let the garbage collector reclaim the value
    s.items = s.items[:last]
    return item, true
}

// Peek returns the top item without removing it.
func (s *Stack[T]) Peek() (T, bool) {
    var zero T
    if len(s.items) == 0 {
        return zero, false
    }
    return s.items[len(s.items)-1], true
}

func (s *Stack[T]) Len() int      { return len(s.items) }
func (s *Stack[T]) IsEmpty() bool { return len(s.items) == 0 }

func main() {
    s := &Stack[int]{}
    s.Push(1)
    s.Push(2)
    s.Push(3)

    for !s.IsEmpty() {
        v, _ := s.Pop()
        fmt.Println(v) // 3, 2, 1
    }
}

A few Go-specific choices worth noting:

Returning (T, bool) follows the same "comma ok" pattern as map lookups. It avoids panics on an empty stack and avoids returning a misleading zero value without telling the caller.
Zeroing the popped slot matters when T is a pointer or contains pointers. Without it, the underlying array keeps a reference and the garbage collector can't free the object.
Pointer receivers are needed because Push and Pop modify the struct.

Real example: balanced brackets

Checking that (, [, and { are properly matched is the classic stack problem. Every opener gets pushed, and every closer must match the most recent opener.

func isBalanced(s string) bool {
    pairs := map[rune]rune{')': '(', ']': '[', '}': '{'}
    stack := &Stack[rune]{}

    for _, ch := range s {
        switch ch {
        case '(', '[', '{':
            stack.Push(ch)
        case ')', ']', '}':
            top, ok := stack.Pop()
            if !ok || top != pairs[ch] {
                return false
            }
        }
    }
    return stack.IsEmpty()
}

func main() {
    fmt.Println(isBalanced("{[()]}")) // true
    fmt.Println(isBalanced("{[(])}")) // false
    fmt.Println(isBalanced("(("))     // false
}

Other places stacks show up: undo/redo, browser history, expression evaluation, depth-first search, and the call stack itself.

Building a queue

A queue needs to add at one end and remove from the other. The simplest version uses a slice again:

type Queue[T any] struct {
    items []T
}

// Enqueue adds an item to the back of the queue.
func (q *Queue[T]) Enqueue(item T) {
    q.items = append(q.items, item)
}

// Dequeue removes and returns the item at the front.
func (q *Queue[T]) Dequeue() (T, bool) {
    var zero T
    if len(q.items) == 0 {
        return zero, false
    }
    item := q.items[0]
    q.items[0] = zero
    q.items = q.items[1:]
    return item, true
}

func (q *Queue[T]) Len() int      { return len(q.items) }
func (q *Queue[T]) IsEmpty() bool { return len(q.items) == 0 }

func main() {
    q := &Queue[string]{}
    q.Enqueue("a")
    q.Enqueue("b")
    q.Enqueue("c")

    for !q.IsEmpty() {
        v, _ := q.Dequeue()
        fmt.Println(v) // a, b, c
    }
}

This works well for small and medium workloads, but there's a catch. q.items[1:] doesn't free the front of the underlying array. Go reclaims that memory only when append eventually reallocates. For long-lived queues with heavy traffic, you may want something better.

A better queue: `container/list`

The standard library includes a doubly linked list, which gives true O(1) operations at both ends:

import "container/list"

type ListQueue[T any] struct {
    l *list.List
}

func NewListQueue[T any]() *ListQueue[T] {
    return &ListQueue[T]{l: list.New()}
}

func (q *ListQueue[T]) Enqueue(item T) {
    q.l.PushBack(item)
}

func (q *ListQueue[T]) Dequeue() (T, bool) {
    var zero T
    front := q.l.Front()
    if front == nil {
        return zero, false
    }
    q.l.Remove(front)
    return front.Value.(T), true
}

func (q *ListQueue[T]) Len() int { return q.l.Len() }

The trade-off is that container/list stores values as any, so you need a type assertion, and each node is a separate allocation. For most programs, either approach is fine. A ring buffer is the high-performance option if you ever need one.

Real example: breadth-first search

BFS explores a graph level by level, which is exactly what FIFO order gives you:

func bfs(graph map[string][]string, start string) []string {
    visited := map[string]bool{start: true}
    queue := &Queue[string]{}
    queue.Enqueue(start)

    var order []string
    for !queue.IsEmpty() {
        node, _ := queue.Dequeue()
        order = append(order, node)

        for _, next := range graph[node] {
            if !visited[next] {
                visited[next] = true
                queue.Enqueue(next)
            }
        }
    }
    return order
}

func main() {
    graph := map[string][]string{
        "A": {"B", "C"},
        "B": {"D"},
        "C": {"D", "E"},
        "D": {"F"},
        "E": {"F"},
    }
    fmt.Println(bfs(graph, "A")) // [A B C D E F]
}

Swap the queue for a stack and you get depth-first search instead. The traversal logic stays the same; only the order of removal changes.

The idiomatic Go queue: channels

If you need a queue between goroutines, you probably don't want to build one at all. A buffered channel is already a thread-safe FIFO queue:

func main() {
    jobs := make(chan int, 10) // capacity of 10

    go func() {
        for i := 1; i <= 5; i++ {
            jobs <- i // enqueue
        }
        close(jobs)
    }()

    for job := range jobs { // dequeue
        fmt.Println("processing job", job)
    }
}

Our slice and list queues are not safe for concurrent use. If multiple goroutines touch them, you'd need a sync.Mutex. Channels give you ordering, blocking, and synchronization for free, which is why they're the go-to for producer/consumer patterns.

Time complexity

Operation	Stack (slice)	Queue (slice)	Queue (`container/list`)
Push / Enqueue	O(1) amortized	O(1) amortized	O(1)
Pop / Dequeue	O(1)	O(1)	O(1)
Peek	O(1)	O(1)	O(1)

"Amortized" means an occasional append has to grow the underlying array, but averaged over many operations the cost per call is constant.

Which one should you use?

Need to reverse things, backtrack, or track nested structure? Use a stack.
Need to process things in the order they arrived, or explore level by level? Use a queue.
Passing work between goroutines? Use a buffered channel.

Wrapping up

Stacks and queues are small enough to implement in a few lines, yet they power some of the most important algorithms in computing. In Go, a slice gets you surprisingly far, generics keep the code reusable, and channels cover the concurrent case.

A good next step is to try implementing a ring-buffer queue, or a thread-safe stack using sync.Mutex. What data structure should I cover next? Let me know in the comments.

JavaScript's Safe Assignment Operator (?=): Hype vs. Reality

Daniel Keya — Sat, 03 Oct 2026 16:40:19 +0000

If you've scrolled through dev.to lately, you've probably seen headlines like "Say goodbye to try/catch!" promoting a new JavaScript operator: ?=, the safe assignment operator.

It's a neat idea, and worth understanding. But before you rewrite your codebase, there's an important caveat: you can't use it, and it's unlikely you ever will. Let's look at what it proposes, why people liked it, and what you can do today to get the same benefits.

The problem: try/catch gets noisy

Error handling around async code often looks like this:

async function loadUser(id) {
  let response;
  try {
    response = await fetch(`/api/users/${id}`);
  } catch (error) {
    console.error("Network error", error);
    return null;
  }

  let user;
  try {
    user = await response.json();
  } catch (error) {
    console.error("Invalid JSON", error);
    return null;
  }

  return user;
}

Every fallible step needs its own try block. Variables have to be declared with let outside the block so they're visible afterward. The happy path gets buried.

The proposal: errors as values

The safe assignment operator proposal suggests a new operator, ?=, that runs the expression on the right and gives you back a tuple of [error, result]:

async function loadUser(id) {
  const [networkError, response] ?= await fetch(`/api/users/${id}`);
  if (networkError) {
    console.error("Network error", networkError);
    return null;
  }

  const [parseError, user] ?= await response.json();
  if (parseError) {
    console.error("Invalid JSON", parseError);
    return null;
  }

  return user;
}

The rules are simple:

If the call succeeds, you get [null, result].
If it throws (or the promise rejects), you get [error, null].

If you've written Go (val, err := doThing()) or used Rust's Result, this will feel familiar. Putting the error first in the tuple nudges you to check it before touching the data.

The proposal also described a Symbol.result protocol, so objects could define how they report success and failure through that tuple shape.

The reality check: it's not happening (at least not as `?=`)

Here's the part many tutorials skip. The proposal started as a draft from a community member. It got a lot of attention on social media and then a lot of pushback on the ECMAScript discussion boards. Critics argued that:

It isn't really about assignment. The operator acts on the expression on the right, but errors can be thrown in other places before that call even happens.
?= looks too much like existing operators, like the nullish coalescing assignment ??=, and is easy to misread.
It only works in assignments and declarations, while a try-expression approach can be used anywhere an expression can.

The proposal's own repository says it will move toward try-expressions as a more idiomatic way to solve the same problem. Several commenters on dev.to posts about ?= point out that the operator was effectively abandoned. So treat any article telling you this is "coming soon" with skepticism.

Syntax like this will not run in Node, Deno, Bun, or any browser:

const [error, data] ?= await fetch(url); // SyntaxError today

What you can do today

The good news is that the idea doesn't need new syntax. A tiny helper gives you the same ergonomics:

async function safe(promiseOrFn) {
  try {
    const value = await (
      typeof promiseOrFn === "function" ? promiseOrFn() : promiseOrFn
    );
    return [null, value];
  } catch (error) {
    return [error, null];
  }
}

const [error, response] = await safe(fetch("/api/data"));
if (error) {
  // handle it
}

And a synchronous version:

function safeSync(fn) {
  try {
    return [null, fn()];
  } catch (error) {
    return [error, null];
  }
}

const [parseError, config] = safeSync(() => JSON.parse(rawText));

There are also small libraries on npm that do the same thing, if you'd rather not maintain your own helper.

A note on TypeScript

You can make this type-safe with a discriminated tuple:

type Result<T, E = unknown> = [error: null, value: T] | [error: E, value: null];

async function safe<T>(promise: Promise<T>): Promise<Result<T>> {
  try {
    return [null, await promise];
  } catch (error) {
    return [error, null];
  }
}

const [error, user] = await safe(getUser(1));
if (error) return;
user; // narrowed to User

Should you adopt this pattern?

It's worth being honest about the trade-offs.

Pros

Flatter, more linear code
Errors are handled right next to the call that caused them
No let declarations hoisted out of try blocks

Cons

You can silently ignore the error if you forget the if (error) check
It's not idiomatic JavaScript, so teammates may find it unfamiliar
Throwing and catching still works better when you want errors to bubble up to a central handler

A good rule of thumb: use the tuple style for local, expected failures (a network call you want to retry, parsing user input), and keep try/catch for unexpected failures that should propagate.

Wrapping up

The safe assignment operator is a good case study in how JavaScript evolves. A catchy idea spreads fast, the community debates it, and the final shape often differs from the first sketch. ?= itself probably won't ship, but the underlying question of how to make error handling less verbose is very much alive, and try-expressions are where that conversation seems to be heading.

In the meantime, a ten-line helper function gets you most of the benefit today.

What do you think: would you use errors-as-values in your JavaScript, or do you prefer good old try/catch? Let me know in the comments.

Using JavaScript to Build Algorithmic Trading Indicators — A Look at Algocdk

Daniel Keya — Fri, 02 Oct 2026 14:47:01 +0000

Algorithmic trading is often associated with Python, C++, or specialized trading platforms.

But JavaScript can also be a powerful language for building trading indicators and automated trading strategies — especially when the trading platform itself runs in the browser.

In this article, we'll look at how JavaScript can be used to:

Process market data
Calculate technical indicators
Render indicators on charts
Detect trading signals
Build automated trading bots
Create custom strategies

I'll also introduce Algocdk, an algorithmic trading platform I built that allows developers to create custom indicators and trading bots using JavaScript.

Visit Algocdk

Why JavaScript for Algorithmic Trading?

When you think about algorithmic trading, Python is probably one of the first languages that comes to mind.

Python has an incredible ecosystem for quantitative analysis, machine learning, and data science.

However, JavaScript has another important advantage:

It runs natively in the browser.

That makes JavaScript particularly interesting for web-based trading applications.

A developer can write code that:

Receives market data
Processes the data
Calculates an indicator
Renders the result on a chart
Generates a trading signal
Sends the signal to a trading engine

All within a web application.

For example, imagine receiving candle data like this:

const candles = [
    {
        open: 100,
        high: 105,
        low: 98,
        close: 103,
        time: 1710000000
    },
    {
        open: 103,
        high: 108,
        low: 101,
        close: 107,
        time: 1710000060
    }
];

JavaScript can immediately process these candles.

const closes = candles.map(candle => candle.close);

console.log(closes);

Output:

[103, 107]

From there, we can calculate almost any technical indicator we want.

What Is a Trading Indicator?

A trading indicator is essentially a mathematical function that transforms market data into useful information.

For example, a Moving Average calculates the average price over a specific number of candles.

A simple moving average with a period of 5 can be represented as:

SMA = (Price1 + Price2 + Price3 + Price4 + Price5) / 5

The indicator can then be displayed on a price chart.

Other popular indicators include:

SMA
EMA
MACD
RSI
Bollinger Bands
Stochastic Oscillator
ATR
VWAP
Moving Average Crossovers

The interesting part is that these indicators don't have to be hardcoded into a trading platform.

They can be written as JavaScript.

Building a Moving Average in JavaScript

Let's start with a simple Simple Moving Average.

function sma(values, period) {
    const result = new Array(values.length).fill(null);

    for (let i = period - 1; i < values.length; i++) {
        let sum = 0;

        for (let j = i - period + 1; j <= i; j++) {
            sum += values[j];
        }

        result[i] = sum / period;
    }

    return result;
}

We can use it with candle data:

const closes = candles.map(c => c.close);

const movingAverage = sma(closes, 20);

console.log(movingAverage);

Now we have a 20-period moving average calculated from the closing prices.

Moving Average Crossover

One of the simplest algorithmic trading concepts is the Moving Average Crossover.

The idea is to compare two moving averages.

For example:

Fast MA = 9 periods
Slow MA = 21 periods

When the fast moving average crosses above the slow moving average:

Fast MA
    /
   /
  /____ Slow MA
 /

the strategy can generate a bullish signal.

When the fast moving average crosses below the slow moving average:

Slow MA
 ______
       \
        \
         \ Fast MA

the strategy can generate a bearish signal.

Here's a simple JavaScript implementation:

function crossover(fast, slow, index) {
    if (index < 1) {
        return null;
    }

    const previousFast = fast[index - 1];
    const previousSlow = slow[index - 1];

    const currentFast = fast[index];
    const currentSlow = slow[index];

    if (
        previousFast <= previousSlow &&
        currentFast > currentSlow
    ) {
        return "BUY";
    }

    if (
        previousFast >= previousSlow &&
        currentFast < currentSlow
    ) {
        return "SELL";
    }

    return null;
}

This is already the foundation of an algorithmic strategy.

The important part is that we're not manually looking at the chart.

The computer is detecting the crossover for us.

Building the Same Idea in Algocdk

This is where Algocdk comes in.

I built Algocdk as a platform where developers can write their own JavaScript indicators and trading bots.

The platform handles the market-data connection and chart environment while your JavaScript code handles the strategy logic.

An Algocdk indicator is a JavaScript object.

For example:

({
    name: "Moving Average",

    color: "#FF4500",

    lineWidth: 2,

    defaultParams: {
        period: 20
    },

    calculate(data, params) {
        const period = params.period || 20;

        const values = new Array(data.length).fill(null);

        for (let i = period - 1; i < data.length; i++) {

            let sum = 0;

            for (
                let j = i - period + 1;
                j <= i;
                j++
            ) {
                sum += data[j].close;
            }

            values[i] = sum / period;
        }

        return values;
    }
})

The important function here is:

calculate(data, params)

data contains the candle history and params contains the indicator parameters.

Algocdk calls the function as new market data becomes available and uses the returned values to render the indicator.

You don't have to build the WebSocket connection or chart renderer yourself.

You focus on the trading logic.

Creating a MACD Indicator

Let's move to something more interesting.

MACD stands for:

Moving Average Convergence Divergence

MACD commonly uses:

Fast EMA = 12
Slow EMA = 26
Signal EMA = 9

The MACD line is:

MACD = EMA(12) - EMA(26)

Then we calculate the signal line:

Signal = EMA(9) of MACD

And finally:

Histogram = MACD - Signal

Let's implement the EMA first.

function ema(values, period) {
    const result = new Array(values.length).fill(null);

    const multiplier = 2 / (period + 1);

    let previous = values[0];

    for (let i = 0; i < values.length; i++) {

        if (i === 0) {
            result[i] = previous;
            continue;
        }

        previous =
            (values[i] - previous) * multiplier +
            previous;

        result[i] = previous;
    }

    return result;
}

Now we can calculate MACD:

function calculateMACD(
    closes,
    fastPeriod = 12,
    slowPeriod = 26,
    signalPeriod = 9
) {

    const fastEMA = ema(closes, fastPeriod);
    const slowEMA = ema(closes, slowPeriod);

    const macd = closes.map((_, i) => {
        return fastEMA[i] - slowEMA[i];
    });

    const signal = ema(macd, signalPeriod);

    const histogram = macd.map((value, i) => {
        return value - signal[i];
    });

    return {
        macd,
        signal,
        histogram
    };
}

We can now use it:

const closes = candles.map(c => c.close);

const result = calculateMACD(closes);

console.log(result.macd);
console.log(result.signal);
console.log(result.histogram);

This is the core logic behind a MACD indicator.

MACD Inside Algocdk

We can turn the same logic into an Algocdk indicator.

({
    name: "MACD",

    hasWindow2: true,

    defaultParams: {
        fastPeriod: 12,
        slowPeriod: 26,
        signalPeriod: 9
    },

    calculate(data, params) {

        const closes = data.map(c => c.close);

        const fast = ema(
            closes,
            params.fastPeriod
        );

        const slow = ema(
            closes,
            params.slowPeriod
        );

        const macd = closes.map((_, i) => {
            return fast[i] - slow[i];
        });

        const signal = ema(
            macd,
            params.signalPeriod
        );

        return {
            macd,
            signal
        };
    }
})

The hasWindow2 property can be used when the indicator should appear in a separate pane below the main price chart.

This is useful for indicators such as:

MACD
RSI
Stochastic
CCI
Momentum

Instead of drawing everything directly over the price candles.

From Indicators to Trading Bots

This is where things become even more interesting.

An indicator tells us what is happening in the market.

A trading bot can take the next step:

If a condition happens → generate a trading signal.

For example:

if (fastMA > slowMA) {
    return "BUY";
}

if (fastMA < slowMA) {
    return "SELL";
}

Inside Algocdk, a bot can implement this using getSignalAt().

A simplified example:

({
    name: "MA Crossover Bot",

    defaultParams: {
        fastPeriod: 9,
        slowPeriod: 21,
        stake: 1,
        duration: 5,
        duration_unit: "t"
    },

    getSignalAt(candles, index, params) {

        if (candles.length < params.slowPeriod) {
            return null;
        }

        const closes =
            candles.map(c => c.close);

        const fast =
            ema(closes, params.fastPeriod);

        const slow =
            ema(closes, params.slowPeriod);

        if (
            fast[index] > slow[index] &&
            fast[index - 1] <= slow[index - 1]
        ) {
            return {
                signal: "buy",
                stake: params.stake,
                duration: params.duration,
                duration_unit: params.duration_unit,
                label: "MA Cross Up"
            };
        }

        if (
            fast[index] < slow[index] &&
            fast[index - 1] >= slow[index - 1]
        ) {
            return {
                signal: "sell",
                stake: params.stake,
                duration: params.duration,
                duration_unit: params.duration_unit,
                label: "MA Cross Down"
            };
        }

        return null;
    }
})

The important distinction is:

Indicator
    ↓
Analyze market data
    ↓
Return calculated values
    ↓
Render on chart

while a bot can do:

Market data
    ↓
Calculate indicators
    ↓
Evaluate strategy
    ↓
Generate signal
    ↓
Execute trade

Algocdk's bot system supports this type of JavaScript strategy architecture.

Why I Built Algocdk

The idea behind Algocdk started with a simple question:

What if traders and developers could create their own trading tools instead of being limited to the indicators provided by a platform?

That led me to build Algocdk.

Algocdk is an algorithmic trading platform focused on giving developers more control over their trading logic.

Instead of requiring developers to build an entire trading infrastructure, the platform provides the environment where JavaScript code can interact with market data, charts, indicators, and trading strategies.

Developers can create:

Custom JavaScript indicators
Moving averages
Oscillators
Trading signals
Automated trading bots
Digit trading strategies
Custom chart visualizations
Backtesting strategies
Bot marketplace projects

The platform currently provides a Developer Guide with examples for custom indicators and trading bots.

The Interesting Part: You Don't Need a Build System

One of the things I wanted to keep simple was the developer experience.

An indicator can be a plain JavaScript file.

For example:

({
    name: "My Indicator",

    calculate(data, params) {

        return data.map(candle => {
            return candle.close;
        });
    }
})

No React.

No npm installation.

No bundler.

No complicated configuration.

Just JavaScript.

Upload the file and the platform can load it as a custom indicator.

This makes experimentation much faster.

JavaScript Gives You More Than Technical Indicators

Once the market data is available inside JavaScript, you aren't restricted to traditional indicators.

You can build your own logic.

For example:

const closes = data.map(c => c.close);

const recent = closes.slice(-20);

const average =
    recent.reduce((sum, value) => sum + value, 0)
    / recent.length;

You could combine this with:

Price action
Moving averages
Volatility
Momentum
Candle patterns
Statistical analysis
Market regimes
Custom mathematical models

You can even create completely custom chart visualizations using JavaScript and Canvas.

Algocdk supports custom drawing through the indicator draw() function, giving developers control over how their indicator is rendered.

From Code to Trading Platform

The bigger vision behind Algocdk isn't simply:

"Here is another chart."

It's:

Give developers the ability to build their own trading logic.

A developer should be able to go from:

Idea
 ↓
JavaScript
 ↓
Indicator
 ↓
Backtest
 ↓
Strategy
 ↓
Trading Bot

without having to build the entire trading infrastructure from scratch.

That is the direction I'm building Algocdk toward.

Example Project

If you're interested in experimenting with this approach, you can visit:

https://algocdk.site

The platform includes a trading interface, custom indicators, JavaScript bots, developer documentation, strategy tools, and other algorithmic trading features.

The developer documentation is also available directly on the platform:

https://algocdk.site/dev-guide

Final Thoughts

JavaScript isn't traditionally the first language people mention when they talk about algorithmic trading.

But for web-based trading systems, it has an interesting advantage:

The language that powers the interface can also power the trading logic.

With a relatively small amount of JavaScript, we can calculate:

SMA
EMA
MACD
RSI
Bollinger Bands

and then use those calculations to create trading strategies and automated bots.

The important part isn't the programming language itself.

It's the ability to turn an idea into a repeatable algorithm.

That's what I'm trying to make easier with Algocdk.

If you're a JavaScript developer interested in algorithmic trading, I'd love to see what you build.

Build the indicator. Test the strategy. Automate the idea.

Explore Algocdk: https://algocdk.site

Disclaimer: Algorithmic trading involves financial risk. The examples in this article are educational and are not financial advice or a guarantee of trading profits.

Day 5 of 30: Message Queues and Why Not Everything Needs an Instant Response

Daniel Keya — Sat, 11 Jul 2026 18:01:53 +0000

Day 1: the big picture. Day 2: databases. Day 3: caching. Day 4: load balancing, and how the load balancer itself becomes a new single point of failure. Today: message queues and asynchronous processing — a topic that quietly reframed how I think about "handling a request."

The Assumption I Didn't Know I Had

Up until today, my mental model of a system was: request comes in, server does the work, response goes out, all in one continuous chain. Synchronous, start to finish.

Today's lesson was basically: that assumption breaks down the moment any part of the work is slow, unreliable, or doesn't need to happen right now. Sending a confirmation email, resizing an uploaded image, generating a report, notifying a third-party service — none of that needs to block the response the user is waiting on.

What a Message Queue Actually Does

A message queue sits between two parts of a system — a producer that creates a task or event, and a consumer that processes it — and holds messages in between until the consumer is ready. The producer doesn't wait around for the consumer to finish; it just drops the message in the queue and moves on.

That single shift — not waiting — is what decoupling really means in practice. The producer and consumer don't need to run at the same speed, be available at the same time, or even know much about each other. They just agree on the shape of the message.

Why This Matters for Scaling

A few reasons this clicked for me as a system design concept, not just an engineering convenience:

Load leveling — if a sudden burst of requests comes in, the queue absorbs the spike. Consumers can keep processing at a steady rate instead of getting overwhelmed all at once. This is the async equivalent of what caching does for reads.
Failure isolation — if the consumer service goes down temporarily, messages just wait in the queue instead of being lost or crashing the producer. When the consumer comes back up, it picks up where it left off.
Independent scaling — producers and consumers can scale separately. If processing is the bottleneck, you add more consumers without touching the producer at all.

This ties directly back to horizontal scaling from Day 1 — message queues are one of the main tools that make horizontal scaling of background work practical, not just scaling the request-handling servers.

Delivery Guarantees: The Part That Gets Complicated Fast

I expected this section to be simple. It was not. Queues generally offer one of a few delivery guarantees:

At-most-once — a message is delivered zero or one times. Fast, but messages can be lost.
At-least-once — a message is guaranteed to be delivered, but might be delivered more than once. Common default, but it means consumers need to be idempotent — processing the same message twice shouldn't cause duplicate side effects (e.g., charging a customer twice).
Exactly-once — the message is delivered exactly one time, no duplicates, no loss. Sounds ideal, but it's genuinely hard to guarantee in a distributed system and usually comes with real performance cost.

The idempotency point stuck with me the most. It's not enough to just add a queue — the consumer logic has to be written defensively, because "at-least-once" is the realistic default in most systems, not "exactly-once."

Queue vs. Pub/Sub

I also drew a distinction I'd been fuzzy on:

Message queue (point-to-point) — each message is processed by exactly one consumer. Good for distributing discrete units of work, like "process this uploaded file."
Pub/Sub (publish-subscribe) — a message is broadcast to all subscribers interested in that event. Good for things like "notify every service that cares that an order was placed" — inventory, billing, and notifications can all react independently to the same event.

Same underlying idea (decoupled, asynchronous communication), but the delivery pattern is fundamentally different depending on whether you need one consumer or many.

What Clicked Today

The biggest mental shift: not every part of a system needs to run inside the request-response cycle. Deciding what's synchronous and what's asynchronous is itself a design decision — and getting it right is often what separates a system that stays responsive under load from one that falls over the moment traffic spikes.

It also reinforced something from Day 3 and Day 4: decoupling components (via caching, load balancers, or queues) keeps showing up as the general strategy for making a system resilient. Different tools, same underlying move.

Tomorrow

Next up: API design and communication patterns — REST vs. GraphQL vs. gRPC, and how services actually talk to each other once you've got more than one of them.

If you've built something with a message queue, I'd love to hear how you handled idempotency — that feels like the kind of thing that's easy to get wrong quietly.

This is part of a 30-day series on learning system design from scratch. Catch up on Day 1, Day 2, Day 3, and Day 4 if you missed them.

title: Day 4 of 30: Load Balancing and the Single Point of Failure Problem

Daniel Keya — Wed, 08 Jul 2026 20:21:42 +0000

Day 1: the big picture. Day 2: databases. Day 3: caching, and how it's really a consistency problem in disguise. Today: load balancing — how traffic actually gets spread across servers, and the ways that can quietly go wrong.

Why Load Balancers Exist

Once you've horizontally scaled (Day 1 flashback) and have multiple servers instead of one, something needs to decide which server handles which request. That's the load balancer's job: sit between the client and your fleet of servers, and distribute incoming traffic so no single server gets overwhelmed while others sit idle.

It sounds simple — "just spread the requests out" — but how you spread them out turns out to matter a lot.

Load Balancing Algorithms

I went through the main ones:

Round robin — requests go to servers in sequence, one after another. Simple, but doesn't account for server load — if one request is expensive and another is cheap, round robin doesn't know or care.
Least connections — send the next request to whichever server currently has the fewest active connections. More adaptive than round robin, especially when requests vary a lot in how long they take.
Weighted round robin / weighted least connections — same ideas as above, but servers with more capacity get proportionally more traffic. Useful when your fleet isn't made of identical machines.
IP hash — the client's IP determines which server it's routed to, so the same client tends to land on the same server. This matters for session persistence (also called "sticky sessions") — some applications need a user's requests to keep hitting the same server, e.g., if session data is stored locally on that server instead of in a shared store.

The thing that stood out: there's no universally "best" algorithm. Round robin is fine when requests are roughly uniform. Least connections is better when they're not. IP hash matters when statefulness matters. It's another trade-off decision, not a default setting.

Layer 4 vs. Layer 7 Load Balancing

This distinction took me a minute to really get:

Layer 4 (transport layer) load balancers route based on IP address and port, without looking at the actual content of the request. Faster, simpler, but less flexible.
Layer 7 (application layer) load balancers can inspect the actual request — HTTP headers, URL paths, cookies — and make smarter routing decisions, like sending /api/video traffic to one set of servers and /api/search to another. More flexible, but more overhead per request.

This maps neatly onto the classic latency vs. flexibility trade-off from Day 1: Layer 4 is faster because it knows less; Layer 7 is smarter because it does more work per request.

Health Checks: How the Load Balancer Knows What's Alive

A load balancer is useless if it keeps sending traffic to a server that's already down. Health checks solve this — the load balancer periodically pings each server (or checks a dedicated health endpoint), and stops routing traffic to any server that fails to respond. This was a good reminder that load balancing isn't just about distribution, it's also about failure detection — the load balancer is quietly doing double duty as a monitoring system.

The Load Balancer Becomes a Single Point of Failure

Here's the part that really made me stop and think: I spent three days learning how to avoid single points of failure by scaling horizontally — only to introduce one giant new single point of failure sitting in front of all those servers. If the load balancer itself goes down, it doesn't matter how many healthy backend servers you have.

The common fix is to not have just one load balancer — instead, you run multiple load balancer instances behind something like DNS round robin or a floating/virtual IP, so there's no single machine the entire system depends on. It's a very "turtles all the way down" realization: every layer you add to solve a scaling problem can itself become the next bottleneck or failure point if you don't also scale and replicate that layer.

What Clicked Today

Load balancing isn't just "spread traffic evenly." It's a small system of its own, with its own trade-offs (routing algorithm, OSI layer, statefulness) and its own failure modes (the load balancer itself needs redundancy). I'm noticing a theme across all four days now: every solution to a scaling problem introduces a new, smaller version of the same problem one layer up.

Tomorrow

Next up: message queues and asynchronous processing — why not everything needs to happen in the request-response cycle, and how queues help decouple services from each other.

If you've dealt with a load balancer misconfiguration that caused uneven traffic or a nasty outage, I'd love to hear about it — feels like exactly the kind of thing that's obvious in hindsight and invisible beforehand.

This is part of a 30-day series on learning system design from scratch. Catch up on Day 1, Day 2, and Day 3 if you missed them.

Day 3 of 30: Caching, and Why Invalidation Really Is That Hard

Daniel Keya — Tue, 07 Jul 2026 20:52:07 +0000

Recap

Day 1 was the big picture (client-server, scaling, latency vs. throughput). Day 2 went into databases (SQL vs. NoSQL, replication, sharding). Today's topic was one I thought I already understood: caching. Turns out I only understood the easy 20% of it.

Why Caching Exists

The core idea is simple: fetching data is expensive (database queries, network calls, computation), so you store a copy of the result somewhere faster and cheaper to access, and serve from that copy instead of redoing the work every time.

But "store a copy somewhere faster" hides a lot of decisions: where do you store it, how long do you keep it, and how do you know when that copy is no longer accurate?

Where Caches Live

I mapped out the different layers where caching shows up in a typical system, and it's more places than I expected:

Client-side caching — browser cache, mobile app cache. Avoids hitting the network at all.
CDN caching — caching static assets (images, scripts, videos) geographically close to the user.
Application-layer caching — an in-memory store like Redis or Memcached sitting between your app server and your database.
Database caching — query result caching or buffer pools built into the database itself.

The pattern I noticed: the closer the cache is to the user, the faster it is, but the more stale it's likely to become and the harder it is to invalidate reliably.

Caching Strategies

A few patterns kept coming up:

Cache-aside (lazy loading) — the application checks the cache first; on a miss, it fetches from the database and writes the result to the cache. Simple, and only caches what's actually requested, but the first request always pays the full cost.
Write-through — data is written to the cache and the database at the same time. Keeps the cache consistent, but adds latency to every write.
Write-behind (write-back) — data is written to the cache immediately and to the database asynchronously later. Fast writes, but riskier if the cache fails before the write reaches the database.

None of these is universally "correct." Cache-aside is the one I'd reach for by default, but write-through makes sense when staleness is unacceptable, and write-behind makes sense when write throughput matters more than durability guarantees.

Cache Invalidation: The Actually Hard Part

This is where the famous line clicked for me: there really are only two hard problems in computer science — cache invalidation, naming things, and off-by-one errors (the joke itself is a joke about how hard naming and counting things are). Invalidation is hard because you're trying to answer "has the source of truth changed?" without constantly re-checking the source of truth, which would defeat the point of caching in the first place.

The strategies I looked at:

TTL (time-to-live) — the cache entry just expires after a fixed time. Simple, predictable, but you're always trading off freshness against cache-hit rate.
Explicit invalidation — when the underlying data changes, the application actively tells the cache to drop or update that entry. More accurate, but adds coupling between every write path and the cache layer.
Event-based invalidation — changes publish an event (e.g., through a message queue), and any cache watching that event invalidates itself. More scalable across multiple services, but adds infrastructure and a new failure mode: what if the event never arrives?

Cache Stampede — A Failure Mode I Hadn't Considered

One thing that stood out: what happens when a popular cache entry expires and a huge number of requests hit the database at the exact same moment because they all missed the cache simultaneously? This is called a cache stampede (or thundering herd), and it's a good example of how a cache — a tool meant to protect your database — can end up hammering it hardest at the worst possible moment if you're not careful. Mitigations like staggered expiration times or "locking" so only one request repopulates the cache while others wait came up as common fixes.

What Clicked Today

Caching looks like a performance optimization on the surface, but it's really a consistency problem wearing a performance-optimization costume. Every caching decision is really a decision about how stale you're willing to let your data be, and for how long, in exchange for speed.

That's the same trade-off shape I kept running into on Day 1 (CAP theorem) and Day 2 (replication). It's starting to feel like most of system design is a small number of trade-offs showing up over and over in different clothes.

Tomorrow

Next up: load balancing — how traffic actually gets distributed across servers, different load balancing algorithms, and where load balancers themselves become a single point of failure if you're not careful.

If you've been burned by a cache invalidation bug in production, I'd genuinely love to hear the story — those tend to be the most memorable ones.

This is part of a 30-day series on learning system design from scratch. Catch up on Day 1 and Day 2 if you missed them.

Day 2 of 30: Databases, Replication, and Sharding published: false tags: systemdesign, beginners, learning, databases

Daniel Keya — Mon, 06 Jul 2026 16:25:47 +0000

Recap

Yesterday was about the big picture — client-server architecture, scaling, latency vs. throughput, and a first look at the CAP theorem. Today I went one layer deeper: databases, and specifically, how they hold up when a system needs to scale.

This turned out to be a bigger topic than I expected. I thought "SQL vs. NoSQL" would be the whole day. It was maybe a third of it.

SQL vs. NoSQL: Less "Which Is Better," More "Which Fits"

I used to think of this as a binary choice where one option was obviously modern and the other outdated. That's the wrong frame. It's really about what shape your data takes and how you need to query it.

SQL (relational) databases — think PostgreSQL, MySQL:

Structured data with defined relationships
Strong consistency guarantees (ACID transactions)
Great when your data has clear relationships and you need to enforce integrity (e.g., financial systems, orders tied to users tied to inventory)

NoSQL databases — think MongoDB, Cassandra, DynamoDB:

Flexible or schema-less data models
Built to scale horizontally more naturally
Great for unstructured or fast-changing data, high write throughput, or massive scale where strict consistency can be relaxed

The real takeaway: the choice isn't about which database is "better" — it's about which trade-offs your system can tolerate. A banking system probably shouldn't sacrifice consistency for speed. A social media feed's like counter probably can.

Replication: Copies for Reliability and Speed

Replication means keeping copies of the same data on multiple machines. I looked at two common patterns:

Leader-follower (master-replica) replication — one node handles writes, and changes propagate to follower nodes that mostly handle reads. This spreads read load and adds redundancy if the leader goes down.
Multi-leader replication — multiple nodes can accept writes, which is more resilient but opens the door to write conflicts that need to be resolved somehow.

The concept that stuck with me most: replication isn't free. The moment you have copies of data, you have to decide how quickly those copies need to agree with each other. That's where I ran straight back into the CAP theorem from yesterday — replication is where "choose consistency or availability" actually shows up in a real design decision.

Sharding: Splitting Data Across Machines

If replication is about copying the same data across machines, sharding is about splitting different data across machines. Instead of one giant database holding everything, you partition data — often by a key like user ID or region — so no single machine has to hold or serve all of it.

A few things I noted:

Sharding key choice matters a lot. Pick a bad key (like something unevenly distributed) and you end up with "hot shards" — some machines overloaded while others sit idle.
Sharding solves a storage/throughput problem, but introduces new ones: queries that need data from multiple shards get more complex, and cross-shard transactions are painful.
This is usually a "last resort" scaling technique — you shard once vertical scaling and replication aren't enough anymore.

Replication + Sharding Together

In real large-scale systems, these aren't either/or — they're often combined. You shard data across multiple database clusters, and within each shard, you replicate for redundancy and read scaling. It was helpful to picture this as a grid: shards along one axis, replicas along the other.

What Clicked Today

The biggest realization: most of these decisions come back to the same question I hit yesterday — what are you willing to give up, and when?

Give up some consistency to get more availability
Give up simplicity to get more scale
Give up flexibility to get stronger guarantees

Database design isn't a checklist. It's a series of trade-off calls based on what the system actually needs to do.

Tomorrow

Next up: caching. Where caches fit in a system, cache invalidation strategies, and why "there are only two hard things in computer science: cache invalidation and naming things" is apparently a very real problem and not just a joke.

If you work with distributed databases day to day — what's a sharding or replication decision that bit you later on? I'd love to hear the war stories.

*This is part of a 30-day series on learning system design from scratch. Catch up on Day 1 if you missed it.*v

Day 1 of 30: Kicking Off My System Design Learning Journey published: false tags: systemdesign, beginners, learning, softwareengineering

Daniel Keya — Sun, 05 Jul 2026 13:37:40 +0000

Why I'm Doing This

I've been writing code professionally for a while now, but there's a gap in my knowledge that keeps showing up in interviews and in my own projects: system design. I can build a feature, ship an API, write clean code — but ask me to design something that needs to handle millions of users, and I start hand-waving.

So I'm giving myself 30 days to fix that. This is Day 1 of a series where I'll document what I learn, the mistakes I make, and the "aha" moments along the way. If you're on a similar journey, follow along — maybe we can learn together.

What Even Is System Design?

At its core, system design is about answering one question: how do you build software that works reliably at scale?

That means thinking about things like:

How does data move between a client and a server?
What happens when one server isn't enough?
How do you keep data consistent across multiple machines?
What breaks first when traffic spikes 100x?
How do you design for failure, not just for the happy path?

It's less about writing code and more about making trade-offs. Every decision in system design — cache vs. no cache, SQL vs. NoSQL, sync vs. async — comes with a cost. Today was about internalizing that mindset shift.

Topics I Covered Today

1. Client-Server Architecture

The absolute foundation. A client sends a request, a server processes it and sends back a response. Simple in theory, but I spent time really understanding the layers involved:

DNS resolution
Load balancers
Application servers
Databases

Mapping out a basic request lifecycle (browser → DNS → load balancer → server → database → back) made a lot of "why do we need X" questions click into place.

2. Vertical vs. Horizontal Scaling

Vertical scaling = throwing more resources (CPU, RAM) at a single machine. Simple, but has a ceiling.
Horizontal scaling = adding more machines. More complex (hello, distributed systems problems), but it's how you actually scale to millions of users.

Key takeaway: almost every large-scale system eventually needs to scale horizontally, and that decision ripples into everything else — session management, data consistency, caching strategy.

3. Latency vs. Throughput

Two terms I always mixed up:

Latency — how long a single request takes
Throughput — how many requests you can handle per unit of time

You can optimize for one at the expense of the other. Understanding which one matters more for a given system (e.g., a stock trading platform cares deeply about latency; a batch data pipeline cares more about throughput) is a design decision in itself.

4. The CAP Theorem (just the surface)

I only scratched the surface here, but the core idea stuck: in a distributed system, you can't have perfect Consistency, Availability, and Partition tolerance all at once. Since network partitions are a fact of life, real systems have to choose between prioritizing consistency or availability when a partition happens.

I know I'll be coming back to this one in more depth later in the series.

My Study Approach for These 30 Days

To keep this from turning into passive video-watching, I'm structuring each day like this:

Learn one core concept (30–45 minutes of reading/videos)
Sketch it out — diagrams, not just notes. System design is visual.
Explain it simply — if I can't explain a concept in plain English, I don't actually understand it yet
Write it down — hence this series

What's Next

Tomorrow I'm diving into databases: SQL vs. NoSQL, when to use each, and how replication and sharding actually work under the hood.

If you've done a deep dive into system design before, I'd love to hear:

What tripped you up early on?
Any resources you'd recommend?
Anything you wish someone had told you on day 1?

Drop a comment below. See you on Day 2.

This is part of a 30-day series on learning system design from scratch. Follow along for the rest of the journey.

Why I Fell in Love with Go — And Why You Should Give It a Chance

Daniel Keya — Thu, 04 Jun 2026 14:16:50 +0000

Tags: go, golang, beginners, programming

If you told me two years ago that I'd be writing a love letter to a programming language, I'd have laughed. But here I am.

Go (or Golang) changed the way I think about software development. And if you haven't tried it yet — or tried it and gave up — I want to make the case for why 2024 is the perfect year to give it a real shot.

What Even Is Go?

Go is an open-source programming language designed at Google by Robert Griesemer, Rob Pike, and Ken Thompson. It was released in 2009, but it has exploded in popularity over the last few years, powering tools you probably use every day: Docker, Kubernetes, Terraform, Prometheus, and more.

It sits in a sweet spot: it has the performance of a compiled language like C, but it feels almost as productive as Python or JavaScript. That's a rare combination.

The Things That Will Surprise You

1. The Simplicity Is the Point

Go has around 25 keywords. Compare that to Java (~50) or C++ (~90+). The language designers made a deliberate choice: less is more.

package main

import "fmt"

func main() {
    fmt.Println("Hello, Gopher! 🐹")
}

That's it. No classes. No inheritance. No decorators. Just clean, readable code. When you read someone else's Go code, you can actually understand it.

2. Concurrency Is a First-Class Citizen

One of Go's killer features is goroutines — lightweight threads that let you run things concurrently without the headaches of traditional threading.

package main

import (
    "fmt"
    "time"
)

func sayHello(name string) {
    time.Sleep(100 * time.Millisecond)
    fmt.Printf("Hello, %s!\n", name)
}

func main() {
    go sayHello("Alice")
    go sayHello("Bob")
    go sayHello("Charlie")

    time.Sleep(500 * time.Millisecond)
}

Launching a goroutine costs just a few kilobytes of memory. You can run hundreds of thousands of them. This makes Go exceptional for building APIs, microservices, and anything that handles lots of simultaneous connections.

3. The Tooling Is Incredible

Go ships with everything you need baked in:

go fmt — formats your code automatically (no more arguments about tabs vs spaces)
go test — runs your tests
go build — compiles to a single binary
go vet — catches common mistakes
go mod — manages dependencies

No decision fatigue. No webpack configs. No 47 different linting tools to choose from. You just... write code.

4. It Compiles to a Single Binary

When you build a Go application, you get one binary file. No runtime to install. No dependencies to bundle. You can copy that file to any Linux server and it will just run.

GOOS=linux GOARCH=amd64 go build -o myapp .

For DevOps and deployment, this is a game-changer.

A Real Example: Building a Simple HTTP Server

Here's a fully working HTTP server in Go. No frameworks. No npm install. Just Go's standard library:

package main

import (
    "fmt"
    "net/http"
)

func helloHandler(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Welcome to Go! You're going to love it here. 🐹")
}

func main() {
    http.HandleFunc("/", helloHandler)
    fmt.Println("Server running on http://localhost:8080")
    http.ListenAndServe(":8080", nil)
}

Run it with go run main.go. That's a production-grade HTTP server in under 15 lines.

"But Go Doesn't Have Generics…"

It does now! Go 1.18 (released in 2022) introduced generics, one of the most requested features in the language's history. The Go team took their time to get it right, and the result is a clean, practical implementation:

func Map[T, U any](slice []T, fn func(T) U) []U {
    result := make([]U, len(slice))
    for i, v := range slice {
        result[i] = fn(v)
    }
    return result
}

The language keeps getting better while staying true to its philosophy of simplicity.

A Word of Encouragement 💪

Learning a new programming language can feel overwhelming. You're going to write weird code at first. You're going to fight the compiler. You're going to wonder why there are no classes and feel slightly lost.

That's completely normal. That's part of the process.

Here's what I want you to know:

The Go community is one of the most welcoming in tech. The error messages are actually helpful. The documentation at pkg.go.dev is outstanding. And the moment things click — the moment you write a concurrent program that handles 10,000 requests per second on a tiny server — it's deeply satisfying.

You don't need to be a senior engineer to learn Go. You don't need a computer science degree. If you can write a for loop in any language, you can start learning Go today.

Where to Start

Here are the best free resources to get going (pun intended):

A Tour of Go — The official interactive tutorial. Do this first.
Go by Example — Hands-on examples for every feature.
The Go Playground — Write and run Go in your browser, no install needed.
Effective Go — Once you know the basics, this will make you write good Go.

Final Thoughts

Go won't solve every problem. It's not the best choice for machine learning, desktop apps, or scripting. But for backend services, CLIs, DevOps tooling, and APIs, it is hard to beat.

If you're a developer looking to add a powerful, pragmatic language to your toolkit — one that will make you more productive and more employable — Go deserves a serious look.

Start small. Build a CLI. Write a small REST API. Deploy it as a single binary and feel the magic.

The Gopher community is waiting for you. 🐹

Did this post inspire you to try Go? Drop a comment below — I'd love to hear what you're building. And if you're already a Gopher, what was the moment Go finally clicked for you?

If you found this helpful, consider following me for more posts on Go, backend development, and software craftsmanship.

Building My First React Component with TypeScript: A Beginner's Breakdown

Daniel Keya — Tue, 02 Jun 2026 21:53:12 +0000

So I just started learning React with TypeScript, and I want to share what I built and — more importantly — what I actually learned from it. If you're also new to this stack, this one's for you.

Here's the component I wrote:

import { useState } from "react";
import { Fragment } from "react/jsx-runtime";

interface Props {
  items: string[];
  heading: string;
}

export default function ListGroup({ items, heading }: Props) {
  const [selectdIndex, setSelectedIndex] = useState(-1);
  return (
    <Fragment>
      <h1>{heading}</h1>
      {items.length === 0 && <p>No item found</p>}
      <ul className="list-group">
        {items.map((item, index) => (
          <li
            className={
              selectdIndex === index
                ? "list-group-item active"
                : "list-group-item"
            }
            key={item}
            onClick={() => {
              setSelectedIndex(index);
            }}
          >
            {item}
          </li>
        ))}
      </ul>
    </Fragment>
  );
}

It's a ListGroup component — a clickable list that highlights whichever item you select. Simple, but packed with fundamentals. Let me walk you through what I learned.

1. TypeScript Interfaces for Props

interface Props {
  items: string[];
  heading: string;
}

In plain React (JavaScript), you can pass anything as props and React won't complain — until something breaks at runtime. TypeScript fixes this by letting you define exactly what a component expects.

Here, I'm saying: "This component must receive an array of strings (items) and a string (heading). Nothing else, nothing less."

The component signature then uses destructuring to pull those props out cleanly:

function ListGroup({ items, heading }: Props)

This is cleaner than writing props.items and props.heading everywhere.

2. useState — Tracking Which Item Is Selected

const [selectdIndex, setSelectedIndex] = useState(-1);

useState is React's way of giving a component memory. Without it, clicking an item would do nothing visible — React wouldn't know anything changed.

-1 is the initial value, meaning nothing is selected yet
selectdIndex holds the current state
setSelectedIndex is the function we call to update it

💡 Quick note: There's a typo here — selectdIndex should be selectedIndex. These things happen when you're learning! TypeScript won't catch variable name typos, only type mismatches.

3. Fragment — Because JSX Needs One Root Element

return (
  <Fragment>
    <h1>{heading}</h1>
    ...
  </Fragment>
);

JSX has a rule: every component must return a single root element. But wrapping everything in an extra <div> just to satisfy that rule can mess up your CSS layout.

Fragment is the solution — it's an invisible wrapper that groups elements without adding anything to the DOM. You can also write it as the shorthand <>...</>.

4. Conditional Rendering

{items.length === 0 && <p>No item found</p>}

This is a neat React pattern. If items.length === 0 is true, React renders the <p> tag. If it's false, React renders nothing. It's the JSX equivalent of an if statement, inline.

5. Rendering a List with `.map()`

{items.map((item, index) => (
  <li key={item}>
    {item}
  </li>
))}

To render a list dynamically in React, you use .map() to transform each array item into a JSX element.

The key prop is important — React uses it internally to track which items have changed, been added, or removed. Without it, you'll get a console warning and potentially buggy re-renders.

6. Dynamic CSS Classes

className={
  selectdIndex === index
    ? "list-group-item active"
    : "list-group-item"
}

This is a ternary operator used to conditionally apply a CSS class. If the current item's index matches selectdIndex, it gets the active class (which highlights it in Bootstrap). Otherwise, it just gets the base class.

It's a clean way to reflect state visually.

7. Handling Click Events

onClick={() => {
  setSelectedIndex(index);
}}

When a list item is clicked, we call setSelectedIndex(index) with the clicked item's index. React then re-renders the component with the new state, and the correct item gets the active class.

Putting It All Together

Here's the full picture of what happens when you use this component:

A parent component passes in an items array and a heading string
The component renders the heading, an empty-state message (if needed), and the list
When you click an item, setSelectedIndex updates the state
React re-renders, and the clicked item gets highlighted

What I'd Improve Next

Fix the typo: selectdIndex → selectedIndex
Use <>...</> shorthand instead of <Fragment> (cleaner to read)
Emit the selected item to the parent via an onSelectItem callback prop — right now the selection is trapped inside the component

Final Thoughts

This small component taught me a lot: TypeScript interfaces, state management, conditional rendering, list rendering, and event handling — all in ~30 lines of code. React with TypeScript felt intimidating at first, but once you see how the pieces fit together, it starts to click.

If you're also just starting out, I hope this breakdown helped. Let me know what you're building in the comments!

Tags: #react #typescript #beginners #webdev

Designing TikTok from Scratch — A System Design Deep Dive

Daniel Keya — Mon, 25 May 2026 22:24:05 +0000

Who is this for? Mid-to-senior engineers preparing for system design interviews, or anyone curious how a short-video platform at billion-user scale actually works under the hood.

Scale We're Designing For

Metric	Number
Monthly active users	1B+
Videos uploaded per day	~34 million
Target feed latency (P99)	~167ms
Peak egress bandwidth	~26 Tbps

1. Requirements

Before drawing a single box, nail down what the system must do — and what it doesn't need to do perfectly on day one.

Functional requirements:

Upload and transcode short videos
Serve a personalized "For You" feed
Like, comment, share, follow
Search videos and creators
Live streaming

Non-functional requirements:

High availability (99.99% uptime)
Sub-200ms feed latency
Horizontal scalability
Global CDN video delivery
Strong eventual consistency

2. High-Level Architecture

The system splits into four major domains: ingestion (upload pipeline), serving (read path), recommendation (ML feed), and social graph.

┌─────────────────────────────────────────────────┐
│              Mobile / Web Clients                │
└─────────────────────┬───────────────────────────┘
                      │
┌─────────────────────▼───────────────────────────┐
│         Global CDN / Edge PoPs                   │
│   Video delivery, static assets, geo-routing    │
└─────────────────────┬───────────────────────────┘
                      │
┌─────────────────────▼───────────────────────────┐
│       API Gateway + Load Balancer                │
│   Auth, rate limiting, routing, TLS termination │
└────────┬────────────┴────────────────┬──────────┘
         │                             │
   ┌─────▼──────┐  ┌──────────────┐  ┌▼────────────────┐
   │  Upload    │  │ Feed Service │  │  Social Graph   │
   │  Service   │  │(pre-compute  │  │    Service      │
   │            │  │ + real-time) │  │                 │
   └─────┬──────┘  └──────┬───────┘  └┬────────────────┘
         │                │            │
   ┌─────▼──────┐  ┌──────▼───────┐  ┌▼────────────────┐
   │ Transcode  │  │Recommendation│  │  Notification   │
   │  Workers   │  │   Engine     │  │    Service      │
   └─────┬──────┘  └──────┬───────┘  └┬────────────────┘
         │                │            │
   ┌─────▼──────┐  ┌──────▼───────┐  ┌▼────────────────┐
   │  Object    │  │ Feature Store│  │  Search Service │
   │  Storage   │  │(Redis+Cassie)│  │ (Elasticsearch) │
   └─────┬──────┘  └──────┬───────┘  └┬────────────────┘
         │                │            │
┌────────▼────────────────▼────────────▼──────────────┐
│              Async Message Bus (Kafka)               │
└──────────┬──────────────┬──────────────┬────────────┘
           │              │              │
    ┌──────▼─────┐ ┌──────▼────┐ ┌──────▼──────┐
    │MySQL/Vitess│ │   Redis   │ │  Cassandra  │
    │(user data, │ │ (counters,│ │ (timelines, │
    │ metadata)  │ │  cache)   │ │  history)   │
    └────────────┘ └───────────┘ └─────────────┘

All services communicate asynchronously via Kafka for non-critical paths.

3. Key Components Explained

CDN + Edge PoPs

TikTok's secret weapon. ~70% of video traffic is served directly from edge nodes in 150+ cities, bypassing origin entirely. It uses Anycast routing to send users to the nearest PoP. Manifest files (playlist URLs) are invalidated within seconds of a video going viral.

Upload Pipeline

Chunked multi-part upload (5 MB chunks) tolerates flaky mobile connections. Workers dedup via SHA-256 before writing. Transcode jobs run on GPU fleets — outputs include 360p, 720p, 1080p, and HEVC variants. Thumbnails and stills are extracted for ML feature generation.

Recommendation Engine

A two-tower neural network:

Tower 1 — encodes user state (watch history, device, time of day, location)
Tower 2 — encodes video features (visual embeddings, audio, caption text)

Dot product gives a relevance score. The model runs online for top-k retrieval, then a ranker applies real-time signals (trending, friend activity) before the feed is assembled.

Feed Assembly (Pre-compute + Real-time Merge)

This is where TikTok differs from Twitter/Instagram:

Celebrity/high-follow accounts — fan-out on write (posts pushed to follower inboxes eagerly)
Regular accounts — fan-out on read (merged at request time)

The feed service merges both lists, injects ML-recommended videos, and applies diversity rules to avoid repetition. Final feed is cached in Redis with a 300s TTL.

Kafka Message Bus

All write events (upload complete, like, follow, watch-complete) are published to Kafka topics. Downstream consumers include:

Analytics pipeline
Notification fan-out
ML feature store updater
Search indexer

Topics are partitioned by user_id for ordered processing per user. This decouples services and allows independent scaling.

Database Strategy

Store	Use Case	Why
MySQL / Vitess	User profiles, video metadata, social graph	ACID, sharded by `user_id`
Redis Cluster	Counters (likes, views), session tokens, feed cache	Sub-millisecond reads
Cassandra	Watch history, timelines, notification logs	Wide-row reads, high write throughput

4. Key Design Trade-offs

Fan-out on Write vs Read

The classic dilemma in social feed systems. TikTok uses a hybrid approach (the "celebrity problem" split):

Fan-out on write (for accounts with millions of followers):

Read path is O(1) — just read the inbox
Fast feed assembly at serving time
Massive write amplification when a celebrity posts

Fan-out on read (for regular users):

No write amplification on post
Storage-efficient
Slower feed assembly if following thousands of accounts

Eventual vs Strong Consistency

Like/view counts can lag by a few seconds — nobody notices. But user authentication tokens and billing events require strong consistency. TikTok segments these into separate storage tiers with different consistency guarantees, accepting complexity for throughput on hot paths.

Push vs Pull for Notifications

Likes and comments use WebSocket push for real-time delivery. Less critical notifications (weekly summaries, suggested follows) use a pull-based batch pipeline that runs every few hours — no need to maintain a persistent connection for a weekly digest email.

5. Back-of-Envelope Estimates

Assumptions: 1B MAU, 500M DAU, avg user watches 45 min/day, avg video = 30 sec ~= 8 MB (720p). 34M uploads/day ~= 400 uploads/sec peak.

Storage:

34M uploads/day x 8 MB x 3 resolutions = ~816 TB/day of new video
With 3x replication over 5 years = ~4.4 EB total raw storage

Feed reads:

500M DAU x 20 feed refreshes/day / 86,400 sec = ~115,000 feed reads/sec
With 95% Redis cache hit rate -> recommendation backend sees ~5,750 rps

Bandwidth:

500M users x 45 min x 2 Mbps (720p) / 86,400 = ~26 Tbps peak egress

This is why TikTok operates its own backbone in many regions and has deep-peering agreements with major ISPs.

6. What Makes TikTok's Architecture Special?

Most social platforms optimize for social graph traversal — show me what people I follow posted. TikTok inverted this: the algorithm is the product. The architecture is built around a recommendation pipeline that must be both blazing-fast and constantly learning from watch signals.

Three things stand out:

Aggressive edge caching — they push video delivery as close to the user as physically possible. The CDN is not a performance optimization; it is the entire delivery strategy.
Real-time ML feedback loops — a video's trajectory is decided in the first 30 minutes based on completion rate signals. A new creator can go viral without any followers.
Microservice isolation — upload, serving, recommendation, and social graph are independently deployable and scalable, preventing any single bottleneck from cascading.

Interview Tips

If you're using this for a system design interview:

Start with requirements — always clarify scale before designing anything
Estimate first — back-of-envelope math shows you understand the constraints
Sketch the high-level diagram — then dive into the component your interviewer cares about
Talk through trade-offs — interviewers want reasoning, not a list of technologies
Bottleneck hunt — proactively identify where the system will break and how you'd fix it

Found this useful? Follow for more system design deep dives — next up: designing YouTube's upload pipeline at scale.

Two tiny functions that make your async code production-ready: `retry` and `timeout`

Daniel Keya — Sun, 24 May 2026 00:01:00 +0000

Every async function you write assumes the network cooperates, the server responds, and the database doesn't hiccup. In production, none of those assumptions hold forever.

Here are two higher-order functions — each under 15 lines — that make any async function resilient without touching its internals.

The problem

You have an async function. Maybe it calls an API, queries a database, or reads a file over the network.

async function fetchUserData(userId) {
    const response = await fetch(`/api/users/${userId}`);
    return response.json();
}

Two things will go wrong eventually:

It will fail intermittently and you'll want to retry it
It will hang indefinitely and you'll want to give up after a deadline

You could wrap every function in retry logic and timeout logic inline. Or you could write it once, properly, and wrap any function you want.

`retry` — automatic reattempts on failure

export function retry(count, callback) {
  return async function (...args) {
    let attempts = 0;
    let lastError;

    while (attempts <= count) {
      try {
        return await callback(...args);
      } catch (err) {
        lastError = err;
        attempts++;
      }
    }

    throw lastError;
  };
}

How it works

retry is a higher-order function — it takes a function and returns a new function with retry behaviour baked in. The original function is untouched.

The while (attempts <= count) condition is deliberate. If count is 3, the loop runs when attempts is 0, 1, 2, 3 — that's 4 total executions: one initial attempt plus three retries. This matches the natural language meaning of "retry 3 times".

On success, return await callback(...args) exits immediately — no more iterations. On failure, the error is stored in lastError and attempts increments. Once the loop exhausts all attempts, the last error is rethrown — not a generic new Error('Max retries reached'), but the actual error the callback produced. Your callers get a meaningful error message, not a wrapper.

Usage

const resilientFetch = retry(3, fetchUserData);

// Works exactly like fetchUserData, but retries up to 3 times on failure
const user = await resilientFetch('user_123');

Why `await` inside `try` matters

try {
    return await callback(...args); // ✓ catches rejected promises
} catch (err) { ... }

Without await, a rejected promise escapes the try/catch entirely:

try {
    return callback(...args); // ✗ returns a pending promise — catch never fires
} catch (err) { ... }

await unwraps the promise inside the try block, so rejections are catchable. This is one of the most common async/await mistakes and retry only works correctly because it gets this right.

`timeout` — give up after a deadline

export function timeout(delay, callback) {
  return async function (...args) {
    const timer = new Promise((_, reject) =>
      setTimeout(() => reject(new Error('timeout')), delay)
    );

    return Promise.race([callback(...args), timer]);
  };
}

How it works

Promise.race resolves or rejects with whichever promise settles first. This function creates a race between two competitors:

callback(...args) — the actual work
timer — a promise that rejects after delay milliseconds

If the callback finishes in time, its value wins and timer becomes irrelevant. If delay milliseconds pass first, timer rejects with Error('timeout') and the callback's eventual result is ignored.

Notice the timer promise is constructed with (_, reject) — it never resolves, only rejects. This ensures the timer can never accidentally win the race with a successful value; it can only interrupt with a failure.

Usage

const limitedFetch = timeout(5000, fetchUserData);

try {
    const user = await limitedFetch('user_123');
} catch (e) {
    if (e.message === 'timeout') {
        console.error('Request took too long');
    }
}

Combining them

Both functions return async functions with the same signature as their input — which means they compose cleanly.

// Retry up to 3 times, but abandon any single attempt after 5 seconds
const resilientFetch = retry(3, timeout(5000, fetchUserData));

await resilientFetch('user_123');

Here's what happens on each attempt:

timeout(5000, fetchUserData) races the fetch against a 5-second timer
If it times out, timeout rejects with Error('timeout')
retry catches that rejection, increments attempts, and tries again
After 3 retries all fail, retry rethrows the last error

Four attempts, each with a 5-second ceiling, maximum 20 seconds total. All from two composable functions and one line of setup.

What makes these worth keeping

They don't modify the original function. fetchUserData is unchanged. You can use it with or without retry/timeout anywhere else.

They forward arguments transparently. ...args passes everything through — the wrapped function behaves identically to the original from the caller's perspective.

They preserve the error. retry rethrows lastError, not a new generic error. timeout rejects with a named Error('timeout') you can check by message. Callers always know what actually went wrong.

They compose. Because both return async functions with matching signatures, you can layer them in any order and they work together without knowing about each other.

The pattern

Both functions follow the same structure:

higherOrderFn(config, callback) {
    return async function (...args) {
        // enhanced behaviour around callback(...args)
    }
}

This is the decorator pattern applied to async functions. You write the enhancement once, and apply it to any async function that needs it — no inheritance, no classes, no modification of the original. Just functions wrapping functions.

It's a small pattern. It shows up everywhere once you start looking for it.

DEV Community: Daniel Keya

Understanding Stacks and Queues in Go

The core idea

Building a stack

Real example: balanced brackets

Building a queue

A better queue: container/list

Real example: breadth-first search

The idiomatic Go queue: channels

Time complexity

Which one should you use?

Wrapping up

JavaScript's Safe Assignment Operator (?=): Hype vs. Reality

The problem: try/catch gets noisy

The proposal: errors as values

The reality check: it's not happening (at least not as ?=)

What you can do today

A note on TypeScript

Should you adopt this pattern?

Wrapping up

Using JavaScript to Build Algorithmic Trading Indicators — A Look at Algocdk

Why JavaScript for Algorithmic Trading?

What Is a Trading Indicator?

Building a Moving Average in JavaScript

Moving Average Crossover

Building the Same Idea in Algocdk

Creating a MACD Indicator

MACD Inside Algocdk

From Indicators to Trading Bots

Why I Built Algocdk

The Interesting Part: You Don't Need a Build System

JavaScript Gives You More Than Technical Indicators

From Code to Trading Platform

Example Project

Final Thoughts

Day 5 of 30: Message Queues and Why Not Everything Needs an Instant Response

The Assumption I Didn't Know I Had

What a Message Queue Actually Does

Why This Matters for Scaling

Delivery Guarantees: The Part That Gets Complicated Fast

Queue vs. Pub/Sub

What Clicked Today

Tomorrow

title: Day 4 of 30: Load Balancing and the Single Point of Failure Problem

Why Load Balancers Exist

Load Balancing Algorithms

Layer 4 vs. Layer 7 Load Balancing

Health Checks: How the Load Balancer Knows What's Alive

The Load Balancer Becomes a Single Point of Failure

What Clicked Today

Tomorrow

Day 3 of 30: Caching, and Why Invalidation Really Is That Hard

Recap

Why Caching Exists

Where Caches Live

Caching Strategies

Cache Invalidation: The Actually Hard Part

Cache Stampede — A Failure Mode I Hadn't Considered

What Clicked Today

Tomorrow

Day 2 of 30: Databases, Replication, and Sharding published: false tags: systemdesign, beginners, learning, databases

Recap

SQL vs. NoSQL: Less "Which Is Better," More "Which Fits"

Replication: Copies for Reliability and Speed

Sharding: Splitting Data Across Machines

Replication + Sharding Together

What Clicked Today

Tomorrow

Day 1 of 30: Kicking Off My System Design Learning Journey published: false tags: systemdesign, beginners, learning, softwareengineering

Why I'm Doing This

What Even Is System Design?

Topics I Covered Today

1. Client-Server Architecture

2. Vertical vs. Horizontal Scaling

3. Latency vs. Throughput

4. The CAP Theorem (just the surface)

My Study Approach for These 30 Days

What's Next

Drop a comment below. See you on Day 2.

Why I Fell in Love with Go — And Why You Should Give It a Chance

A better queue: `container/list`

The reality check: it's not happening (at least not as `?=`)

5. Rendering a List with `.map()`

`retry` — automatic reattempts on failure

Why `await` inside `try` matters

`timeout` — give up after a deadline