Building a Distributed Rate Limiter from Scratch

#backend #distributedsystems #opensource

Introduction

There comes a point while learning software engineering where using a technology isn't enough anymore.

For me, that technology was Redis. I had used Redis before in small projects. I knew it was fast, great for caching, and commonly used in backend systems. But there was one thing I couldn't confidently explain:

How do distributed applications actually implement rate limiting?

I understood the concept of rate limiting, but not the engineering behind it. Instead of watching another tutorial, I decided to build one from scratch.

That project eventually became Aegis—my first open-source Python package.

Why Build Another Rate Limiter?

Honestly, I wasn't trying to build a better rate limiter. There are already plenty of mature libraries that solve this problem far better than something I could build alone.

My goal was much simpler. I wanted to understand how these systems actually work under the hood.

That meant asking questions like:

How do different rate limiting algorithms behave?
Why is Redis commonly used?
What causes race conditions?
Why do people use Lua scripts with Redis?
How would I design a library that other developers could actually use?

The only way I knew how to answer those questions was by building everything myself.

Starting Small

Like most projects, Aegis didn't start with Redis or distributed systems. It started with a simple in-memory implementation.

I first implemented four common rate limiting algorithms:

Fixed Window
Sliding Window
Token Bucket
Leaky Bucket

Getting each algorithm working taught me that they all solve the same problem in different ways.

Some are simple. Some are more accurate. Some allow bursts of traffic. Some smooth traffic over time.

Understanding those trade-offs was one of the most interesting parts of the project.

Making It Distributed

Once the in-memory implementation was working, I started thinking about a more realistic scenario.

What happens when your application runs on multiple servers? An in-memory counter only exists inside a single process. If two servers each keep their own counters, the rate limiter isn't really shared anymore.

That's where Redis came in. Redis became the shared source of truth, allowing every application instance to work with the same state.

But moving the logic to Redis introduced another challenge-Race conditions.

Two requests arriving at almost the same time could both read the same value before either one updated it.

To solve that problem, I implemented the Redis algorithms using Lua scripts, allowing the entire operation to execute atomically on the Redis server. It was one of the biggest learning moments during this project.

Beyond the Algorithms

At some point, I realized Aegis was becoming more than just a collection of algorithms. So I spent time improving the architecture.

Some of the things I added were:

A shared Redis Lua base class
Separate Redis and in-memory backends
A simple SDK API
Unit and integration tests
GitHub Actions for automated testing
Black and Ruff for formatting and linting
Packaging and publishing to PyPI

These weren't the most exciting features to build, but they made the project feel much more complete.

What I Learned

Looking back, Aegis taught me much more than rate limiting.

It taught me how to think about distributed systems, software design, testing, packaging, and maintaining an open-source project. More importantly, it reminded me that the best way to understand a complex topic is often to build it yourself.

There are still plenty of things I want to improve, and Aegis is far from finished. But I'm happy with where it has taken me so far.

Final Thoughts

This project started with a simple question:

"How does distributed rate limiting actually work?"

Trying to answer that question led me through Redis, Lua, multiple algorithms, software architecture, testing, CI/CD, and eventually publishing my first open-source Python package.

If you're learning backend engineering, I'd definitely recommend picking a problem that interests you and building it from scratch.

You'll probably learn much more than any tutorial could teach.

If you'd like to check out the project:

⭐ GitHub: https://github.com/Nilotpal04/aegis

📦 PyPI: https://pypi.org/project/aegis-rl/0.1.1/

You can install it with: