Building a Distributed Rate Limiter with Python, Redis, NGINX and RabbitMQ

This article explores how we can enforce rate-limiting in a distributed system, especially when multiple workers are making request to the same API. This demo project is implemented using Python, Redis, RabbitMQ and NGNIX

The Problem

Let's consider a simple scenario involving two companies.

Company A is a used-car marketplace that needs to fetch updated market prices for 1,000 vehicles.

Company B provides a vehicle pricing API, but allows only 50 requests per second.

Company A can use multiple workers to fetch prices faster. However, if each worker independently sends requests, the combined traffic could exceed Company B's limit.

So we need to build a solution which address below question.

1. How can Company B protect its API from excessive traffic?
2. How can Company A coordinate requests across multiple workers to respect the shared rate limit?

The Architecture

Company A publishes vehicle IDs to RabbitMQ. Independent Python workers consume these messages and fetch prices from Company B.

Redis coordinates the outgoing request rate across workers, while NGINX protects Company B from excessive incoming traffic.

Protecting Company B with NGINX

Company B which provide the pricing API using FastAPI and placing NGINX in front of it for rate limiting.

NGINX supports request rate limiting using a leaky-bucket algorithm.

Here is the main NGINX configuration:

limit_req_zone $binary_remote_addr zone=pricing_limit:10m rate=50r/s;

server {
    location /prices/ {
        limit_req zone=pricing_limit burst=50;

        proxy_pass http://company_b_api;
    }
}

Let's break down the important parts.

• $binary_remote_addr identifies clients by IP address. Requests from the same IP share the same limit.
• zone=pricing_limit:10m creates a shared memory zone where NGINX tracks request rates.
• rate=50r/s configures the permitted request rate to 50 requests per second.
• burst=50 allows temporary excess requests to wait in a queue, up to the configured burst capacity.
• proxy_pass forwards accepted requests to Company B's FastAPI service.

Without nodelay, NGINX delays excess requests to smooth incoming traffic. If requests exceed the allowed burst capacity, they are rejected.

Distributed Workers with RabbitMQ

Company A has /sync endpoint when it is called, the application publishes 1,000 vehicle IDs to RabbitMQ.

Instead of fetching all prices directly inside the API request, independent Python worker processes consume the messages.

This allows the API and background processing to operate independently.

I tested the setup with three separate consumer processes, all reading from the same RabbitMQ queue.

Coordinating Python Workers with Redis

In this project, I used 3 python workers which has their own local rate limiter allowing 50 requests per second.

Together, they could potentially send 150 requests per second.

To avoid this, I used Redis as shared storage for rate limiting.

The implementation uses the Python limits library with a moving-window strategy.

import asyncio

from limits import parse
from limits.aio.storage import RedisStorage
from limits.aio.strategies import MovingWindowRateLimiter

storage = RedisStorage(
    "async+redis://localhost:6349/0",
    implementation="redispy",
)

limiter = MovingWindowRateLimiter(storage)
limit = parse("50/second")

async def acquire():
    while True:
        allowed = await limiter.hit(
            limit,
            "company-b-pricing-api",
        )

        if allowed:
            return

        await asyncio.sleep(0.02)

Every worker connects to the same Redis instance and uses the same rate-limit key.

Before calling Company B, the worker must acquire a permit.

If the limit is reached, it waits and tries again.

This allows multiple independent workers to coordinate around the same request quota.

Processing Messages with Async Python

Each consumer follows a simple flow:

1. Receive a vehicle ID from RabbitMQ.
2. Acquire a permit from Redis.
3. Call Company B's pricing API using HTTPX.
4. Acknowledge the RabbitMQ message after a successful response.

The main processing logic looks like this:

async for message in messages:
    vehicle_id = message.body.decode()

    await rate_limiter.acquire()

    response = await client.get(
        f"{provider_url}/prices/{vehicle_id}"
    )

    response.raise_for_status()

    price = response.json()
    print(vehicle_id, price["market_price"])

    await message.ack()

The acknowledgment happens only after the API request succeeds.

If the worker crashes before acknowledgment, RabbitMQ can redeliver the message. In a real system, we would also need idempotent processing to handle possible duplicates.

Key Takeaways

• NGINX protects the provider API by controlling incoming request traffic.
• Redis coordinates a shared rate limit across independent worker processes.
• RabbitMQ decouples API requests from background processing, allowing consumers to scale independently.
• Acknowledgments support reliable processing, while idempotency is needed to safely handle redeliveries.

For production, I would also consider retries with backoff, dead-letter queues, monitoring, and global concurrency limits.

The main takeaway for me is that rate limiting is not only about restricting requests. In distributed systems, it is also about coordinating multiple independent workers that share the same external API quota.