Building a Zero-Allocation, Nanosecond Distributed Rate Limiter in Go

Rate limiting seems simple on paper until your microservice hits 100k requests/second, Redis experiences a transient network hiccup, or an attacker spoofs their X-Forwarded-For header to bypass your limits entirely.

While building high-throughput services in Go, I noticed that many existing rate-limiting libraries suffer from three major issues:

1. Global Lock Contention: Single sync.Mutex architectures causing bottleneck spikes during concurrent request surges.
2. Garbage Collection (GC) Pressure: Heap allocations (B/op) on every limit evaluation triggering GC pauses.
3. Cascading Outages: When Redis goes down, applications either crash or get hit by a Thundering Herd when Redis recovers.

To solve these exact engineering pain points, I created distlimit an open-source, ultra-fast, and resilient distributed rate-limiting library for Go.

Here is how it works under the hood and why it executes in sub-100 nanoseconds.

⚡ 1. Zero-Allocation & 64-Sharded Map Engine

To achieve true nanosecond performance, distlimit completely eliminates memory allocations during evaluation:

BenchmarkSlidingLog_EvaluateMemory-2         20.36 ns/op    0 B/op    0 allocs/op
BenchmarkTokenBucket_EvaluateMemory-2        46.18 ns/op    0 B/op    0 allocs/op
BenchmarkLeakyBucket_EvaluateMemory-2        47.76 ns/op    0 B/op    0 allocs/op
BenchmarkFixedWindow_EvaluateMemory-2        77.27 ns/op    0 B/op    0 allocs/op
BenchmarkSlidingCounter_EvaluateMemory-2      129.10 ns/op    0 B/op    0 allocs/op

Instead of using a single global lock, distlimit divides the in-memory map into 64 independent shards hashed via FNV-1a:

shard_index=FNV-1a(key) (mod 64)

This drastically distributes lock contention across 64 CPU pathways, resulting in up to 64x higher concurrent throughput.

🛡️ 2. Dual-Tier Resilience with Half-Open Circuit Breaker

What happens if Redis fails? Most rate limiters either block all traffic or dump all requests straight to your database.

distlimit features a Hybrid Driver (Redis Primary + Memory Fallback) equipped with a Half-Open Circuit Breaker:

[ Incoming Request ]
         │
         ▼
 ┌───────────────┐      FAIL      ┌─────────────────────────┐
 │ Redis Primary │ ─────────────> │ Memory Fallback (Sharded│
 └───────────────┘                └─────────────────────────┘
         │                                     │
    RECOVERY MODE                              │
         │ (Atomic CAS Single-Request Probe)   │
         ▼                                     ▼
 ┌───────────────┐                ┌─────────────────────────┐
 │ Redis Warmed  │ <───────────── │ Prevents Thundering     │
 │ Up Safely     │                │ Herd Spike              │
 └───────────────┘                └─────────────────────────┘

go
When Redis recovers after an outage cool-off, an Atomic Compare-And-Swap (CAS) mechanism allows only one single request to probe Redis. All other concurrent requests continue safely on the memory fallback until Redis is confirmed healthy. This prevents the dreaded Thundering Herd phenomenon.

🔐 3. Security Hardened: Anti-IP Spoofing & Redis Cluster Safe

Security was a non-negotiable priority when designing distlimit:

• Anti-IP Spoofing Shield: Middlewares do NOT blindly trust X-Forwarded-For or X-Real-IP. Headers are strictly inspected against CIDR-validated subnet rules (WithTrustedProxies).
• Redis Cluster Hash Tags {}: Key generation automatically wraps keys in Redis Hash Tags (distlimit:{key}) to prevent CROSSSLOT cluster routing errors out of the box.

🔌 4. Pluggable Strategy Architecture

distlimit ships with 5 standard algorithms that can be swapped without touching your storage driver:

• Token Bucket: Perfect for burst-friendly APIs.
• Leaky Bucket: Smooth traffic shaping for third-party rate quotas.
• Fixed Window: Ultra-fast counter reset protection.
• Sliding Window Log: 100% exact precision for strict quotas.
• Sliding Window Counter: $O(1)$ memory weighted average.

---

💻 Show Me The Code!

Here is how easy it is to protect a Fiber v3 app with CIDR proxy verification:

package main

import (
    "time"

    "[github.com/balramadan/distlimit](https://github.com/balramadan/distlimit)"
    "[github.com/balramadan/distlimit/algorithm/tokenbucket](https://github.com/balramadan/distlimit/algorithm/tokenbucket)"
    "[github.com/balramadan/distlimit/driver/memory](https://github.com/balramadan/distlimit/driver/memory)"
    distlimitfiber "[github.com/balramadan/distlimit/middleware/fiber](https://github.com/balramadan/distlimit/middleware/fiber)"
    "[github.com/gofiber/fiber/v3](https://github.com/gofiber/fiber/v3)"
)

func main() {
    // 1. Initialize 64-Sharded Memory Driver
    memDriver := memory.New(5 * time.Minute)

    // 2. Configure Limiter: 100 reqs/min using Token Bucket
    limiter, _ := distlimit.New(
        memDriver,
        distlimit.WithLimit(100),
        distlimit.WithWindow(1*time.Minute),
        distlimit.WithAlgorithm(tokenbucket.New()),
    )

    app := fiber.New()

    // 3. Attach Middleware with Anti-IP Spoofing Shield
    app.Use(distlimitfiber.New(
        limiter,
        distlimitfiber.WithTrustedProxies([]string{"10.0.0.0/8", "172.16.0.0/12"}),
    ))

    app.Get("/api/ping", func(c fiber.Ctx) error {
        return c.SendString("pong")
    })

    app.Listen(":3000")
}

Support for Gin, Echo v4/v5, net/http, Chi, and gRPC is also built-in!

🚀 Try It Out & Support Open Source

distlimit is 100% open-source under the MIT license and ready for production workloads.

• 📦 Go Module: go get github.com/balramadan/distlimit@v1.1.1
• ⭐️ GitHub Repository: github.com/balramadan/distlimit
• 📖 Documentation: Docs

If you find this project helpful or plan to use it in your Go stack, leaving a Star ⭐️ on GitHub would mean the world to me and help other developers discover it!

What rate-limiting strategies are you currently using in your microservices? Let's discuss in the comments below! 👇