Ilya Masliev

Posted on Feb 5

I Felt Like a Clown Wiring 5 Libraries Just to Build a Resilient API Client

#python #microservices #architecture #performance

So I wrote one that unifies everything.

The moment it broke me

I just wanted a simple API client.

import httpx

async def fetch_user(user_id: str):
    async with httpx.AsyncClient() as client:
        r = await client.get(f"https://api.example.com/users/{user_id}")
        return r.json()

That lasted about 5 minutes.

Because real APIs:

rate limit you
timeout
return 503
sometimes completely die
and retries can DDoS your own service

So I did what every Python dev does.

I started stacking libraries.

The decorator tower of doom

First: rate limiting.

Then: retry.

Then: circuit breaker.

And suddenly my function looked like this:

@breaker
@retry(...)
@sleep_and_retry
@limits(...)
async def fetch_user(...):
    ...

And I hated it.

Not because it didn’t work — but because it didn’t scale.

Problems:

3+ libraries
fragile decorator ordering
conflicting abstractions
async quirks
painful testing
scattered observability
dependency sprawl

And this was for one function.

Now imagine 10 APIs. Per-user limits. Background jobs. Webhooks.

You’re no longer writing business logic.

You’re babysitting resilience glue code.

The idea: resilience as a pipeline

What if resilience wasn’t decorator soup?

What if every call flowed through a single orchestrator?

from limitpal import (
    AsyncResilientExecutor,
    AsyncTokenBucket,
    RetryPolicy,
    CircuitBreaker
)

executor = AsyncResilientExecutor(
    limiter=AsyncTokenBucket(capacity=10, refill_rate=100/60),
    retry_policy=RetryPolicy(max_attempts=3),
    circuit_breaker=CircuitBreaker(failure_threshold=5)
)

result = await executor.run("user:123", api_call)

No decorators.
No stacking libraries.
No fragile glue.

One execution pipeline.

That’s what LimitPal is.

What LimitPal actually gives you

LimitPal is a toolkit for building resilient Python clients and services.

It combines:

rate limiting (Token / Leaky bucket)
retry with exponential backoff + jitter
circuit breaker
composable limiters
a resilience executor that orchestrates everything

And:

full async + sync support
zero dependencies
thread-safe by default
deterministic time control for tests
key-based isolation (per-user / per-IP / per-tenant)

The goal isn’t more features.

The goal is fewer moving parts.

The resilience pipeline (this is the key idea)

Every call goes through:

Circuit breaker check
→ Rate limiter
→ Execute + retry loop
→ Record result

This ordering matters.

You’re not just “adding retry”.

You’re designing failure behavior as a system.

breaker stops cascading failures
limiter protects infrastructure
retry handles temporary issues
executor keeps it coherent

One mental model instead of five.

The testing problem nobody talks about

Time-based logic is brutal to test.

Traditional approach:

time.sleep(1)

Slow. Flaky. Non-deterministic.

LimitPal uses a pluggable clock.

So tests become:

clock.advance(1.0)

Instant. Deterministic.

You can simulate minutes of retries in milliseconds.

For teams that care about reliability, this is a game changer.

Real example

A resilient HTTP client in ~10 lines:

executor = AsyncResilientExecutor(
    limiter=AsyncTokenBucket(capacity=10, refill_rate=100/60),
    retry_policy=RetryPolicy(max_attempts=3, base_delay=0.5),
    circuit_breaker=CircuitBreaker(failure_threshold=5)
)

async def fetch():
    return await httpx.get("https://api.example.com")

result = await executor.run("api", fetch)

You automatically get:

burst control
exponential retry
cascading failure protection
clean async semantics

No decorator tower.

When should you use this?

Use LimitPal if you:

build API clients
call flaky third-party services
run background jobs
need per-user limits
care about deterministic tests
want clean async support

If you only need retry — smaller libs are fine.

If you need composition, that’s the niche.

Part 2: internals

This post is about the idea.

In Part 2 I’ll go deep into:

how the executor pipeline works
circuit breaker state machine
clock abstraction design
composite limiter architecture
failure modeling

Because resilience isn’t magic.

It’s architecture.

Try it

pip install limitpal

Docs:
https://limitpal.readthedocs.io/

Repo:
https://github.com/Guli-vali/limitpal

If it saves you pain — stars are welcome ⭐

DEV Community