DEV Community

Emre Kadir Dağdelen
Emre Kadir Dağdelen

Posted on Originally published at dagdelean.dev

Building the Polly that Python never had

Building the Polly that Python never had

Every app that calls external APIs faces the same four problems: transient
failures, dead dependencies, rate limits, and slow responses. Java solved
this with resilience4j. .NET solved it with Polly. Python never had a
unified answer: tenacity does retries, pybreaker does circuit breaking
(sync only, aging), and everything else you wire by hand.

These patterns only pay off when they compose: a timeout per attempt,
retries that respect an open circuit, a fallback that absorbs the rest.
Wiring that from three libraries with three philosophies is the code
nobody wants to own. So I built nopanic.

One decorator API, sync and async

from nopanic import compose, fallback, retry, circuit_breaker, timeout, backoff, CircuitOpen

llm = circuit_breaker(failure_threshold=0.5, reset_timeout=20.0, name="llm")

resilient = compose(
    fallback(lambda e: "temporarily unavailable", on=(ConnectionError, CircuitOpen, TimeoutError)),
    retry(attempts=3, on=(ConnectionError, TimeoutError), backoff=backoff.full_jitter(0.2)),
    llm,
    timeout(30.0),
)

@resilient
async def ask(prompt: str) -> str: ...
Enter fullscreen mode Exit fullscreen mode

Reading inside out: each attempt gets 30 seconds, outcomes feed the
breaker, transient failures retry with jitter, and whatever is left
becomes a degraded answer instead of a traceback. The same decorators
work on plain sync functions. Zero runtime dependencies.

Design choices that mattered

No wrapper exceptions. Exhausted retries re-raise the original error.
Your except ConnectionError: keeps working.

Injectable time. Breakers, rate limiters and caches take a clock=
parameter, so the test suite (141 tests) runs in under two seconds with
zero sleeps.

Observability as a stream. events.subscribe() sees everything every
policy does. With no listeners the cost is one attribute read (~0.1us).

Measured overhead. Every policy costs 0.15 to 1.3 microseconds per
call on the success path. A fast HTTP round trip is 5,000+ us.

Validation: retrofitting a 232k-star repo

To test the API against reality, I took network scripts from
geekcomputers/Python and added resilience: three decorators per call
site, no restructuring. The exercise even improved the design: my first
retry policy naively retried a 403, which led to the documented
"retry 429/5xx, never other 4xx" recipe.

The bug that only exists on Windows

The best story came from CI. Tests passed everywhere except Windows on
Python 3.11/3.12. After reproducing locally and instrumenting, the data
showed time.sleep(0.05) waking after 47ms by the monotonic clock, and
sub-quantum sleeps returning instantly. My retry honored a server's
Retry-After by sleeping exactly that long, then racing the deadline and
losing, burning attempts against a still-closed breaker window.

The fix belongs in the library, not the test: "retry after X" means "not
before X", so honored hints now sleep X * 1.05 + 50ms, still capped so a
hostile server cannot park your client. If your code races deadlines
exactly, Windows will eventually teach you the same lesson.

Adaptive rate limiting: stop hardcoding guesses

The newest addition applies TCP's AIMD congestion control to API quotas:
a 429 cuts your rate multiplicatively, successes recover it additively,
and Retry-After blocks the bucket for exactly that long. You converge on
the rate the server actually sustains.

pip install nopanic — GitHub: github.com/dagdelenemre/nopanic
Feedback and issues welcome, especially from Polly and resilience4j users.

Top comments (0)