Tried 3 ways to handle retries in Python. This one actually makes sense.
Scraping APIs that randomly fail taught me retries matter.
Tried a bunch of approaches. Most were garbage.
Manual While Loop (My First Attempt)
Started obvious:
attempts = 0
max_attempts = 3
while attempts < max_attempts:
try:
response = requests.get(url)
response.raise_for_status()
break
except requests.exceptions.RequestException:
attempts += 1
if attempts >= max_attempts:
raise
Worked for one endpoint.
Copy pasted this everywhere. Had 15 files with the same loop. Changed retry logic once, had to update 15 files.
Dumb.
Tenacity Library (Overkill)
Found the tenacity library. Looked professional:
from tenacity import retry, stop_after_attempt, wait_exponential
@retry(stop=stop_after_attempt(3),
wait=wait_exponential(multiplier=1, min=2, max=10))
def fetch_data(url):
response = requests.get(url)
response.raise_for_status()
return response.json()
This worked fine but felt like using a forklift to move a chair.
Had to install extra deps. Exponential backoff config was confusing for simple cases. Team kept asking what multiplier meant. Not worth it honestly.
Simple Decorator (What I Use Now)
Built my own tiny decorator:
import time
from functools import wraps
def retry(times=3, delay=1):
def decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
for attempt in range(times):
try:
return func(*args, **kwargs)
except Exception as e:
if attempt == times - 1:
raise
time.sleep(delay)
return None
return wrapper
return decorator
@retry(times=3, delay=2)
def fetch_api(url):
response = requests.get(url)
response.raise_for_status()
return response.json()
This is like 15 lines. No deps. Anyone reading my code understands it instantly.
Want different retry counts per function? Just change the decorator params. Want exponential backoff? Add delay * (attempt + 1) to the sleep.
Used this pattern building scrapers with the ParseForge API. Sometimes endpoints hiccup and need a second attempt. This decorator makes that brain dead simple.
When to Use What
Manual loop: Never. Just don't.
Tenacity: If you need complex retry logic like jitter or conditional retries based on error type. Or if your company already uses it.
Simple decorator: Pretty much everything else. APIs that occasionally fail. Network requests. Anything where just trying again solves it
Top comments (0)