DEV Community

Benji 🍙
Benji 🍙

Posted on

TIL that the `requests` library supports automatic retries with exponential backoff

You can use a customized Adapter and force several retries with an exponential backoff factor on all HTTP/HTTPS requests. See example below:

import requests
from requests import adapters
from urllib3.util import Retry

# Create a transport adapter with a custom retry strategy.
retries = Retry(
    total=3,
    backoff_factor=3,
    status_forcelist=[500, 502, 503, 504]
)
adapter = adapters.HTTPAdapter(max_retries=retries)

# Ensure adapter is used for both HTTP and HTTPS requests.
session = requests.Session()
session.mount('https://', adapter)
session.mount('http://', adapter)

# Testing the retry mechanism
response = session.get("http://httpbin.org/status/500")
Enter fullscreen mode Exit fullscreen mode

This returns the error below:

RetryError: HTTPConnectionPool(host='httpbin.org', port=80): Max retries exceeded with url: /status/500 (Caused by ResponseError('too many 500 error responses'))
Enter fullscreen mode Exit fullscreen mode

The unfortunate thing is that there doesn't seem to be a way to tell how many times the above mechanism has attempted to retry, only when all attempts have been exhausted

Reference

https://stackoverflow.com/a/47475019/4477547

Image of Timescale

🚀 pgai Vectorizer: SQLAlchemy and LiteLLM Make Vector Search Simple

We built pgai Vectorizer to simplify embedding management for AI applications—without needing a separate database or complex infrastructure. Since launch, developers have created over 3,000 vectorizers on Timescale Cloud, with many more self-hosted.

Read more →

Top comments (0)

Sentry image

See why 4M developers consider Sentry, “not bad.”

Fixing code doesn’t have to be the worst part of your day. Learn how Sentry can help.

Learn more