Your Python script hangs forever on requests.get() because there is no default timeout. Here is the fix

#python #webdev #tutorial #beginners

I had a cron job that was supposed to finish in 2 minutes. It was still "running" 9 hours later. No error, no log line, nothing. The cause was one requests.get(url) call to a server that accepted the connection and then never answered.

The surprising part: this is documented behavior. The requests library has no default timeout. If you do not set one, the call can wait indefinitely.

The one-line fix

import requests

r = requests.get("https://example.com/api", timeout=10)

Now the call raises requests.exceptions.Timeout instead of hanging.

Connect timeout vs read timeout

timeout accepts a tuple: (connect, read).

r = requests.get(url, timeout=(3.05, 10))

connect: how long to wait to establish the TCP connection.
read: how long to wait between bytes from the server.

Careful: the read timeout is not a total deadline. A server that sends one byte every 9 seconds will never trip a 10-second read timeout. If you need a hard cap on total time, track it yourself:

import time, requests

def get_with_deadline(url, total=30):
    start = time.monotonic()
    with requests.get(url, stream=True, timeout=(3.05, 10)) as r:
        chunks = []
        for chunk in r.iter_content(8192):
            chunks.append(chunk)
            if time.monotonic() - start > total:
                raise TimeoutError(f"exceeded {total}s total")
        return b"".join(chunks)

Stop repeating yourself: set it once on a Session

Typing timeout= on every call is how one gets forgotten. A small adapter applies a default to every request on a session, and urllib3's Retry handles transient errors:

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

class TimeoutAdapter(HTTPAdapter):
    def __init__(self, *args, timeout=(3.05, 10), **kwargs):
        self.timeout = timeout
        super().__init__(*args, **kwargs)

    def send(self, request, **kwargs):
        if kwargs.get("timeout") is None:
            kwargs["timeout"] = self.timeout
        return super().send(request, **kwargs)

def make_session():
    retry = Retry(
        total=3,
        backoff_factor=0.5,                    # 0.5s, 1s, 2s between tries
        status_forcelist=[429, 500, 502, 503, 504],
        allowed_methods=["GET", "HEAD"],       # only retry safe methods
    )
    s = requests.Session()
    adapter = TimeoutAdapter(max_retries=retry)
    s.mount("https://", adapter)
    s.mount("http://", adapter)
    return s

session = make_session()
r = session.get("https://example.com/api")   # timeout applied automatically

Two details worth knowing:

allowed_methods matters. Retrying a POST can create duplicate records, so the example only retries GET and HEAD.
Retry honors the Retry-After header on 429/503 responses by default, so you back off when the server tells you to.

Quick checklist

Every network call has a timeout. No exceptions.
Use a (connect, read) tuple, with connect short.
Need a total deadline? Track elapsed time while streaming.
Retry only idempotent methods.

Sources

Requests docs, Timeouts: https://requests.readthedocs.io/en/latest/user/advanced/#timeouts
urllib3 Retry reference: https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html#urllib3.util.Retry

What is the longest a "temporary" hang has gone unnoticed in one of your scripts, and how did you finally find it?

DEV Community