DEV Community

RecallRun
RecallRun

Posted on Originally published at recallrun.dev

OpenAI Python SDK errors, retries and timeouts: what it retries for you and what it can't

Network calls fail. Rate limits get hit, servers have bad minutes, requests are malformed, keys expire.

The OpenAI Python SDK already retries the retryable errors for you (connection problems, 408, 409, 429 and 5xx) with exponential backoff. Your job is three things: set sensible timeouts, handle the errors that retrying cannot fix, and log the request id so a failure can be traced later.

1. The exception hierarchy

openai.APIError
├── APIConnectionError          -> retried automatically
│   └── APITimeoutError         -> retried automatically
└── APIStatusError
    ├── 400 BadRequestError          -> fix the request
    ├── 401 AuthenticationError      -> fix the config
    ├── 403 PermissionDeniedError    -> fix the config
    ├── 404 NotFoundError            -> fix the request
    ├── 422 UnprocessableEntityError -> fix the request
    ├── 429 RateLimitError           -> retried automatically
    └── 5xx InternalServerError      -> retried automatically
Enter fullscreen mode Exit fullscreen mode

2. Automatic retries, observed

fake_client below is a small stub that scripts HTTP responses, so you can watch the retry logic without spending tokens or waiting for a real outage.

import openai
from fake_openai import fake_client

# a 429 then a 500, then success: the SDK retries with backoff and you never see the errors
client = fake_client(script=[{"status": 429}, {"status": 500}, "Recovered."], max_retries=2)
r = client.responses.create(model="gpt-5.6-luna", input="hello")
print(r.output_text, "| HTTP requests made:", len(client.fake.requests))

# more failures than retries: the error reaches your code
client = fake_client(script=[{"status": 429, "message": "Rate limit reached for requests"}] * 5, max_retries=2)
try:
    client.responses.create(model="gpt-5.6-luna", input="hello")
except openai.RateLimitError as e:
    print(type(e).__name__, e.status_code, "| request id:", e.request_id,
          "| requests made:", len(client.fake.requests))

# a 400 is NOT retried: sending the same bad request again would fail again
client = fake_client(script=[{"status": 400, "message": "Invalid value for 'model'"}], max_retries=2)
try:
    client.responses.create(model="gpt-5.6-luna", input="hello")
except openai.BadRequestError as e:
    print(type(e).__name__, e.status_code, e.message, "| requests made:", len(client.fake.requests))
Enter fullscreen mode Exit fullscreen mode

Output:

Recovered. | HTTP requests made: 3
RateLimitError 429 | request id: req_fake_8 | requests made: 3
BadRequestError 400 Error code: 400 - {'error': {'message': "Invalid value for 'model'", 'type': 'fake_error', 'code': None}} | requests made: 1
Enter fullscreen mode Exit fullscreen mode

Three requests for one call in the first case, and the 400 went out exactly once.

3. Timeouts

import openai
from fake_openai import fake_client

client = fake_client(script=[{"delay": 2.0}, "too late"], timeout=0.5, max_retries=0)
try:
    client.responses.create(model="gpt-5.6-luna", input="summarise this long report")
except openai.APITimeoutError as e:
    print("timed out:", type(e).__name__)

# per-call override for a known-slow request (e.g. high reasoning effort)
client = fake_client(script=[{"delay": 0.8}, "done"], timeout=0.5, max_retries=0)
r = client.with_options(timeout=2.0).responses.create(model="gpt-5.6-luna", input="hard question")
print("with a longer timeout:", r.output_text)
Enter fullscreen mode Exit fullscreen mode

Output:

timed out: APITimeoutError
with a longer timeout: done
Enter fullscreen mode Exit fullscreen mode

The default timeout is 10 minutes. That suits long reasoning calls, not a web request where a user is waiting. Set a timeout per use case (interactive: 30-60 s; background: longer), stream interactive responses so users see progress, and remember a retried timeout multiplies the wait: worst case = timeout × (retries + 1).

4. A handling pattern for applications

import logging
import openai

log = logging.getLogger("assistant")

def answer(question: str) -> str:
    try:
        r = client.responses.create(model=MODEL, input=question, max_output_tokens=800)
    except openai.RateLimitError:
        return "We're busy right now. Please try again in a minute."      # already retried by the SDK
    except openai.APITimeoutError:
        return "That took too long. Try a shorter question."
    except openai.APIConnectionError:
        log.exception("network problem reaching OpenAI")
        return "Temporary connection problem."
    except openai.BadRequestError as e:
        log.error("bad request %s: %s", e.request_id, e.message)          # a bug in OUR code
        return "Sorry, something went wrong."
    except (openai.AuthenticationError, openai.PermissionDeniedError):
        log.critical("OpenAI credentials rejected")                        # page someone
        raise
    if r.status == "incomplete":
        log.warning("incomplete response %s: %s", r.id, r.incomplete_details)
    return r.output_text
Enter fullscreen mode Exit fullscreen mode

Order matters: APITimeoutError is a subclass of APIConnectionError, so catch it first.

Error Meaning Action
401 / 403 bad, revoked or under-privileged key alert; never retry in a loop
400 / 422 your request is invalid (parameters, schema, too long) fix the code; log the request id
404 unknown model, file or response id check names and ids (models get retired)
429 rate limit or quota exceeded retried automatically; if it persists, cut concurrency or raise limits; check billing
5xx, connection, timeout transient server or network trouble retried automatically; degrade gracefully if it persists

Recap

  • The SDK retries connection errors and 408/409/429/5xx (max_retries, default 2) with backoff.
  • Other 4xx errors mean fix your request; retrying won't help.
  • Set timeouts per use case; the default is long.
  • Log request_id and check status == "incomplete".

This is part 12 of a free 20-part OpenAI Python SDK guide. The full series (streaming, structured outputs, function calling, evals, cost) is at recallrun.dev/guides/openai-sdk.

Top comments (1)

Collapse
 
acemeridian profile image
Ace Meridian •

"Worst case = timeout × (retries + 1)" holds when the server goes quiet, not when it answers slowly. The SDK passes timeout to httpx, which applies it to each connect, write and read, so timeout=2.0 bounds each socket read, not the request.

I checked it with openai 2.43.0 and httpx 0.28.1 on Python 3.14, using OpenAI(timeout=2.0, max_retries=2) and responses.create against a local fake endpoint:

  • Server reads the request and never answers: openai.APITimeoutError after 7.8 s, with three requests in the server log. That is your formula (6 s) plus the backoff sleeps.
  • Server sends the 200 and headers at once, then the JSON body one byte every 1.5 s for the first 16 bytes: the call returned done after 24.5 s. One request, no timeout, no retry. Every byte restarts the 2 s read timer, so anything upstream that dribbles bytes can hold an "interactive" call far past what the timeout suggests.

A wall-clock cap closes that gap. With AsyncOpenAI(timeout=2.0, max_retries=2):

async with asyncio.timeout(5):
    r = await client.responses.create(model=MODEL, input="hello")
Enter fullscreen mode Exit fullscreen mode

On the trickling server that raised at 5.0 s. It raises the builtin TimeoutError, though, and on 2.43.0 neither class subclasses the other (issubclass(openai.APITimeoutError, TimeoutError) is False), so an async port of the answer() chain in section 4 needs its own except TimeoutError: branch next to APITimeoutError, or the cap surfaces as an unhandled error.

Could fake_openai script a delay per byte? Then the slow-body case would be one more entry in your section 3 script.