Network calls fail. Rate limits get hit, servers have bad minutes, requests are malformed, keys expire.
The OpenAI Python SDK already retries the retryable errors for you (connection problems, 408, 409, 429 and 5xx) with exponential backoff. Your job is three things: set sensible timeouts, handle the errors that retrying cannot fix, and log the request id so a failure can be traced later.
1. The exception hierarchy
openai.APIError
├── APIConnectionError -> retried automatically
│ └── APITimeoutError -> retried automatically
└── APIStatusError
├── 400 BadRequestError -> fix the request
├── 401 AuthenticationError -> fix the config
├── 403 PermissionDeniedError -> fix the config
├── 404 NotFoundError -> fix the request
├── 422 UnprocessableEntityError -> fix the request
├── 429 RateLimitError -> retried automatically
└── 5xx InternalServerError -> retried automatically
2. Automatic retries, observed
fake_client below is a small stub that scripts HTTP responses, so you can watch the retry logic without spending tokens or waiting for a real outage.
import openai
from fake_openai import fake_client
# a 429 then a 500, then success: the SDK retries with backoff and you never see the errors
client = fake_client(script=[{"status": 429}, {"status": 500}, "Recovered."], max_retries=2)
r = client.responses.create(model="gpt-5.6-luna", input="hello")
print(r.output_text, "| HTTP requests made:", len(client.fake.requests))
# more failures than retries: the error reaches your code
client = fake_client(script=[{"status": 429, "message": "Rate limit reached for requests"}] * 5, max_retries=2)
try:
client.responses.create(model="gpt-5.6-luna", input="hello")
except openai.RateLimitError as e:
print(type(e).__name__, e.status_code, "| request id:", e.request_id,
"| requests made:", len(client.fake.requests))
# a 400 is NOT retried: sending the same bad request again would fail again
client = fake_client(script=[{"status": 400, "message": "Invalid value for 'model'"}], max_retries=2)
try:
client.responses.create(model="gpt-5.6-luna", input="hello")
except openai.BadRequestError as e:
print(type(e).__name__, e.status_code, e.message, "| requests made:", len(client.fake.requests))
Output:
Recovered. | HTTP requests made: 3
RateLimitError 429 | request id: req_fake_8 | requests made: 3
BadRequestError 400 Error code: 400 - {'error': {'message': "Invalid value for 'model'", 'type': 'fake_error', 'code': None}} | requests made: 1
Three requests for one call in the first case, and the 400 went out exactly once.
3. Timeouts
import openai
from fake_openai import fake_client
client = fake_client(script=[{"delay": 2.0}, "too late"], timeout=0.5, max_retries=0)
try:
client.responses.create(model="gpt-5.6-luna", input="summarise this long report")
except openai.APITimeoutError as e:
print("timed out:", type(e).__name__)
# per-call override for a known-slow request (e.g. high reasoning effort)
client = fake_client(script=[{"delay": 0.8}, "done"], timeout=0.5, max_retries=0)
r = client.with_options(timeout=2.0).responses.create(model="gpt-5.6-luna", input="hard question")
print("with a longer timeout:", r.output_text)
Output:
timed out: APITimeoutError
with a longer timeout: done
The default timeout is 10 minutes. That suits long reasoning calls, not a web request where a user is waiting. Set a timeout per use case (interactive: 30-60 s; background: longer), stream interactive responses so users see progress, and remember a retried timeout multiplies the wait: worst case = timeout × (retries + 1).
4. A handling pattern for applications
import logging
import openai
log = logging.getLogger("assistant")
def answer(question: str) -> str:
try:
r = client.responses.create(model=MODEL, input=question, max_output_tokens=800)
except openai.RateLimitError:
return "We're busy right now. Please try again in a minute." # already retried by the SDK
except openai.APITimeoutError:
return "That took too long. Try a shorter question."
except openai.APIConnectionError:
log.exception("network problem reaching OpenAI")
return "Temporary connection problem."
except openai.BadRequestError as e:
log.error("bad request %s: %s", e.request_id, e.message) # a bug in OUR code
return "Sorry, something went wrong."
except (openai.AuthenticationError, openai.PermissionDeniedError):
log.critical("OpenAI credentials rejected") # page someone
raise
if r.status == "incomplete":
log.warning("incomplete response %s: %s", r.id, r.incomplete_details)
return r.output_text
Order matters: APITimeoutError is a subclass of APIConnectionError, so catch it first.
| Error | Meaning | Action |
|---|---|---|
| 401 / 403 | bad, revoked or under-privileged key | alert; never retry in a loop |
| 400 / 422 | your request is invalid (parameters, schema, too long) | fix the code; log the request id |
| 404 | unknown model, file or response id | check names and ids (models get retired) |
| 429 | rate limit or quota exceeded | retried automatically; if it persists, cut concurrency or raise limits; check billing |
| 5xx, connection, timeout | transient server or network trouble | retried automatically; degrade gracefully if it persists |
Recap
-
The SDK retries connection errors and 408/409/429/5xx (
max_retries, default 2) with backoff. - Other 4xx errors mean fix your request; retrying won't help.
- Set timeouts per use case; the default is long.
-
Log
request_idand checkstatus == "incomplete".
This is part 12 of a free 20-part OpenAI Python SDK guide. The full series (streaming, structured outputs, function calling, evals, cost) is at recallrun.dev/guides/openai-sdk.
Top comments (1)
"Worst case = timeout × (retries + 1)" holds when the server goes quiet, not when it answers slowly. The SDK passes
timeoutto httpx, which applies it to each connect, write and read, sotimeout=2.0bounds each socket read, not the request.I checked it with openai 2.43.0 and httpx 0.28.1 on Python 3.14, using
OpenAI(timeout=2.0, max_retries=2)andresponses.createagainst a local fake endpoint:openai.APITimeoutErrorafter 7.8 s, with three requests in the server log. That is your formula (6 s) plus the backoff sleeps.doneafter 24.5 s. One request, no timeout, no retry. Every byte restarts the 2 s read timer, so anything upstream that dribbles bytes can hold an "interactive" call far past what the timeout suggests.A wall-clock cap closes that gap. With
AsyncOpenAI(timeout=2.0, max_retries=2):On the trickling server that raised at 5.0 s. It raises the builtin
TimeoutError, though, and on 2.43.0 neither class subclasses the other (issubclass(openai.APITimeoutError, TimeoutError)isFalse), so an async port of theanswer()chain in section 4 needs its ownexcept TimeoutError:branch next toAPITimeoutError, or the cap surfaces as an unhandled error.Could
fake_openaiscript a delay per byte? Then the slow-body case would be one more entry in your section 3 script.