AIFast Hub

Posted on Aug 13 Edited on Aug 16

Debugging 401, 429, and 5xx Errors in OpenAI-Compatible APIs: A Practical Field Guide

#api #python #openai #debugging

A failed AI request often gets reduced to one line in an application log:

openai.APIStatusError: Error code: 502

That line is almost useless on its own.

A request may pass through an SDK, DNS, TLS, a reverse proxy, an API gateway, and an upstream model before the response reaches your code. A 401, 429, or 502 tells you where to start, but it does not tell you what happened.

The fastest debugging method I know is to stop treating every failure as "the API is down." First capture the raw response. Then classify the failure. Only after that should you decide whether a retry makes sense.

Capture the wire evidence first

SDK exceptions are convenient, but they hide details you need during an incident. Reproduce one failing call with curl before changing application code:

export OPENAI_BASE_URL="https://www.aifast.hk/v1"
export OPENAI_API_KEY="replace-with-your-key"
export OPENAI_MODEL="copy-an-exact-model-id-from-your-provider"

curl -sS \
  -D /tmp/ai-response.headers \
  -o /tmp/ai-response.body \
  -w $'http=%{http_code}\ndns=%{time_namelookup}\nconnect=%{time_connect}\ntls=%{time_appconnect}\nfirst_byte=%{time_starttransfer}\ntotal=%{time_total}\n' \
  "$OPENAI_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"$OPENAI_MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"Reply with OK\"}],\"max_tokens\":8}"

This leaves you with three useful artifacts:

/tmp/ai-response.headers
/tmp/ai-response.body
DNS, connection, TLS, first-byte, and total timings

Do not paste the complete Authorization header into a ticket. Keep the status, request ID, rate-limit headers, error code, and timestamps. Redact credentials and user data.

A small shell helper makes the capture repeatable:

printf '%s\n' '--- headers ---'
sed -E 's/^(set-cookie:|authorization:).*/\1 [REDACTED]/I' /tmp/ai-response.headers
printf '%s\n' '--- body ---'
python3 -m json.tool /tmp/ai-response.body 2>/dev/null || cat /tmp/ai-response.body

If curl fails before it receives an HTTP status, you are not debugging a 401, 429, or 5xx yet. Check DNS, TLS, proxy variables, and network reachability first.

dig +short "$(python3 - <<'PY'
from urllib.parse import urlparse
import os
print(urlparse(os.environ['OPENAI_BASE_URL']).hostname)
PY
)"

curl -Iv "$OPENAI_BASE_URL/models"

A 401 is an identity or routing problem

A 401 means an HTTP server received the request and refused the credentials. It is not proof that the network is broken.

I check these in order:

Did the request go to the expected host?
Does OPENAI_BASE_URL include the required /v1 prefix exactly once?
Did the shell actually export the key used by this process?
Does the key belong to this provider, project, or organization?
Is an intermediate gateway stripping or replacing Authorization?
Does the account or key have an IP allowlist?

Check the variables without printing the secret:

python3 - <<'PY'
import hashlib
import os
from urllib.parse import urlparse

base = os.environ.get("OPENAI_BASE_URL", "")
key = os.environ.get("OPENAI_API_KEY", "")

print("scheme:", urlparse(base).scheme)
print("host:", urlparse(base).hostname)
print("path:", urlparse(base).path)
print("key_present:", bool(key))
print("key_length:", len(key))
print("key_fingerprint:", hashlib.sha256(key.encode()).hexdigest()[:12] if key else "missing")
PY

The fingerprint lets two operators confirm they are using the same key without exposing it.

Next, test the lightest authenticated endpoint the provider exposes:

curl -sS -i "$OPENAI_BASE_URL/models" \
  -H "Authorization: Bearer $OPENAI_API_KEY"

Interpret the result carefully:

/models returns 401: focus on the key, account, host, and gateway.
/models works but generation returns 401 or 403: inspect model permissions and routing policy.
The response is HTML: you may have hit a website route, WAF page, login page, or proxy error instead of the API.
The response is 404: verify the path before rotating keys.

Rotating a key before confirming the host is a common waste of time. A perfect key sent to the wrong service still fails.

A 429 is not one problem

Many applications treat every 429 as "requests per minute exceeded." That is too coarse.

A 429 can mean:

a short-window request-rate limit
a token-rate limit
a concurrency limit
an account usage or spend limit
depleted balance or quota
a provider-specific safety throttle

Read the response body and headers before retrying:

python3 - <<'PY'
import json
from pathlib import Path

body = Path("/tmp/ai-response.body").read_text(errors="replace")
try:
    data = json.loads(body)
except json.JSONDecodeError:
    print(body[:1000])
else:
    error = data.get("error", data)
    for key in ("type", "code", "message", "param"):
        print(f"{key}: {error.get(key)}")
PY

grep -iE '^(retry-after|x-ratelimit-|ratelimit-|x-request-id|request-id):' \
  /tmp/ai-response.headers

The error code matters more than the broad HTTP status. Retrying a temporary rate limit can work. Retrying an exhausted balance or enforced spend limit cannot restore access.

A safe retry loop respects Retry-After, adds jitter, and stops after a small number of attempts:

import email.utils
import random
import time
from datetime import datetime, timezone

import requests


def retry_delay(response, attempt):
    value = response.headers.get("retry-after")
    if value:
        try:
            return max(0.0, float(value))
        except ValueError:
            try:
                target = email.utils.parsedate_to_datetime(value)
                return max(0.0, (target - datetime.now(timezone.utc)).total_seconds())
            except (TypeError, ValueError):
                pass

    return min(20.0, (2 ** attempt) + random.random())


def post_with_bounded_retry(url, headers, payload, attempts=4):
    for attempt in range(attempts):
        response = requests.post(url, headers=headers, json=payload, timeout=(5, 90))

        if response.status_code != 429:
            response.raise_for_status()
            return response.json()

        try:
            code = response.json().get("error", {}).get("code", "")
        except ValueError:
            code = ""

        permanent = {
            "insufficient_quota",
            "organization_spend_limit_exceeded",
            "project_spend_limit_exceeded",
            "organization_usage_limit_exceeded",
        }
        if code in permanent or attempt == attempts - 1:
            response.raise_for_status()

        time.sleep(retry_delay(response, attempt))

One trap: the official SDK already retries some failures. If you add another retry layer around it, four application attempts can turn into many more wire requests. During debugging, set SDK retries explicitly so the count is observable:

import os

from openai import OpenAI

client = OpenAI(
    api_key=os.environ["OPENAI_API_KEY"],
    base_url=os.environ["OPENAI_BASE_URL"],
    max_retries=0,
)

Then decide where retry ownership belongs. I prefer one retry layer, not an SDK loop hidden inside a job-worker loop hidden inside a queue retry.

Split 5xx by where it originated

5xx means a server failed, but "server" may refer to your reverse proxy, the API gateway, or the upstream model provider.

Start with the response shape:

JSON in the provider's normal error schema suggests the request reached the API layer.
An nginx or CDN HTML page suggests an edge or gateway failure.
An empty body with a gateway-branded header often means the upstream connection failed.
A request ID from the provider is evidence the provider saw the call.

Then compare timing:

Large DNS or TLS time: network path or certificate issue.
Fast 502: gateway rejected or could not open the upstream connection.
Response near a fixed duration such as 30 or 60 seconds: proxy or load-balancer timeout is likely.
Long first-byte time followed by 504: upstream generation exceeded a gateway deadline.

Run two controls before declaring an outage:

# Control 1: authenticated discovery
curl -sS -o /dev/null -w '%{http_code} %{time_total}\n' \
  "$OPENAI_BASE_URL/models" \
  -H "Authorization: Bearer $OPENAI_API_KEY"

# Control 2: minimal non-streaming generation
curl -sS -o /tmp/minimal.json -w '%{http_code} %{time_total}\n' \
  "$OPENAI_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"$OPENAI_MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"OK\"}],\"max_tokens\":4,\"stream\":false}"

If discovery works and only one model fails, the gateway is reachable. Test another model listed as available by the provider before changing DNS or TLS settings. If every model fails with the same gateway-generated response, collect evidence and check the provider's live status information.

For AIFast, the English model compatibility and status checker is one way to separate a model-specific maintenance event from a client-side configuration failure. Treat it as a provider-specific example; the same method applies to any gateway with a live model catalog and status source.

Streaming needs a separate test

A non-streaming request can succeed while streaming fails through a buffering proxy.

Use curl -N so output is not buffered by the client:

curl -N -sS \
  "$OPENAI_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"$OPENAI_MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"Count from 1 to 5, one number at a time\"}],\"stream\":true}"

What to watch:

All chunks arrive at once: a proxy may be buffering Server-Sent Events.
The connection closes without a terminal marker: inspect idle timeouts and upstream resets.
Non-streaming succeeds but streaming gets 502: compare gateway buffering and timeout policies.
Chunks are valid but your SDK crashes: save the raw event stream and inspect parser assumptions.

Do not use a single 200 non-streaming response as proof that an OpenAI-compatible endpoint supports your production workload.

Log failures in a form another person can use

A useful incident record answers six questions:

{
  "timestamp_utc": "2026-08-13T07:30:00Z",
  "operation": "chat.completions",
  "model": "provider-model-id",
  "stream": false,
  "http_status": 502,
  "error_code": "upstream_error",
  "request_id": "req_redacted",
  "attempt": 1,
  "duration_ms": 31240,
  "base_url_host": "api.example.com"
}

Keep the API key, prompt text, cookies, and personal data out of the log. If prompts are needed for reproduction, store a sanitized fixture separately.

In Python, catch connection failures separately from status failures:

import os

import openai

try:
    response = client.chat.completions.create(
        model=os.environ["OPENAI_MODEL"],
        messages=[{"role": "user", "content": "Reply with OK"}],
        max_tokens=8,
    )
    print("request_id:", response._request_id)
except openai.APIConnectionError as exc:
    print("transport_error:", repr(exc.__cause__))
    raise
except openai.APIStatusError as exc:
    print("status:", exc.status_code)
    print("request_id:", exc.request_id)
    print("body:", exc.response.text[:1000])
    raise

The request ID is the best bridge between your logs and a provider's logs. Record it on failures and successful canary requests.

The decision tree I use

When a production request fails, I follow this order:

No HTTP status: inspect DNS, TLS, proxy, and connection errors.
401: verify host, /v1, key fingerprint, project, and gateway header forwarding.
429: read error.code and rate headers. Retry only temporary limits.
5xx: identify the response owner, compare timings, run a second model and a non-streaming control.
Streaming-only failure: inspect buffering, idle timeout, and event framing.
Preserve the request ID, UTC time, model, operation, status, duration, and sanitized body.

The main lesson is boring but useful: status codes are categories, not diagnoses. A blind retry may hide a transient failure, multiply traffic during an outage, or delay the discovery that an account has no usable quota.

Capture the wire evidence first. The fix usually becomes much smaller after that.

DEV Community