48-Hour Field Notes: The Header the Tool Loop Never Saw

The lab clock started when the fifth poll came back 429 and the agent asked for the same status again, 800 milliseconds later. Nothing in the prompt was wrong. The HTTP layer had already told the caller to wait. The model never saw that sentence.

This is a 48-hour field note from a local harness, not a customer story and not a benchmark. The question was narrow: if an AI tool loop treats every non-2xx the same, does Retry-After ever reach the next model turn? The short answer is no, unless you copy the header into the tool result on purpose.

I stood up a tiny status service, pointed a naive poller at it, then compared that poller with a loop that actually honored the header. The service is deliberately rude. After four successful GETs from one client key it starts answering 429 with Retry-After: 3. That is enough to watch a tool-calling agent melt a free server without needing a load generator.

The analogy that stuck: Retry-After is a traffic cop holding a palm out at an intersection. A tool schema that only returns status and body is a driver who only looks at the color of the light. The palm is still there. The intersection still exists.

What I tried in the first twelve hours

The first pass was a stdlib server so the only moving parts were HTTP and a dict. No framework. No queue. One process, one lock, one map from client key to hit count.

# retry_after_lab.py — run: python retry_after_lab.py
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from collections import defaultdict
import threading, json, time

HITS = defaultdict(int)
LOCK = threading.Lock()
LIMIT = 4  # 429 after this many GETs per client key

class Handler(BaseHTTPRequestHandler):
    def log_message(self, fmt, *args):
        return

    def do_GET(self):
        if self.path != "/status":
            self.send_response(404); self.end_headers(); return
        key = self.headers.get("X-Client-Key", "anon")
        with LOCK:
            HITS[key] += 1
            n = HITS[key]
        if n > LIMIT:
            self.send_response(429)
            self.send_header("Retry-After", "3")
            self.send_header("Content-Type", "application/json")
            self.end_headers()
            self.wfile.write(json.dumps({
                "error": "rate_limited",
                "hits": n,
            }).encode())
            return
        self.send_response(200)
        self.send_header("Content-Type", "application/json")
        self.end_headers()
        self.wfile.write(json.dumps({
            "state": "open",
            "hits": n,
            "ts": time.time(),
        }).encode())

if __name__ == "__main__":
    ThreadingHTTPServer(("127.0.0.1", 8765), Handler).serve_forever()

Curl confirmed the contract before any model entered the room. Four 200s. Then a 429 with a header the shell could print and a JSON body that looked like an ordinary error.

python retry_after_lab.py &
for i in 1 2 3 4 5 6; do
  echo "--- $i"
  curl -sD - -H "X-Client-Key: lab" http://127.0.0.1:8765/status
  echo
done

That part was boring in the useful way. The header was stable. The body was small. The failure mode I cared about was not the server. It was the next hop: a tool wrapper that serializes only the JSON body into the model transcript.

What broke when the loop started calling tools

Most agent runtimes expose HTTP as a function with a name, a URL, and a parsed body. Status code sometimes makes it through. Response headers often do not. I labeled the following wrapper as a proposal that matches what I have seen in traces; it is not a dump from a production agent.

# Proposed tool result — headers dropped on purpose
def http_get(url: str, headers: dict | None = None) -> dict:
    import urllib.request
    req = urllib.request.Request(url, headers=headers or {})
    try:
        with urllib.request.urlopen(req, timeout=5) as resp:
            return {
                "ok": True,
                "status": resp.status,
                "body": resp.read().decode(),
            }
    except urllib.error.HTTPError as e:
        return {
            "ok": False,
            "status": e.code,
            "body": e.read().decode(),
        }

Feed that dict back as a tool result and the model receives {"ok": false, "status": 429, "body": "{\"error\": \"rate_limited\", \"hits\": 5}"}. There is no sleep instruction in that object. A reasonable next token sequence is another http_get on the same URL. The cop's palm never entered the chat.

I then ran a second client that did what a rushed agent does: retry on any failure, 200ms apart, no jitter, same client key. Six extra GETs arrived inside two seconds. The server did its job. The client did not.

# naive_poller.py — this is the broken loop under test
import json, time, urllib.request, urllib.error

URL = "http://127.0.0.1:8765/status"
HEADERS = {"X-Client-Key": "naive"}

def once():
    req = urllib.request.Request(URL, headers=HEADERS)
    try:
        with urllib.request.urlopen(req, timeout=5) as resp:
            return resp.status, dict(resp.headers), resp.read().decode()
    except urllib.error.HTTPError as e:
        return e.code, dict(e.headers), e.read().decode()

for i in range(10):
    status, headers, body = once()
    print(f"{i} status={status} retry_after={headers.get('Retry-After')} body={body}")
    if status == 200:
        break
    time.sleep(0.2)  # the bug: ignores Retry-After

The printout is the whole lesson. retry_after=3 sits in the same line as the sleep of 0.2. The number is visible to Python. It is invisible to the model unless the tool result copies it. That is the seam.

A correct tool result is not clever. It is wider.

# Proposed tool result — header survives into the next model turn
{
  "ok": False,
  "status": 429,
  "retry_after_seconds": 3,
  "instruction": "Do not call http_get again until retry_after_seconds have elapsed.",
  "body": {"error": "rate_limited", "hits": 5}
}

Even that is not enough if the runtime retries the tool below the model. Some orchestrators catch ok: false and re-invoke the same function before a new completion is requested. Then the header is in the JSON and still unused. The sleep has to live in the orchestrator, not in the prompt. Models are bad metronomes.

The artifact: a probe that fails closed

The second day was a test, not a dashboard. The probe records timestamps of 429s from one client key. If two 429s land closer than Retry-After minus a 150ms clock skew, the test fails. That is the contract I would actually keep.

# test_retry_after.py — run after the server is up
import time, urllib.request, urllib.error, json, sys

URL = "http://127.0.0.1:8765/status"
KEY = "probe"

def hit():
    req = urllib.request.Request(URL, headers={"X-Client-Key": KEY})
    try:
        with urllib.request.urlopen(req, timeout=5) as resp:
            return resp.status, dict(resp.headers), time.monotonic()
    except urllib.error.HTTPError as e:
        return e.code, dict(e.headers), time.monotonic()

# Warm the limiter
for _ in range(5):
    status, headers, ts = hit()

assert status == 429, status
wait = float(headers.get("Retry-After", "0"))
assert wait >= 1, headers

# Immediate retry must still be 429
status2, _, ts2 = hit()
assert status2 == 429, status2
gap = ts2 - ts
if gap < wait - 0.15:
    # This assert is the point of the lab.
    print(f"FAIL: second 429 arrived after {gap:.3f}s, header asked for {wait}s")
    sys.exit(1)

time.sleep(wait)
status3, _, _ = hit()
print(json.dumps({"after_wait_status": status3, "gap_before_sleep": round(gap, 3)}))
# Naive pollers never reach this line with a 200 unless LIMIT resets.

I would not ship this as a load test. It is a contract test for the client. The server is a fixture. If you point the same probe at a public API you do not own, you are no longer debugging a tool loop. You are contributing to the 429s you claim to study.

A compact decision table sat on the desk for the rest of the window. Status 429 plus Retry-After means sleep in the orchestrator, then one retry, then stop. Status 429 without the header means exponential backoff with a cap, then stop. Status 200 with an application error in the body is not a retry at the HTTP layer. Status 204 is success with an empty body, not a parse failure. Mixing those four is how an agent turns a polite limiter into a retry storm.

Where a free model lane and a free server actually helped

The harness above runs on a laptop. The reason to move it off the laptop is overnight traces: you want the model turns, the tool JSON, and the HTTP log in one place while you sleep. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is an open-source coding-agent project that, as supplied for this write-up, offers free model access and a free server option. I used that pairing as a place to rerun the same poller against the same fixture without inventing a GPU story. I am not attaching model names, token ceilings, hardware, or duration claims here. Those numbers go stale in a week and this note is about a header, not a plan sheet.

The useful part was logistical. Start the fixture, start the naive poller, start a second poller that copies Retry-After into the tool result, and keep the transcripts. If the free server is busy or undefined in capacity, the test still holds locally. The product is a convenience for the method, not the method.

One pass I would repeat: log the tool arguments as raw strings before JSON parse. Several traces showed a function call that looked like {"url": "http://127.0.0.1:8765/status" with no closing brace. The runtime retried the parser, then retried the HTTP call, then the limiter engaged. Truncated tool JSON and ignored headers compound. They look like two bugs. They are one pipeline with no backpressure.

What I would repeat, and what I would not

I would keep the fixture tiny. Four 200s and a 429 is enough. I would keep Retry-After in the tool schema as a first-class field, not a hope that the model will read body. I would keep the sleep in process code. I would keep the probe that fails when two 429s are too close.

I would not let the model choose the sleep duration from vibes. I would not parse only the body. I would not run this against a host I do not operate. I would not treat a free shared server as an SLO. Unknown caps are not infinite caps. They are unknown.

Limitations are blunt. This lab does not measure throughput, cost, or quality of any model. It does not prove that every agent framework drops headers; it shows how a common wrapper shape makes the drop inevitable. Clock skew, HTTP/2, proxies that strip Retry-After, and servers that send a date instead of a delta are all out of scope. If your limiter returns 503 with no header, this probe will not save you.

Who should skip this approach: anyone using the poller as a weapon against a third-party API; anyone who needs guaranteed remote capacity; anyone hoping a prompt will replace a mutex. If your tool results are already {status, headers, body} and your orchestrator sleeps, you do not need the article. You need the assert.

Forty-eight hours later the fixture still returns 429 on the fifth hit. The naive loop still ignores the cop. The probe still fails closed. That is the whole note. If you want a sandbox that already wires free model access to a free server lane, MonkeyCode is one place to rerun the harness — the tests above do not depend on it.