TL;DR — When an agent fails, the model's final answer is the least useful thing to look at. Agent Black Box captures every OpenAI Agents SDK run as structured spans, renders it as a timeline, replays any run against GPT-5.6 with captured tool outputs stubbed in (side-effect-free), and turns the corrected behavior into a portable regression eval. Built solo in four days for OpenAI Build Week 2026. Live demo: blackbox.kopachelli.dev · 99s video: youtu.be/rxBLF6QtJkE.
When an agent fails in production, the real cause is usually several steps before the final message: a bad handoff, malformed function arguments, an unexpected guardrail result, or an MCP call that returned something the next agent misread. The raw JSON has the evidence — it just doesn't tell the story.
That's the problem I built Agent Black Box to solve: a self-hostable flight recorder for OpenAI agents. Capture a run as structured spans, inspect it as a timeline, replay it safely on GPT-5.6, and turn the corrected behavior into a regression eval.
Start with the one path that must work
With four days, a normal feature backlog would have been dangerous. I treated one user journey as sacred:
seeded failing run → timeline → failing span → replay on GPT-5.6 → diff red→green → generate eval → Run all green
Every technical choice had to protect that path; anything that threatened it was cut or deferred. The canonical incident is small but realistic: a two-agent support flow hands off to a refund agent that calls refund_order with the malformed amount "12.OO". That failure is reproducibly generated through the real Python Agents SDK Runner and tracing lifecycle using a local scripted model — so anyone can produce it without an API key or model variance. The replay, by contrast, is a live GPT-5.6 operation.
A thin capture layer, not a second agent framework
The Python integration consumes the Agents SDK's public TracingProcessor callbacks directly. Register AbbProcessor, and completed spans are normalized and exported as one authenticated batch when the trace ends — in a background worker, so callback or transport failures never alter the host agent.
The wrapper is deliberately thin. It doesn't replace Runner or invent a tracing format; it preserves IDs, parentage, timing, and errors, then maps everything into five span types: turn, tool_call, handoff, guardrail, mcp_call. At the server boundary, Phoenix does the unglamorous work that makes an observability tool trustworthy:
- spans are idempotent on run + external span ID;
- input/output/error bodies are recursively redacted before persistence;
- raw project keys are never stored — only uniquely indexed SHA-256 digests;
- bodies over 256 KiB become a bounded truncation sentinel, not broken JSON;
- PostgreSQL transactions + advisory locks keep concurrent version allocation consistent.
Phoenix PubSub announces new runs and LiveView re-queries PostgreSQL as the source of truth. No separate frontend framework, no client-side store to reconcile.
The important replay boundary
"Replay" is easy to over-claim. Agent Black Box does not pretend model generation is deterministic — GPT-5.6 stays live and its wording may vary. The deterministic boundary is captured-tool execution.
The engine rebuilds the conversation up to the point before the original terminal answer, reconstructs strict tool definitions, and calls GPT-5.6 through the Responses API with Programmatic Tool Calling. When the model emits a client-owned function call, the harness matches it to the captured sequence and returns the stored output. The real tool never runs — so replaying a refund, an email, or a DB mutation never repeats the side effect.
The seeded failure exposed a nice edge case: its malformed args were rejected before refund_order executed, so there was no historical output to return. Rather than falsify the original span, the trace carries an explicitly labeled safe replay_stub_output used only when the captured output is absent; if neither exists, the engine records a divergence instead of invoking the tool.
The final verdict needs no extra model call: a failed source followed by a successful child replay is deterministically labeled FAILURE FIXED. The judge sees a stable verdict while the underlying generation stays honestly live.
Turning a debugging session into a test
A green replay is useful once. A saved eval makes it useful on every future deploy.
Agent Black Box compacts the captured source plus its latest successful replay and asks GPT-5.6 for a strict Structured Output that must match a closed, versioned assertion schema and pass local validation before anything is stored. Synthesis focuses on replay-stable structure: a tool was called, a JSON-Pointer argument matched an expected scalar, the run stayed within a turn bound, and no span errored. Run all re-replays each eval and checks assertions locally — no model-as-judge — and the stored contract is portable JSON, not an Elixir closure, leaving a clean path to a future CI runner.
This came from a real bug: the first generated eval copied full prose from a replay; a later replay phrased the same result differently and correctly failed the string check. The fix wasn't a looser green button — it was separating stochastic prose from durable structural behavior.
Safety and cost are part of the feature
Captured traces can hold credentials, private prompts, and expensive replay contexts, so safety is on the product path, not in deployment notes. Every OpenAI call goes through one audited AgentBlackBox.OpenAI module: store: false by default, pinned service tier for predictable accounting, explicit low reasoning for the demo, and one frozen cacheable prefix under a stable non-PII key. Before any network I/O, a supervised in-process ledger reserves a conservative cost bound, settles from reported usage, and charges the full bound when a sent request's outcome becomes unknowable — with a provider-side project cap as the restart-safe outer limit. The public demo runs in demo mode: ingestion returns 403 while replay/eval stay interactive on the known dataset.
Why Phoenix, PostgreSQL, and NixOS
Phoenix LiveView was a deadline choice, not a novelty one: one app owns ingestion, persistence, replay supervision, PubSub, and the streamed UI, and tests swap the OpenAI boundary for Mox while exercising the whole path offline. The hosted instance is the same release packaged natively for a NixOS fleet — beamPackages.mixRelease builds it, systemd runs it under a dynamic user, sops + LoadCredential deliver secrets, PostgreSQL 17 uses local peer auth, and Caddy is the only public listener. Publication was fail-closed (a hostname-scoped Caddy 503 until loopback acceptance passed), and Docker Compose remains the portable self-host route.
What Codex changed about the build
Codex didn't replace product ownership — it made a disciplined, evidence-heavy solo workflow possible under a deadline. One primary Codex task held the substantive implementation, driven by an AGENTS.md operating contract that defined the sacred path, coding rules, safety boundaries, and session protocol; bounded specialist agents reviewed isolated risks (replay semantics, redaction, deployment, UI, submission claims) while the primary task owned integration. Every unit of work had a Linear issue before implementation, decisions became immutable ADRs when made, and worklogs recorded evidence and cost.
The deeper lesson: an agent works far better with explicit invariants than with "build the app." The contract turned scope, truthfulness, and completion evidence into inputs the coding system could reason about.
Lessons I'd carry forward
- Build backward from one undeniable outcome. The sacred path kept infrastructure subordinate to product value.
- Name the deterministic boundary precisely. Captured tools can be deterministic and side-effect-free even when live model generation is not.
- Prefer structural evals to prose snapshots. Stable behavioral invariants survive harmless wording changes.
- Fail closed around money, secrets, and public state.
- Make decisions durable when they happen. An ADR directory is useful to humans and coding agents.
Try the complete path
Zero setup: open the live seeded demo, pick error-bad-tool-args, select the red refund_order span, replay it on GPT-5.6, generate an eval from the successful replay, and hit Run all — the whole capture → replay → eval loop in about a minute. It's self-hostable: one Phoenix + PostgreSQL release on a single box.
Agent Black Box started as a way to make one malformed tool call understandable. The useful result is broader: a concrete pattern for going from opaque agent history → safe live replay → a regression contract another human (or coding agent) can inspect and run.
Disclosure: I built Agent Black Box, and I drafted this write-up with AI assistance (writing and editing). Every technical claim here is verified against the running system.
Top comments (0)