DEV Community

Cover image for The OpenAI Agents SDK in Production: What It Gives You and What It Hides
Gabriel Anhaia
Gabriel Anhaia

Posted on

The OpenAI Agents SDK in Production: What It Gives You and What It Hides

#agents #ai #llm #python

A support team wires up their first agent on a Friday. By Monday, tickets are triaged, refund requests escalate to an agent with Stripe access, and abuse reports route to a read-only agent holding a policy doc. No state machine. No graph compilation. About two hundred lines of Python, and a tracing UI that already shows every run with every tool call labelled.

That is the OpenAI Agents SDK doing exactly what it was built to do. It is also the moment where a lot of teams stop reading the docs, because the thing works. Six months later, the same team is trying to correlate an agent trace with the HTTP request that triggered it, and the answer is: you can't, not without rebuilding the part the SDK quietly did for you.

This post is about that gap. The abstractions that earn their place, and the ones that hide behavior you will eventually need to see.

The package is not the package you think

The thing you install is openai-agents. It is not the openai package, and the import trips people up on day one.

pip install openai-agents
Enter fullscreen mode Exit fullscreen mode 
from agents import Agent, Runner
Enter fullscreen mode Exit fullscreen mode

The plain openai package still does raw chat, responses, embeddings, moderations. If you want an agent from it, you write the tool-call loop yourself: call the model, read tool_calls, dispatch to your functions, append results, call again, stop when the model returns a final answer. Three of the four bugs in your first hand-rolled agent live in that loop.

The Agents SDK owns the loop. The mental model is small: an Agent plus a Runner. You describe the agent declaratively and hand it to the Runner, which runs until the model returns a final output or a guardrail trips.

from agents import Agent, Runner

agent = Agent(
    name="Assistant",
    instructions="You are a terse assistant.",
)

result = Runner.run_sync(agent, "Explain a semaphore.")
print(result.final_output)
Enter fullscreen mode Exit fullscreen mode

run_sync is the blocking entry point. In production you want Runner.run (async, streams events). The result object also carries new_items, raw_responses, and last_agent. Log all four in structured form. When an eval fails three weeks from now, you will want to know which agent emitted the bad answer.

The version pin matters more here than in most libraries. The SDK is young and cuts roughly one minor a month. Signatures have broken between minors, so treat every upgrade as potentially breaking. Pin it, and upgrade on a branch with your evals wired up.

# pyproject.toml
dependencies = [
    "openai-agents==0.13.6",
    "openai>=1.50",
    "pydantic>=2.6",
]
Enter fullscreen mode Exit fullscreen mode

Handoffs: the primitive that sells the SDK

A handoff is a delegation from one agent to another, modelled to the LLM as a regular tool call. When the model decides to hand off, the SDK swaps the active agent mid-run (new instructions, new tools, same conversation) and keeps going. You write no routing code.

from agents import Agent, Runner, function_tool

@function_tool
def issue_refund(order_id: str, cents: int) -> str:
    """Issue a refund for an order, in cents."""
    return f"Refunded {cents}c on {order_id}"

refunds_agent = Agent(
    name="RefundsSpecialist",
    instructions=(
        "Handle refunds. Never refund over 5000c "
        "without escalating. Use issue_refund."
    ),
    tools=[issue_refund],
)

triage_agent = Agent(
    name="Triage",
    instructions=(
        "Read the message. If it is a refund, hand off "
        "to RefundsSpecialist. Otherwise answer briefly."
    ),
    handoffs=[refunds_agent],
)

result = Runner.run_sync(
    triage_agent, "Refund order A-4421, never shipped."
)
print(result.last_agent.name)  # RefundsSpecialist
Enter fullscreen mode Exit fullscreen mode

The triage agent has no tools of its own, only handoffs. The SDK renders each target as a tool named transfer_to_<agent_name>, so the model picks a handoff the same way it picks any tool call. Your routing logic lives in the instructions.

That is the trade, and it cuts both ways. You add a new specialist by appending to a Python list. No graph recompile, no edge rewiring. But your control flow now lives inside prompts. A router in code that misroutes is a bug you unit-test. A handoff that misroutes is a bug you catch in evals, if you catch it at all.

The rule of thumb that holds up: if the routing is something a domain expert could sketch in five edges on a napkin, a handoff ships faster and reads fine. If it has more than a handful of branches, or depends on structured state the model should not second-guess (an invoice status, a legal-hold flag), put it behind deterministic code. The pattern that works in practice is soft routing at the top, hard invariants in the tools. The triage agent hands off; the specialist's issue_refund reads the feature flag and the policy check before it touches Stripe. The LLM does the soft part. Python enforces the load-bearing part where your tests can reach it.

Guardrails run in parallel, and only where you wire them

Guardrails are not blocking middleware. They are concurrent validators that can trip an exception mid-run.

from agents import (
    Agent, Runner, GuardrailFunctionOutput, input_guardrail,
)
from pydantic import BaseModel

class PIICheck(BaseModel):
    contains_ssn: bool

pii_detector = Agent(
    name="PIIDetector",
    instructions="Set contains_ssn=true if text has a US SSN.",
    output_type=PIICheck,
)

@input_guardrail
async def block_ssn(ctx, agent, user_input):
    result = await Runner.run(pii_detector, user_input)
    check = result.final_output
    return GuardrailFunctionOutput(
        output_info=check,
        tripwire_triggered=check.contains_ssn,
    )
Enter fullscreen mode Exit fullscreen mode

When you call Runner.run, the SDK kicks off guardrails as asyncio tasks alongside the main agent's first LLM call. The latency cost is roughly the max of the two, not the sum. That is why the guardrail judge should be a small, cheap model, not your main reasoning model. Use the expensive one for both and you double per-turn cost for no reason.

Here is the part that bites teams. Guardrails are targeted, not blanket. Input guardrails fire only for the first agent in a chain; after a handoff they do not re-run. Output guardrails fire only for the agent that produces the final output. Tool guardrails run on every function-tool call but not on handoffs, because handoffs are a separate pipeline. If you assumed a guardrail wraps every agent transition, it does not. To enforce a policy across the whole chain you need output guardrails on the final agent, or an explicit check in each agent's instructions.

Sessions persist turns, not tool progress

A Session is the persistent conversation. Pass one to Runner.run and successive calls share history.

from agents import Agent, Runner, SQLiteSession

session = SQLiteSession("user-42", "sessions.db")
agent = Agent(name="Assistant", instructions="Be terse.")

await Runner.run(agent, "My name is Gabriel.", session=session)
await Runner.run(agent, "What is my name?", session=session)
Enter fullscreen mode Exit fullscreen mode

SQLiteSession is batteries-included; for a production backend, the RedisSession in agents.extensions.memory is the one you reach for. What a session does not give you is checkpointing of mid-run tool calls. If your agent dies halfway through a five-tool sequence, the session resumes from the last persisted turn, not the last persisted tool. If exactly-where-it-crashed resumption is load-bearing for you, that is a checkpointer's job, and the Agents SDK does not have one. Reach for a graph framework instead.

The default provider is OpenAI. You can point it at Claude.

The SDK defaults to OpenAI's Responses API, which buys stateful reasoning and hosted tools. It is not locked to OpenAI models. The LiteLLM adapter points any Agent at another provider, and for a cheap guardrail judge or a long-context summariser, Claude is a sensible default.

from agents import Agent
from agents.extensions.models.litellm_model import LitellmModel

judge = Agent(
    name="Judge",
    instructions="Score the answer 1-5 for policy compliance.",
    model=LitellmModel(model="anthropic/claude-sonnet-4-5"),
)
Enter fullscreen mode Exit fullscreen mode

The catch to know before you commit: non-OpenAI calls do not show up in the built-in platform tracer. The moment you go multi-provider, half your run stops appearing in the one observability surface the SDK gave you for free.

TypeScript gets the same shapes

The SDK is not Python-only. @openai/agents ships the same primitives for Node and the browser, which matters if your product surface is already TypeScript.

import { Agent, run } from "@openai/agents";

const agent = new Agent({
  name: "Assistant",
  instructions: "You are a terse assistant.",
});

const result = await run(agent, "Explain a semaphore.");
console.log(result.finalOutput);
Enter fullscreen mode Exit fullscreen mode

Handoffs, tools, and guardrails carry over with camelCase names. The tracing caveat below is identical: the built-in tracer writes to OpenAI's platform, not yours.

Tracing: the best thing and the thing that hides the most

Run anything and open platform.openai.com/traces. Every run is a trace, every generation a span, every tool call and handoff and guardrail a span, inputs and outputs captured. You configured nothing. You did not set an OTLP endpoint or import a tracing module.

For the first two weeks of a project there is nothing faster. It is also the abstraction that hides the most, because your traces live inside OpenAI's platform, not your stack. They are not correlated with your HTTP traces, your database spans, your queue latency. If your SRE team is on Grafana or Datadog and expects to pivot from an agent trace to the upstream request, they cannot. Not without shipping OpenTelemetry anyway. And the platform UI enforces a retention window, so eval replays and post-mortems need your own storage regardless.

The bridge is short. OpenInference wraps the SDK and emits OTel GenAI spans to any OTLP endpoint, preserving the handoff hierarchy so waterfall views render without custom span munging.

from openinference.instrumentation.openai_agents import (
    OpenAIAgentsInstrumentor,
)
OpenAIAgentsInstrumentor().instrument()
Enter fullscreen mode Exit fullscreen mode

Do this on day one if you know you will need it. Retrofitting after you have built dashboards that read from platform traces is the migration nobody schedules until it is urgent.

What the SDK actually gives you

The SDK is the fastest path to a working multi-agent prototype on OpenAI models. What it hides is the seam between "works in the demo" and "runs in your stack." None of that is a flaw. It is the shape of the trade: shipping speed and hosted tooling, in exchange for coupling and abstractions you have to see through when the system grows past the demo. Know which side of that trade is load-bearing for you before you pick it, not six months in.

If this helped, both books in The AI Engineer's Library go deeper than a post can. Agents in Production walks the SDK's primitives end to end, including the handoff and guardrail edge cases above and when to reach for a graph instead. Observability for LLM Applications is the companion for the tracing fork: OpenTelemetry as the vendor-neutral wire, backend pricing, and the eval loop that catches a misrouted handoff before your users do.

The AI Engineer's Library — Observability for LLM Applications and Agents in Production, side by side

Top comments (0)