Not a Python tutorial: the patterns that bite in production agents

This is not a Python tutorial. If you can already write a decorator, skip ahead. The agent loop does not care that you know the language. It cares whether your AWS SDK call just froze every other request your process was supposed to be handling while it waited on a model.

Rung 1 of the AI Architect Roadmap is Foundations, and it has sat marked as a gap since the page went up: the Python and tooling that matter once code is running an agent loop in production, not what an intro course teaches. This closes it. Five patterns, all of them things I've either hit myself or watched a production Bedrock workload hit, in roughly the order they bite.

boto3 is synchronous, and that is expensive inside an event loop

boto3 has no native async support. bedrock-runtime.converse_stream() returns a plain synchronous EventStream, and the call itself is a blocking network round trip. Call it directly from inside an asyncio agent loop and you have not made an async mistake in the abstract, you have stopped every other coroutine in that process from making progress for the entire duration of the model's response, streaming or not.

The fix is aioboto3, which wraps aiobotocore to give you an async Bedrock client with the same converse_stream() shape, this time awaited. The gotcha that catches people even after they switch: since aioboto3 8.0, .client() is an async context manager, and opening a fresh one inside every request re-authenticates and re-establishes the connection each time, the same tax you were trying to avoid, just hidden one layer down. The client, not just the Session, has to live for the process's lifetime, held open with an AsyncExitStack:

import aioboto3
from contextlib import AsyncExitStack

session = aioboto3.Session()
exit_stack = AsyncExitStack()
client = None  # populated at startup, reused by every request

async def startup():
    global client
    client = await exit_stack.enter_async_context(session.client("bedrock-runtime"))

async def shutdown():
    await exit_stack.aclose()

async def handle_turn(prompt: str) -> str:
    response = await client.converse_stream(
        modelId="eu.anthropic.claude-sonnet-4-6",
        messages=[{"role": "user", "content": [{"text": prompt}]}],
    )
    async for event in response["stream"]:
        ...  # yields control back to the loop between chunks

Call startup() once when the process boots and shutdown() once when it stops (a framework's lifespan hook, if it has one). Every request in between reuses the same client and the same underlying connection pool.

If you cannot touch the call site (a library that only exposes the sync client), the fallback is loop.run_in_executor(None, sync_call) to push the blocking call onto a thread pool instead of the event loop thread. It costs a thread, but it stops the freeze.

Typed tool signatures are not optional decoration

The Bedrock Converse API's toolSpec takes an inputSchema that must be real JSON Schema, with type: "object" at the top level and a required array naming which properties are mandatory. A bare Python type hint on your tool function does not become this automatically if you are calling Bedrock directly with boto3. Nothing in boto3 converts def get_weather(city: str, unit: str = "celsius") into a schema on its own, you build one by hand, via a Pydantic model's model_json_schema(), or by using an agent framework that does the conversion for you (Strands Agents' @tool decorator parses a function's type hints and docstring into the schema automatically, which is exactly why reaching for boto3 directly instead of a framework is a decision worth making on purpose, not the path of least resistance).

Get the required array wrong, drop a type, or leave a field's description empty, and the failure is not a clean exception. The model either omits an argument it needed, or invents a plausible-looking value for a field whose constraints it was never told, and the tool call fails validation two layers downstream from where the real bug is. AWS's own guidance is explicit that the description is what the model uses to decide when a tool applies, not just what it does, so a thin schema does not just risk a malformed call, it risks the model never reaching for the tool at all. A correct schema is not a security boundary either: it constrains shape, not content, so a syntactically valid string can still be a path-traversal or injection payload the schema never sees. Validate and sanitise inside the tool handler regardless of how tight the schema looks.

Streaming to a browser is a second translation, not a continuation

Bedrock's stream is a boto3 EventStream of typed chunk events, not a wire format a browser understands. If you are proxying that stream to a frontend over Server-Sent Events, you are writing a translator, not a pass-through: every chunk has to become its own data: ...\n\n frame, and the frame boundary matters as much as the content.

The failure mode looks identical to a hung connection: your backend is emitting frames correctly, but something between it and the browser is buffering, an ASGI server, a reverse proxy, a gzip middleware that waits for enough bytes before flushing. curl --no-buffer against your own endpoint is the fastest way to tell the difference between "the server is not sending anything yet" and "the server sent it, something in the middle is holding it."

Where the GIL actually still matters

Python 3.13 shipped a free-threaded build experimentally in October 2024. Python 3.14, a year later, promoted it to officially supported under PEP 779. It is still opt-in: the default build keeps the GIL on, you have to explicitly install or build the free-threaded variant, and as of this year package compatibility across the ecosystem sits at roughly half, not all, of what people actually depend on.

For most agent workloads that barely matters, because the loop is I/O-bound: it spends nearly all of its time waiting on a Bedrock response, not burning CPU, and asyncio was already the right tool for that, GIL or no GIL. Where it still bites is CPU-bound work sharing the same process as the loop, local tokenisation, embedding a document, post-processing a response with a regex pass over a few hundred KB of text. That work still serialises behind the GIL on a stock interpreter no matter how many coroutines you write, and I have watched a "slow Bedrock call" get blamed on the model when the actual bottleneck was a synchronous embedding step competing with the agent loop for the same core.

The order these actually bite in

1. A synchronous boto3 call inside the agent loop, freezing every other request.
2. A fresh aioboto3 client per call, after the async fix, quietly re-paying the connection cost.
3. A tool schema that drifted from the Python type hints it was meant to describe.
4. An assumption that a boto3 stream is already browser-ready SSE.
5. CPU-bound work blamed on "the model" when it is the GIL serialising a step that shares the process with the loop.

Rung 2, the ML you actually need to operate an LLM rather than train one, is still open. If there is a Python pattern that has bitten you in a production agent that is not on this list, I'd genuinely like to hear which one.

This post is paired with Introducing the AI Architect Roadmap, and Evals for Production AI, the series currently climbing rung 7, continues. Find me on LinkedIn or via rajmurugan.com.