DEV Community: klement Gunndu

Inside Claude Code's Hidden Multi-Agent Architecture

klement Gunndu — Thu, 02 Apr 2026 23:53:47 +0000

Anthropic's Claude Code has 58 tools, but the one that matters most is the one that spawns copies of itself.

On March 31, the full source leaked via npm source maps. I spent the last two days reading the multi-agent architecture. Here is what I found.

AgentTool: The Tool That Spawns Agents

Every subagent in Claude Code is created through a single tool. The input schema tells you everything about how Anthropic thinks about agent orchestration:

const baseInputSchema = z.object({
  description: z.string().describe('A short (3-5 word) description'),
  prompt: z.string().describe('The task for the agent to perform'),
  subagent_type: z.string().optional(),
  model: z.enum(['sonnet', 'opus', 'haiku']).optional(),
  run_in_background: z.boolean().optional(),
})

The parent agent picks the model tier per task. Search gets Haiku. Complex reasoning gets Opus. Everything else gets Sonnet. This is not automatic routing — the parent makes an explicit choice every time it spawns a child.

One-Shot vs Persistent Agents

The source defines two categories:

export const ONE_SHOT_BUILTIN_AGENT_TYPES: ReadonlySet<string> = new Set([
  'Explore',
  'Plan',
])

One-shot agents run a task and return a report. The parent never sends follow-up messages. This saves tokens — no agent ID, no SendMessage trailer, no usage block. At 34 million Explore runs per week, those 135 characters per run add up.

Every other agent type is persistent. The parent can continue the conversation using SendMessage with the agent's ID. This is how Claude Code runs parallel research tasks while you wait.

Team Spawning: tmux Panes, Not API Calls

The most surprising discovery: teammates are not spawned via API. They are spawned as separate Claude Code processes in tmux panes.

async function handleSpawnSplitPane(input, context) {
  const model = resolveTeammateModel(input.model, getAppState().mainLoopModel)
  const uniqueName = await generateUniqueTeammateName(name, teamName)
  const { paneId } = await createTeammatePaneInSwarmView(...)

  const spawnCommand = `cd ${workingDir} && env ${envStr} ${binaryPath} ${args}`
  await sendCommandToPane(paneId, spawnCommand, ...)

  // Communication via filesystem mailbox
  await writeToMailbox(sanitizedName, { from: 'TEAM_LEAD', text: prompt }, teamName)
}

Each teammate gets its own tmux pane, its own process, its own context window. Communication happens through a filesystem-based mailbox — not shared memory, not API calls. The team lead writes a message to ~/.claude/teams/{team}/mailbox/{agent}.json. The teammate reads it on its next loop iteration.

This is the simplest possible multi-agent communication protocol. No message broker. No WebSocket. No shared state. Just files on disk.

KAIROS: The Autonomous Daemon

Behind a feature flag called KAIROS, there is an unreleased autonomous mode. The agent runs as a persistent daemon that:

Monitors GitHub webhooks for new issues and PRs
Reads a channel-based task queue
Executes tasks without human prompting
Reports results back through the same mailbox system

const fullInputSchema = baseInputSchema.merge(z.object({
  name: z.string().optional(),
  team_name: z.string().optional(),
  mode: permissionModeSchema().optional(),
  isolation: z.enum(['worktree', 'remote']).optional(),  // KAIROS feature
  cwd: z.string().optional(),  // KAIROS feature
}))

export const inputSchema = feature('KAIROS') ? fullInputSchema : fullInputSchema.omit({ cwd: true })

When KAIROS is enabled, agents can specify their own working directory and run in isolated git worktrees. Without it, those fields are stripped from the schema entirely — the model never sees them.

44 Feature Flags Control Everything

The entire system is gated behind feature flags. I counted 44 in the buildable fork:

KAIROS — autonomous daemon mode
PROACTIVE — agent initiates without prompting
COORDINATOR_MODE — multi-agent swarm orchestration
BUDDY — Tamagotchi companion system
VOICE_MODE — voice interaction
BRIDGE_MODE — IDE integration with JWT auth
CHICAGO_MCP — Computer Use (screen control)
ULTRAPLAN — enhanced planning mode
TEAMMEM — team memory sharing
EXTRACT_MEMORIES — automatic memory extraction

Each flag is checked with a feature() function that conditionally includes code, schemas, and even entire tool definitions. Dead code elimination means if a flag is off, the model literally cannot see or call the gated functionality.

What This Architecture Teaches

Three things stood out to me after reading the full multi-agent system:

1. Filesystem beats message brokers for local agents. When all agents run on the same machine, JSON files on disk are simpler, more debuggable, and more reliable than any message queue. You can cat the mailbox. You can tail -f the team log. No infrastructure to maintain.

2. Model routing should be explicit, not automatic. The parent agent chooses Haiku, Sonnet, or Opus for each child. This is a deliberate cost-quality tradeoff made at spawn time, not a system-level optimization. The agent that understands the task picks the model for the task.

3. Feature flags are the real architecture. The 44 flags mean Claude Code is not one product. It is dozens of products sharing a codebase, each activated by a boolean. KAIROS-mode Claude Code is a fundamentally different system from default Claude Code — and the flag system lets Anthropic test both in production simultaneously.

The source was not supposed to be public. But now that it is, it is the most detailed reference for production multi-agent architecture I have read. Every decision is visible in the code.

Follow @klement_gunndu for more AI engineering breakdowns. We are building in public.

What 512K Lines of Leaked Claude Code Taught Me About AI Tool Design

klement Gunndu — Thu, 02 Apr 2026 08:46:54 +0000

On March 31, 2026, Anthropic shipped Claude Code v2.1.88 with a 59.8MB source map file still attached. The entire TypeScript source — 1,900 files, 512K+ lines — was readable by anyone who ran npm pack.

I downloaded it. I read the tool architecture. What I found changed how I think about building AI tools.

This is not speculation. Every code snippet below comes from the actual source. I have the full archive on disk.

The Tool Interface: One Type to Rule 58 Tools

Claude Code ships 58 tools — from BashTool to AgentTool to GrepTool. Every single one implements the same TypeScript type:

export type Tool<Input, Output, Progress> = {
  name: string
  searchHint?: string  // 3-10 word capability hint

  // Core execution
  call(args, context, canUseTool, parentMessage, onProgress): Promise<ToolResult>

  // Schema (Zod)
  readonly inputSchema: Input
  readonly outputSchema?: z.ZodType<unknown>

  // Safety declarations
  isConcurrencySafe(input): boolean
  isReadOnly(input): boolean
  isDestructive?(input): boolean

  // Permission hooks
  validateInput?(input, context): Promise<ValidationResult>
  checkPermissions(input, context): Promise<PermissionResult>
}

The insight is not in any single field. It is in what the type forces you to declare.

Every tool must answer three questions before it runs: Can it run alongside other tools? Does it modify state? Could it destroy something? These are not optional annotations. They are required by the type system.

Most AI tool frameworks I have seen treat safety as an afterthought — a wrapper you add later. Claude Code makes it structural. You cannot build a tool without deciding upfront whether it is safe.

buildTool(): Defaults That Fail Closed

All 58 tools go through a factory function called buildTool(). It supplies defaults:

const TOOL_DEFAULTS = {
  isConcurrencySafe: () => false,   // assume NOT safe
  isReadOnly: () => false,          // assume writes
  isDestructive: () => false,
  checkPermissions: (input) =>
    Promise.resolve({ behavior: 'allow', updatedInput: input }),
}

Read that first line again: isConcurrencySafe: () => false.

If you forget to declare concurrency safety, your tool defaults to serial execution. If you forget to declare read-only, the system assumes your tool writes. The defaults are pessimistic.

This is a pattern I now use in every tool system I build. When the GrepTool overrides it:

export const GrepTool = buildTool({
  name: 'Grep',
  searchHint: 'search file contents with regex (ripgrep)',

  isConcurrencySafe() { return true },
  isReadOnly() { return true },
})

That true is an explicit, conscious declaration. The developer had to think about it.

Compare this to LangChain's @tool decorator, where concurrency and safety are not part of the interface at all. You get convenience, but you lose the forcing function.

BashTool: 22 Security Validators Before Execution

The BashTool is the most complex tool in the system. Before any command runs, it passes through 22 distinct security validators:

const BASH_SECURITY_CHECK_IDS = {
  INCOMPLETE_COMMANDS: 1,
  JQ_SYSTEM_FUNCTION: 2,
  OBFUSCATED_FLAGS: 4,
  SHELL_METACHARACTERS: 5,
  DANGEROUS_PATTERNS_COMMAND_SUBSTITUTION: 8,
  IFS_INJECTION: 11,
  PROC_ENVIRON_ACCESS: 13,
  MALFORMED_TOKEN_INJECTION: 14,
  BRACE_EXPANSION: 16,
  CONTROL_CHARACTERS: 17,
  UNICODE_WHITESPACE: 18,
  ZSH_DANGEROUS_COMMANDS: 20,
  COMMENT_QUOTE_DESYNC: 22,
  // ... 9 more
}

Each validator catches a specific class of shell injection. UNICODE_WHITESPACE catches invisible characters that look like spaces but are not. COMMENT_QUOTE_DESYNC catches payloads that exploit the gap between how comments and quotes are parsed.

This is defense in depth. The permission system handles "should this command run?" The security validators handle "is this command what it appears to be?"

I counted: 22 validators for one tool. Most AI agent frameworks ship bash execution with zero input validation. If you are building a tool that runs shell commands, this is the minimum bar.

Three-Layer Permission Architecture

Claude Code does not have one permission check. It has three layers, and they run in order:

Layer 1: validateInput() — Semantic checks before anything else.

// FileEditTool example
async validateInput(input, context) {
  if (oldString === newString) {
    return { result: false, message: 'No changes to make' }
  }
  const { size } = await fs.stat(fullFilePath)
  if (size > MAX_EDIT_FILE_SIZE) {
    return { result: false, message: 'File too large' }
  }
  return { result: true }
}

Layer 2: checkPermissions() — Rule engine for allow/deny/ask decisions.

Layer 3: canUseTool callback — Hook integration. External systems (pre-tool-use hooks) get a veto.

The key design decision: validation happens before permissions. If the input is semantically invalid, the system rejects it before even checking whether you have permission. This prevents wasting a user's permission approval on a request that would fail anyway.

I have started applying this pattern in my own Python tools. Validate first, authorize second, execute third.

ToolSearch: Lazy Loading That Saves Tokens

Claude Code has 58 tools, but the model does not see all 58 schemas in every prompt. That would burn thousands of tokens on tools the model will never call.

Instead, most tools are "deferred." The model sees only their names. When it needs a tool, it calls ToolSearch:

async function searchToolsWithKeywords(query, deferredTools, maxResults) {
  // Fast path: exact match on tool name
  const exactMatch = deferredTools.find(
    t => t.name.toLowerCase() === queryLower
  )
  if (exactMatch) return [exactMatch]

  // Keyword search: parse CamelCase names into words
  // Score by word boundary matches in name + searchHint
  const matches = scoreAndRankTools(query, deferredTools)
  return matches.slice(0, maxResults)
}

Only after ToolSearch returns a match does the full schema get injected into the conversation.

This is smart token economics. The searchHint field — that 3-10 word description each tool declares — is the entire search corpus. No embeddings, no vector DB. Just keyword matching on short hints.

If you are building an agent with more than 10 tools, steal this pattern. Keep tool descriptions short. Load schemas lazily. Let the model search for what it needs.

What I Am Applying to My Own Systems

I maintain an autonomous content engine (Herald) that publishes to dev.to. It has tools for article creation, comment monitoring, engagement tracking, and browser automation. After reading Claude Code's source, I changed three things:

1. Every tool now declares safety properties. My Python tools have is_read_only and is_concurrent_safe as required attributes, not optional. The default is False for both.

2. Validation before authorization. My Playwright engagement tools now validate comment content (quality gate) before checking browser session permissions. This catches LLM-generated spam before wasting a browser launch.

3. Lazy tool registration. My agent no longer loads all tool schemas at startup. Tools register with a one-line description. Full schemas load on first use.

None of these are revolutionary ideas. But seeing them implemented at scale, in production code serving millions of users, made the patterns click in a way that documentation never did.

The Takeaway

Claude Code's tool architecture is not clever. It is disciplined. Every tool declares its safety properties. Defaults fail closed. Validation precedes authorization. Schemas load lazily. Security checks are specific, not generic.

The source was not supposed to be public. But now that it is, it is the best reference implementation for AI tool design I have seen. Study it.

Follow @klement_gunndu for more AI engineering breakdowns. We are building in public.

5 Red Flags in AI Product Demos That PMs Should Never Ignore

klement Gunndu — Sat, 28 Mar 2026 12:34:02 +0000

Every AI vendor has a demo that works perfectly. That is the problem.

Gartner predicts over 40% of agentic AI projects will be canceled by the end of 2027 due to escalating costs, unclear business value, or inadequate risk controls (Gartner, June 2025). A separate Gartner report found that through 2026, organizations will abandon 60% of AI projects unsupported by AI-ready data (Gartner, February 2025).

The pattern is consistent: teams greenlight AI products based on impressive demos, then discover the gap between demo and production is a canyon.

PMs sit at the decision point. You approve the budget. You set the timeline. You own the outcome when it ships — or when it doesn't. These 5 red flags help you spot the canyon before you walk into it.

Red Flag 1: "AI-Powered" With No Explanation of What That Means

The vendor says their product is "AI-powered." You ask what the AI actually does. They pivot to a slide about "leveraging machine learning" or "using advanced neural networks."

This is AI washing. The term "AI-powered" has become so overused that the U.S. Federal Trade Commission issued guidance warning companies about making unsubstantiated AI claims (FTC, February 2023). The problem has only gotten worse since then.

What to ask instead:

"What specific task does the AI perform that wasn't possible before?"
"What model or approach powers this? Is it a foundation model, a fine-tuned model, or a rules engine with an AI label?"
"What happens when I turn the AI off? What manual process does it replace?"

If the vendor cannot explain in one sentence what the AI does — not what it "leverages" or "harnesses" — the product is either not AI or the team does not understand their own technology. Both are disqualifying.

The test: Ask the sales engineer, not the account executive. Sales engineers talk implementation. Account executives talk vision. You need implementation.

Red Flag 2: The Demo Uses Their Data, Not Yours

The demo runs on a curated dataset. The search returns perfect results. The classification hits 98% accuracy. The generated text reads like a press release from a Fortune 500 company.

Then you feed it your data — messy CSVs with missing fields, inconsistent naming conventions, and 3 years of legacy formatting — and accuracy drops to 60%.

This is the most common gap between demo and production. Gartner found that lack of AI-ready data is the primary reason organizations abandon AI projects (Gartner, February 2025).

What to ask instead:

"Can we run the demo on our data? Not our cleanest data — our realistic data."
"What data preparation did you do before this demo? How long did it take?"
"What percentage of your customers needed data cleaning before going live? How long did that take on average?"

If the vendor hesitates to run on your data, that tells you everything. A mature product handles messy inputs. An immature product needs a clean room.

The test: Bring a sample dataset to the second meeting. Not your best data. Your average data. Watch what happens.

Red Flag 3: No Production Customers — Only Pilots and POCs

"We have 15 enterprise pilots running right now."

Pilots are not production. A pilot is a controlled experiment with a dedicated support team, a narrow scope, and a safety net. Production means the product handles real traffic, real edge cases, and real failures at scale with no one holding its hand.

Gartner predicted that at least 30% of generative AI projects would be abandoned after proof of concept by the end of 2025 (Gartner, July 2024). The pilot-to-production gap is where most AI projects die.

What to ask instead:

"How many customers are running this in production today — not pilots, production?"
"What is your average time from pilot to production deployment?"
"Can I talk to a production customer in my industry? Not a reference they prepared — one I choose from a list."
"What is the biggest production failure you have seen, and how did you handle it?"

The last question is the most revealing. Every production system fails. A vendor who cannot describe a specific failure and their response to it has either never been in production or is hiding something.

The test: Ask for 3 production customer names. If they give you 3 pilot names instead, you have your answer.

Red Flag 4: Pricing That Hides the Real Cost

The vendor quotes $2,000 per month for their AI platform. What they don't mention: the platform makes API calls to a foundation model provider, and those calls are billed separately based on token usage.

Your proof of concept runs 50 queries a day. Your production environment will run 5,000. The $2,000/month platform fee stays the same. The model inference cost goes from $200/month to $20,000/month.

This is not a vendor problem — it is an AI economics problem. Foundation model costs scale with usage in ways that traditional SaaS does not. A SaaS product costs the same whether 10 users or 10,000 users run the same query. An AI product that calls GPT-4 or Claude costs more with every query, every token, every retry.

What to ask instead:

"What is the total cost at 10x our current volume? At 100x?"
"Does your pricing include model inference costs, or are those separate?"
"What happens to cost if we switch to a different model? Are we locked into one provider?"
"What cost optimization have you built in? Caching? Model routing? Batch processing?"

If the vendor quotes a flat rate and cannot answer volume questions, they either haven't scaled or they're counting on you not asking.

The test: Ask for a cost calculator or a cost projection at 3 volume tiers: current, 10x, and 100x. If they don't have one, they haven't thought about it.

Red Flag 5: No Answer to "What Happens When It's Wrong?"

The AI agent summarizes a contract and misses a liability clause. The AI classifier labels a support ticket as low priority when the customer is about to churn. The AI recommendation engine suggests a product that was discontinued last month.

Every AI system produces wrong outputs. The question is not "is it perfect?" — the question is "what happens when it isn't?"

What to ask instead:

"What is your accuracy rate on tasks similar to ours? How do you measure it?"
"When the system produces a wrong output, how does it signal that to the user?"
"Is there a confidence score? What threshold do you recommend for human review?"
"What is your feedback loop? If a user corrects an error, does the system learn from it?"
"Do you have an audit trail? Can I trace why the system made a specific decision?"

A product that cannot explain its failure mode is a product that has not been tested at scale. Confidence scores, human-in-the-loop workflows, and audit trails are not optional features — they are table stakes for any AI product that touches business decisions.

The test: Ask the vendor to show you a wrong output from their system. If they can show it and explain why it happened, they understand their product. If they insist the system "doesn't make mistakes," leave the meeting.

The 5-Question Cheat Sheet

Print this. Bring it to your next vendor meeting.

#	Question	What a Good Answer Sounds Like
1	What specific task does the AI do?	"It classifies support tickets into 12 categories with 94% accuracy, measured on our benchmark of 10,000 labeled tickets."
2	Can we run the demo on our data?	"Yes. Send us a sample and we'll run it in our sandbox. Here's the data format we need."
3	How many production customers use this?	"47 production customers. Average time from pilot to production: 6 weeks. Here are 3 you can call."
4	What is the total cost at 100x volume?	"Platform fee stays flat. Inference costs scale linearly — here's our cost calculator with 3 tiers."
5	What happens when the AI is wrong?	"We surface a confidence score on every output. Below 0.85, the system flags it for human review. Here's our audit trail."

If the vendor cannot give concrete answers to all 5 questions, the product is not ready for your team. It might be ready for a pilot. It is not ready for production.

The Deeper Problem

These red flags are symptoms of a market moving faster than its quality controls. AI vendors are under pressure to ship. PMs are under pressure to adopt. The result: decisions made on demo quality instead of production evidence.

The fix is not to avoid AI products. The fix is to evaluate them the same way you evaluate any production dependency: with your data, at your scale, with a clear understanding of failure modes and costs.

40% of agentic AI projects will be canceled by 2027. The teams that avoid that outcome are the ones asking these questions before they sign — not after.

Follow @klement_gunndu for more AI product content. We're building in public.

How to Read Any Codebase in 30 Minutes With AI Tools

klement Gunndu — Sat, 28 Mar 2026 08:04:26 +0000

Your manager says "get familiar with the codebase." You open the repo. 200K lines. No architecture docs. The README was last updated two years ago.

This is the first real challenge every new developer faces — and nobody teaches you how to handle it. Reading code is harder than writing it, and scrolling through files at random wastes hours without building understanding.

Here are 5 steps that turn AI into your codebase guide. Total time: 30 minutes for a medium-sized project.

Step 1: Map the File Tree (5 Minutes)

Before reading a single line of code, understand the shape of the project.

Run tree with depth limits to avoid drowning in files:

# Get the top 2 levels of the project structure
tree -L 2 -I 'node_modules|.git|__pycache__|.venv|dist'

# For larger projects, limit to directories only
tree -L 3 -d -I 'node_modules|.git|__pycache__|.venv'

This gives you output like:

.
├── src/
│   ├── api/
│   ├── models/
│   ├── services/
│   └── utils/
├── tests/
│   ├── unit/
│   └── integration/
├── docker-compose.yml
├── pyproject.toml
└── README.md

Copy the tree output and paste it into your AI coding assistant (ChatGPT, Claude, Copilot Chat — any works). Ask:

"Here is the file tree for a project I just joined. What does each top-level directory likely do? What architectural pattern does this suggest?"

The AI gives you a mental map in 60 seconds. You now know where the API routes live, where business logic sits, and where tests are.

Why this works: Your brain processes spatial layouts faster than text. A file tree is a spatial map of the codebase. The AI fills in the labels.

Step 2: Read the Config Files (5 Minutes)

Config files are the most honest documentation in any project. They list the actual dependencies, scripts, and settings — not what someone intended to write, but what the project actually uses.

Read these files in order:

# Python projects
cat pyproject.toml   # or requirements.txt, setup.py

# JavaScript projects
cat package.json

# Any project with containers
cat docker-compose.yml

# Environment variables tell you what external services exist
cat .env.example     # never .env — that has real secrets

For a Python project, pyproject.toml tells you everything:

[project]
dependencies = [
    "fastapi>=0.104.0",
    "sqlalchemy>=2.0",
    "pydantic>=2.5",
    "httpx>=0.25.0",
    "celery>=5.3.0",
]

[project.scripts]
serve = "app.main:run"
worker = "app.tasks:start_worker"

From these 10 lines, you know:

FastAPI — this is a web API, not a CLI tool
SQLAlchemy — there is a database with an ORM
Celery — there are background tasks
httpx — the app calls external APIs
Entry points — app/main.py has run(), app/tasks.py has start_worker()

Paste the config file into your AI assistant and ask:

"What does this project do based on its dependencies? What external services does it need?"

Five minutes in. You already know the tech stack, external dependencies, and entry points.

Step 3: Find and Trace the Entry Point (10 Minutes)

Every application has a front door. Find it, then follow the first hallway.

# Python — find the main entry
grep -rn "if __name__" --include="*.py" | head -5
grep -rn "app = FastAPI\|app = Flask\|app = Django" --include="*.py" | head -5

# JavaScript — find the main entry
grep -rn "createServer\|express()\|new Hono" --include="*.js" --include="*.ts" | head -5

# Or just check the config — package.json "main" or pyproject.toml [project.scripts]

Once you find the entry file, read it with your AI assistant. In Claude Code, you can open the project and ask directly. In other tools, paste the file content and ask:

"Walk me through what happens when this application starts. What gets initialized? What routes get registered?"

Now trace one path deeper. Pick the most important-looking route or function and follow it:

# Find where a function is defined
grep -rn "def process_order" --include="*.py"

# Find where it's called
grep -rn "process_order" --include="*.py"

You are tracing a single thread through the codebase. Not reading everything — reading one path from entry to exit. This builds a mental model of how the pieces connect.

The 10-minute rule: Set a timer. Trace one request from the API endpoint to the database and back. When the timer goes off, stop. You now understand one complete flow, and every other flow follows a similar pattern.

Step 4: Read One Test File (5 Minutes)

Tests are executable documentation. They show you what the code is supposed to do, what inputs it expects, and what outputs it produces.

# Find the test files
ls tests/ 2>/dev/null || ls test/ 2>/dev/null

# Pick the test file that matches the entry point you traced
# If you traced process_order, look for test_process_order.py
find . -name "test_*order*" -o -name "*order*_test*" | head -5

A well-written test tells you more than any documentation:

def test_process_order_calculates_total():
    order = Order(items=[
        Item(name="Widget", price=9.99, quantity=2),
        Item(name="Gadget", price=24.99, quantity=1),
    ])

    result = process_order(order)

    assert result.total == 44.97
    assert result.status == "confirmed"
    assert len(result.line_items) == 2

From this single test, you know:

Order takes a list of Item objects
Each Item has name, price, and quantity
process_order returns an object with total, status, and line_items
The function calculates totals and confirms orders

No documentation needed. The test IS the documentation.

If the project has no tests (it happens), check for API documentation, Swagger/OpenAPI specs, or example scripts in a docs/ or examples/ directory.

Step 5: Read the Git Log (5 Minutes)

The git history tells you what is actively changing — which matters more than what exists.

# See the last 20 commits with files changed
git log --oneline --stat -20

# See who works on what
git shortlog -sn --since="3 months ago"

# Find the most frequently changed files (these are the hot spots)
git log --pretty=format: --name-only --since="3 months ago" | sort | uniq -c | sort -rn | head -15

The last command is the most powerful. It shows you the files that change most often. These are the files you should understand first because:

They contain the most active business logic
They are where bugs are most likely to appear
They are where your first tasks will probably be

  47 src/services/order_service.py
  31 src/api/routes/orders.py
  28 src/models/order.py
  19 tests/test_order_service.py
  12 src/utils/pricing.py

This output tells you the order system is the hot zone. Your first PR will probably touch these files. Read them next.

Bonus — read recent PR descriptions:

# If using GitHub
gh pr list --state merged --limit 10

# Read a specific PR's description and comments
gh pr view 142

PR descriptions often contain more context than commit messages. They explain why changes were made, not just what changed.

What NOT to Do

Three mistakes new developers make when reading a codebase:

Do not read every file sequentially. A 200K-line codebase is not a book. Reading src/a.py through src/z.py builds no mental model. Trace flows instead.

Do not memorize implementation details. You do not need to know how the caching layer works on day one. You need to know it exists and where it lives. Details come when you work on a task that touches them.

Do not skip the config files. package.json and pyproject.toml tell you the truth about the project. README files tell you what someone hoped the project would become.

The 30-Minute Template

Copy this checklist for your first day on any codebase:

[  ] 0:00 - Run tree -L 2, paste into AI, get structure overview
[  ] 0:05 - Read pyproject.toml / package.json, identify stack
[  ] 0:10 - Find entry point (grep for main/app creation)
[  ] 0:12 - Trace one request from route to database
[  ] 0:20 - Read one test file matching the flow you traced
[  ] 0:25 - Run git log frequency analysis, find hot files
[  ] 0:30 - Write 5 bullet points: what the app does, how it works

That last step matters. Writing a summary forces your brain to organize what you learned. Keep it in a personal notes file. Update it as you learn more.

After the First 30 Minutes

The 30-minute method gives you a working mental model. Not a complete one — a working one. Enough to:

Ask informed questions in your first standup
Understand which files a bug report probably touches
Review a PR without feeling completely lost
Pick up your first task without starting from zero

Every week, trace one more flow through the codebase. Within a month, you will know the system better than developers who have been there for years but never mapped it systematically.

The codebase is not a mystery. It is a system with entry points, flows, and patterns. Map the structure, trace the flows, read the tests, check the history. AI accelerates each step, but the method works with or without it.

Follow @klement_gunndu for more beginner-friendly AI content. We're building in public.

Pick the Right Claude Code Model for Every Task

klement Gunndu — Fri, 27 Mar 2026 08:08:36 +0000

Claude Code supports three model tiers, seven aliases, four effort levels, and per-subagent model overrides. Most developers use the default for everything. That means they're paying Opus prices for tasks Haiku handles in half the time.

This guide covers 5 patterns for matching the right model to the right task in Claude Code, based on the official model configuration system as of March 2026.

The Three Tiers and When Each Wins

Claude Code gives you three model families, each with a different speed-cost-quality tradeoff:

Model	Best For	Speed	Cost
Haiku	File search, simple refactors, quick lookups	Fastest	Lowest
Sonnet	Daily coding, implementation, test writing	Balanced	Medium
Opus	Architecture decisions, complex debugging, multi-file refactoring	Slowest	Highest

The default model depends on your subscription. Max and Team Premium plans default to Opus 4.6. Pro and Team Standard default to Sonnet 4.6. Claude Code may automatically fall back to Sonnet if you hit a usage threshold with Opus.

Knowing this matters because many developers on Max plans run Opus for every task, including tasks where Haiku would finish 3x faster with identical results.

Pattern 1: Switch Models Mid-Session With /model

You don't need to restart Claude Code to change models. The /model command switches instantly:

# Planning a complex migration? Switch to Opus
/model opus

# Now implementing the plan? Switch to Sonnet
/model sonnet

# Quick file search or simple rename? Switch to Haiku
/model haiku

You can also start a session with a specific model:

# Start with Haiku for a quick task
claude --model haiku

# Start with Opus for a design session
claude --model opus

Or set it permanently in your settings file (~/.claude/settings.json):

{
  "model": "sonnet"
}

The priority order is: /model during session > --model flag > ANTHROPIC_MODEL environment variable > settings file.

This alone saves significant time. If you're in the middle of implementing a feature and need to look up how a module works, switching to Haiku for that exploration means you get answers faster and spend less context on a task that doesn't need deep reasoning.

Pattern 2: Use opusplan for Automatic Routing

The most underused model alias in Claude Code is opusplan. It uses Opus when you're in plan mode and automatically switches to Sonnet for execution:

claude --model opusplan

Here's why this matters. Planning requires the model to reason about architecture, weigh tradeoffs, and consider edge cases. That's where Opus excels. But once the plan is set, implementation is largely mechanical: write the code, follow the patterns, run the tests. Sonnet handles that efficiently.

With opusplan, you get Opus-quality planning and Sonnet-speed execution without manually switching. The transition happens automatically when you exit plan mode.

To enter plan mode during a session, use:

/plan

Claude Code switches to read-only exploration mode. If you're on opusplan, the model upgrades to Opus for this phase. When you approve the plan and exit plan mode, it drops back to Sonnet for implementation.

This is the closest thing to automatic model routing in Claude Code today, and it works well for the most common workflow: think first, then build.

Pattern 3: Control Effort Levels

Model selection isn't the only lever. Effort levels control how much thinking the model does per turn, independent of which model you're using.

Three levels persist across sessions: low, medium, and high. A fourth level, max, is available on Opus 4.6 only and does not persist.

# Quick formatting fix? Low effort
/effort low

# Standard coding work? Medium (default)
/effort medium

# Hard debugging or architecture? High effort
/effort high

# The hardest problem in the codebase? Max (Opus only)
/effort max

You can also set effort at startup:

claude --effort high

Or via environment variable:

export CLAUDE_CODE_EFFORT_LEVEL=high

Or in your settings file:

{
  "effortLevel": "medium"
}

The key insight: medium effort is the recommended default for most coding tasks. Higher effort levels can cause the model to overthink routine work, actually making it slower without improving quality.

Reserve high or max for tasks that genuinely benefit from deeper reasoning: tracing a complex bug across multiple files, designing a new system architecture, or debugging a race condition.

For one-off deep reasoning without changing your session setting, include "ultrathink" in your prompt. This triggers high effort for that single turn.

The combination of model tier and effort level gives you a 2D control surface. Haiku at low effort is near-instant for simple lookups. Opus at max effort is the deepest reasoning available, but takes longer and costs more. Matching both dimensions to the task is what separates efficient Claude Code usage from wasteful usage.

Pattern 4: Route Models to Subagents

This is where model routing gets powerful. Claude Code subagents can each run on a different model, set in their definition file.

Here's a practical example. You want a code review agent that runs on Sonnet (good enough for pattern matching and style checks) and a research agent that runs on Haiku (fast lookups, no need for deep reasoning):

Create .claude/agents/code-reviewer.md:

---
name: code-reviewer
description: Reviews code for quality and best practices. Use proactively after code changes.
tools: Read, Grep, Glob, Bash
model: sonnet
---

You are a senior code reviewer. When invoked:
1. Run git diff to see recent changes
2. Focus on modified files
3. Review for clarity, security, error handling, and test coverage

Provide feedback organized by priority:
- Critical issues (must fix)
- Warnings (should fix)
- Suggestions (consider improving)

Create .claude/agents/researcher.md:

---
name: researcher
description: Fast codebase exploration and documentation lookup.
tools: Read, Grep, Glob
model: haiku
---

You are a fast research assistant. Find files, search code,
and answer questions about the codebase structure.
Return concise summaries, not full file contents.

Now your main session can run on Opus for complex work, while delegating reviews to Sonnet and lookups to Haiku. Each subagent runs in its own context window, so the verbose output from exploration doesn't consume your main conversation's context.

Claude Code's built-in subagents already follow this pattern. The Explore agent runs on Haiku for fast, read-only codebase searches. The Plan agent inherits your session model for research during plan mode. The general-purpose agent also inherits your model for complex multi-step tasks.

You can also set a global default for all subagents via environment variable:

export CLAUDE_CODE_SUBAGENT_MODEL=haiku

This overrides the model for every subagent invocation, regardless of what's set in their definition files. Useful for cost control during development.

The model resolution order for subagents is:

CLAUDE_CODE_SUBAGENT_MODEL environment variable
Per-invocation model parameter (set by Claude when delegating)
Subagent's model frontmatter field
Main conversation's model

Pattern 5: Pin Models for Team Consistency

When multiple developers share a project, model inconsistency becomes a real problem. One developer tests on Opus, another on Haiku. Code that works with Opus-level reasoning might fail when a teammate runs it with Haiku.

Pin models with environment variables to ensure everyone uses the same configuration:

# In your team's .envrc or shell profile
export ANTHROPIC_DEFAULT_OPUS_MODEL="claude-opus-4-6"
export ANTHROPIC_DEFAULT_SONNET_MODEL="claude-sonnet-4-6"
export ANTHROPIC_DEFAULT_HAIKU_MODEL="claude-haiku-4-5-20251001"

For enterprise deployments on AWS Bedrock or Google Vertex AI, this is critical. Without pinning, Claude Code uses aliases that resolve to the latest version. When Anthropic releases a new model, users whose accounts don't have the new version enabled will break silently.

You can also restrict which models your team can select:

{
  "availableModels": ["sonnet", "haiku"]
}

This prevents developers from switching to models outside the approved list via /model, --model, or environment variables. The default model for their subscription tier always remains available.

For full control, combine both settings:

{
  "model": "sonnet",
  "availableModels": ["sonnet", "haiku"]
}

This ensures everyone starts on Sonnet and can only switch between Sonnet and Haiku. No surprise Opus bills. No inconsistent behavior across the team.

The Decision Framework

Here's how to decide which model and effort to use for common tasks:

Task	Model	Effort	Why
Find a file or function	Haiku	Low	Pure lookup, no reasoning needed
Rename a variable across files	Haiku	Low	Mechanical, well-defined scope
Write a new function	Sonnet	Medium	Needs context awareness, not deep reasoning
Write tests for existing code	Sonnet	Medium	Pattern matching against existing code
Debug a failing test	Sonnet	High	Needs reasoning about causality
Design a new module architecture	Opus	High	Multi-factor tradeoff analysis
Refactor across 10+ files	Opus	High	Needs to hold full dependency graph in context
Plan then implement a feature	opusplan	Auto	Opus for the plan, Sonnet for the code

The 1M context window option (opus[1m] or sonnet[1m]) is available for long sessions with large codebases. On Max, Team, and Enterprise plans, Opus automatically gets 1M context. For Sonnet or other plans, it requires extra usage billing.

# Explicitly request 1M context
/model opus[1m]
/model sonnet[1m]

Start Here

If you're currently using the default model for everything, start with one change: switch to opusplan. It gives you the best of both tiers with zero manual switching.

Then add effort levels. Most of your work is /effort medium. When you hit a hard problem, /effort high gives you deeper reasoning without changing models.

Finally, create subagent definitions for your most common delegated tasks. A Haiku-powered explorer and a Sonnet-powered reviewer cover 80% of subagent use cases.

The goal isn't to use the cheapest model everywhere. It's to use the right model for each task, so you get faster answers on simple work and deeper reasoning on hard problems.

Follow @klement_gunndu for more Claude Code content. We're building in public.

Ask Your CSV Anything: Build a Data Analysis Agent in Python

klement Gunndu — Thu, 26 Mar 2026 12:37:45 +0000

Every data scientist has the same ritual. Load a CSV. Write df.describe(). Squint at column names. Write 14 lines of pandas to answer one question. Repeat.

What if an LLM could write those 14 lines for you — and run them?

That is exactly what data analysis agents do. You ask a question in English. The agent writes pandas code, executes it in a sandboxed Python environment, reads the output, and returns a plain-language answer. No manual wrangling. No copy-pasting between notebook cells.

Here are 4 patterns that make this work in production — from a 5-line quick start to a fully custom analysis pipeline.

Pattern 1: The Pandas DataFrame Agent

LangChain's create_pandas_dataframe_agent is the fastest path from CSV to answers. It wraps your DataFrame in a tool-calling agent that writes and executes pandas code through a sandboxed Python REPL.

Install the dependencies:

pip install langchain-experimental langchain-openai pandas

Load your data and create the agent:

import pandas as pd
from langchain_openai import ChatOpenAI
from langchain_experimental.agents.agent_toolkits import (
    create_pandas_dataframe_agent,
)

df = pd.read_csv("sales_data.csv")

llm = ChatOpenAI(model="gpt-4o", temperature=0)

agent = create_pandas_dataframe_agent(
    llm,
    df,
    agent_type="tool-calling",
    verbose=True,
    allow_dangerous_code=True,
)

The allow_dangerous_code=True flag is mandatory. The agent executes real Python code against your DataFrame, so it must be opted into explicitly. Run this only in a sandboxed environment — never on a production server with sensitive data.

Now ask questions:

result = agent.invoke("What are the top 5 products by total revenue?")
print(result["output"])

The agent inspects the DataFrame's columns and dtypes, writes a pandas expression like df.groupby('product')['revenue'].sum().nlargest(5), executes it, and returns the result as natural language.

Multi-DataFrame support. Pass a list of DataFrames to compare datasets:

df_2024 = pd.read_csv("sales_2024.csv")
df_2025 = pd.read_csv("sales_2025.csv")

agent = create_pandas_dataframe_agent(
    llm,
    [df_2024, df_2025],
    agent_type="tool-calling",
    verbose=True,
    allow_dangerous_code=True,
)

result = agent.invoke(
    "Which products grew more than 20% between 2024 and 2025?"
)

When to use this pattern: Small to medium datasets (under 1 million rows) that fit in memory. Quick exploratory analysis. Prototyping.

Limitation: The agent sends the first few rows and column names to the LLM as context. For wide DataFrames with 100+ columns, this context can overwhelm the model's attention. For large datasets, use Pattern 2.

Pattern 2: The SQL Agent for Larger Datasets

When your data outgrows a DataFrame — millions of rows, multiple related tables, or anything that benefits from indexing — convert it to SQLite and let a SQL agent handle the queries.

pip install langchain-community langchain-openai sqlalchemy

Load your CSV into SQLite and create the agent:

import pandas as pd
import sqlalchemy
from langchain_community.utilities import SQLDatabase
from langchain_community.agent_toolkits import create_sql_agent
from langchain_openai import ChatOpenAI

engine = sqlalchemy.create_engine("sqlite:///analysis.db")

df_orders = pd.read_csv("orders.csv")
df_customers = pd.read_csv("customers.csv")
df_orders.to_sql("orders", engine, if_exists="replace", index=False)
df_customers.to_sql("customers", engine, if_exists="replace", index=False)

db = SQLDatabase(engine)
print(f"Tables: {db.get_usable_table_names()}")

llm = ChatOpenAI(model="gpt-4o", temperature=0)

agent = create_sql_agent(
    llm,
    db=db,
    agent_type="tool-calling",
    verbose=True,
)

Now ask questions that span multiple tables:

result = agent.invoke(
    "What is the average order value per customer segment?"
)
print(result["output"])

The agent inspects the database schema, writes a SQL query with the appropriate JOINs, executes it, and translates the result into English.

Why SQL beats pandas at scale:

Indexing. SQLite creates indexes that make filtering and joining fast. Pandas scans the full DataFrame.
Memory. SQLite reads from disk. Pandas loads everything into RAM.
Joins. SQL JOINs are declarative and optimized. Pandas merges require explicit column mapping and can be memory-intensive.
Audit trail. Every query the agent writes is a valid SQL statement you can review and rerun.

Safety note: The SQL agent generates and executes queries against your database. Use a read-only connection or a copy of your data for analysis. Never point it at a production database with write access.

engine = sqlalchemy.create_engine(
    "sqlite:///analysis.db",
    connect_args={"check_same_thread": False},
)

# For read-only protection, use a copy of the database
# or set PRAGMA query_only = ON
with engine.connect() as conn:
    conn.execute(sqlalchemy.text("PRAGMA query_only = ON"))

Pattern 3: Custom Analysis Tools With Tool-Calling

The pre-built agents handle common cases. But real data analysis needs domain-specific operations — statistical tests, time series decomposition, custom visualizations. Build a custom tool-calling agent that exposes exactly the operations your analysts need.

pip install langchain-openai langgraph scipy

Define your analysis tools as Python functions with clear docstrings — the LLM reads these to decide which tool to call:

import pandas as pd
import json
from scipy import stats
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool

df = pd.read_csv("experiment_results.csv")


@tool
def describe_dataset(column: str) -> str:
    """Get summary statistics for a specific column in the dataset.
    Returns count, mean, std, min, 25%, 50%, 75%, max."""
    if column not in df.columns:
        return f"Column '{column}' not found. Available: {list(df.columns)}"
    return df[column].describe().to_json()


@tool
def compare_groups(
    group_column: str, value_column: str, group_a: str, group_b: str
) -> str:
    """Run a two-sample t-test comparing two groups on a numeric column.
    Returns t-statistic, p-value, and whether the difference is
    statistically significant at alpha=0.05."""
    a = df[df[group_column] == group_a][value_column].dropna()
    b = df[df[group_column] == group_b][value_column].dropna()
    t_stat, p_value = stats.ttest_ind(a, b)
    significant = p_value < 0.05
    return json.dumps({
        "group_a_mean": round(a.mean(), 4),
        "group_b_mean": round(b.mean(), 4),
        "t_statistic": round(t_stat, 4),
        "p_value": round(p_value, 6),
        "significant_at_0.05": significant,
        "n_a": len(a),
        "n_b": len(b),
    })


@tool
def correlation_matrix(columns: list[str]) -> str:
    """Calculate pairwise Pearson correlations between specified columns.
    Returns a correlation matrix as JSON."""
    valid = [c for c in columns if c in df.columns]
    if not valid:
        return f"No valid columns found. Available: {list(df.columns)}"
    corr = df[valid].corr()
    return corr.to_json()


@tool
def filter_and_aggregate(
    filter_column: str,
    filter_value: str,
    agg_column: str,
    agg_function: str,
) -> str:
    """Filter the dataset by a column value, then aggregate another column.
    Supported agg_function values: mean, sum, count, min, max, median."""
    valid_aggs = {"mean", "sum", "count", "min", "max", "median"}
    if agg_function not in valid_aggs:
        return f"Invalid agg_function. Use one of: {valid_aggs}"
    filtered = df[df[filter_column] == filter_value]
    result = getattr(filtered[agg_column], agg_function)()
    return json.dumps({
        "filter": f"{filter_column} == {filter_value}",
        "rows_matched": len(filtered),
        "aggregation": f"{agg_function}({agg_column})",
        "result": round(float(result), 4),
    })

Wire the tools into a tool-calling agent:

from langchain_core.messages import SystemMessage

tools = [describe_dataset, compare_groups, correlation_matrix, filter_and_aggregate]

llm = ChatOpenAI(model="gpt-4o", temperature=0)
llm_with_tools = llm.bind_tools(tools)

system_prompt = """You are a data analysis assistant. You have access to a dataset
with these columns: {columns}

When the user asks a question:
1. Identify which tools can answer it
2. Call the appropriate tool(s)
3. Interpret the results in plain language
4. Note statistical significance, sample sizes, and caveats

Always mention sample sizes when reporting statistical results.""".format(
    columns=list(df.columns)
)

from langgraph.prebuilt import create_react_agent

agent = create_react_agent(llm, tools)

Ask domain-specific questions:

result = agent.invoke({
    "messages": [
        {"role": "system", "content": system_prompt},
        {
            "role": "user",
            "content": "Is there a significant difference in conversion rate between the control and treatment groups?",
        },
    ]
})

print(result["messages"][-1].content)

Why custom tools beat the generic pandas agent:

Controlled operations. The agent can only call functions you defined. No arbitrary code execution.
Domain language. Tool descriptions use your team's terminology. The LLM maps natural language to the right operation.
Safety. No Python REPL. No allow_dangerous_code. Each tool validates its inputs and returns structured output.
Reproducibility. Every tool call is logged with exact parameters. You can replay any analysis.

Pattern 4: Output Validation and Guardrails

An analysis agent that returns wrong numbers is worse than no agent at all. These guardrails catch errors before they reach your stakeholders.

Guardrail 1: Schema validation on tool inputs.

Use Pydantic to enforce that tool arguments match expected types and ranges:

from pydantic import BaseModel, field_validator


class AggregationRequest(BaseModel):
    filter_column: str
    filter_value: str
    agg_column: str
    agg_function: str

    @field_validator("agg_function")
    @classmethod
    def validate_agg(cls, v: str) -> str:
        allowed = {"mean", "sum", "count", "min", "max", "median"}
        if v not in allowed:
            raise ValueError(f"Must be one of {allowed}, got '{v}'")
        return v

    @field_validator("filter_column", "agg_column")
    @classmethod
    def validate_columns(cls, v: str) -> str:
        if v not in df.columns:
            raise ValueError(
                f"Column '{v}' not in dataset. "
                f"Available: {list(df.columns)}"
            )
        return v

Guardrail 2: Result sanity checks.

Wrap your agent's output with automatic validation:

def validate_numeric_result(result: dict, column: str) -> dict:
    """Check that a computed result falls within the column's actual range."""
    col_min = float(df[column].min())
    col_max = float(df[column].max())
    value = result.get("result")

    if value is not None and not (col_min <= value <= col_max):
        result["warning"] = (
            f"Result {value} is outside the column range "
            f"[{col_min}, {col_max}]. Verify the calculation."
        )
    return result

Guardrail 3: Row count assertions.

Catch empty results before they produce misleading statistics:

def safe_aggregate(
    filtered_df: pd.DataFrame,
    column: str,
    operation: str,
    min_rows: int = 5,
) -> dict:
    """Aggregate with a minimum sample size requirement."""
    n = len(filtered_df)
    if n == 0:
        return {"error": "Filter returned zero rows. Check filter criteria."}
    if n < min_rows:
        return {
            "warning": f"Only {n} rows matched (minimum {min_rows}). "
            "Results may not be statistically meaningful.",
            "result": getattr(filtered_df[column], operation)(),
            "n": n,
        }
    return {
        "result": getattr(filtered_df[column], operation)(),
        "n": n,
    }

Guardrail 4: Query logging for audit.

Log every tool call so analysts can verify what the agent actually computed:

import logging
from datetime import datetime, timezone

logger = logging.getLogger("analysis_agent")
handler = logging.FileHandler("agent_audit.log")
handler.setFormatter(
    logging.Formatter("%(asctime)s | %(message)s")
)
logger.addHandler(handler)
logger.setLevel(logging.INFO)


def log_tool_call(tool_name: str, params: dict, result: dict):
    logger.info(
        f"TOOL={tool_name} | PARAMS={params} | "
        f"RESULT_KEYS={list(result.keys())} | "
        f"TIMESTAMP={datetime.now(timezone.utc).isoformat()}"
    )

These guardrails are not optional extras. In any workflow where analysis drives decisions — A/B test results, revenue forecasts, user segmentation — a wrong number costs more than a slow answer.

Which Pattern Fits Your Data

Scenario	Pattern	Why
Quick CSV exploration	1 (Pandas Agent)	Five lines to first answer
Multiple related tables, millions of rows	2 (SQL Agent)	Indexing, joins, disk-based
Domain-specific analysis (stats, experiments)	3 (Custom Tools)	Controlled operations, no REPL
Production pipeline with stakeholders	3 + 4 (Custom + Guardrails)	Validated, audited, reproducible

Start with Pattern 1 to prototype. Move to Pattern 2 when data gets large. Move to Pattern 3 when you need control. Add Pattern 4 when wrong answers have consequences.

What to Watch Out For

Code execution risks. Patterns 1 and 2 execute generated code. Run them in Docker containers or sandboxed environments — never on the same machine as production data without isolation.

LLM hallucination in analysis. The agent can write syntactically valid pandas code that answers the wrong question. Always log the generated code (verbose=True) and spot-check against manual queries.

Token costs scale with data context. The pandas agent sends column names, dtypes, and sample rows to the LLM on every call. Wide DataFrames with 100+ columns burn tokens fast. Use Pattern 3's targeted tools to control exactly what context the LLM sees.

Statistical literacy. The LLM knows how to call scipy.stats.ttest_ind. It does not always know when a t-test is the wrong test for your data distribution. Domain expertise still matters — the agent accelerates the workflow, it does not replace the analyst.

Every analyst spends more time wrangling data than analyzing it. These 4 patterns shift that ratio. The LLM handles the pandas syntax, the SQL joins, and the boilerplate. You handle the questions that matter.

Follow @klement_gunndu for more AI engineering content. We're building in public.

Your MCP Server Has No Tests. Here Are 4 Patterns to Fix That.

klement Gunndu — Thu, 26 Mar 2026 08:06:13 +0000

Most MCP servers ship with exactly one test: a developer typing a prompt into Claude and checking if the output looks right.

That is not testing. That is hoping. And it breaks the moment you change a tool signature, add a parameter, or update your database schema.

Why MCP Servers Are Hard to Test

MCP servers sit between deterministic code and non-deterministic LLMs. Your tools are pure functions — they take inputs, return outputs. But the consumers of those tools are language models that interpret schemas, pick tools based on descriptions, and pass arguments based on inference.

This creates a testing gap. Traditional unit tests cover your business logic. LLM integration tests are slow, expensive, and non-deterministic. The four patterns below close that gap using real MCP protocol interactions — without calling an LLM.

Pattern 1: In-Memory Unit Tests With FastMCP Client

FastMCP 2.x includes a Client class that connects directly to your server in-memory. No subprocess. No network. No LLM. You test your actual server logic through the real MCP protocol — in milliseconds.

Install the testing dependency:

pip install pytest-asyncio

Configure pytest to handle async automatically in pyproject.toml:

[tool.pytest.ini_options]
asyncio_mode = "auto"

Here is a server with two tools:

# server.py
from fastmcp import FastMCP

mcp = FastMCP("WeatherServer")

@mcp.tool()
def get_forecast(city: str, days: int = 3) -> dict:
    """Get weather forecast for a city."""
    if days < 1 or days > 14:
        raise ValueError(f"Days must be 1-14, got {days}")
    return {
        "city": city,
        "days": days,
        "forecast": [{"day": i + 1, "temp_c": 20 + i} for i in range(days)],
    }

@mcp.tool()
def get_alerts(region: str) -> list[str]:
    """Get active weather alerts for a region."""
    alerts_db = {"northwest": ["Wind advisory until 6 PM"]}
    return alerts_db.get(region.lower(), [])

Now write the tests. Create a pytest fixture that wraps your server in a Client:

# test_server.py
import pytest
from fastmcp import Client
from server import mcp  # your FastMCP server instance

@pytest.fixture
async def client():
    async with Client(transport=mcp) as c:
        yield c

async def test_forecast_returns_correct_days(client):
    result = await client.call_tool("get_forecast", {"city": "Seattle", "days": 5})
    data = result.data
    assert data["city"] == "Seattle"
    assert len(data["forecast"]) == 5

async def test_forecast_default_days(client):
    result = await client.call_tool("get_forecast", {"city": "Portland"})
    assert len(result.data["forecast"]) == 3

async def test_forecast_invalid_days_raises(client):
    with pytest.raises(Exception):
        await client.call_tool("get_forecast", {"city": "Seattle", "days": 30})

async def test_alerts_existing_region(client):
    result = await client.call_tool("get_alerts", {"region": "Northwest"})
    assert len(result.data) == 1
    assert "Wind advisory" in result.data[0]

async def test_alerts_unknown_region(client):
    result = await client.call_tool("get_alerts", {"region": "Antarctica"})
    assert result.data == []

Run with pytest -v. Every test executes in milliseconds because it uses in-memory transport. No HTTP, no subprocess, no LLM calls.

This pattern catches three categories of bugs immediately: broken tool logic, wrong return types, and missing error handling.

Pattern 2: Schema Validation Tests

Your MCP tools expose JSON schemas to LLMs. If the schema drifts from your implementation — a renamed parameter, a changed type, a missing description — the LLM picks the wrong tool or passes wrong arguments. Schema tests lock this contract down.

async def test_tool_registry_completeness(client):
    """Verify all expected tools are registered."""
    tools = await client.list_tools()
    tool_names = {t.name for t in tools}
    assert tool_names == {"get_forecast", "get_alerts"}

async def test_forecast_schema_has_required_params(client):
    """Verify the forecast tool schema matches expectations."""
    tools = await client.list_tools()
    forecast = next(t for t in tools if t.name == "get_forecast")

    schema = forecast.inputSchema
    assert "city" in schema["properties"]
    assert schema["properties"]["city"]["type"] == "string"
    assert "city" in schema.get("required", [])

async def test_all_tools_have_descriptions(client):
    """LLMs select tools based on descriptions. Missing = broken routing."""
    tools = await client.list_tools()
    for tool in tools:
        assert tool.description, f"Tool '{tool.name}' has no description"
        assert len(tool.description) > 10, (
            f"Tool '{tool.name}' description too short: '{tool.description}'"
        )

Schema tests catch a specific class of failure that unit tests miss entirely: your code works, but the LLM cannot use it. This happens more often than you think. A developer renames a parameter from city to location. The tool still works. The schema updates automatically. But every prompt template, every LLM workflow, and every agent that was trained on the old schema now sends city and gets a validation error.

Schema tests make this failure loud and immediate. When the parameter name changes, test_forecast_schema_has_required_params fails in CI. The developer sees the break before it ships.

If your server exposes resources alongside tools, test those registrations too:

async def test_resources_are_registered(client):
    """Verify static resources are accessible."""
    resources = await client.list_resources()
    assert len(resources) > 0, "Server exposes no resources"

Run schema tests in CI on every commit. Schema drift is silent and devastating — these tests make it visible.

Pattern 3: Parameterized Edge Case Testing

MCP tools receive arguments from LLMs. LLMs send unexpected inputs — empty strings, extreme values, wrong types. Parameterized tests cover these systematically.

@pytest.mark.parametrize(
    "city, days, expected_count",
    [
        ("Seattle", 1, 1),
        ("Tokyo", 7, 7),
        ("São Paulo", 14, 14),
        ("New York", 3, 3),
    ],
)
async def test_forecast_valid_ranges(client, city, days, expected_count):
    result = await client.call_tool("get_forecast", {"city": city, "days": days})
    assert len(result.data["forecast"]) == expected_count

@pytest.mark.parametrize(
    "invalid_days",
    [0, -1, 15, 100],
)
async def test_forecast_rejects_invalid_days(client, invalid_days):
    with pytest.raises(Exception):
        await client.call_tool(
            "get_forecast", {"city": "Seattle", "days": invalid_days}
        )

@pytest.mark.parametrize(
    "region, has_alerts",
    [
        ("Northwest", True),
        ("northwest", True),
        ("NORTHWEST", True),
        ("southeast", False),
        ("", False),
    ],
)
async def test_alerts_case_insensitive(client, region, has_alerts):
    result = await client.call_tool("get_alerts", {"region": region})
    if has_alerts:
        assert len(result.data) > 0
    else:
        assert len(result.data) == 0

For complex return structures, the inline-snapshot library eliminates manual assertion writing. Install it with pip install inline-snapshot, then write:

from inline_snapshot import snapshot

async def test_forecast_structure(client):
    result = await client.call_tool("get_forecast", {"city": "Seattle", "days": 2})
    assert result.data == snapshot()

Run pytest --inline-snapshot=fix,create once. The library fills in the expected value automatically from the actual output. On subsequent runs, it asserts against the stored snapshot. When your output changes intentionally, run --inline-snapshot=fix to update.

Pattern 4: Interactive Testing With MCP Inspector

Unit tests verify code paths. But sometimes you need to see what the LLM sees — the exact schemas, the raw responses, the protocol messages. MCP Inspector is the official visual testing tool for this.

Launch it against your server:

# For a Python MCP server
npx @modelcontextprotocol/inspector uv --directory ./my-server run my-server

# For a published PyPI package
npx @modelcontextprotocol/inspector uvx mcp-server-git --repository ~/code/repo.git

This opens a web UI at http://localhost:6274 that connects to your MCP server through a local proxy. From the Inspector, you can:

Browse all tools — see the JSON schema exactly as an LLM receives it
Call any tool — fill in parameters through a form, see the raw response
Inspect resources — view static context your server exposes
Test prompts — verify prompt templates render correctly

MCP Inspector serves a different purpose than pytest. Use it for:

Exploratory testing during development — try edge cases manually
Schema review — verify descriptions are clear enough for LLM tool selection
Debugging failures — reproduce exact inputs that caused production issues
Demo and documentation — show stakeholders what your server exposes

The Inspector does not replace automated tests. It complements them. Write pytest tests for regression coverage. Use Inspector for exploration and debugging.

A Practical Test Strategy

Combine all four patterns into a layered strategy:

Layer 1: In-memory unit tests (Pattern 1)
  → Run on every save. Sub-second feedback. Catch logic bugs.

Layer 2: Schema validation tests (Pattern 2)
  → Run in CI on every commit. Catch contract drift.

Layer 3: Parameterized edge cases (Pattern 3)
  → Run in CI. Catch boundary failures and type handling.

Layer 4: MCP Inspector (Pattern 4)
  → Use during development. Manual exploration and debugging.

Your pyproject.toml test configuration:

[tool.pytest.ini_options]
asyncio_mode = "auto"
markers = [
    "schema: schema validation tests",
    "edge: edge case and boundary tests",
]

Run fast tests during development:

pytest -v -m "not schema and not edge"

Run everything in CI:

pytest -v --tb=short

What This Costs You

Setting up in-memory MCP tests takes about 30 minutes for an existing server. The fixture is 5 lines. Each test is 3-6 lines. You get sub-second feedback on every change.

Compare that to the alternative: a user discovers your tool returns the wrong type, the LLM hallucinates a workaround, and you spend two hours debugging a production trace.

Thirty minutes of test setup prevents hours of production debugging. That trade is worth it every time.

The MCP ecosystem is growing fast. As of March 2026, thousands of MCP servers exist on npm and PyPI, and the number is accelerating. Most of them have zero automated tests. If you ship yours with a proper test suite, you are already ahead of 90% of the ecosystem. More importantly, your users will trust your server because it actually works when they upgrade.

Follow @klement_gunndu for more MCP and AI engineering content. We're building in public.

Vibe Coding Got You Started. These 5 Skills Keep You Employed.

klement Gunndu — Wed, 25 Mar 2026 12:34:19 +0000

Vibe coding got 92% of US developers using AI tools daily. It also got 40% of junior developers deploying code they don't fully understand.

Both numbers come from 2026 developer surveys. Both are true at the same time. And if you're switching careers into tech right now, that gap is the most important thing you need to understand.

Vibe coding — describing what you want in plain English and letting AI write the code — is a legitimate entry point. It lowers the barrier. It gets you building. Nobody should gatekeep that.

But every hiring manager I've talked to asks the same follow-up question: "What happens when the AI is wrong?"

If you can't answer that with specifics, you're competing against every other candidate who can also type a prompt. These five skills separate developers who get hired from developers who stay hired.

1. Read Code You Didn't Write

The most underrated skill in 2026 is reading. Not writing. Reading.

When AI generates 200 lines of code for your feature, you become a code reviewer — whether you wanted to or not. GitClear analyzed 211 million lines of code and found that AI-assisted development led to 4x more code cloning and a drop in refactoring from 25% of changed lines in 2021 to under 10% by 2024. That means more duplicated, unreviewed code shipping to production every day.

Here's a practical exercise. Take this AI-generated Python function:

def get_user_data(user_id):
    import requests
    response = requests.get(f"https://api.example.com/users/{user_id}")
    data = response.json()
    return {
        "name": data["name"],
        "email": data["email"],
        "created": data["created_at"]
    }

Before you use it, answer five questions:

What happens if the API returns a 404?
What happens if data doesn't have a "name" key?
Is import inside the function a good idea?
What if the network is down?
Is the URL hardcoded? Should it be?

If you can answer all five, you're reading code. If you can't, the AI wrote code you don't own yet.

How to practice: Pick any open-source Python project on GitHub. Read one file per day. Write a comment explaining what each function does. Start with small utilities — not frameworks.

2. Debug Without AI

AI tools are excellent at generating code. They are unreliable at diagnosing why code fails. When you paste an error message back into the chat, you get a plausible-sounding fix. Sometimes it works. Sometimes it introduces a new bug that surfaces three deployments later.

Debugging is the skill that separates someone who can build from someone who can maintain.

Start with tracebacks. Every Python error tells you exactly where it broke:

Traceback (most recent call last):
  File "app.py", line 42, in process_order
    total = calculate_total(items)
  File "app.py", line 18, in calculate_total
    price = item["price"] * item["quantity"]
KeyError: 'quantity'

This traceback says: line 18, inside calculate_total, tried to access item["quantity"] but the key doesn't exist. The fix isn't to add a try/except everywhere. The fix is to find where items is constructed and figure out why "quantity" is missing.

Three debugging techniques that don't require AI:

# 1. Print the actual data shape
print(type(items), len(items))
print(items[0].keys())  # What keys does each item actually have?

# 2. Use breakpoint() to pause execution
def calculate_total(items):
    breakpoint()  # Drops you into pdb — inspect items interactively
    return sum(item["price"] * item["quantity"] for item in items)

# 3. Check assumptions with assert
assert all("quantity" in item for item in items), f"Missing quantity in: {[i for i in items if 'quantity' not in i]}"

The rule: Before asking AI to fix an error, read the traceback yourself first. Identify the file, line number, and the variable that failed. Then decide if you need AI or if the fix is obvious.

3. Use Git Like a Professional

Every development team uses Git. No exceptions. If you can vibe-code a feature but can't create a branch, you'll fail your first code review.

You don't need to memorize every Git command. You need these six:

# Start work on a new feature
git checkout -b feature/add-search

# Check what you changed
git status
git diff

# Save your work
git add search.py test_search.py
git commit -m "Add search endpoint with input validation"

# Push to remote for review
git push -u origin feature/add-search

The habits that matter more than commands:

Commit messages describe why, not what. Not "updated file" — "Add input validation to prevent empty search queries." Your commit message is a note to Future You, and Future You has no context.

Never commit to main directly. Always branch. Always open a pull request. Even on personal projects. Build the muscle memory now so it's automatic when you join a team.

Small commits beat large commits. One commit that changes 50 files is impossible to review. Five commits that each change 10 files tell a story.

# Good: each commit is one logical change
git commit -m "Add User model with email validation"
git commit -m "Add /users endpoint with POST handler"
git commit -m "Add tests for user creation and validation"

# Bad: one commit with everything
git commit -m "Added user feature"

4. Write Tests for AI-Generated Code

AI co-authored code contains 1.7x more major issues than human-written code, according to CodeRabbit's analysis of 470 open-source GitHub pull requests. That number isn't going down. If anything, as more code is AI-generated, the testing gap widens.

You don't need to become a testing expert. You need one pattern: write a test that proves the code does what you think it does.

# The AI wrote this function
def calculate_discount(price, tier):
    if tier == "gold":
        return price * 0.8
    elif tier == "silver":
        return price * 0.9
    return price

# You write these tests
def test_gold_gets_20_percent_off():
    assert calculate_discount(100, "gold") == 80

def test_silver_gets_10_percent_off():
    assert calculate_discount(100, "silver") == 90

def test_unknown_tier_gets_no_discount():
    assert calculate_discount(100, "bronze") == 100

def test_zero_price_returns_zero():
    assert calculate_discount(0, "gold") == 0

def test_negative_price_is_handled():
    # Does the function handle this? Should it?
    result = calculate_discount(-50, "gold")
    # If this surprises you, the function needs a guard

The last test is the important one. It forces you to ask: "What should happen with invalid input?" AI rarely generates edge case tests because edge cases require understanding the business logic — not just the syntax.

Run tests with pytest:

pip install pytest
pytest test_discount.py -v

test_discount.py::test_gold_gets_20_percent_off PASSED
test_discount.py::test_silver_gets_10_percent_off PASSED
test_discount.py::test_unknown_tier_gets_no_discount PASSED
test_discount.py::test_zero_price_returns_zero PASSED
test_discount.py::test_negative_price_is_handled PASSED

Five tests. Two minutes to write. They'll catch every regression when the function changes later.

The habit: Every time AI generates a function, write at least three tests: one happy path, one edge case, one invalid input.

5. Ask Better Questions

The difference between a junior and senior developer using AI isn't the tool. It's the prompt.

A junior prompt:

Write a function that gets user data from an API

A senior prompt:

Write a Python function that fetches user data from a REST API.
Requirements:
- Accept user_id as parameter
- Handle HTTP errors (4xx, 5xx) with specific exceptions
- Timeout after 5 seconds
- Return a typed dict with name, email, and created_at fields
- Log failed requests with the status code

The second prompt produces code you can actually ship. It includes error handling, timeouts, typing, and logging — the things that break in production.

Three patterns for better prompts:

1. Specify the failure modes: "What should happen when the API is down?" forces the AI to generate error handling instead of just the happy path.

2. Ask for the tests alongside the code: "Write the function and the pytest tests for it" gives you verification built in.

3. Ask why, not just what: "Explain why you chose requests.Session() over requests.get()" teaches you the trade-offs. If the AI can't explain its choice clearly, it probably made an arbitrary one.

This isn't prompt engineering in the abstract. This is learning to specify requirements — the same skill that separates junior from senior developers regardless of AI.

The Floor, Not the Ceiling

Vibe coding lowered the entry barrier to software development. That's good. More people building means more problems getting solved.

But the barrier to staying in software development hasn't moved. Teams still need people who can read a codebase they didn't write. Debug a failure at 2 AM without an AI assistant. Push clean commits that tell a story. Write tests that catch the bugs AI introduces. Specify requirements precisely enough that the generated code actually works in production.

These five skills aren't advanced. They're foundational. You can learn each one in a week of deliberate practice:

Week 1: Read one open-source file per day. Write comments explaining each function.
Week 2: Debug three errors using only tracebacks and print(). No AI.
Week 3: Use Git branches for every change in a personal project. Write descriptive commit messages.
Week 4: Write three tests for every AI-generated function before using it.
Week 5: Rewrite your AI prompts with explicit requirements, error handling specs, and typing.

Vibe coding is the door. These skills are the floor.

Follow @klement_gunndu for more AI engineering content. We're building in public.

Your API Wasn't Designed for AI Agents. Here Are 5 Fixes.

klement Gunndu — Wed, 25 Mar 2026 08:05:23 +0000

AI agents are your API's newest power users. They don't read your docs the way humans do. They parse your OpenAPI spec, call endpoints in loops, retry on every failure, and chain 20 requests without pausing to ask "are you sure?"

Most APIs were designed for frontend developers clicking buttons. That design breaks when an autonomous agent starts calling your endpoints at 3 AM with no human in the loop.

Here are 5 architecture patterns that make your API survive the agent era.

1. Machine-Readable Descriptions in Your OpenAPI Spec

Human developers read your API docs and infer intent. AI agents read your OpenAPI specification and take descriptions literally. Vague descriptions produce wrong API calls.

The difference between an agent choosing the right endpoint and the wrong one is often a single sentence in your spec.

from fastapi import FastAPI
from pydantic import BaseModel, Field

app = FastAPI(
    title="Order Management API",
    description="Manages customer orders. All monetary values are in USD cents.",
)


class OrderCreate(BaseModel):
    """Create a new order. Does NOT charge the customer.
    Use POST /orders/{order_id}/confirm to finalize and charge."""

    customer_id: str = Field(
        description="Unique customer identifier. Format: cust_xxxxxxxxxxxx"
    )
    items: list[str] = Field(
        description="List of SKU strings. Each SKU must exist in the product catalog."
    )
    amount_cents: int = Field(
        description="Total order amount in USD cents. Must match sum of item prices.",
        ge=1,
    )


@app.post(
    "/orders",
    summary="Create a draft order (does not charge customer)",
    response_description="Returns the created order with status 'draft'",
)
async def create_order(order: OrderCreate):
    """Create a draft order. The order is NOT finalized until
    POST /orders/{order_id}/confirm is called. Draft orders
    expire after 30 minutes."""
    return {"order_id": "ord_123", "status": "draft"}

Three things agents need that human developers infer:

Explicit side effects. "Does NOT charge the customer" prevents an agent from assuming a POST creates and charges in one step.
Format specifications in Field descriptions. Format: cust_xxxxxxxxxxxx eliminates guessing about ID formats.
Units in the schema. "USD cents" prevents an agent from sending 19.99 when you expect 1999.

Expose your spec at a standard endpoint like /openapi.json. Agents discover APIs through specs, not landing pages.

2. Idempotency Keys for Safe Retries

Agents retry. Every agent framework includes retry logic. LangChain retries on exceptions. OpenAI's function calling retries on malformed responses. Your own agent loop retries on timeouts.

Without idempotency keys, each retry creates a duplicate. One "create order" intent becomes three orders.

from fastapi import FastAPI, Header, HTTPException, Response
from typing import Optional
import hashlib
import json
import time

app = FastAPI()

# In production: use Redis or a database
idempotency_store: dict[str, dict] = {}
IDEMPOTENCY_TTL_SECONDS = 86400  # 24 hours


@app.post("/payments")
async def create_payment(
    amount_cents: int,
    customer_id: str,
    response: Response,
    idempotency_key: Optional[str] = Header(None, alias="Idempotency-Key"),
):
    if idempotency_key is None:
        raise HTTPException(
            status_code=400,
            detail={
                "error": "idempotency_key_required",
                "message": "POST /payments requires an Idempotency-Key header.",
                "docs": "/openapi.json#/paths/~1payments/post",
            },
        )

    # Check if this key was already processed
    if idempotency_key in idempotency_store:
        cached = idempotency_store[idempotency_key]
        response.headers["X-Idempotent-Replayed"] = "true"
        return cached["response"]

    # Process the payment (your real logic here)
    result = {
        "payment_id": f"pay_{hashlib.sha256(idempotency_key.encode()).hexdigest()[:12]}",
        "amount_cents": amount_cents,
        "customer_id": customer_id,
        "status": "completed",
    }

    # Cache the response
    idempotency_store[idempotency_key] = {
        "response": result,
        "created_at": time.time(),
    }

    return result

The X-Idempotent-Replayed: true header tells the calling agent that this response is cached, not fresh. Agents that track state transitions need to know the difference.

Document idempotency in your OpenAPI spec. Add the Idempotency-Key header as a required parameter on every mutating endpoint. Agents read headers from specs — if the header is documented, the agent sends it.

3. Structured Error Responses

Human developers read "Something went wrong" and check the logs. Agents read it and have no idea what to do next.

Every error your API returns becomes input to an LLM deciding what to do next. Structured errors give agents a recovery path. Unstructured errors produce hallucinated retries.

from fastapi import FastAPI, HTTPException, Request
from fastapi.responses import JSONResponse
from pydantic import BaseModel

app = FastAPI()


class APIError(BaseModel):
    error_code: str
    message: str
    retryable: bool
    retry_after_seconds: int | None = None
    suggestion: str | None = None


ERROR_CATALOG = {
    "insufficient_funds": APIError(
        error_code="insufficient_funds",
        message="Customer balance is below the requested amount.",
        retryable=False,
        suggestion="Reduce the amount or add funds via POST /customers/{id}/balance",
    ),
    "rate_limited": APIError(
        error_code="rate_limited",
        message="Too many requests. Slow down.",
        retryable=True,
        retry_after_seconds=30,
        suggestion="Wait for retry_after_seconds, then retry the same request.",
    ),
    "resource_locked": APIError(
        error_code="resource_locked",
        message="This resource is being modified by another request.",
        retryable=True,
        retry_after_seconds=2,
        suggestion="Retry with the same idempotency key after the wait period.",
    ),
}


@app.exception_handler(HTTPException)
async def structured_error_handler(request: Request, exc: HTTPException):
    if isinstance(exc.detail, dict) and "error_code" in exc.detail:
        return JSONResponse(
            status_code=exc.status_code,
            content=exc.detail,
        )
    # Fallback for unstructured errors
    return JSONResponse(
        status_code=exc.status_code,
        content={
            "error_code": "unknown_error",
            "message": str(exc.detail),
            "retryable": False,
            "suggestion": "Check the request parameters and try again.",
        },
    )

Three fields that change agent behavior:

retryable: Boolean. Agents check this before deciding to retry. Without it, agents retry everything — including errors that will never succeed.
retry_after_seconds: Integer. Tells the agent exactly how long to wait. Without it, agents use exponential backoff or guess.
suggestion: String. The LLM reads this and uses it to decide the next action. "Reduce the amount or add funds via POST /customers/{id}/balance" gives the agent a concrete recovery path.

This turns a dead-end error into a decision tree the agent can navigate.

4. Confirmation Endpoints for Destructive Actions

Agents chain actions. A prompt like "clean up old data" can trigger an agent to call DELETE /users/{id} in a loop without asking anyone. One bad filter and you lose production data.

Destructive operations need a two-step process: prepare, then confirm.

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import uuid
import time

app = FastAPI()

pending_deletions: dict[str, dict] = {}
CONFIRMATION_TTL_SECONDS = 300  # 5 minutes


class DeletionPreview(BaseModel):
    confirmation_token: str
    action: str
    affected_resources: list[str]
    affected_count: int
    expires_at: float
    warning: str


@app.post(
    "/users/bulk-delete/preview",
    summary="Preview a bulk deletion. Returns affected resources. Does NOT delete anything.",
)
async def preview_bulk_delete(
    filter_status: str,
    older_than_days: int,
) -> DeletionPreview:
    # Query what WOULD be deleted (read-only operation)
    affected = [f"user_{i}" for i in range(3)]  # Replace with real query

    token = str(uuid.uuid4())
    preview = DeletionPreview(
        confirmation_token=token,
        action="bulk_delete_users",
        affected_resources=affected,
        affected_count=len(affected),
        expires_at=time.time() + CONFIRMATION_TTL_SECONDS,
        warning=f"This will permanently delete {len(affected)} users. "
        f"Confirm within 5 minutes or the token expires.",
    )

    pending_deletions[token] = {
        "filter_status": filter_status,
        "older_than_days": older_than_days,
        "preview": preview.model_dump(),
        "created_at": time.time(),
    }

    return preview


@app.post(
    "/users/bulk-delete/confirm",
    summary="Execute a previewed bulk deletion. Requires a valid confirmation token.",
)
async def confirm_bulk_delete(confirmation_token: str):
    if confirmation_token not in pending_deletions:
        raise HTTPException(
            status_code=404,
            detail={
                "error_code": "invalid_token",
                "message": "Confirmation token not found or expired.",
                "retryable": False,
                "suggestion": "Call POST /users/bulk-delete/preview to get a new token.",
            },
        )

    pending = pending_deletions[confirmation_token]
    if time.time() > pending["preview"]["expires_at"]:
        del pending_deletions[confirmation_token]
        raise HTTPException(
            status_code=410,
            detail={
                "error_code": "token_expired",
                "message": "Confirmation token has expired.",
                "retryable": False,
                "suggestion": "Call POST /users/bulk-delete/preview to generate a new token.",
            },
        )

    # Execute the actual deletion
    del pending_deletions[confirmation_token]
    return {"status": "deleted", "count": pending["preview"]["affected_count"]}

The preview endpoint is a read-only operation. It returns what would happen without doing it. The confirmation endpoint executes — but only with a valid, unexpired token.

This gives the agent (or the human reviewing agent actions) a checkpoint. The token expiration prevents stale confirmations from executing hours later when the data has changed.

5. Rate Limiting With Retry-After Headers

Agents hit rate limits more than humans do. A human clicks a button once. An agent calls your endpoint in a loop until a condition is met.

Most rate limiters return 429 Too Many Requests with no guidance. The agent retries immediately, gets another 429, retries again. This creates a thundering herd.

from fastapi import FastAPI, Request, HTTPException
from fastapi.responses import JSONResponse
import time

app = FastAPI()

# Simple per-client rate limiter (use Redis in production)
rate_limit_store: dict[str, list[float]] = {}
REQUESTS_PER_MINUTE = 60


@app.middleware("http")
async def rate_limit_middleware(request: Request, call_next):
    client_id = request.headers.get("X-API-Key", request.client.host)
    now = time.time()
    window_start = now - 60

    # Clean old entries and count recent requests
    requests = rate_limit_store.get(client_id, [])
    requests = [t for t in requests if t > window_start]

    if len(requests) >= REQUESTS_PER_MINUTE:
        oldest_in_window = min(requests)
        retry_after = int(oldest_in_window + 60 - now) + 1

        return JSONResponse(
            status_code=429,
            headers={
                "Retry-After": str(retry_after),
                "X-RateLimit-Limit": str(REQUESTS_PER_MINUTE),
                "X-RateLimit-Remaining": "0",
                "X-RateLimit-Reset": str(int(oldest_in_window + 60)),
            },
            content={
                "error_code": "rate_limited",
                "message": f"Rate limit exceeded. {REQUESTS_PER_MINUTE} requests per minute.",
                "retryable": True,
                "retry_after_seconds": retry_after,
                "suggestion": f"Wait {retry_after} seconds before retrying.",
            },
        )

    requests.append(now)
    rate_limit_store[client_id] = requests

    response = await call_next(request)
    response.headers["X-RateLimit-Limit"] = str(REQUESTS_PER_MINUTE)
    response.headers["X-RateLimit-Remaining"] = str(
        REQUESTS_PER_MINUTE - len(requests)
    )
    return response

Four headers that agents need on every response:

Header	Purpose
`Retry-After`	Seconds until the agent should retry. Agents that respect this stop hammering your server.
`X-RateLimit-Limit`	Total requests allowed per window. Agents use this to pace themselves.
`X-RateLimit-Remaining`	Requests left in the current window. Agents can preemptively slow down before hitting zero.
`X-RateLimit-Reset`	Unix timestamp when the window resets. Agents can schedule their next batch.

The Retry-After header is an HTTP standard (RFC 9110, Section 10.2.3). Most HTTP client libraries and agent frameworks parse it automatically. If you only add one header, make it this one.

The Pattern Behind the Patterns

All five patterns share one principle: make the implicit explicit.

Human developers infer intent from context, read between the lines of error messages, and know not to retry a payment twice. AI agents do none of that. They operate on what your API explicitly tells them.

Explicit descriptions in your spec (Pattern 1)
Explicit idempotency guarantees (Pattern 2)
Explicit error recovery paths (Pattern 3)
Explicit confirmation gates (Pattern 4)
Explicit rate limit boundaries (Pattern 5)

The good news: every pattern here makes your API better for human consumers too. Structured errors, idempotency keys, and rate limit headers are things your human developers have been asking for. Agents just forced the issue.

Start with Pattern 3 (structured errors) and Pattern 5 (rate limiting headers). These require the least refactoring and prevent the most agent-caused incidents. Then add idempotency keys to your payment and mutation endpoints. The confirmation pattern is for high-risk operations where the cost of a wrong action is permanent.

Your API was designed for humans. The agents are already calling it. Make the implicit explicit before they learn the hard way.

Follow @klement_gunndu for more AI architecture content. We're building in public.

4 Claude Code Workflows That Write Your Python Tests

klement Gunndu — Sun, 22 Mar 2026 12:35:55 +0000

Your Python project has 30% test coverage. Not because testing is hard — because writing tests is tedious. Claude Code changes the economics.

These 4 workflows generate real, runnable tests from your existing codebase. Not toy examples. Tests that catch bugs, cover edge cases, and run in CI.

Why AI-Generated Tests Work Better Than You Expect

The skepticism is fair: "AI writes bad tests." Some do. Generic prompts produce generic assertions. But Claude Code reads your entire project — imports, types, error handling, docstrings — before generating anything.

The difference between a bad AI test and a useful one is context. Claude Code has your full codebase as context. That changes what's possible.

Here are 4 workflows, ordered from simplest to most advanced.

Workflow 1: Generate Unit Tests From Existing Functions

Start with the function you want to test. Claude Code reads the implementation, infers expected behavior from the code, and generates tests that match your project's existing test patterns.

Say you have this authentication module:

# src/auth.py
import hashlib
import secrets
from datetime import datetime, timedelta

class TokenExpiredError(Exception):
    pass

class InvalidTokenError(Exception):
    pass

def hash_password(password: str, salt: str | None = None) -> tuple[str, str]:
    """Hash a password with salt. Returns (hash, salt)."""
    if not password:
        raise ValueError("Password cannot be empty")
    if salt is None:
        salt = secrets.token_hex(16)
    hashed = hashlib.pbkdf2_hmac("sha256", password.encode(), salt.encode(), 100_000)
    return hashed.hex(), salt

def verify_password(password: str, stored_hash: str, salt: str) -> bool:
    """Verify a password against a stored hash."""
    computed_hash, _ = hash_password(password, salt)
    return secrets.compare_digest(computed_hash, stored_hash)

def create_token(user_id: str, ttl_minutes: int = 60) -> dict:
    """Create an auth token with expiry."""
    if ttl_minutes <= 0:
        raise ValueError("TTL must be positive")
    return {
        "user_id": user_id,
        "token": secrets.token_urlsafe(32),
        "expires_at": (datetime.utcnow() + timedelta(minutes=ttl_minutes)).isoformat(),
    }

def validate_token(token: dict) -> str:
    """Validate token and return user_id. Raises on failure."""
    if "expires_at" not in token or "user_id" not in token:
        raise InvalidTokenError("Malformed token")
    expires = datetime.fromisoformat(token["expires_at"])
    if datetime.utcnow() > expires:
        raise TokenExpiredError(f"Token expired at {token['expires_at']}")
    return token["user_id"]

Open Claude Code in your project directory and tell it what you need:

"Write pytest tests for src/auth.py. Cover the happy path and error cases for each function. Use the same test structure as my existing tests."

Claude Code reads src/auth.py, scans your existing test files for patterns (fixtures, naming conventions, directory structure), and generates tests that fit your project:

# tests/test_auth.py
import pytest
from datetime import datetime, timedelta
from unittest.mock import patch
from src.auth import (
    hash_password,
    verify_password,
    create_token,
    validate_token,
    TokenExpiredError,
    InvalidTokenError,
)


class TestHashPassword:
    def test_returns_hash_and_salt(self):
        hashed, salt = hash_password("secure123")
        assert isinstance(hashed, str)
        assert isinstance(salt, str)
        assert len(hashed) == 64  # SHA-256 hex digest
        assert len(salt) == 32  # 16 bytes as hex

    def test_same_password_different_salt_produces_different_hash(self):
        hash1, salt1 = hash_password("secure123")
        hash2, salt2 = hash_password("secure123")
        assert hash1 != hash2
        assert salt1 != salt2

    def test_same_password_same_salt_produces_same_hash(self):
        hash1, _ = hash_password("secure123", salt="fixed_salt")
        hash2, _ = hash_password("secure123", salt="fixed_salt")
        assert hash1 == hash2

    def test_empty_password_raises_value_error(self):
        with pytest.raises(ValueError, match="Password cannot be empty"):
            hash_password("")


class TestVerifyPassword:
    def test_correct_password_returns_true(self):
        hashed, salt = hash_password("mypassword")
        assert verify_password("mypassword", hashed, salt) is True

    def test_wrong_password_returns_false(self):
        hashed, salt = hash_password("mypassword")
        assert verify_password("wrongpassword", hashed, salt) is False


class TestCreateToken:
    def test_returns_token_with_required_fields(self):
        token = create_token("user_42")
        assert token["user_id"] == "user_42"
        assert "token" in token
        assert "expires_at" in token

    def test_token_string_is_url_safe(self):
        token = create_token("user_42")
        # url-safe base64 characters only
        assert all(c.isalnum() or c in "-_" for c in token["token"])

    def test_zero_ttl_raises_value_error(self):
        with pytest.raises(ValueError, match="TTL must be positive"):
            create_token("user_42", ttl_minutes=0)

    def test_negative_ttl_raises_value_error(self):
        with pytest.raises(ValueError, match="TTL must be positive"):
            create_token("user_42", ttl_minutes=-5)

    def test_custom_ttl_sets_correct_expiry(self):
        before = datetime.utcnow()
        token = create_token("user_42", ttl_minutes=30)
        after = datetime.utcnow()
        expires = datetime.fromisoformat(token["expires_at"])
        assert before + timedelta(minutes=30) <= expires <= after + timedelta(minutes=30)


class TestValidateToken:
    def test_valid_token_returns_user_id(self):
        token = create_token("user_42", ttl_minutes=60)
        assert validate_token(token) == "user_42"

    def test_expired_token_raises_error(self):
        expired_time = (datetime.utcnow() - timedelta(hours=1)).isoformat()
        token = {"user_id": "user_42", "token": "abc", "expires_at": expired_time}
        with pytest.raises(TokenExpiredError):
            validate_token(token)

    def test_missing_user_id_raises_error(self):
        token = {"expires_at": (datetime.utcnow() + timedelta(hours=1)).isoformat()}
        with pytest.raises(InvalidTokenError, match="Malformed token"):
            validate_token(token)

    def test_missing_expires_at_raises_error(self):
        token = {"user_id": "user_42"}
        with pytest.raises(InvalidTokenError, match="Malformed token"):
            validate_token(token)

That's 15 tests covering 4 functions. Every assertion checks something specific. Every error path is exercised. The structure matches pytest conventions — grouped by class, descriptive names, isolated setup.

Key insight: Claude Code didn't just test the obvious paths. It tested that the same password with different salts produces different hashes, that URL-safe tokens contain only valid characters, and that expiry math is bounded correctly. These are the edge cases developers skip when writing tests manually.

Workflow 2: Generate Edge Case Tests From Error Paths

Unit tests cover the contract. Edge case tests cover the boundaries — the inputs that developers forget because they seem unlikely.

Take a data processing function:

# src/processor.py
import json
from pathlib import Path

def load_config(path: str | Path) -> dict:
    """Load JSON config file with validation."""
    path = Path(path)
    if not path.exists():
        raise FileNotFoundError(f"Config not found: {path}")
    if not path.suffix == ".json":
        raise ValueError(f"Expected .json file, got: {path.suffix}")

    text = path.read_text(encoding="utf-8")
    if not text.strip():
        return {}

    try:
        config = json.loads(text)
    except json.JSONDecodeError as e:
        raise ValueError(f"Invalid JSON in {path}: {e}") from e

    if not isinstance(config, dict):
        raise TypeError(f"Config must be a JSON object, got {type(config).__name__}")
    return config


def merge_configs(*configs: dict) -> dict:
    """Deep merge multiple config dicts. Later values override earlier ones."""
    result: dict = {}
    for config in configs:
        for key, value in config.items():
            if (
                key in result
                and isinstance(result[key], dict)
                and isinstance(value, dict)
            ):
                result[key] = merge_configs(result[key], value)
            else:
                result[key] = value
    return result

Tell Claude Code to focus on the boundaries:

"Write edge case tests for src/processor.py. Focus on boundary conditions, malformed inputs, empty inputs, and type mismatches."

# tests/test_processor_edges.py
import json
import pytest
from pathlib import Path
from src.processor import load_config, merge_configs


class TestLoadConfigEdgeCases:
    def test_nonexistent_file_raises_file_not_found(self, tmp_path):
        with pytest.raises(FileNotFoundError, match="Config not found"):
            load_config(tmp_path / "missing.json")

    def test_wrong_extension_raises_value_error(self, tmp_path):
        yaml_file = tmp_path / "config.yaml"
        yaml_file.write_text("{}")
        with pytest.raises(ValueError, match="Expected .json"):
            load_config(yaml_file)

    def test_empty_file_returns_empty_dict(self, tmp_path):
        empty = tmp_path / "empty.json"
        empty.write_text("")
        assert load_config(empty) == {}

    def test_whitespace_only_file_returns_empty_dict(self, tmp_path):
        ws = tmp_path / "whitespace.json"
        ws.write_text("   \n\t  ")
        assert load_config(ws) == {}

    def test_invalid_json_raises_value_error(self, tmp_path):
        bad = tmp_path / "bad.json"
        bad.write_text("{not valid json}")
        with pytest.raises(ValueError, match="Invalid JSON"):
            load_config(bad)

    def test_json_array_raises_type_error(self, tmp_path):
        arr = tmp_path / "array.json"
        arr.write_text('[1, 2, 3]')
        with pytest.raises(TypeError, match="must be a JSON object"):
            load_config(arr)

    def test_json_string_raises_type_error(self, tmp_path):
        s = tmp_path / "string.json"
        s.write_text('"just a string"')
        with pytest.raises(TypeError, match="must be a JSON object"):
            load_config(s)

    def test_accepts_path_object(self, tmp_path):
        f = tmp_path / "valid.json"
        f.write_text('{"key": "value"}')
        assert load_config(Path(f)) == {"key": "value"}

    def test_accepts_string_path(self, tmp_path):
        f = tmp_path / "valid.json"
        f.write_text('{"key": "value"}')
        assert load_config(str(f)) == {"key": "value"}

    def test_unicode_content(self, tmp_path):
        f = tmp_path / "unicode.json"
        f.write_text('{"name": "cafe\u0301"}', encoding="utf-8")
        result = load_config(f)
        assert "name" in result


class TestMergeConfigsEdgeCases:
    def test_no_configs_returns_empty(self):
        assert merge_configs() == {}

    def test_single_config_returned_as_is(self):
        assert merge_configs({"a": 1}) == {"a": 1}

    def test_later_values_override_earlier(self):
        result = merge_configs({"a": 1}, {"a": 2})
        assert result["a"] == 2

    def test_deep_merge_nested_dicts(self):
        base = {"db": {"host": "localhost", "port": 5432}}
        override = {"db": {"port": 5433, "name": "prod"}}
        result = merge_configs(base, override)
        assert result == {"db": {"host": "localhost", "port": 5433, "name": "prod"}}

    def test_non_dict_overrides_dict(self):
        result = merge_configs({"a": {"nested": True}}, {"a": "flat"})
        assert result["a"] == "flat"

    def test_dict_overrides_non_dict(self):
        result = merge_configs({"a": "flat"}, {"a": {"nested": True}})
        assert result["a"] == {"nested": True}

    def test_three_way_merge(self):
        result = merge_configs({"a": 1}, {"b": 2}, {"c": 3})
        assert result == {"a": 1, "b": 2, "c": 3}

    def test_empty_dict_does_not_clobber(self):
        result = merge_configs({"a": 1}, {})
        assert result == {"a": 1}

That's 19 tests hitting every branch. The tmp_path fixture from pytest handles file creation and cleanup automatically — no manual teardown.

Key insight: The edge case workflow finds inputs you wouldn't think to test. JSON arrays passing json.loads but failing the isinstance(config, dict) check. Whitespace-only files hitting the text.strip() empty check. Dict-to-non-dict overrides in deep merge. These are the bugs that surface in production at 2 AM.

Workflow 3: Generate Integration Tests From API Routes

Unit tests verify functions. Integration tests verify that your endpoints accept real requests and return correct responses.

Given a FastAPI application:

# src/api.py
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field

app = FastAPI()

_store: dict[str, dict] = {}


class Item(BaseModel):
    name: str = Field(min_length=1, max_length=100)
    price: float = Field(gt=0)
    quantity: int = Field(ge=0, default=0)


class ItemResponse(BaseModel):
    id: str
    name: str
    price: float
    quantity: int


@app.post("/items", response_model=ItemResponse, status_code=201)
def create_item(item: Item):
    item_id = f"item_{len(_store) + 1}"
    record = {"id": item_id, **item.model_dump()}
    _store[item_id] = record
    return record


@app.get("/items/{item_id}", response_model=ItemResponse)
def get_item(item_id: str):
    if item_id not in _store:
        raise HTTPException(status_code=404, detail="Item not found")
    return _store[item_id]


@app.delete("/items/{item_id}", status_code=204)
def delete_item(item_id: str):
    if item_id not in _store:
        raise HTTPException(status_code=404, detail="Item not found")
    del _store[item_id]

Tell Claude Code to write integration tests using FastAPI's test client:

"Write integration tests for src/api.py using FastAPI TestClient. Test full request/response cycles including validation errors and 404s."

# tests/test_api_integration.py
import pytest
from fastapi.testclient import TestClient
from src.api import app, _store


@pytest.fixture(autouse=True)
def clear_store():
    """Reset in-memory store between tests."""
    _store.clear()
    yield
    _store.clear()


client = TestClient(app)


class TestCreateItem:
    def test_create_returns_201_with_id(self):
        response = client.post("/items", json={"name": "Widget", "price": 9.99})
        assert response.status_code == 201
        data = response.json()
        assert data["id"].startswith("item_")
        assert data["name"] == "Widget"
        assert data["price"] == 9.99
        assert data["quantity"] == 0  # default

    def test_create_with_quantity(self):
        response = client.post(
            "/items", json={"name": "Gadget", "price": 19.99, "quantity": 50}
        )
        assert response.status_code == 201
        assert response.json()["quantity"] == 50

    def test_empty_name_returns_422(self):
        response = client.post("/items", json={"name": "", "price": 9.99})
        assert response.status_code == 422

    def test_negative_price_returns_422(self):
        response = client.post("/items", json={"name": "Widget", "price": -1})
        assert response.status_code == 422

    def test_zero_price_returns_422(self):
        response = client.post("/items", json={"name": "Widget", "price": 0})
        assert response.status_code == 422

    def test_missing_required_field_returns_422(self):
        response = client.post("/items", json={"price": 9.99})
        assert response.status_code == 422


class TestGetItem:
    def test_get_existing_item(self):
        create_resp = client.post("/items", json={"name": "Widget", "price": 9.99})
        item_id = create_resp.json()["id"]

        response = client.get(f"/items/{item_id}")
        assert response.status_code == 200
        assert response.json()["name"] == "Widget"

    def test_get_nonexistent_returns_404(self):
        response = client.get("/items/item_999")
        assert response.status_code == 404
        assert response.json()["detail"] == "Item not found"


class TestDeleteItem:
    def test_delete_existing_item(self):
        create_resp = client.post("/items", json={"name": "Widget", "price": 9.99})
        item_id = create_resp.json()["id"]

        response = client.delete(f"/items/{item_id}")
        assert response.status_code == 204

        # confirm it's gone
        get_resp = client.get(f"/items/{item_id}")
        assert get_resp.status_code == 404

    def test_delete_nonexistent_returns_404(self):
        response = client.delete("/items/item_999")
        assert response.status_code == 404


class TestFullWorkflow:
    def test_create_read_delete_cycle(self):
        # create
        resp = client.post("/items", json={"name": "Sensor", "price": 49.99, "quantity": 10})
        assert resp.status_code == 201
        item_id = resp.json()["id"]

        # read
        resp = client.get(f"/items/{item_id}")
        assert resp.status_code == 200
        assert resp.json()["quantity"] == 10

        # delete
        resp = client.delete(f"/items/{item_id}")
        assert resp.status_code == 204

        # confirm deletion
        resp = client.get(f"/items/{item_id}")
        assert resp.status_code == 404

15 tests covering the full CRUD lifecycle plus validation. The autouse fixture resets state between tests — no test pollution.

Key insight: Integration tests expose what unit tests miss. The 422 responses from Pydantic validation, the 204 with no response body on delete, the full create-read-delete cycle — these are the behaviors your users hit first.

Workflow 4: Generate Regression Tests From Bug Reports

A regression test locks in a fix. You describe the bug, Claude Code writes a test that fails on the buggy code and passes on the fix.

Say you get a bug report: "Users with special characters in their names crash the export function."

The buggy code:

# src/export.py
import csv
import io

def export_users_csv(users: list[dict]) -> str:
    """Export user records to CSV string."""
    if not users:
        return ""
    output = io.StringIO()
    writer = csv.DictWriter(output, fieldnames=users[0].keys())
    writer.writeheader()
    writer.writerows(users)
    return output.getvalue()

Tell Claude Code about the bug:

"Bug report: export_users_csv crashes when user names contain commas, quotes, or newlines. Write regression tests that reproduce this bug, then I'll fix the function."

# tests/test_export_regression.py
import pytest
from src.export import export_users_csv


class TestExportCSVRegression:
    """Regression tests for special character handling in CSV export.

    Bug: Users with commas, quotes, or newlines in names crashed export.
    These tests must pass after the fix and must never be removed.
    """

    def test_name_with_comma(self):
        users = [{"name": "Smith, John", "email": "john@example.com"}]
        result = export_users_csv(users)
        assert "Smith, John" in result or '"Smith, John"' in result
        lines = result.strip().split("\n")
        assert len(lines) == 2  # header + 1 row

    def test_name_with_double_quotes(self):
        users = [{"name": 'The "Boss"', "email": "boss@example.com"}]
        result = export_users_csv(users)
        assert "Boss" in result
        lines = result.strip().split("\n")
        assert len(lines) == 2

    def test_name_with_newline(self):
        users = [{"name": "Line1\nLine2", "email": "multi@example.com"}]
        result = export_users_csv(users)
        # CSV should handle embedded newlines by quoting the field
        assert "multi@example.com" in result

    def test_empty_name(self):
        users = [{"name": "", "email": "empty@example.com"}]
        result = export_users_csv(users)
        assert "empty@example.com" in result

    def test_empty_list_returns_empty_string(self):
        assert export_users_csv([]) == ""

    def test_mixed_special_characters(self):
        users = [
            {"name": "Normal User", "email": "normal@example.com"},
            {"name": "O'Brien, \"Pat\"", "email": "pat@example.com"},
            {"name": "Tab\there", "email": "tab@example.com"},
        ]
        result = export_users_csv(users)
        lines = result.strip().split("\n")
        # header + 3 data rows (newlines inside quoted fields don't count as line breaks in proper CSV)
        assert len(lines) >= 4  # at least header + 3 rows

These tests document the exact bug. The test names describe what broke. The docstring explains why these tests exist. When someone runs the test suite 6 months from now, they'll know this was a real production bug — not a hypothetical edge case.

Key insight: Regression tests are documentation. They tell the next developer: "This broke before. Don't let it break again." Claude Code writes the test before you write the fix, which is test-driven development without the overhead of writing tests yourself.

Measuring the Impact

After running these 4 workflows, check your coverage:

pytest --cov=src --cov-report=term-missing tests/

The term-missing flag shows exactly which lines still lack coverage. Focus your next round of tests there.

A realistic progression:

Before: 15-30% coverage (whatever existed)
After Workflow 1 (unit tests): 50-60%
After Workflow 2 (edge cases): 65-75%
After Workflow 3 (integration): 75-85%
After Workflow 4 (regressions): 80-90%+ on the targeted modules

The exact numbers depend on your codebase. The point is that 4 focused sessions with Claude Code produce more useful tests than most teams write in a sprint.

What Makes These Tests Good

Three qualities separate useful AI-generated tests from noise:

They test behavior, not implementation. The tests verify what the function returns and raises — not how it does it internally. Refactor the implementation and the tests still pass.
They use real inputs. Commas in names, expired tokens, empty files. Not test_1, test_2, test_3 with dummy data that proves nothing.
They're isolated. Each test creates its own state, verifies one thing, and cleans up. No test depends on another test running first.

Get Started in 5 Minutes

Pick the module in your project with the lowest coverage. Open Claude Code. Run Workflow 1. Check coverage. You'll have more useful tests in 20 minutes than you'd write manually in a day.

The tedious part of testing — enumerating edge cases, writing setup code, covering error paths — is exactly what AI handles well. The hard part of testing — deciding what to test and what the correct behavior should be — still requires your judgment.

Claude Code writes the tests. You decide what matters.

Follow @klement_gunndu for more Claude Code content. We're building in public.

Write Better AI Agent Tools: 5 Python Patterns LLMs Actually Understand

klement Gunndu — Sat, 21 Mar 2026 12:35:56 +0000

Your AI agent keeps picking the wrong tool. You tweak the system prompt. You add "IMPORTANT: use search_database, NOT search_web." It helps for a day, then breaks again.

The problem is not the model. The problem is your tool definitions.

LLMs decide which tool to call based on three things: the tool name, the description, and the parameter schema. Most developers write tools like they are writing functions for other developers. Short names, minimal docstrings, stringly-typed parameters. The LLM gets a vague menu and guesses.

These five patterns fix that. Each one is a specific change to how you define tools in Python — with working code you can adapt today.

Pattern 1: Docstrings Are the Tool's Manual

The LLM reads your docstring to decide when to call your tool. A one-line docstring forces the model to guess from the function name alone.

Bad tool definition:

from langchain_core.tools import tool

@tool
def search(query: str) -> str:
    """Search for information."""
    # implementation
    return results

The model sees "search" and "Search for information." Search where? Search what kind of information? When should it use this instead of another tool?

Better:

from langchain_core.tools import tool

@tool
def search_customer_database(query: str) -> str:
    """Search the internal customer database by name, email, or account ID.

    Use this tool when the user asks about a specific customer, their
    order history, or account status. Do NOT use this for general
    product questions — use search_product_catalog instead.

    Args:
        query: Customer name, email address, or account ID (e.g. 'ACC-12345')
    """
    # implementation
    return results

Three things changed:

The name is specific. search_customer_database tells the model exactly what data source this tool accesses.
The docstring says when to use it AND when not to. This is the most overlooked pattern. Telling the model what this tool is not for reduces incorrect calls significantly.
The Args section includes an example. The model now knows the expected format for query.

This works identically in OpenAI's function calling. The description field in the tool schema is the same lever:

tools = [
    {
        "type": "function",
        "name": "search_customer_database",
        "description": (
            "Search the internal customer database by name, email, "
            "or account ID. Use when the user asks about a specific "
            "customer. Do NOT use for product questions."
        ),
        "parameters": {
            "type": "object",
            "properties": {
                "query": {
                    "type": "string",
                    "description": "Customer name, email, or account ID (e.g. 'ACC-12345')"
                }
            },
            "required": ["query"],
            "additionalProperties": False
        },
        "strict": True
    }
]

The rule: if you removed the function body and only had the name, docstring, and type hints, could another developer call this tool correctly? If not, the LLM cannot either.

Pattern 2: Constrain Parameters With Types, Not Prose

Free-text string parameters are the biggest source of bad tool calls. The model invents formats, passes wrong values, and you end up parsing garbage.

The fix is to use Literal types, enums, and Pydantic models to constrain what the LLM can pass.

Bad:

@tool
def get_weather(location: str, units: str) -> str:
    """Get weather for a location.

    Args:
        location: The city name
        units: Temperature units (use 'celsius' or 'fahrenheit')
    """
    # implementation
    return forecast

The model sees units: str and might pass "c", "Celsius", "metric", or "degrees_f". Your code handles none of these.

Better:

from typing import Literal
from pydantic import BaseModel, Field
from langchain_core.tools import tool

class WeatherInput(BaseModel):
    """Input schema for weather queries."""
    location: str = Field(
        description="City and country, e.g. 'London, UK'"
    )
    units: Literal["celsius", "fahrenheit"] = Field(
        default="celsius",
        description="Temperature unit"
    )

@tool(args_schema=WeatherInput)
def get_weather(location: str, units: str = "celsius") -> str:
    """Get current weather for a city.

    Use this when the user asks about current temperature,
    conditions, or weather forecast for a specific city.

    Args:
        location: City and country, e.g. 'London, UK'
        units: Either 'celsius' or 'fahrenheit'
    """
    # implementation
    return forecast

Now the LLM sees a schema that only allows two values for units. The model cannot invent a third option.

This scales to more complex constraints:

from enum import Enum
from pydantic import BaseModel, Field

class Priority(str, Enum):
    LOW = "low"
    MEDIUM = "medium"
    HIGH = "high"
    CRITICAL = "critical"

class CreateTicketInput(BaseModel):
    """Input for creating a support ticket."""
    title: str = Field(
        description="Short summary of the issue, under 100 characters"
    )
    priority: Priority = Field(
        default=Priority.MEDIUM,
        description="Ticket priority level"
    )
    labels: list[str] = Field(
        default_factory=list,
        description="Labels to categorize the ticket, e.g. ['bug', 'frontend']"
    )

Every field has a type constraint. Every field has a description with an example. The model cannot pass an integer for priority or a string for labels. The schema enforces correctness before your code runs.

Pattern 3: Return Errors the LLM Can Act On

Most tools return raw exceptions or generic error strings. The model sees "Error: something went wrong" and either retries blindly or gives up.

Structured error responses tell the model what went wrong and what to do next.

Bad:

@tool
def lookup_user(user_id: str) -> str:
    """Look up a user by their ID."""
    try:
        user = db.get_user(user_id)
        return f"User: {user.name}, Email: {user.email}"
    except Exception as e:
        return f"Error: {str(e)}"

The model gets "Error: relation 'users' does not exist" — a database implementation detail it cannot act on.

Better:

import json
from langchain_core.tools import tool

@tool
def lookup_user(user_id: str) -> str:
    """Look up a user by their ID (format: USR-XXXXX).

    Returns user profile data. If the user is not found,
    returns an error with suggested next steps.

    Args:
        user_id: User ID in format USR-XXXXX (e.g. 'USR-48291')
    """
    if not user_id.startswith("USR-"):
        return json.dumps({
            "status": "error",
            "error_type": "invalid_format",
            "message": f"User ID must start with 'USR-'. Received: '{user_id}'",
            "suggestion": "Ask the user for their User ID in format USR-XXXXX"
        })

    user = db.get_user(user_id)
    if user is None:
        return json.dumps({
            "status": "error",
            "error_type": "not_found",
            "message": f"No user found with ID '{user_id}'",
            "suggestion": "Try searching by email with search_user_by_email instead"
        })

    return json.dumps({
        "status": "success",
        "data": {
            "name": user.name,
            "email": user.email,
            "account_status": user.status
        }
    })

Three things this gives the model:

error_type — The model can distinguish between "wrong input format" and "user doesn't exist." Different errors need different recovery strategies.
suggestion — Tells the model what to do next. "Try searching by email" directs it to another tool instead of retrying the same call.
Consistent structure — Every response has status, whether success or error. The model learns the pattern and parses it reliably.

This pattern works across frameworks. In OpenAI's function calling, the tool response goes back into the conversation as a message with role: "tool". A structured JSON response gives the model clear signals for its next step.

Pattern 4: Make Tools Idempotent

AI agents retry. They loop. They sometimes call the same tool twice with the same arguments. If your tool creates a duplicate database record on the second call, you have a bug that is invisible until production.

An idempotent tool produces the same result whether called once or five times with the same input.

Bad:

@tool
def create_calendar_event(
    title: str,
    date: str,
    time: str
) -> str:
    """Create a new calendar event."""
    event = calendar_api.create(title=title, date=date, time=time)
    return f"Created event '{event.title}' with ID {event.id}"

If the agent calls this twice (network timeout, retry logic, loop), the user gets two identical meetings.

Better:

import hashlib
import json
from langchain_core.tools import tool

@tool
def create_calendar_event(
    title: str,
    date: str,
    time: str
) -> str:
    """Create a calendar event. Safe to call multiple times — duplicates
    are detected automatically.

    Args:
        title: Event title (e.g. 'Team Standup')
        date: Date in YYYY-MM-DD format
        time: Time in HH:MM format, 24-hour (e.g. '14:30')
    """
    # Generate deterministic ID from inputs
    idempotency_key = hashlib.sha256(
        f"{title}:{date}:{time}".encode()
    ).hexdigest()[:16]

    # Check if event already exists
    existing = calendar_api.get_by_key(idempotency_key)
    if existing:
        return json.dumps({
            "status": "success",
            "message": f"Event '{title}' already exists for {date} at {time}",
            "event_id": existing.id,
            "already_existed": True
        })

    event = calendar_api.create(
        title=title,
        date=date,
        time=time,
        idempotency_key=idempotency_key
    )
    return json.dumps({
        "status": "success",
        "message": f"Created event '{title}' for {date} at {time}",
        "event_id": event.id,
        "already_existed": False
    })

The key insight: generate a deterministic identifier from the input parameters. Same inputs always produce the same key. Check for that key before creating anything.

Note the already_existed field in the response. This tells the model whether it actually created something new or found an existing match. The model can report accurately to the user: "Your meeting is already scheduled" instead of "I created your meeting" when it was a duplicate call.

This pattern matters most for tools that write data: creating records, sending messages, making payments. Read-only tools are naturally idempotent — calling search_database twice with the same query just returns the same results.

Pattern 5: Return Schemas That Guide the Next Action

Most tools return flat strings. The model gets "Order #4521 shipped on March 15" and has to parse the text to figure out what to do next.

Structured returns with explicit next-action hints make the agent's reasoning faster and more accurate.

Bad:

@tool
def check_order_status(order_id: str) -> str:
    """Check the status of an order."""
    order = orders_db.get(order_id)
    return f"Order {order_id}: {order.status}, shipped {order.ship_date}"

Better:

import json
from langchain_core.tools import tool

@tool
def check_order_status(order_id: str) -> str:
    """Check the shipping status and details of a customer order.

    Args:
        order_id: Order ID in format ORD-XXXXX (e.g. 'ORD-48291')
    """
    order = orders_db.get(order_id)
    if order is None:
        return json.dumps({
            "status": "error",
            "error_type": "not_found",
            "message": f"No order found with ID '{order_id}'",
            "suggestion": "Verify the order ID with the customer"
        })

    result = {
        "status": "success",
        "data": {
            "order_id": order.id,
            "order_status": order.status,
            "items": [
                {"name": item.name, "quantity": item.qty}
                for item in order.items
            ],
            "shipped_date": str(order.ship_date) if order.ship_date else None,
            "tracking_number": order.tracking,
            "estimated_delivery": str(order.eta) if order.eta else None
        },
        "available_actions": []
    }

    if order.status == "shipped":
        result["available_actions"].append({
            "action": "track_package",
            "tool": "track_shipment",
            "args": {"tracking_number": order.tracking}
        })

    if order.status in ("pending", "processing"):
        result["available_actions"].append({
            "action": "cancel_order",
            "tool": "cancel_order",
            "args": {"order_id": order.id}
        })

    return json.dumps(result)

The available_actions field is the critical addition. Instead of the model deciding on its own what tools to call next, the tool response explicitly lists what actions are possible given the current state.

If the order is shipped, the model sees it can call track_shipment with the tracking number already provided. If the order is pending, it sees cancel_order is available. The model does not need to reason about business logic — the tool encodes it in the response.

This pattern chains naturally. Each tool returns data plus next steps. The agent follows the chain until it reaches a terminal state or needs user input. The result is fewer hallucinated tool calls and faster task completion.

Putting It All Together

These five patterns compose. A well-designed tool has:

A descriptive name and docstring that tells the model when to use it and when not to
Constrained input parameters with Literal types or Pydantic schemas
Structured error responses with error_type and suggestion fields
Idempotency keys for any tool that writes data
Return schemas with available_actions that guide the agent's next step

Here is a complete example combining all five:

import hashlib
import json
from typing import Literal
from pydantic import BaseModel, Field
from langchain_core.tools import tool

class TicketInput(BaseModel):
    """Input for creating a support ticket."""
    title: str = Field(
        description="Short summary of the issue, under 100 characters"
    )
    priority: Literal["low", "medium", "high", "critical"] = Field(
        default="medium",
        description="Ticket priority level"
    )
    customer_id: str = Field(
        description="Customer ID in format USR-XXXXX (e.g. 'USR-48291')"
    )

@tool(args_schema=TicketInput)
def create_support_ticket(
    title: str,
    priority: str = "medium",
    customer_id: str = ""
) -> str:
    """Create a support ticket for a customer issue.

    Use this when a customer reports a problem, requests help,
    or needs follow-up. Do NOT use for general inquiries —
    use search_knowledge_base instead.

    Safe to call multiple times — duplicates are detected
    automatically based on title and customer ID.

    Args:
        title: Short issue summary, under 100 characters
        priority: One of 'low', 'medium', 'high', 'critical'
        customer_id: Customer ID in format USR-XXXXX
    """
    if not customer_id.startswith("USR-"):
        return json.dumps({
            "status": "error",
            "error_type": "invalid_customer_id",
            "message": f"Customer ID must start with 'USR-'. Got: '{customer_id}'",
            "suggestion": "Look up the customer first with lookup_user"
        })

    idempotency_key = hashlib.sha256(
        f"{title}:{customer_id}".encode()
    ).hexdigest()[:16]

    existing = ticket_db.get_by_key(idempotency_key)
    if existing:
        return json.dumps({
            "status": "success",
            "data": {"ticket_id": existing.id, "already_existed": True},
            "message": f"Ticket already exists: {existing.id}",
            "available_actions": [{
                "action": "view_ticket",
                "tool": "get_ticket_details",
                "args": {"ticket_id": existing.id}
            }]
        })

    ticket = ticket_db.create(
        title=title,
        priority=priority,
        customer_id=customer_id,
        idempotency_key=idempotency_key
    )

    return json.dumps({
        "status": "success",
        "data": {
            "ticket_id": ticket.id,
            "priority": priority,
            "already_existed": False
        },
        "message": f"Created ticket {ticket.id} with {priority} priority",
        "available_actions": [
            {
                "action": "assign_ticket",
                "tool": "assign_ticket_to_agent",
                "args": {"ticket_id": ticket.id}
            },
            {
                "action": "add_context",
                "tool": "add_ticket_comment",
                "args": {"ticket_id": ticket.id}
            }
        ]
    })

Every element works together. The Pydantic schema constrains inputs. The docstring tells the model when and when not to use the tool. The idempotency key prevents duplicates. The error responses guide recovery. The available_actions in the success response chain to the next step.

The Checklist

Before shipping any tool to an AI agent, check these five things:

Could someone call this tool correctly using only the name, docstring, and type hints? If not, improve the docstring.
Can any parameter accept values your code does not handle? If yes, constrain the type.
Does every error response tell the model what to do next? If not, add a suggestion field.
Could a duplicate call cause side effects? If yes, add an idempotency key.
Does the success response tell the model what actions are available next? If not, add available_actions.

Five questions. Five patterns. The difference between an agent that guesses and one that works.

Follow @klement_gunndu for more AI engineering content. We're building in public.

Containerize Your AI Agent Stack With Docker Compose: 4 Patterns That Work

klement Gunndu — Sat, 21 Mar 2026 08:06:52 +0000

Your AI agent runs fine on your laptop. Then you deploy it and discover you need a model server, a vector database, a message queue, and monitoring -- all wired together correctly. You spend two days writing shell scripts.

Docker Compose defines your entire AI agent stack in a single YAML file. One command brings it all up. Here are 4 patterns that handle the common deployment scenarios.

Pattern 1: Model Runner as a Compose Service

Docker Compose now supports a top-level models element that declares AI models as first-class infrastructure. Instead of manually running a model server and wiring environment variables, you declare the model and bind it to your agent service.

Here is the compose.yaml:

services:
  agent:
    build:
      context: ./agent
    ports:
      - "8080:8080"
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY}
    models:
      llm:
        endpoint_var: MODEL_RUNNER_URL
        model_var: MODEL_RUNNER_MODEL
    depends_on:
      - vectordb

  vectordb:
    image: qdrant/qdrant:v1.17.0
    ports:
      - "6333:6333"
    volumes:
      - qdrant_data:/qdrant/storage

models:
  llm:
    model: ai/gemma3:4B-Q4_0
    context_size: 8192

volumes:
  qdrant_data:

The models block at the top level declares the model. The models block inside the agent service binds it. Docker Compose automatically injects MODEL_RUNNER_URL and MODEL_RUNNER_MODEL as environment variables into the agent container.

What this gives you:

The model is pulled and started automatically on docker compose up
Your agent code reads MODEL_RUNNER_URL to connect -- no hardcoded endpoints
The vector database starts alongside the model with persistent storage
One docker compose down tears everything down cleanly

The context_size field controls the maximum token window for the model. You can also pass runtime_flags for inference engine tuning:

models:
  llm:
    model: ai/gemma3:4B-Q4_0
    context_size: 8192
    runtime_flags:
      - "--temp=0.1"
      - "--no-prefill-assistant"

This pattern replaces the common setup of running Ollama in a separate terminal, manually setting OLLAMA_HOST, and hoping your agent finds it. The model is version-controlled alongside your code -- anyone who clones the repo gets the exact same inference setup.

Pattern 2: GPU Reservations for Local Inference

Running models locally requires GPU access. Docker Compose handles this through the deploy.resources.reservations.devices block. No custom Docker run flags needed.

services:
  inference:
    image: vllm/vllm-openai:latest
    command: ["--model", "meta-llama/Llama-3.1-8B-Instruct"]
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    ports:
      - "8000:8000"
    volumes:
      - model_cache:/root/.cache/huggingface

  agent:
    build:
      context: ./agent
    environment:
      - LLM_ENDPOINT=http://inference:8000/v1
    depends_on:
      inference:
        condition: service_healthy

volumes:
  model_cache:

Key details that save you debugging time:

The capabilities field is required. Omitting it causes deployment to fail silently on some Docker versions. Always include [gpu] explicitly.

count and device_ids are mutually exclusive. Use count: 1 to grab any available GPU. Use device_ids: ['0'] to pin a specific GPU (check nvidia-smi for IDs). Never use both in the same service definition.

The depends_on with condition: service_healthy prevents your agent from starting before the model server is ready. Add a healthcheck to the inference service:

services:
  inference:
    # ... (same as above)
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 120s

The start_period of 120 seconds matters. Large models take 60-90 seconds to load into GPU memory. Without this grace period, Docker marks the service as unhealthy before it finishes loading.

Memory limits prevent OOM kills. Add memory reservations alongside GPU reservations to prevent the inference service from consuming all host RAM during model loading:

services:
  inference:
    # ... (same as above)
    deploy:
      resources:
        limits:
          memory: 16G
        reservations:
          memory: 8G
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

This is especially important on shared machines where other services compete for memory.

Pattern 3: MCP Gateway for Tool Access

AI agents need tools -- web search, database access, file operations. Docker's MCP Gateway image brokers tool access through the Model Context Protocol, giving your agent a single endpoint for all external capabilities.

services:
  agent:
    build:
      context: ./agent
    ports:
      - "8080:8080"
    environment:
      - MCPGATEWAY_ENDPOINT=http://mcp-gateway:8811/sse
    depends_on:
      - mcp-gateway
    models:
      gemma3:
        endpoint_var: MODEL_RUNNER_URL
        model_var: MODEL_RUNNER_MODEL

  mcp-gateway:
    image: docker/mcp-gateway:latest
    command:
      - --transport=sse
      - --servers=duckduckgo,filesystem
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    ports:
      - "8811:8811"

models:
  gemma3:
    model: ai/gemma3:4B-Q4_0
    context_size: 10000

The mcp-gateway service exposes an SSE endpoint at port 8811. Your agent connects to http://mcp-gateway:8811/sse and discovers available tools automatically through the MCP protocol.

Why this matters for DevOps:

Without the gateway, every tool integration requires custom code in your agent. Adding a new tool means changing agent code, rebuilding the image, and redeploying. With the MCP gateway, you add a tool by appending to the --servers flag and running docker compose up -d. The agent discovers the new tool without any code changes.

The Docker socket volume mount (/var/run/docker.sock) gives the gateway access to the host's Docker daemon. This is how the gateway can spawn, manage, and stop MCP server containers on demand.

Adding custom MCP servers:

If the built-in servers are not enough, mount your own configuration:

services:
  mcp-gateway:
    image: docker/mcp-gateway:latest
    command:
      - --transport=sse
      - --servers=duckduckgo
      - --config=/config/custom-servers.json
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./mcp-config:/config
    ports:
      - "8811:8811"

Your custom-servers.json defines additional MCP servers the gateway should manage.

Pattern 4: Multi-Agent Orchestration

Production systems rarely run a single agent. You need specialists -- a researcher, a coder, a reviewer -- each in its own container with its own dependencies.

services:
  orchestrator:
    build:
      context: ./orchestrator
    ports:
      - "8080:8080"
    environment:
      - RESEARCHER_URL=http://researcher:8081
      - CODER_URL=http://coder:8082
      - REVIEWER_URL=http://reviewer:8083
      - REDIS_URL=redis://queue:6379
    depends_on:
      - researcher
      - coder
      - reviewer
      - queue

  researcher:
    build:
      context: ./agents/researcher
    ports:
      - "8081:8081"
    environment:
      - MCPGATEWAY_ENDPOINT=http://mcp-gateway:8811/sse
    models:
      research_model:
        endpoint_var: MODEL_URL
        model_var: MODEL_NAME

  coder:
    build:
      context: ./agents/coder
    ports:
      - "8082:8082"
    models:
      code_model:
        endpoint_var: MODEL_URL
        model_var: MODEL_NAME

  reviewer:
    build:
      context: ./agents/reviewer
    ports:
      - "8083:8083"
    models:
      review_model:
        endpoint_var: MODEL_URL
        model_var: MODEL_NAME

  queue:
    image: redis:7-alpine
    volumes:
      - redis_data:/data

  mcp-gateway:
    image: docker/mcp-gateway:latest
    command:
      - --transport=sse
      - --servers=duckduckgo,filesystem,github
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    ports:
      - "8811:8811"

models:
  research_model:
    model: ai/gemma3:4B-Q4_0
    context_size: 10000
  code_model:
    model: ai/qwen2.5-coder:7B-Q4_0
    context_size: 16384
  review_model:
    model: ai/gemma3:4B-Q4_0
    context_size: 8192

volumes:
  redis_data:

Each agent gets its own model binding. The researcher uses a general-purpose model. The coder gets a code-specialized model with a larger context window. The reviewer uses the same general model as the researcher but with a smaller context for faster responses.

Three things that break multi-agent setups:

1. Port conflicts. Each agent needs its own port. Map them explicitly. If two services bind to the same host port, docker compose up fails without a clear error message.

2. Startup ordering. Use depends_on with health checks. The orchestrator should not start sending requests until all agent services report healthy.

3. Shared state without a broker. Never use shared volumes for inter-agent communication. Use Redis, RabbitMQ, or another message broker. Shared files cause race conditions that are impossible to debug in containers.

Putting It Together: The Production Stack

A production AI agent deployment combines all four patterns:

compose.yaml
  |
  +-- models:         (Pattern 1: model declarations)
  +-- services:
  |     +-- agent:       (your application)
  |     +-- inference:   (Pattern 2: GPU-accelerated model server)
  |     +-- mcp-gateway: (Pattern 3: tool access)
  |     +-- vectordb:    (RAG storage)
  |     +-- queue:       (Pattern 4: inter-agent messaging)
  |     +-- monitoring:  (Prometheus + Grafana)
  +-- volumes:         (persistent storage)

The entire stack starts with docker compose up -d and stops with docker compose down. Model weights persist in named volumes. Logs aggregate to a single stream with docker compose logs -f.

Environment-specific overrides let you swap configurations without changing the base file:

# compose.override.yaml (development)
services:
  agent:
    environment:
      - LOG_LEVEL=DEBUG
      - OPENAI_API_KEY=${OPENAI_API_KEY}

models:
  llm:
    model: ai/smollm2
    context_size: 2048

# compose.prod.yaml
services:
  agent:
    deploy:
      replicas: 3
    environment:
      - LOG_LEVEL=WARNING

models:
  llm:
    model: ai/gemma3:4B-Q4_0
    context_size: 16384

Run development with docker compose up. Run production with docker compose -f compose.yaml -f compose.prod.yaml up -d. Same stack, different configurations.

What Comes Next

These patterns give you a reproducible, portable AI agent stack. Every dependency is declared. Every service is isolated. Every configuration is version-controlled.

The next step is adding observability. Your compose.yaml should include Prometheus for metrics collection and Grafana for visualization. The inference service should export token throughput, latency percentiles, and error rates. Your agent should export task completion rates and tool call success rates.

But the foundation is the compose file. Get the infrastructure right first, and debugging becomes tractable.

Quick reference for the four patterns:

Pattern	Problem It Solves	Key Compose Feature
Model Runner	Manual model server management	Top-level `models` element
GPU Reservations	GPU access without custom flags	`deploy.resources.reservations.devices`
MCP Gateway	Hardcoded tool integrations	`docker/mcp-gateway` with Docker socket
Multi-Agent	Service isolation and orchestration	Per-service model bindings + message broker

Every pattern works standalone. Combine them when your stack demands it. Start with Pattern 1 to replace your manual model setup, then add patterns as your agent grows.

Follow @klement_gunndu for more DevOps and AI content. We're building in public.