DEV Community: Yaohua Chen

A Claude Code Skills Stack: How to Combine Superpowers, gstack, and GSD Without the Chaos

Yaohua Chen — Mon, 06 Apr 2026 23:30:15 +0000

One article to compare the frameworks, see where they overlap, and land on a stable three-layer practice.

Introduction

Claude Code has quickly become one of the most widely adopted AI coding tools. Individual developers, startups, and large engineering teams alike have integrated it into their daily workflows—writing production code, reviewing pull requests, debugging, and shipping features at a pace that was hard to imagine a year ago. As usage has scaled, so has the ecosystem around it. Claude Skills—composable, auto-invoked instruction sets that shape how the agent plans, builds, and verifies—have emerged as one of the most important extension points in Claude Code. They let you go beyond one-off prompts and encode repeatable workflows directly into the agent's behavior. In fact, Anthropic has doubled down on this direction: the latest version of Claude Code consolidates the previously separate "slash commands" and "skills" systems into a single, unified skills format, signaling that skills are now the canonical way to extend the agent.

With Skills now central to the experience, the community has rallied around a handful of open-source frameworks that package best practices into ready-made skill sets. The two most discussed stacks are Superpowers and gstack. Installing both sounds easy; in practice they can conflict, and piling frameworks on without a plan often makes the setup less stable, not more. So where do they differ, and how should you choose?

This post does three things:

Compare Superpowers and gstack on repos, features, and philosophy—the material below on stars, skill lists, and trade-offs.
Add a third layer many guides skip: GSD as a context / spec stabilizer so long-running work does not drift (informed by Tricia Notes Editorial’s three-layer framing).
End with a single playbook: who owns decision, context, and execution, and how to cherry-pick skills without blowing up token use or cognitive load.

The useful question is not only “Superpowers or gstack?” but: what are you missing—decision-making, durable context, or execution?

In one line: gstack thinks, GSD stabilizes, Superpowers executes.

Orientation: Three Layers, Not Only Two

What stays stable in practice is often not picking one framework over another, but a three-way division of labor.

Layer	Stack	Role
Decision / roles	gstack	Judgment from CEO, design, architecture, QA-style lenses—not only “how to code.”
Context / spec	GSD	Keeps spec, status, boundaries, and long-horizon context from rotting.
Execution	Superpowers	Requirement clarification → plan → TDD → acceptance as a closed loop.

How each is “strong”:

Superpowers — How work gets done; smooth execution loop.
gstack — What to do and whether it should be done; richer role-based judgment.
GSD — Not drifting; steadier specs and context over long chains.

Both Superpowers and gstack have gone viral. On the surface they add process to AI; in use, they help you think clearly about what matters. When the model codes fast, that is exactly when you need clear requirements and stable context—that is what most people still overlook.

Superpowers vs gstack: Quick Facts

Superpowers (GitHub ~137K stars)

Repository: obra/superpowers
An Agent Skills framework and software development methodology: 14 built-in skills across brainstorming, planning, TDD, execution, and verification.

gstack (GitHub ~65K stars)

Repository: garrytan/gstack
From YC CEO Garry Tan, open source.
Philosophy: a team beside you—CEO, designer, eng manager, release manager, doc engineer, QA, and more—23 opinionated tools (product thinking, CEO review, architecture review, real browser testing, design review, security audits, etc.).
Garry has claimed 600K+ lines of production code (35% tests) in 60 days, part-time while running YC full-time.

Stars are a weak proxy: high star count does not mean every skill fits your workflow.

Feature Comparison (Superpowers vs gstack)

Category	Superpowers	gstack
Product brainstorming	brainstorming	/office-hours, /plan-ceo-review
Architecture planning	writing-plans	/plan-eng-review, /autoplan
Design	—	/design-consultation, /plan-design-review, /design-shotgun, /design-html
Development execution	executing-plans, subagent-driven-development, dispatching-parallel-agents	—
Testing	test-driven-development	/qa, /qa-only
Debugging	systematic-debugging	/investigate
Code review	requesting-code-review, receiving-code-review	/review, /codex
Verification & acceptance	verification-before-completion, finishing-a-development-branch	/ship, /land-and-deploy, /canary, /document-release
Security	—	/cso, /careful, /freeze, /guard, /unfreeze
Observability	—	/learn, /retro
Browser testing	—	/browse, /connect-chrome, /setup-browser-cookies
Git worktrees	using-git-worktrees	—
Skill management	using-superpowers, writing-skills	/gstack-upgrade
Performance	—	/benchmark
Deployment	—	/setup-deploy

Coverage differs a lot; quantity is not the point—design philosophy is.

Design Philosophy: “How” vs “What” (and Where GSD Fits)

Superpowers — focused on how code gets built

The workflow centers on high-quality output: clarify, plan, TDD (tests before implementation), verify. Checkpoints at each step—little room to skip. In practice it feels disciplined: you ask for X, it tends to build X. Engineers who already know what to build often find that empowering.

(Execution-layer detail from hands-on use: strong process and steady execution; small tasks can still feel **heavy* because the full rhythm applies even to tiny asks.)*

gstack — focused on what and what not to do

Before heavy coding, flows like /office-hours walk requirements; CEO and engineering reviews stress-test the approach. It is not only code—it can run real browser tests from a user angle. Rough split:

Decision layer: /office-hours, /plan-ceo-review, /plan-eng-review
Execution layer: /review, /qa, /ship, etc.

gstack shines when requirements are still fuzzy—PMs, indies, or “think while building.” Caveat: turning all roles on can feel bloated; decision skills also burn serious tokens (see below).

GSD — context / spec, not another “team chart”

GSD is not “install another team.” It is context engineering: goals, specs, status, boundaries, and summaries anchored so context rot slows down. Short demos hide this; long projects show it—when context wobbles, output scatters; that is state, not only “bad execution.”

gstack thinks but is not, by itself, a long-term context vault.
Superpowers executes but is not, by itself, a spec/context system.
GSD fills that gap so chains stay coherent.

Three-Way Comparison (Problems, Not “Who Wins”)

Dimension	Superpowers	gstack	GSD
Core question	How to get things done	What to do; whether it should	How to keep the project from diverging
Layer	Execution	Decision / roles	Context / spec
Strongest fit	Planning, TDD, acceptance loop	Multi-perspective judgment, review, QA	Context engineering; stable state
Best for	Clear requirements	Think-while-building	Long chains / many iterations
Common pain	Front-loaded process can feel heavy (details below)	Bloated and token-hungry when fully enabled (details below)	Little standalone “shipping” value on its own (details below)
Role	Own execution	Own decision-making	Own long-term context

Common Pain Points in Detail

Superpowers — front-loaded process can feel heavy. Every task, no matter how small, runs through the full cycle: clarify requirements, draft a plan, write tests first, then implement, then verify. For a large feature this rhythm pays off handsomely. For a two-line config fix or a quick copy change, the same ceremony kicks in and you end up spending more time on process than on the actual change. The overhead does not scale down with task size, so small requests can feel disproportionately slow.

gstack — bloated and token-hungry when fully enabled. Each gstack role (CEO, designer, architect, QA, etc.) injects its own perspective and prompts into the context. Turn them all on and a single execution-layer skill can consume 10K+ tokens before any real code is written. Daily usage burns through tokens fast, and the back-and-forth between multiple “virtual team members” can make even straightforward tasks feel sluggish and redundant. You may also encounter irrelevant meta-questions (e.g. “Are you applying to become a YC company?”) while your codebase is being scanned—artifacts of the framework’s opinionated persona layer.

GSD — little standalone “shipping” value. GSD excels at keeping specs, goals, and state anchored across long sessions. But if you use it alone, it does not directly produce code, run tests, or open a PR. It is a stabilizer, not a builder. Without an execution layer (Superpowers) or a decision layer (gstack) alongside it, GSD manages context that nothing acts on—useful plumbing, but no visible output. Its value only becomes apparent when paired with tools that actually ship work.

Practical takeaway: they are complements, not substitutes—Superpowers executes, gstack decides, GSD stabilizes specs and context over time.

Strengths, Weaknesses, and Friction

Superpowers

Strengths: Brainstorming and overall workflow feel solid; full process even on small asks can become smooth once habitual; execution and TDD are strong.
Weaknesses: Weaker spots are often early decision skills (e.g. planning/brainstorming) compared to gstack’s decision layer—hence many people pair gstack’s front end with Superpowers’ execution.

gstack

Strengths: Decision layer—/office-hours, /plan-ceo-review, /plan-eng-review—stand out for positioning and approach review.
Weaknesses: Execution feels rougher vs Superpowers; token cost is real—a single execution-layer skill can cost 10K+ tokens, and heavy scans can feel like noisy “process” rather than help.

The analogy

Superpowers is a scalpel — precise and efficient.

gstack is a full clinic — from diagnosis to aftercare.

Use the metaphor to choose depth: narrow execution vs full-spectrum product and review.

Consolidated Best Practices

1. Choose skills deliberately—do not install everything

Skill counts spiral easily (Superpowers today, gstack tomorrow, another stack next week). Selective deployment beats volume; random invocation feels unstable and inflates surface-level “skill count” without clarity.

Underlying idea: both stacks are experiments in Harness Engineering. The mindset is leverage strengths, cover weaknesses—not “I want it all.”

2. Decision vs execution (the classic split)—then add context when needed

gstack for the decision layer (cherry-picked):

Prioritize high-value flows: e.g. /office-hours, /plan-ceo-review, /plan-eng-review for requirements and alignment—avoid over-investing in every role.

Superpowers for the execution layer:

Prefer Superpowers as the base for TDD, plans-as-executed, verification—optionally de-emphasize its own heavy decision skills if gstack already covers that phase, so small tasks do not inherit double process.

GSD when the chain diverges:

If work spreads across sessions and threads, add GSD so spec and state stay anchored—not for flash, for anti-drift.

3. Stable workflow (three steps)

Decision → gstack — Start with /office-hours to stress-test the idea, then run /plan-ceo-review for a founder-level sanity check and /plan-eng-review to lock architecture and data flow. If design matters, add /plan-design-review. The goal: decide what to build and whether to build it before touching code.
Context → GSD — Once the decision is made, use GSD (v2) to anchor the plan: PROJECT.md for what the project is, DECISIONS.md for architectural choices, KNOWLEDGE.md for cross-session rules and patterns, and milestone roadmaps (M001-ROADMAP.md) for sliced execution. These v2 artifacts keep spec, status, and boundaries stable so context does not rot between sessions. (The original GSD uses REQUIREMENTS.md, ROADMAP.md, and STATE.md instead.)
Execution → Superpowers — With clear requirements and stable context in place, hand off to Superpowers’ execution loop: brainstorming (if lightweight refinement is still needed), writing-plans → executing-plans for implementation, test-driven-development for the RED-GREEN-REFACTOR cycle, requesting-code-review / receiving-code-review for review, and verification-before-completion → finishing-a-development-branch to close the loop. For parallel work, use dispatching-parallel-agents or subagent-driven-development.

Merged tagline: gstack handles thinking, Superpowers handles doing, GSD keeps long context honest. Combining the strong decision slice of gstack with Superpowers’ execution (and GSD when needed) keeps skill count and collisions under control—similar to the author’s experience building a small tool on a weekend with a curated mix.

4. Final heuristics

Requirements still fuzzy → start with gstack (decision).
Work keeps diverging across the chain → add GSD (context).
You want execution steady and closed-loop → lean on Superpowers (execution).

Stop asking only: “Superpowers or gstack?” Ask: Am I missing decision, context, or execution?

Closing:

Skills are not stronger because you install more—they are stronger when you combine the right pieces for the gap you actually have and understand what each layer does, then assemble a workflow that is yours.

References

Superpowers — github.com/obra/superpowers
gstack — github.com/garrytan/gstack
GSD (Get Shit Done) — github.com/gsd-build/get-shit-done (original) | github.com/gsd-build/gsd-2 (v2, standalone CLI)

From IDE to AGaaS: How Cursor Cloud Agents Bring the OpenClaw Model to Your Slack

Yaohua Chen — Tue, 24 Mar 2026 00:07:17 +0000

TL;DR

Cursor's Cloud Agents let you delegate coding tasks — bug fixes, feature work, test writing — directly from a Slack message. The agent spins up a remote VM, clones your repo, writes the code, runs your tests, and opens a Pull Request on GitHub. You never open an IDE. This post walks you through the full setup — from Slack integration to your first hands-off pull request — and then examines where the technology shines, where it falls short, and where the AGaaS market is heading next.

What Is the OpenClaw Model — and Why Should You Care?

OpenClaw refers to an emerging paradigm in AI-assisted development where a cloud-hosted coding agent operates autonomously and headlessly — meaning it doesn't need a local IDE, a human at the keyboard, or even a screen. You give it a task in natural language, and it handles the full software development lifecycle (clone → code → test → commit → PR) on its own.

AGaaS (Agent-as-a-Service) is the broader industry term for this pattern: instead of installing AI tooling locally, you interact with a managed agent through everyday interfaces like Slack, Teams, or a web dashboard.

Cursor's Cloud Agents are a production-ready implementation of this model. If you're already using Cursor as your IDE, you can now step outside the IDE entirely and operate as a manager — assigning tasks from Slack and reviewing the output as Pull Requests.

How Cloud Agents Work Under the Hood?

Before diving into setup, here's what happens when you type @Cursor revise the README.md file to make it more professional and beginner-friendly in Slack:

Headless Execution on Isolated VMs

Traditionally, Cursor ran locally — consuming your RAM, competing for your CPU. Cloud Agents move the execution layer to a remote, isolated Virtual Machine. When a task is triggered, the agent provisions a sandboxed VM, clones your GitHub repository into it, and does all the work in the background. Your local machine stays completely free.

Each VM comes pre-loaded with a production-grade development environment:

Component	Specification
OS	Ubuntu 24.04.4 LTS (Noble Numbat), Linux kernel 6.12.58+, x86_64
Hardware	4 CPU cores, 15 GB RAM, ~126 GB disk (overlay filesystem)
Runtimes	Python 3.12.3, Node.js v22.22.1
Toolchain	Git 2.43.0, GitHub CLI 2.81.0, Bash
Workspace	Your repo cloned at `/workspace`, running as user `ubuntu`

You can verify this yourself by asking the agent about its environment. Here's what that looks like in a real Slack conversation:

Slack Thread as Context Window

This isn't a basic chatbot that only reads your one-line prompt. Cursor's Slack integration behaves like a teammate who's been reading the whole conversation:

If your team has been discussing a bug in a thread — sharing stack traces, debating approaches, pasting logs — the agent ingests all of it when you tag @Cursor.
It synthesizes the thread context and implements a fix that reflects the team's consensus, not just your single message.

Autonomous Testing via "Computer Use"

Because the agent has its own VM with a full desktop environment, it doesn't just write code and hope for the best:

It can start your dev server, open a headless browser, and click through UI flows to visually verify the fix.
If tests fail or the UI breaks, it self-corrects before submitting the Pull Request.

Now that you understand what's happening behind the scenes, let's set it up. The whole process takes about 15 minutes.

Step-by-Step Setup

Prerequisites

Before you begin, make sure you have the following in place:

Requirement	Details
Cursor subscription	Cloud Agents require a paid plan — Pro ($20/mo), Pro+, Ultra, or Teams. Check your plan at cursor.com/pricing.
GitHub account	Your repository must be hosted on GitHub or GitLab. You need read-write access to the repo.
Slack workspace	You need admin permissions (or the ability to request app installation) in your Slack workspace.
Existing test suite	Recommended but not required. The agent can run your tests automatically if they exist (e.g., `npm test`, `pytest`, `go test`).

Step 1: Connect Slack to Cursor

Open the Cursor Dashboard at cursor.com/dashboard.
Navigate to the Integrations & MCP tab.
Click Connect next to Slack. This launches an OAuth flow that installs the Cursor bot into your Slack workspace.
Authorize the requested permissions (read messages in channels where the bot is invited, post replies).

Step 2: Connect Your GitHub Repository

In the same Dashboard, go to the Cloud Agents > Default Repositories > Manage Repositories section.
Click Add Repository and authenticate with GitHub.
Select the repository (or repositories) you want the Cloud Agent to access.
Grant the agent permission to create branches and open Pull Requests.

Step 3: Configure the Cloud Agent Environment

Before triggering tasks from Slack, configure the Cloud Agent's development environment and defaults in the Cursor dashboard. Navigate to Cloud Agents in the left sidebar.

3a. Set Your Defaults

Under the My Settings tab, configure the following:

Setting	What It Controls	Example Value
Default Model	The AI model the agent uses when no model is specified in the task. Higher-tier models produce better code.	`Opus 4.6 High Fast`
Default Repository	The GitHub repo the agent targets when no repo is mentioned in the Slack message.	`chen115y/MLOpsLearning`
Base Branch	The branch the agent creates feature/fix branches from. Leave empty to use the repo's default branch.	`main`
Branch Prefix	Prepended to every branch the agent creates, making agent-authored branches easy to filter.	`cursor/`

3b. Set Up a Development Environment

For repositories with complex dependencies (Python data-science stacks, system libraries, database services), click Add Environment button. This launches a very simple setup agent that provisions and validates the VM:

Once all fields are filled, click Start For Free to start the VM provisioning. The setup agent will analyze the repository and provision the VM accordingly.

Tip: You can add multiple environments for different repos. If the setup agent reports warnings (e.g., deprecated API calls in older notebooks), these are pre-existing code issues, not environment problems — the snapshot is still safe to save.

Step 4: Create a Channel and Invite the Bot

In Slack, create a dedicated channel for agent-assisted work (e.g., #engineering-triage, #cursor-tasks, or #bug-reports).
Simply mention @Cursor in the channel with any prompt — the bot joins automatically when the Slack app is installed (Step 1). No separate invite is needed.
You can also type @Cursor help to see available commands, or @Cursor settings to configure channel-level defaults.

Step 5: Configure Cursor Rules (the Agent's Playbook)

This is the most important step. Without rules, the agent will make reasonable guesses about your codebase conventions. With rules, it follows your team's standards precisely.

Create a .cursor/rules/triage.mdc file in your repository root, for example:

---
description: "Rules for Slack-triggered bug triage and feature tasks"
globs:
  - "**/*"
alwaysApply: true
---

# Agent Behavior for Slack Tasks

## Bug Triage Protocol
1. Read the full Slack thread for context, including any error logs or stack traces.
2. Search the codebase to locate the relevant source files.
3. Identify the root cause before writing any fix.
4. Write the fix following existing code patterns in the repository.
5. Use the project's standard error-handling approach (check for existing wrappers).

## Testing Requirements
- Run the full test suite: `npm run test` (or the project's equivalent).
- If no tests exist for the changed code, write at least one unit test covering the fix.
- Do not submit a PR if tests fail. Debug and fix until green.

## Git and PR Conventions
- Create a new branch from `main` with the format: `fix/<short-description>`.
- Never push directly to `main` or `develop`.
- PR title format: `fix: <concise description of the change>`
- Include a summary of the root cause and fix in the PR description.
- Reply to the original Slack thread with the PR link and a brief explanation.

## Out of Scope
- Do not modify CI/CD configuration files without explicit approval.
- Do not upgrade dependencies unless the fix requires it.
- If the issue is unclear, ask clarifying questions in the Slack thread before proceeding.

You can create additional rule files for different workflows — feature development, refactoring, documentation — each with its own conventions.

Step 6: Run Your First Agent Task

With everything connected, you're ready to give the agent its first job. Post a message in your channel (or reply in an existing thread) and tag @Cursor with a clear task description. The agent picks it up, executes the work on its remote VM, and reports back — all within the same Slack thread.

Here's a real example. A user asks the agent to revise a repository's README to make it more professional and beginner-friendly. Within minutes, the agent replies with a structured breakdown of every change it made — reorganized navigation, plain-language introductions, typo fixes, new formatting — along with the commit diff (+338 / -190 lines):

The user asks the agent to make a commit and push the changes directly to the remote repository. Once the work is done, the agent confirms it has committed and pushed the changes directly to the remote repository, and provides a link to verify on GitHub:

Want to see how the agent reasoned through the task? Click the "Open in Web" button in the Slack message to open the full agent session. This view shows the agent's step-by-step thought process — the file diff it analyzed, the to-do list it created for itself (commit, push), and the detailed revision plan it followed:

And to close the loop, here's the GitHub repository immediately after. Notice the README.md row — updated "1 minute ago" by cursoragent with the commit message matching exactly what the agent described in Slack:

No IDE opened. No branch created manually. No code written by hand. One Slack message in, a polished commit out.

Writing Effective Cursor Rules: A Deeper Look

The example above worked smoothly because the task was straightforward. But as you start assigning more complex work — multi-file refactors, feature additions, cross-cutting bug fixes — the quality of the agent's output depends heavily on how well you've defined your team's standards. That's where Cursor Rules go from "nice to have" to essential.

Step 5 introduced the basic format. Here we'll look at patterns that make rules genuinely effective at scale.

Scope rules by file type. Use the globs field to apply different rules to different parts of your codebase:

---
description: "Frontend component conventions"
globs:
  - "src/components/**/*.tsx"
---
- Use functional components with hooks, never class components.
- All components must have a corresponding .test.tsx file.
- Use the project's design system tokens for colors and spacing.

---
description: "API route conventions"
globs:
  - "src/api/**/*.ts"
---
- Validate all request bodies with zod schemas.
- Return consistent error response shapes: { error: string, code: number }.
- Log errors with the structured logger, not console.log.

Be specific about what the agent should not do. Guardrails prevent expensive mistakes:

## Boundaries
- Never delete database migration files.
- Never modify environment variable files (.env, .env.local).
- If a change requires more than 5 files, stop and ask for confirmation in Slack.

At this point you have the full toolkit: the agent is connected, the environment is configured, and the rules are in place. But having the setup working and knowing where to rely on it are two different things. Let me share what I've learned from using this in practice.

Where Cloud Agents Shine — and Where They Don't (Yet)

The Real Unlock: Work Anytime, Anywhere

Here's what changed my daily workflow more than any single feature: I no longer need to be at my desk, or even awake, for code to get written.

Think about that for a moment. It's 11 PM and a teammate in another timezone drops a bug report in Slack with a Datadog trace attached. Before Cloud Agents, that bug sat untouched until someone opened their laptop the next morning, cloned the repo, reproduced the issue, wrote the fix, ran the tests, and pushed a PR. That's a minimum 30-minute context-switch tax — and that's if the person was already familiar with the code.

Now? I glance at the Slack notification on my phone, type @Cursor investigate and fix this, and go back to sleep. By morning, there's a PR waiting for review with a clear explanation of the root cause. The agent read the error trace, found the offending line, wrote the fix, confirmed the tests pass, and opened the PR — all while I was unconscious.

This isn't just about convenience. It fundamentally changes when and where software development can happen. You can triage bugs from an airport lounge with nothing but your phone. You can delegate a documentation overhaul while you're deep in a design review. You can assign test-writing tasks to the agent on Friday afternoon and come back Monday to a PR that covers the gaps you've been meaning to address for weeks. The agent doesn't get tired, doesn't lose context, and doesn't need to "get back into the zone" after lunch.

What the Agent Handles Well Today

The sweet spot for Cloud Agents is any task where the goal is clearly defined and the scope is contained. Bug fixes are the most natural fit — especially when someone has already done the diagnostic work and there's an error trace, a stack dump, or a reproduction path sitting in the Slack thread. The agent can read that context, locate the relevant source files, and produce a targeted fix without anyone needing to spell out which file to open. It's remarkably good at this.

Test coverage is another area where the agent earns its keep. Most teams know they should be writing more tests, but nobody wants to write the fifteenth unit test for a utility function. Hand that to the agent. It reads the existing code, infers the expected behavior, and generates tests that follow whatever patterns your codebase already uses — pytest, jest, go test, you name it. It's not glamorous work, but it's exactly the kind of high-value, low-creativity task that agents are built for.

Small-to-medium feature additions work well too, as long as the spec is clear. "Add a CSV export button to the billing page that calls the existing exportService" is a great agent task. "Make the app feel more modern" is not — that requires taste, iteration, and subjective judgment that the agent can't provide.

The same applies to code refactoring. If you can describe the before and after state clearly — "rename all instances of getUserData to fetchUserProfile across the codebase" or "extract the validation logic from the controller into a dedicated middleware" — the agent will handle it methodically and consistently. And documentation updates? The agent writes clean, structured prose. Give it a README that's fallen out of date, and it'll cross-reference the actual codebase to produce documentation that matches reality.

Where You Still Need the IDE

That said, Cloud Agents aren't a replacement for sitting down with your code — at least not yet. There are categories of work where human judgment, rapid iteration, and architectural intuition still matter more than raw execution speed.

Large architectural changes are the clearest example. If a task spans multiple services, touches database schemas, modifies CI/CD pipelines, and requires coordinating changes across a dozen files in a specific order, the agent can get lost. It doesn't have the mental model of your system's dependency graph that you've built up over months of working in the codebase. It might fix one file in a way that breaks three others, then chase its tail fixing those. For these tasks, you want a human architect in the driver's seat, possibly using the agent for individual sub-tasks, but directing the overall strategy.

Exploratory prototyping is another area where the agent falls short. When you're experimenting — trying out a new library, playing with different UI layouts, iterating on an API design — you need a tight feedback loop. You write a few lines, run it, see what happens, change direction, try something else. That back-and-forth is the creative engine of prototyping, and it doesn't translate well to "write a Slack message and wait for a PR." The latency alone kills the creative flow.

Security-sensitive code deserves human eyes, full stop. The agent can write functionally correct authentication logic, but it won't catch the subtle timing-attack vulnerability or the OAuth misconfiguration that a security-conscious engineer would flag during a manual review. Use the agent to write the boilerplate, but review every line yourself before it touches production auth flows.

And anything requiring visual design judgment — pixel-perfect UI work, animation tuning, responsive layout decisions — still demands a human with a browser open, resizing windows, and squinting at spacing. The agent can generate the JSX and CSS, but it can't tell you whether the result feels right.

Making the Most of Imperfect Results

Here's a practical pattern that works well: the 90% handoff. The agent doesn't need to produce a perfect PR every time. If it gets 90% of the way there — the logic is right but it missed an edge case, or the implementation is solid but the naming isn't quite what you'd choose — you can pull the agent's remote session directly into your local Cursor IDE and finish the last stretch yourself. You don't start over. You continue right where the agent left off, with all the files already modified and the context preserved.

And when the agent goes in the wrong direction entirely? Course-correct in the same Slack thread. Reply with something like @Cursor stop. The issue is in the middleware, not the controller. Look at src/middleware/auth.ts instead. The agent re-reads the full thread, incorporates your feedback, and adjusts its approach. Think of it less like a tool that either works or doesn't, and more like a junior developer who's fast and tireless but occasionally needs steering.

Going Further: MCP Integrations for Closed-Loop Automation

So far, every workflow in this post has followed the same pattern: a human writes a Slack message, the agent does the work, and a PR appears on GitHub. That's already powerful — but it still requires someone to initiate each task. What if the agent could respond to events across your entire toolchain without waiting for a Slack prompt?

That's where the Model Context Protocol (MCP) comes in. MCP lets the agent interact with external tools beyond Slack and GitHub. By adding MCP servers, you can build a fully closed-loop system:

Jira / Linear: The agent automatically creates a ticket, links it to the PR, and transitions the issue status.
Datadog / Sentry: The agent queries your monitoring tools directly to pull error traces without anyone needing to paste them into Slack.
Confluence / Notion: The agent updates your team's documentation when it changes an API contract.

This turns the workflow from a Slack → PR pipeline into a Slack → Ticket → PR → Docs → Status Update pipeline — with zero manual handoff.

MCP integrations are where Cloud Agents start to feel less like a developer tool and more like infrastructure. And that shift — from tool to infrastructure — is exactly what's happening across the industry.

The Road Ahead: AGaaS and Where the Market Is Going

From Novelty to Infrastructure

What Cursor has shipped with Cloud Agents is impressive, but it's also clearly early. If you zoom out from the specifics of this one product, a much larger shift is taking shape: Agent-as-a-Service (AGaaS) is becoming a real infrastructure category, not just a buzzword.

The core idea is straightforward — instead of every developer installing AI tooling on their local machine and managing prompts, context windows, and model versions themselves, you subscribe to a managed agent that lives in the cloud, integrates with your existing tools, and operates autonomously on your behalf. Cursor is one implementation, but the pattern is bigger than any single vendor.

What Customers Actually Need (and What's Missing)

If you've followed along with this post and tried the setup yourself, you've probably already noticed a few gaps. These aren't criticisms — they're the natural rough edges of a category that's still being defined. But they point directly at where the market is heading.

Multi-repository orchestration. Today, each Cloud Agent task targets a single repository. But real-world features often span a frontend repo, a backend API, a shared library, and an infrastructure-as-code repo. The next generation of AGaaS platforms will need to coordinate changes across multiple repos atomically — opening linked PRs that reference each other and can be merged together.

Persistent agent memory. Right now, each task starts fresh. The agent doesn't remember that it fixed a similar bug last week, or that your team prefers a particular error-handling pattern, or that the last three PRs it opened for this repo all needed the same test fixture adjustment. Future agents will build a persistent understanding of your codebase, your team's preferences, and your project's history — getting better at their job over time, just like a human teammate does.

Richer feedback loops beyond Slack. Slack is a natural starting point because it's where engineering teams already communicate. But imagine triggering agent tasks from a Jira ticket transition, a Sentry alert threshold, a failing CI check, or a monitoring dashboard anomaly. The agent becomes a first-responder that patches issues before a human even notices them. Some of this is possible today through MCP integrations, but it's still manual plumbing — it should be turnkey.

Customizable execution environments at scale. The environment setup flow shown in Step 3 is a solid start, but enterprise teams need more. Think GPU-enabled VMs for ML codebases, pre-configured database fixtures for integration testing, VPN access to internal services, and compliance-scoped environments that restrict which external packages the agent can install. As AGaaS matures, the execution environment will need to match the complexity of real enterprise infrastructure.

Cost transparency and resource governance. When an agent spins up a VM, runs your test suite, and interacts with a paid AI model for 15 minutes, who pays for what? Teams need clear visibility into per-task cost breakdowns — compute, model tokens, API calls — and the ability to set budgets, quotas, and approval gates for expensive operations. This is table stakes for enterprise adoption.

Market Convergence

It's worth noting that Cursor isn't the only player moving in this direction. GitHub Copilot has introduced its own agent mode. Amazon Q Developer (formerly CodeWhisperer) has evolved toward autonomous capabilities. Smaller players like Devin, Cosine, and Factory are building agent-first platforms from scratch. The competitive pressure is accelerating the category.

What's emerging is a spectrum: at one end, lightweight copilot-style suggestions embedded in your editor; at the other end, fully autonomous agents that operate headlessly across your entire development workflow. Most teams will use both, for different tasks, at different times. The interesting question isn't which tool wins — it's how the boundaries between human-driven and agent-driven work shift over the next two to three years.

For engineering leaders, the strategic play is clear: start experimenting now. The teams that build fluency with agent-assisted workflows today — who learn which tasks to delegate, how to write effective agent rules, and how to review agent-produced code efficiently — will have a significant velocity advantage as these tools mature.

References

From Prompts to Real Files: A Developer's Guide to AI File Generation

Yaohua Chen — Mon, 16 Mar 2026 22:11:50 +0000

Ask ChatGPT to "create a sales report PDF with a revenue chart." A year ago, it would paste some markdown and wish you luck. Today, it spins up a sandboxed Python environment, runs reportlab and matplotlib, and hands you a real, downloadable PDF file.

This is the shift from text generation to artifact generation -- and every major LLM vendor now supports it through their API. Claude, OpenAI, and Gemini each give developers a way to prompt an LLM and get back actual files: PDFs, spreadsheets, charts, slide decks, whatever you can create with Python.

This post walks through the universal pattern behind file generation, then shows you exactly how to do it with each vendor -- working code included.

The Universal Pattern

Despite different APIs, all three vendors follow the same three-step architecture:

Every vendor-specific implementation is a variation on this flow. The details change, but three concepts repeat everywhere:

Tool declaration -- you opt in to code execution by including a specific tool in your API request. It's never on by default.
Sandboxed execution -- the LLM's code runs in an isolated container with no internet access. Common libraries (pandas, matplotlib, reportlab) come pre-installed.
File retrieval -- each vendor has a different mechanism to get the bytes out. Some give you a file ID to download; others return bytes inline.

Once you internalize this pattern, learning any vendor's API is just a matter of mapping it to these three steps.

Claude: Code Execution + Files API

Claude's file generation is the most full-featured option for document creation. It provides a persistent container with full bash access, a rich set of pre-installed document libraries, and a clean Files API for uploads and downloads.

Generating a PDF from a Prompt

Enable the code_execution_20250825 tool, send your prompt, then extract file IDs from the response and download them through the Files API.

import anthropic

client = anthropic.Anthropic()

# Step 1: Request with code execution enabled
response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=4096,
    tools=[{"type": "code_execution_20250825", "name": "code_execution"}],
    messages=[{
        "role": "user",
        "content": "Create a one-page PDF sales report with a revenue chart for Q1 2026."
    }]
)

# Step 2: Extract file IDs from the response
file_ids = []
for block in response.content:
    if block.type == "bash_code_execution_tool_result":
        result = block.content
        if result.type == "bash_code_execution_result":
            for item in result.content:
                if hasattr(item, "file_id"):
                    file_ids.append(item.file_id)

# Step 3: Download each generated file
for file_id in file_ids:
    content = client.beta.files.download(file_id)
    metadata = client.beta.files.retrieve_metadata(file_id)
    content.write_to_file(metadata.filename)
    print(f"Saved: {metadata.filename}")

The response content blocks have a nested structure: you're looking for bash_code_execution_tool_result blocks, which contain bash_code_execution_result objects, which contain items with file_id attributes. The files.download() call gives you the raw bytes; retrieve_metadata() gives you the original filename.

Why bash_code_execution? When you include the code_execution_20250825 tool, Claude actually gets two sub-tools: bash_code_execution (run shell commands) and text_editor_code_execution (create and edit files). To generate a file, Claude typically writes a Python script with the text editor sub-tool, then runs it via bash. The result block is named after whichever sub-tool produced the output -- and since it's the bash execution that creates the final file, that's the block type you parse. This is also why Claude has full bash access unlike the other vendors: it's not running Python in a restricted interpreter, it's executing real shell commands. The _20250825 tool version introduced this bash/text-editor split, replacing the earlier _20250522 version that was Python-only.

Uploading a CSV, Getting Back a Chart + PDF

To process your own data, upload via the Files API first, then attach the file to your prompt alongside the code execution tool.

import anthropic

client = anthropic.Anthropic()

# Upload your input file
uploaded = client.beta.files.upload(file=open("sales_data.csv", "rb"))

# Send the file + prompt with code execution
response = client.beta.messages.create(
    model="claude-sonnet-4-6",
    betas=["files-api-2025-04-14"],
    max_tokens=4096,
    messages=[{
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "Analyze this sales CSV. Create a bar chart of revenue by region "
                        "and save it as 'revenue_chart.png'. Also generate a one-page PDF "
                        "summary report of the key findings."
            },
            {"type": "container_upload", "file_id": uploaded.id},
        ],
    }],
    tools=[{"type": "code_execution_20250825", "name": "code_execution"}],
)

# Download all generated files
for block in response.content:
    if block.type == "bash_code_execution_tool_result":
        result = block.content
        if result.type == "bash_code_execution_result":
            for item in result.content:
                if hasattr(item, "file_id"):
                    content = client.beta.files.download(item.file_id)
                    metadata = client.beta.files.retrieve_metadata(item.file_id)
                    content.write_to_file(metadata.filename)
                    print(f"Downloaded: {metadata.filename}")

A single prompt can produce multiple files. In this case, you'll get both the PNG chart and the PDF report. Always iterate the full response -- never assume a single file.

Container Reuse: The Key to Iteration Workflows

Claude containers persist for 30 days. When your first request creates a container, the response includes a container.id. Pass it to subsequent calls and Claude picks up right where it left off -- all files from the previous request are still on disk.
# First call creates the container
response1 = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=4096,
    messages=[{"role": "user", "content": "Generate a sales report PDF."}],
    tools=[{"type": "code_execution_20250825", "name": "code_execution"}],
)
container_id = response1.container.id

# Subsequent calls reuse the same container
response2 = client.messages.create(
    container=container_id,
    model="claude-sonnet-4-6",
    max_tokens=4096,
    messages=[{"role": "user", "content": "Update the chart on page 2 to use a pie chart instead."}],
    tools=[{"type": "code_execution_20250825", "name": "code_execution"}],
)
This enables "conversational file editing" -- users can iterate on documents without re-uploading data or starting from scratch.

Pre-installed Libraries

Claude's sandbox comes with the document generation essentials: reportlab (PDFs), python-docx (Word), python-pptx (PowerPoint), openpyxl (Excel), pandas, matplotlib, pillow, pypdf, pdfplumber, seaborn, scipy, and scikit-learn. Since Claude has full bash access, you can also pip install anything else you need during the session.

OpenAI: Responses API + Code Interpreter

OpenAI's Responses API (the successor to the deprecated Assistants API) uses the Code Interpreter tool for file generation. The pattern is similar to Claude, but the response structure and file retrieval mechanism differ.

Generating a CSV with Code Interpreter

Enable the code_interpreter tool, then parse container_file_citation annotations from the response to find generated files.

from openai import OpenAI

client = OpenAI()

# Step 1: Request with code interpreter enabled
response = client.responses.create(
    model="gpt-5.2",
    tools=[{
        "type": "code_interpreter",
        "container": {"type": "auto"}
    }],
    input="Generate a CSV file named 'q1_report.csv' with 10 rows of financial data."
)

# Step 2: Extract file references from annotations
# The response structure nests deep: output → message → content → output_text → annotations
for item in response.output:
    if item.type == "message":
        for content_block in item.content:
            if content_block.type == "output_text":
                for annotation in content_block.annotations:
                    if annotation.type == "container_file_citation":
                        # Step 3: Download from the container endpoint
                        file_data = client.containers.files.content.retrieve(
                            file_id=annotation.file_id,
                            container_id=annotation.container_id
                        )
                        with open(annotation.filename, "wb") as f:
                            f.write(file_data.read())
                        print(f"Downloaded: {annotation.filename}")

The annotation traversal is the trickiest part. Don't try to shortcut it with response.output_text -- that gives you a plain string with citation markers, not the actual file references.

Uploading a File, Transforming It

Upload via the standard Files API, then pass the file ID in the container config.

from openai import OpenAI

client = OpenAI()

# Upload the file
uploaded = client.files.create(
    file=open("sales_data.csv", "rb"),
    purpose="user_data"
)

# Pass it to code interpreter via container config
response = client.responses.create(
    model="gpt-5.2",
    tools=[{
        "type": "code_interpreter",
        "container": {
            "type": "auto",
            "file_ids": [uploaded.id]
        }
    }],
    input="Analyze this sales CSV. Create a bar chart of revenue by region and save it as a PNG."
)

# Download generated files from annotations
for item in response.output:
    if item.type == "message":
        for content_block in item.content:
            if content_block.type == "output_text":
                for annotation in content_block.annotations:
                    if annotation.type == "container_file_citation":
                        file_data = client.containers.files.content.retrieve(
                            file_id=annotation.file_id,
                            container_id=annotation.container_id
                        )
                        with open(annotation.filename, "wb") as f:
                            f.write(file_data.read())
                        print(f"Downloaded: {annotation.filename}")

You can also request higher memory tiers -- 1g (default), 4g, 16g, or 64g -- by setting "memory_limit" in the container config. Useful when processing large datasets.

OpenAI Gotchas

The cfile_ 404 trap. Generated files have IDs prefixed with cfile_. If you try to download them using the standard client.files.content() endpoint, you'll get a 404. You must use client.containers.files.content.retrieve() instead. This has tripped up every developer at least once.

20-minute container expiry. OpenAI containers are ephemeral -- they expire after 20 minutes of inactivity. Download your files immediately after generation. There is no 30-day persistence like Claude.

Missing annotations fallback. There's a known edge case where container_file_citation annotations don't appear in the response. When this happens, check response.output for items of type code_interpreter_call and inspect their outputs for file references:
if not file_refs:
    for item in response.output:
        if item.type == "code_interpreter_call":
            for output_item in getattr(item, "outputs", []):
                if hasattr(output_item, "file_id"):
                    # Download using output_item.file_id and output_item.container_id
                    pass

Gemini: Inline Results + Structured Output

Gemini takes a fundamentally different approach. It doesn't return downloadable file artifacts with file IDs. Instead, code execution results come back inline -- matplotlib charts as raw image bytes, everything else as text or JSON.

This isn't a technical limitation -- Google has the infrastructure to build containers and file artifact systems. The gap is strategic. Google's file generation story lives in Google Workspace, not in the developer API:

Gemini in Docs generates full first drafts from prompts, matching writing styles and pulling data from Gmail, Drive, and the web.
Gemini in Sheets builds entire spreadsheets from natural language and auto-populates cells with live data.
Gemini in Slides generates themed slides, with full presentation generation from a single prompt on the roadmap.

This makes business sense for Google. Anthropic and OpenAI are API-first companies -- their revenue comes from developers using their APIs, so building sandboxes and file download endpoints directly serves their customers. Google's revenue comes from Workspace subscriptions. When Gemini generates a spreadsheet in Workspace, it creates a Google Sheet (not an .xlsx), keeping users in the Google ecosystem. An API that produces vendor-neutral files would undermine that.

The practical implication: Gemini's API-level file generation gap is unlikely to close anytime soon. The structured output and inline image patterns below are the right long-term approaches, not temporary workarounds.

For developers, this means Gemini is best suited for quick charts and data transforms, while complex document creation belongs with Claude or OpenAI.

Generating a Chart (Inline Image)

Enable the code_execution tool, then extract image bytes directly from the response parts.

from google import genai
from google.genai import types

client = genai.Client(api_key="YOUR_API_KEY")

response = client.models.generate_content(
    model="gemini-2.5-flash",
    config=types.GenerateContentConfig(
        tools=[types.Tool(code_execution=types.ToolCodeExecution)]
    ),
    contents="Generate a bar chart of quarterly revenue: Q1=$2.1M, Q2=$2.8M, Q3=$3.2M, Q4=$3.9M."
)

# Gemini returns results inline -- no separate download step
for part in response.candidates[0].content.parts:
    if part.executable_code:
        print("Code ran:", part.executable_code.code[:80], "...")
    if part.code_execution_result:
        print("Output:", part.code_execution_result.output)
    if part.as_image() is not None:
        with open("revenue_chart.png", "wb") as f:
            f.write(part.as_image().image_bytes)
        print("Chart saved as revenue_chart.png")

No file IDs, no download endpoints. The image bytes are right there in the response. For text/data output, it shows up in code_execution_result.output.

Structured Output for CSV Generation

Gemini's strongest file generation pattern is actually indirect: get structured JSON data back, then format it locally with whatever library you prefer.

import json
import pandas as pd
from google import genai
from google.genai import types

client = genai.Client(api_key="YOUR_API_KEY")

# Ask for structured JSON output
response = client.models.generate_content(
    model="gemini-2.5-flash",
    config=types.GenerateContentConfig(response_mime_type="application/json"),
    contents="Return a JSON array of 10 tech companies with fields: name, ticker, market_cap, sector."
)

# Convert to CSV locally -- you control the formatting
data = json.loads(response.text)
df = pd.DataFrame(data)
df.to_csv("tech_companies.csv", index=False)
print(f"Saved {len(df)} rows to tech_companies.csv")

This "structured output" approach gives you 100% control over formatting and is the most reliable way to produce files from Gemini. Let the model do what it's good at (data generation), and handle the file formatting yourself.

30-Second Execution Timeout

Gemini's code execution sandbox has a hard 30-second timeout. This makes it ideal for quick chart generation and data transforms, but rules it out for heavy document creation tasks like multi-page PDF reports or complex PowerPoint decks. For those, use Claude or OpenAI.

Which API for What?

Feature	Claude	OpenAI	Gemini
Sandbox Type	Reusable container (30-day expiry)	Ephemeral container (20-min idle timeout)	Stateless sandbox (30s timeout)
Resources	5 GiB disk, 5 GiB RAM, 1 CPU	Up to 64 GB RAM (tiered)	Token-limited (inline output)
Shell Access	Full bash	Python only	Python only
File Download	Files API (`files.download()`)	Container endpoint (`containers.files.content.retrieve()`)	Inline in response (no download step)
Best Use Case	Complex documents (PDF, DOCX, PPTX)	Heavy data processing + file gen	Quick charts and data transforms
`pip install`	Yes (bash access)	No (isolated sandbox)	No (isolated sandbox)

The short version:

Complex documents (PDF reports, slide decks, Word docs with formatting): Claude. The pre-installed document libraries and 30-day container persistence make it the best fit.
Large dataset processing (crunching big CSVs, Excel transformations): OpenAI. The ability to request up to 64 GB of RAM is unmatched.
Quick visualizations (charts, graphs, simple data summaries): Gemini. Inline image return means fewer API calls and faster turnaround.
Maximum formatting control: Any model's Structured Output mode. Get JSON data back, render locally with your own libraries.

The Self-Hosted Alternative: Run Your Own Sandbox

The three vendor APIs above all run code in their infrastructure. You send a prompt, they spin up a container, and they hand you back the file. This is convenient, but it means your data leaves your network, you're bound by each vendor's sandbox limits (30-second timeouts, no internet, fixed library sets), and you pay per-execution fees.

There's a fourth option: run the sandbox yourself. In this pattern, you call any LLM API to generate code (without enabling the vendor's code execution tool), then execute that code locally in an isolated environment on your own machines. You get the same prompt-to-file workflow, but you control the execution environment.

Why Self-Host?

Data residency. In regulated industries (healthcare, finance, government), sending code and data to a third-party sandbox may violate compliance requirements. A local sandbox keeps everything on your infrastructure.
No vendor sandbox limits. You choose the timeout, the RAM, the disk, the installed libraries. Need 10 minutes of execution time? A GPU? Network access to internal services? Your sandbox, your rules.
Cost at scale. Vendor sandbox pricing is per-session or per-hour. At high volume, running your own execution infrastructure can be significantly cheaper.
Model flexibility. Since you're decoupling "generate the code" from "run the code," you can use any LLM -- including open-source models, fine-tuned models, or your own -- to produce the Python script. The sandbox doesn't care where the code came from.

Tools for Building It

Two open-source projects have emerged as the leading options for sandboxed code execution:

E2B uses Firecracker microVMs (the same technology behind AWS Lambda) to isolate each execution in its own lightweight VM with a dedicated kernel -- stronger isolation than Docker containers. E2B offers a managed cloud service, but you can also self-host on your own GCP or Linux infrastructure using their Terraform-based deployment. The Python and JavaScript SDKs make it straightforward to spin up a sandbox, run code, and retrieve files programmatically.

exec-sandbox takes the fully-local approach. It runs untrusted code in ephemeral QEMU microVMs with hardware acceleration (KVM on Linux, HVF on macOS). No cloud dependency -- code never leaves your machine. Warm-pool latency is 1-2ms, and it supports Python, JavaScript, and shell execution. It's designed for air-gapped environments where sending code to any external service is a non-starter.

The Architecture Shift

The key difference is that self-hosting decouples code generation from code execution. With vendor APIs, the LLM both writes and runs the code in a single API call. With a self-hosted sandbox, you split these into two steps:

Call the LLM API for text/code generation (no code execution tool needed).
Extract the generated Python script from the response.
Execute it in your local sandbox (E2B, exec-sandbox, or even a locked-down Docker container).
Retrieve the output files from the sandbox filesystem.

Here's a concrete example using E2B as the sandbox and Anthropic as the LLM. Notice there's no code execution tool in the API call -- we just ask Claude to write a script, then run it ourselves:

import re
from anthropic import Anthropic
from e2b_code_interpreter import Sandbox

# Step 1: Ask the LLM to generate a Python script (no code execution tool)
client = Anthropic()
response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=4096,
    messages=[{
        "role": "user",
        "content": "Write a Python script that uses matplotlib to create a bar chart "
                   "of quarterly revenue (Q1=$2.1M, Q2=$2.8M, Q3=$3.2M, Q4=$3.9M) "
                   "and saves it as 'revenue_chart.png'. Return only the script, "
                   "no explanation."
    }]
)

# Step 2: Extract the Python code from the response
code = response.content[0].text
match = re.search(r"```

python\n(.*?)

```", code, re.DOTALL)
if match:
    code = match.group(1)

# Step 3: Execute it in an E2B sandbox
with Sandbox.create() as sbx:
    execution = sbx.run_code(code)

    if execution.error:
        print(f"Error: {execution.error.value}")
    else:
        # Step 4: Download the generated file from the sandbox
        file_content = sbx.files.read("/home/user/revenue_chart.png", format="bytes")
        with open("revenue_chart.png", "wb") as f:
            f.write(file_content)
        print("Saved: revenue_chart.png")

You can swap Anthropic for OpenAI, genai.Client, or any other LLM client -- the sandbox doesn't care where the code came from. You can also upload input files to the sandbox before execution using sbx.files.write(), mirroring the upload-then-process pattern from the vendor APIs.

E2B's default code-interpreter template comes with matplotlib, pandas, numpy, scikit-learn, pillow, openpyxl, python-docx, seaborn, and dozens of other common libraries pre-installed -- similar to the vendor sandboxes. If you need additional packages, you can either install them at runtime with sbx.commands.run("pip install <package>"), or build a custom template with your dependencies baked in so every sandbox starts ready to go.

This is more work to build, but it gives you full control over execution, security, and cost. It also means you can use Gemini or any other model that doesn't offer file artifacts -- you just need the model to write good Python, and your sandbox handles the rest.

Production Tips

If you're building file generation into a real product, a few hard-won lessons:

1. Sanitize filenames. The LLM chooses the filename based on the prompt. A creative user (or an adversarial one) can craft prompts that produce filenames with path traversal characters. Always strip or validate filenames before writing to disk. os.path.basename() is your friend.

2. Handle multi-file responses. A single prompt like "make a PDF report and an Excel spreadsheet of the raw data" can produce two or more files. Always iterate the full response -- never assume exactly one file comes back.

3. Persist container IDs for edit workflows. Claude's 30-day containers enable a powerful pattern: users can say "update the chart on page 2" in a follow-up message, and the LLM picks up the original file from the persistent container. Store the container_id alongside the conversation thread in your database.

4. Set timeouts generously. Code execution is significantly slower than text generation. Simple files might take 30-60 seconds; complex multi-file generation (especially PPTX with embedded charts) can take 5-15 minutes. Don't use your standard API timeout.

5. All sandboxes are offline. None of the three vendors allow network access from within the sandbox. All data must be uploaded or included in the prompt. You can't pip install on OpenAI or Gemini (Claude is the exception -- it has bash access). You can't fetch URLs. Plan accordingly.

Conclusion

File generation via LLM APIs follows a universal pattern across all three major vendors:

Claude excels at complex document creation with its 30-day persistent containers, full bash access, and pre-installed document libraries.
OpenAI offers the most compute headroom with up to 64 GB of RAM, making it ideal for heavy data processing tasks.
Gemini is the fastest path to charts and visualizations, returning inline image bytes with no separate download step.

Try it yourself: Build a CLI tool that takes a prompt and a desired output format, routes to the best vendor based on file type (PDFs to Claude, big data to OpenAI, charts to Gemini), and saves the result locally. You'll touch all three APIs and internalize the patterns in a single afternoon.

Official Documentation

Skills Required for Building AI Agents in 2026

Yaohua Chen — Wed, 25 Feb 2026 19:44:03 +0000

Why Agent Development Is Harder Than You Think

An Agent is conceptually simple: take the one-question-one-answer model of an LLM and add a loop. The model reasons about what to do next, calls external tools, feeds results back into itself, and repeats until the task is complete. A while loop plus tool-calling — that's the skeleton.

But between "working demo" and "production product" lies an engineering chasm. OAuth flows, tool design, error cascading across multi-step tasks, runaway costs, context window management, evaluation, multi-Agent coordination, model capability bottlenecks, and framework trade-offs — these nine challenges are where Agent development actually gets hard. API calls account for roughly 5% of the total effort; the other 95% is everything else.

For a detailed walkthrough of each challenge, see the companion piece: Is AI Agent Development Just About Calling APIs?

The question this post addresses is different: given that Agent development is hard, what skills do you actually need to succeed at it in 2026?

The Skill Shift: From Writing Code to Shaping Problems

Inspired by a Story: How an Intern Outperformed a Senior Engineer?

Shubham Saboo — Senior AI Product Manager at Google Cloud, founder of Unwind AI, and co-author of Google's Introduction to Agents whitepaper — recently shared an experience from a startup where he serves as an advisor. Something happened that overturned everyone's assumptions.

A senior engineer received a task and followed the traditional workflow: understand requirements, design architecture, write code, debug, and test. Three days later, he delivered a technically flawless solution -- clean code, clear logic, fully compliant with engineering standards.

An intern completed the same task in a single afternoon.

It wasn't that the intern had superior technical skills. Quite the opposite -- his coding experience was far less than the senior engineer's. But he did something fundamentally different: he defined the problem clearly enough, then let Claude Code do the rest.

This scenario reveals a harsh reality: when AI can complete implementation-level work quickly and accurately, the bottleneck shifts entirely upstream. The value is no longer "Can you write this code?" but rather "Can you decompose the problem to a level where AI almost never makes mistakes?"

An even more striking example comes from inside Anthropic. They had Opus 4.6 build a C compiler using a team of Agents, then essentially stepped back. Two weeks later, it could run on the Linux kernel -- 100,000 lines of working Rust code, without a single line written by a human.

The researcher leading this project, Nicholas Carlini — a research scientist at Anthropic known for his work on adversarial machine learning — did only one thing: problem decomposition. He broke down the vague goal of "build a compiler" into 16 precisely defined subtasks, each with clear inputs, outputs, and success criteria. Then 16 Agents, each handling its own piece, completed the entire compiler.

The real leverage isn't in writing code -- it's in breaking problems down to the point where AI almost never gets it wrong.

Four Skills That Are No Longer Differentiating

Shubham argues that four capabilities that once commanded high salaries for developers are rapidly losing their power as differentiators — not because they're useless, but because AI has made them table stakes:

Writing code from scratch. Agents write faster and produce fewer bugs. The ability to hand-write code still matters as foundational knowledge, but it's no longer what sets great developers apart.
Boilerplate code and project scaffolding. A single prompt generates them instantly.
Memorizing syntax and APIs. Extended context windows have already solved this problem.
Translating specifications into code. Now, the specification itself is the code.

These skills were once valuable because implementation itself was hard. They required years of training and justified six-figure salaries. But implementation is no longer the bottleneck — it's becoming the easy part.

Yet the entire industry is still optimizing around the old bottleneck. Most companies' job descriptions still emphasize "proficient in Java," "familiar with Spring framework," "5+ years of development experience." These criteria are losing relevance at a visible pace.

Value has migrated to five new skills.

The Five Skills That Truly Matter in 2026

I am tryiing to answer this question. This isn't theoretical speculation -- it's what I has witnessed firsthand when developing AI solutions in the past 2 years, in the open-source community, and through countless experiences building Agents.

1. Problem Shaping

Turning vague goals into executable tasks -- this skill separates people who "play around with AI" from those who actually build products with it.

"Build me a dashboard" is not a task; it's a wish. Problem shaping breaks it into twelve specific, testable subtasks: What data does this dashboard display? What decisions does it support? What must the user understand within the first three seconds? Each sub-problem has clear inputs, clear outputs, and clear success criteria.

When you decompose a vague goal into precise sub-problems, the Agent's execution quality transforms entirely. It no longer needs to guess your intent -- it just follows clear instructions.

How to practice problem shaping:

Start with the desired output and work backwards — what does "done" look like?
For each subtask, define three things: the input it receives, the output it produces, and how you'll know it succeeded.
If a subtask is still ambiguous enough that two people would interpret it differently, break it down further.
Verify your decomposition by asking: could a competent person with zero context about this project execute each subtask from the description alone?

2. Context Design

Agent output quality is directly proportional to the quality of context you provide.

Poor context: "Build me a customer support agent."

Good context: "The target users are SaaS customers considering canceling their subscriptions who have already tried the help documentation but failed. The tone should be empathetic yet efficient -- avoid excessive apologies and robotic responses. Here are 3 real cases that received five-star ratings and 2 cases that received complaints. Edge cases requiring human escalation include: billing disputes over $500, account security issues, and legal compliance matters. The success metric is resolving the issue within 4 messages without escalation."

The difference isn't in prompt engineering tricks. It's in information density, boundary conditions, success criteria, and understanding of real-world scenarios.

A context design checklist:

Who is the target user, and what is their state of mind?
What does the desired tone sound like? Provide 2–3 real examples, not adjectives.
What are the edge cases that require special handling or human escalation?
What does success look like, in measurable terms?
What are the most common failure modes, and how should the Agent handle them?

3. Aesthetic Judgment

When ten options are in front of you, knowing that nine of them won't work.

Shubham recently had Antigravity build a bargaining simulator for his repository: two Agents negotiating a used car deal, each with a distinct personality, live-streaming the entire process. The first version ran perfectly -- clean code, no errors, both sides going back and forth. Technically complete.

He rejected it in thirty seconds.

The interface was just a plain chat window. The negotiation process read like a log file -- no personality tension, no emotional highs and lows, no dramatic moments of "Shark Steve holding the line against Cool-Hand Casey pretending to walk away." It worked as software; it failed as an experience.

An Agent can build anything you describe, but it cannot judge what is worth describing. Agents optimize for correctness; humans optimize for "Would anyone actually want to use this?"

4. Agent Orchestration

Knowing when to use one Agent, when to use multiple, when to run them in parallel, when to run them sequentially, when to add guardrails, and when to let go.

Three core patterns:

Sequential pipeline: Agent A completes its task and passes the output to Agent B. Best for scenarios with dependencies between steps.
Coordinator + specialist team: A lead Agent dispatches tasks and integrates results. Best for complex tasks requiring quality control.
Parallel execution + merge: Multiple Agents handle independent tasks simultaneously, with results consolidated at the end. Best for scenarios with no dependencies between subtasks.

Most people default to sequential workflows because they feel "safer." But knowing when to parallelize and when to introduce a coordinator determines whether your workflow finishes in five minutes or drags on for an hour.

A practical rule of thumb: If two subtasks don't share state — neither reads what the other writes — they can run in parallel. If one subtask's output determines what the next subtask even is, they must be sequential. And if you have more than three parallel Agents whose outputs need to be merged, introduce a coordinator to avoid contradictory results.

5. Knowing When NOT to Use an Agent

Not every problem needs an Agent.

Need to reformat JSON? Hand it to Gemini 3 Flash -- done in ten seconds.
Text replacement across ten files? A lightweight model handles it in seconds.
A bug you already fully understand? Fixing it yourself is faster than explaining it to an Agent.

True capability is matching the right tool to the problem. Complex problems get Agents. Simple problems get models. Obvious problems get your keyboard.

Conway's Law Restructured in the Age of AI

In the classic book The Mythical Man-Month, Fred Brooks proposed a famous insight: a software system's architecture will inevitably mirror the communication structure of the organization that built it. This became known as Conway's Law.

Building AI agents is essentially restructuring Conway's Law with AI.

In traditional software development, the speed of delivering a feature depends on team size, communication efficiency, and technical debt. You need frontend engineers, backend engineers, QA engineers, countless meetings to align requirements, and long develop-test-fix cycles.

In the Agent era, this chain is compressed. One person plus 16 Agents can build a compiler in two weeks. One intern plus Claude Code can accomplish in an afternoon what took a senior engineer three days.

Organizational structure is no longer the bottleneck. The quality of problem definition is.

This is why Shubham says the best developers of 2026 look more like film directors than programmers. They set the scene, cast the actors, and know when to call "cut." They don't write every line of dialogue -- they shape the entire production.

The essence of programming is shifting from "writing" to "orchestrating."

Three Limitations You Must Know

Although Agents sound like magic, you must be aware of three limitations when applying them in practice.

1. Agent quality is highly dependent on problem definition. If you cannot decompose the problem clearly enough, the Agent will consistently produce outputs in the wrong direction. This isn't the Agent's fault -- it's a problem-shaping problem. Before you master this skill, Agents may actually slow you down.

2. Context design requires deep business understanding. Writing a good CLAUDE.md or .cursor/rules file requires you to truly understand the product's worldview, users' pain points, and success criteria. This understanding cannot be rushed -- it can only be accumulated through repeated shipping and observing real user behavior.

3. Aesthetic judgment cannot be learned from books. It comes from repeated shipping, observing real user behavior, and developing sensitivity to the gap between "it works" and "it's worth using." Without this accumulation, Agents will help you rapidly produce a large volume of things that are "technically correct but experientially failed."

State Management: Problem Shaping Applied to Execution

All five skills above come into sharpest focus in one practical engineering challenge: state management. An Agent that can plan is worthless if it can’t track its own progress. Without a progress-tracking mechanism, Agents fall into "hallucination loops" — repeating steps, losing track of the original goal, or confidently declaring a task complete when it’s half-done.

This is where all five skills converge — applied not to a product or a user-facing feature, but to the Agent itself. Each of the four patterns below draws on a different combination of skills:

1. The "Plan-Act-Observe" Loop (ReAct pattern). (Skill #1 Problem Shaping + Skill #2 Context Design) Instead of handing the Agent a giant task list, force it to update its internal state after every single action. The Agent explains what it intends to do (Thought), calls a tool (Action), receives the raw result (Observation), then compares that result against the original plan (Status Update). The loop itself is problem shaping — breaking execution into atomic Thought→Action→Observation cycles. The status update after each cycle is context design — ensuring the Agent's next decision is informed by accurate, structured state rather than stale memory.

2. Dynamic Task Graphs. (Skill #1 Problem Shaping + Skill #4 Agent Orchestration) For complex workflows, static to-do lists break down. Use a directed acyclic graph (DAG) or dynamic task queue where each task carries a status (PENDING, IN_PROGRESS, COMPLETED, FAILED), dependencies are tracked explicitly (Task B doesn’t start until Task A succeeds), and intermediate variables are stored in a scratchpad — like a URL found in Step 1 that’s needed in Step 5. Defining each node with clear inputs, outputs, and success criteria is problem shaping. Deciding which nodes run in parallel versus sequentially, and how results flow between them, is agent orchestration.

3. The Critic Node. (Skill #3 Aesthetic Judgment + Skill #4 Agent Orchestration) In multi-Agent architectures, it helps to have a supervisor that reviews outputs rather than just trusting the worker’s self-assessment. The Worker executes and reports "I’m done." The Critic checks whether the goal was actually achieved. A shared Global State stores the current version of truth. This is the Coordinator pattern from Skill #4 applied to quality control — but the Critic’s evaluation criteria come from Skill #3: knowing when output is "technically correct" but not actually good enough. Without aesthetic judgment baked into the Critic’s rubric, it degrades into a syntax checker.

4. Checkpointing and Self-Correction. (Skill #1 Problem Shaping + Skill #5 Knowing When NOT to Use an Agent) Progress tracking isn’t just about moving forward — it’s about knowing when to turn back. If an observation returns an error, the Agent should update the plan rather than crash — that’s problem shaping in real time, re-decomposing the remaining work based on new information. And if an Agent is 50 steps deep into what should be a 5-step task, it’s "lost in the woods" and needs a reset. Budget monitoring (tokens, turns, or wall-clock time) prevents runaway execution. Recognizing when to abort an Agent run and switch to a simpler tool — or fix the issue manually — is Skill #5 in action.

A practical implementation tip: (Skill #2 Context Design) Prepend a status summary to every LLM call — original goal, completed steps, current step, remaining steps. This is context design at its most literal: engineering the information the Agent sees at every turn. This "external state" acts as a rhythmic beat that keeps the context window focused on the finish line, counteracting the "Agentic Amnesia" problem described in the companion piece.

Putting It Into Practice

I close with a poignant statement: "These skills cannot be acquired through reading. They come from practice."

I offer five concrete exercises:

Review your last five Agent outputs. Write down what you would change and why.
Write a CLAUDE.md for your current project -- even if it only takes 30 minutes.
The next time you face a vague requirement, break it into 10 subtasks before writing a prompt.
Take a sequential workflow and identify which steps can run in parallel.
For one week, log every task where you used an Agent but a simple prompt would have sufficed.

Open your most recent project and ask yourself: Are you spending more time writing code, or shaping problems?

Conclusion

The ten engineering challenges of building AI agents haven't gone away. But the response to them has fundamentally shifted.

Twenty years ago, the scarce resource was implementation skill — the ability to translate an idea into working code. That scarcity justified years of training, specialized hiring, and the entire structure of software teams. Today, Agents handle implementation at speed and quality that rivals senior engineers. The scarce resource has moved upstream: the ability to decompose problems precisely, design rich context, exercise aesthetic judgment, orchestrate multi-Agent workflows, and know when to reach for a simpler tool.

This isn't a prediction about the future. It's a description of what's already happening — an intern shipping in an afternoon, a compiler built without a human writing a single line of code, organizations discovering that their bottleneck is problem definition, not programming talent.

The developers who thrive in this era won't be the ones who write the most code. They'll be the ones who ask the best questions, shape the clearest problems, and know when the Agent's output is good enough — and when it isn't.

The skills have shifted. The question is whether you'll shift with them.

References

Berkeley Function-Calling Leaderboard — Tool-calling accuracy benchmarks across models (~77.5% top accuracy). berkeley-function-call-leaderboard
Galileo Research — Findings on error cascading in multi-step Agent tasks. galileo.ai
LangChain State of AI Agents Report — Survey data on Agent evaluation practices (52% offline evaluation, 37% online evaluation). blog.langchain.dev
UC Berkeley MAST Framework — Analysis of 1,600+ Agent traces showing 41–86.7% multi-Agent failure rates, with 79% of failures from orchestration. arxiv.org
Microsoft Azure SRE Case Study — Production experience scaling from 50+ sub-Agents to 5 core tools. techcommunity.microsoft.com
Anthropic Agent Evaluation Blog (January 2025) — Challenges in systematically evaluating Agent behavior. anthropic.com/research
Nicholas Carlini — C Compiler with Opus — Building a C compiler with 16 Agents producing 100,000 lines of Rust. nicholas.carlini.com
Shubham Saboo / Unwind AI — theunwindai.com
Boston Consulting Group — Research showing fewer than 20% of enterprise Agent projects achieve expected ROI. bcg.com
Alibaba Cloud Engineering Blog — Data showing AI completes 30% of work in production Agent systems, with 70% being tool engineering. alibabacloud.com/blog
Spotify Engineering — Experience with context window limits in code Agent development. engineering.atspotify.com
Manus Team — Four framework rebuilds for context engineering. manus.im
Fred Brooks, The Mythical Man-Month — Origin of Conway's Law and organizational structure insights. wikipedia.org

Is AI Agent Development Just About Calling APIs? Where's the Real Difficulty?

Yaohua Chen — Wed, 25 Feb 2026 19:40:02 +0000

The Bottom Line First

Calling APIs is indeed the entirety of Agent development — just like cooking is indeed putting ingredients in a pot. Technically correct, but it perfectly explains why some people produce Michelin-star dishes while others produce culinary disasters.

Saying the conclusion without explanation is meaningless. Let's actually build an Agent and walk through it together. But before diving in, let's take 30 seconds to clarify what an Agent actually is.

What Is an Agent, Exactly?

The original interaction model with large language models (LLMs) was simple: you ask a question, it gives an answer. One question, one answer, done. If you wanted it to do something complex, you had to manually break tasks into small pieces and feed them one round at a time. You were the "orchestrator"; the LLM was just a passive tool that responded on demand.

What an Agent does is fundamentally one thing: it adds a loop to this question-and-answer model. The model no longer just answers you once. Instead, it judges "what else do I need to do," calls external tools to get results, feeds those results back to itself, thinks about what to do next, and repeats until the task is complete. This loop transforms a large model from a "responder" into an "executor."

Agent Execution Loop:

User Input → LLM Reasoning → Need to call a tool?
                                      │
                    ┌─── Yes ─────────┘─── No ───┐
                    ▼                             ▼
           Select Appropriate Tool         Task Complete?
                    │                             │
                    ▼                        Yes  ▼
           Call External Tool          Return Final Result
         ┌──────────────────┐
         │  Check Emails    │
         │  Check Calendar  │
         │  Create Meeting  │
         └──────────────────┘
                    │
                    ▼
         Get Tool Return Results
                    │
                    ▼
           Update Context ──────────────────────────────┐
                                                        │
                                              (loop continues)

Conceptually, it's that simple. A while loop plus tool-calling capability — that's your Agent skeleton. So many people read this and think, "There's no real technical depth here?" True, the skeleton is simple. But making that loop run stably, reliably, and efficiently in the real world — that is the real engineering challenge.

Let's walk through it for real. Say you want to build an Agent that manages your schedule: read emails, check calendars, arrange meetings. Doesn't sound complicated, right? Let's look at what you encounter at each step.

Step 1: Call the API — Done in 10 Minutes

This step really is easy. Install an SDK, write a few lines of code, pass user input to the model, get back a result. If you've used the OpenAI or Claude API, you could write it blindfolded. You don't even need to write code yourself — open an AI coding tool like Claude Code or Cursor, describe your requirements in natural language, and they'll scaffold the project for you. Define a few tools — check calendar, read emails, create meeting — write the JSON schema, and the model can call them.

It runs. You ask it "what meetings do I have tomorrow?", it calls the calendar tool, gets the result, and reads it back in natural language. Perfect. You think: Agent development isn't that hard, maybe I can ship this in a week.

I've had this feeling before. 20 years ago when I first learned C# development, I dragged a few controls onto a Windows Form and had a running App — I thought Windows Form development was no big deal either.

In theory, those AI coding Agents could handle every step ahead for you too. But in practice, every problem you encounter from here on isn't about how to write the code — it's about what code should be written. To really understand where Agent development gets hard, let's keep walking.

Step 2: Connect to Real APIs — The Nightmare Begins

In the demo you used mock data. Now you need to connect to real email and calendar services. Each user might use something different: Outlook, Gmail, hotmail, etc. Let's simplify and just connect to Microsoft's Graph API — it's accessible domestically and Outlook is mainstream in enterprise.

The first problem arrives immediately: OAuth. Your users must authorize your application to access their Microsoft account. You need to register an app in Azure AD, handle OAuth redirects, securely store refresh tokens, and auto-refresh when tokens expire. None of this has anything to do with the LLM, but without it, your Agent can't take its first step. Microsoft's permissions model alone (delegated permissions vs. application permissions) can eat half a day of research.

Then come the API edge cases. Microsoft Graph returns email lists paginated — 10 items per page by default, up to 50. Your Agent gets the first page without knowing how many more pages exist, and it will give you a confident-sounding conclusion based on just those 10 emails. Ask "did anyone email me last week about Project A?" — the actual email is on page 3, but the Agent confidently tells you "no." You can add a tool to check the next page, but then you need to add a tool to check the next page, and so on.

Rate limiting is another problem. Microsoft Graph's throttling strategy is complex, with different thresholds per app, per user, and per resource type. If your Agent calls it a dozen times in a complex task, it will easily hit a 429 error. What happens then? The model doesn't know what "429 Too Many Requests" means — it just thinks the tool call failed and starts guessing reasons. And this is only for one provider. To build a real product, every provider (Gmail, hotmail, etc.) has its own authentication system and API design. The workload multiplies.

The Tool Design Problem: Connecting the API is only half of the tool-call equation. The other half is how to design the tool itself — and this is trickier than you'd expect.

What should your "search emails" tool look like? If it's too rigid — only supporting sender-based queries — a user saying "find last week's emails about Project A" will fail immediately. So you add keyword search, time range filtering, attachment filtering? The more parameters, the more complex the schema, and the more likely the model is to fill things in wrong or miss fields. Berkeley's Function-Calling benchmark found that the more tools and the more complex the parameters, the worse model accuracy becomes. Smaller models degrade dramatically as tool count grows — BFCL data shows that models like Llama 3.1 8B can handle a modest number of tools but start failing unpredictably once tool count exceeds their capacity threshold.

On the other end, if you design a generic "search" tool that covers everything, the model won't know what to put in it. It might pass calendar query parameters to the email search tool, or call "send email" when it should "create a meeting." There's no right answer for tool granularity — too fine and user needs aren't covered, too coarse and the model can't handle it. The only way is to iterate in your specific context.

Tool description text matters enormously. For the same functionality, a description written as "Search emails" vs. "Search the user's Outlook inbox by keyword, sender, date range, or attachment presence. Returns a list of matching emails sorted by date" produces dramatically different model accuracy. In short, you don't just need to write code to implement a tool — you need to learn to write a manual for the model, and whether that manual is good or bad, you can only verify through repeated testing.

A lot of research puts it clearly with data: in production-grade Agent systems, AI completes only 30% of the work, and the remaining 70% is tool engineering. What you think of as "calling an API" is mostly spent on the design and integration work surrounding that API.

Step 3: Multi-step Tasks — Errors Start Snowballing

Good — the API is connected and basically working. Now try a slightly more complex request: "Find a time slot next week when everyone is free, schedule a project review meeting, and then email all attendees."

This task requires: querying multiple people's calendar availability, finding the intersection, creating a meeting invite, drafting an email, and sending it. Five or six steps, each depending on the previous one's result.

Here's the problem. Berkeley's Function-Calling Leaderboard (BFCL) shows that even the best models struggle with tool-calling accuracy — top scores hover around 80% on overall benchmarks, and accuracy drops further as tool count and parameter complexity increase. That means roughly 1 in 5 calls has an error. The probability of a five-step task completing entirely correctly? About 0.8 to the fifth power — less than 33%. Your Agent has roughly a two-thirds chance of going wrong at some step.

Worse, Galileo's research found that early small errors amplify through later steps. Say the model misparses a date format in step one and reads Tuesday as Wednesday. Every subsequent step builds on that error. It creates a meeting at the wrong time, then sends everyone an email notification with the wrong time. One small hallucination triggers a cascade of wrong actions.

At this point you realize: you need to add validation logic between each step, rollback mechanisms, and confirmation loops. None of this is taught in any LLM's API documentation.

Step 4: Guardrails — The Invisible Security Risk

And there's a deeper problem lurking here that most people don't think about until it's too late: guardrails. Your scheduling Agent has permissions to send emails, create meetings, and modify calendars. What happens when it hallucinates a participant name and sends a meeting invite to the wrong person? Or confidently deletes a calendar block because it "optimized" your schedule?

OWASP classifies this as "Excessive Agency" (LLM06:2025) — one of the top security threats in LLM applications. It breaks down into three failure modes: excessive functionality (your Agent has access to 50 actions when it only needs 5), excessive permissions (your Agent can modify any calendar, not just the user's), and excessive autonomy (the Agent sends emails and creates meetings without any human confirmation gate).

In practice, you need to separate "read" tools from "write" tools and put explicit approval gates on write operations. High-stakes actions — sending external emails, deleting calendar entries, modifying shared resources — should run in a "dry run" mode where the Agent describes what it would do and waits for human confirmation before executing. You need to design for rapid rollback, because the question isn't if your Agent will take a wrong action — it's when. And you need to enforce the principle of least privilege: your Agent should request only the minimum API permissions it needs, not broad access "just in case."

None of this is glamorous engineering. But skip it, and one hallucinated email from your Agent can undo months of user trust.

Step 5: Open It to Real Users — The Bill Scares You Awake

You tested the first three steps in your development environment and things seemed fine. But once you open the Agent to real users, the nightmare comes from a direction you never anticipated: the bill.

You used Claude Sonnet or GPT-4o for development testing — great results, a few cents per complex task, no pain. But with real users, hundreds of requests per day, each averaging four or five tool call rounds, each carrying substantial context — you look at the monthly bill and see a small feature burning thousands of dollars a month. What if user volume grows ten times?

You think: a user saying "what meetings do I have tomorrow?" — does that really need the most powerful model? That's overkill.

So you start thinking about model routing: different tasks use different base models. Simple queries go to cheap small models (Haiku, GPT-4o mini, Gemini Flash); complex multi-step reasoning goes to large models (Claude Sonnet, GPT-4o, Gemini Pro). But who judges complexity?

Use a large model to judge? That costs money too.
Use a rule engine? Works for simple cases, but user inputs are endlessly variable and rules always have gaps.
Use a small model as a classifier? Now you've added another model component that needs tuning and maintenance.

And different models vary enormously in their tool-calling capabilities. A tool schema that works on Claude Sonnet may have parameters filled in wrong on Haiku. JSON that runs perfectly on GPT-4o may fail to parse on open-source models. Every time you swap a model, your carefully tuned prompts and tool descriptions may need to be re-adapted. This is why many teams eventually find that the token money saved doesn't cover the labor cost of multi-model adaptation.

To put concrete numbers on this: Claude Sonnet costs \$3/\$15 per million input/output tokens, while Claude Haiku costs \$0.25/\$1.25 — a 12x to 60x difference. GPT-4o vs. GPT-4o mini has a similar spread. Mid-sized Agent deployments easily burn \$1K–\$5K per month in token costs alone; complex Agents consuming 5–10 million tokens monthly aren't unusual. One underrated optimization: prompt caching. Anthropic's prefix caching can reduce costs by up to 90% and latency by 85% for repeated long prompts — a massive win for Agents that include the same system prompt and tool definitions in every call.

And cost isn't the only scaling problem — latency hits you just as hard. A multi-step scheduling task that checks four people's calendars, finds a common slot, creates a meeting, and sends emails can easily take 30–45 seconds end-to-end. Technically correct, but your users experience it as broken. The biggest UX win is streaming intermediate results: instead of a 45-second black box, show "Checking Alice's calendar... Found 3 available slots... Confirming with Bob..." — the total time is the same, but the perceived wait drops dramatically. Parallelizing independent tool calls (check all four calendars simultaneously instead of sequentially) helps with actual latency. But the hard tradeoff remains: smaller, faster models hallucinate more, so you can't just throw Haiku at everything to speed things up.

Cost optimization looks like an operations problem, but it's actually an architecture problem. You need to make the model-calling layer pluggable from the very beginning — something most people never think about when writing a demo.

Step 6: Context Management — Your Agent Starts "Forgetting"

After a while, you notice a strange problem: the Agent "drifts" during long tasks. You give it a complex task requiring seven or eight conversation rounds, and by rounds four or five, it starts forgetting the original requirements and constraints.

This is what the industry calls "Agentic Amnesia." Research data is clear: when tasks are split across multiple conversation rounds, model performance degrades significantly — and without memory management strategies, Agents lose track of constraints, requirements, and earlier results as context accumulates.

The reason is that LLM context windows are finite. Every tool call's input and output consumes context space. Query five people's calendars, each returning a large JSON payload, and the context window is mostly full. Spotify's engineering team hit the exact same pitfall building a code Agent: once the context window filled up, the Agent "lost its direction" and forgot the original task after a few rounds.

You need to start doing Context Engineering. Anthropic defines it as "curating exactly what content goes into a limited context window from an ever-changing universe of information." In plain terms, it's the LLM version of memory management: you dynamically decide what the model "sees" at each reasoning step and what it "forgets." Which historical information gets compressed into summaries? Which key constraints must always be preserved? Which tool return values can be discarded?

The Manus team rebuilt their entire framework four times to get this right. Four times. They called this process "stochastic gradient descent" — inelegant, but effective.

There's also a subtler trap: research shows context length and hallucination rate are positively correlated. The longer the input, the more likely the model is to hallucinate. For Agent tasks that require large contexts, this is nearly an unresolvable structural paradox.

One emerging solution to this problem is Agent Skills, a mechanism pioneered by Anthropic. Where Context Engineering is about managing what's in the context window, Skills are about not putting things there in the first place. A Skill is a modular package of instructions, workflows, and best practices (typically a SKILL.md file plus optional scripts) that an Agent loads on demand. Think of it as pluggable expertise — a "Tax Compliance Skill" or a "Cloud Migration Skill" that transforms a general-purpose Agent into a domain specialist, without bloating the context window for every other task.

The design uses progressive disclosure: an Agent can have dozens of Skills installed but only loads the 2–3 it needs for any given task. This directly mitigates the context window pressure that causes Agentic Amnesia. Skills also enable composability — combining a code-review Skill with a git-automation Skill produces an Agent that can review and commit code without anyone writing explicit coordination logic.

The impact on the ecosystem has been rapid. OpenAI adopted structurally identical Skills for ChatGPT and Codex CLI. Microsoft's Semantic Kernel implements an equivalent "Plugins" abstraction. Marketplaces like SkillsMP have emerged with hundreds of thousands of community-built Skills. Anthropic has positioned Agent Skills as an open standard — and the convergence across platforms suggests it's becoming the standard abstraction for packaging Agent capabilities, much like MCP became the standard for Agent-to-tool communication.

Step 7: Want to Test It? You Don't Even Know How

At this point, your Agent barely works. But how do you determine whether it's "actually good" vs. "just barely functional"?

Traditional software development has mature testing methodologies: unit tests, integration tests, end-to-end tests — inputs are deterministic, expected outputs are deterministic. But an Agent's input space is open-ended (users can say anything) and its output is non-deterministic (the model generates different text each time). LangChain's blog put it perfectly: "every input is an edge case" — a challenge traditional software has never faced.

You might think to use LLM-as-judge to evaluate LLM outputs. A Hacker News developer explained the problem clearly: using a judge with the same architecture as the system being tested maximizes the probability of systematic bias. The judge and the tested Agent share exactly the same blind spots.

Anthropic's January blog also acknowledged: Agent interactions involving tool calls, state modifications, and behavior adjustments based on intermediate results are precisely the capabilities that make Agents useful — and simultaneously make them almost impossible to evaluate systematically.

The data is stark. LangChain's State of AI Agents survey (1,300+ professionals, 2025) found only about half of organizations run offline evaluations, and fewer than a quarter combine both offline and online evaluations. A multi-dimensional analysis of major Agent benchmarks found a 37% performance gap between lab testing and production environments — with reliability dropping from 60% to 25% in real-world conditions. An Agent that tests great in your dev environment may behave completely differently in users' hands.

Anyone who's done client-side development will understand this pain: your Agent might handle a request perfectly today, and fail on the same request tomorrow. Users can accept missing features — they can't accept inconsistency.

And evaluation is only half the story — the other half is observability in production. Evaluation tests what you expect the Agent to do; observability shows what it actually does with real users. When a user reports "the Agent scheduled my meeting at the wrong time," you need to trace back through every tool call: what calendar data was retrieved, what the LLM reasoned, what meeting parameters were generated, and why the wrong time was selected. Without tool call tracing, latency monitoring, and cost/token budget tracking, you're debugging blind. That "37% performance gap" between lab and production? Observability is how you find it. Tools like LangSmith and Arize have emerged specifically for this, but many teams still discover production failures only when users complain.

Step 8: Add Multi-Agent Collaboration? Complexity Explodes

Your scheduling Agent is working well, and you start thinking: could you add more specialized Agents? One for email, one for calendar, one for meeting notes, one for scheduling coordination. Clear division of labor, each handling its domain — sounds reasonable, right?

Microsoft's Azure SRE team went down this path. They initially built a massive system with 100+ tools and 50+ sub-Agents, and hit a pile of unexpected problems: the orchestrator Agent couldn't find the right sub-Agent (the correct one was "buried three hops away"); a buggy sub-Agent didn't just crash itself — it dragged down the entire reasoning chain; Agents kicked responsibility back and forth in infinite loops. They eventually scaled down to 5 core tools and a few general-purpose Agents, and the system became more reliable.

Their core lesson: scaling from one Agent to five doesn't multiply complexity by four — it grows exponentially. UC Berkeley's MAST framework analyzed 1,600+ Agent traces and found that 41–86.7% of multi-Agent systems fail in production, and 79% of problems come from the orchestration and coordination layer, not the technical implementation. How to divide work and how to communicate between Agents is far harder than how to write the code.

There are established orchestration patterns — sequential chains, concurrent fan-out, hierarchical supervisor models — and each has tradeoffs. ICLR 2025 research found that hierarchical architectures (one coordinator delegating to specialists) show only a 5.5% performance drop when individual Agents malfunction, compared to 10.5–23.7% for flatter architectures. This explains why Microsoft eventually simplified to a supervisor model. The practical advice is almost counterintuitive: start with fewer, more capable Agents rather than many specialized ones, and only decompose when a single Agent demonstrably can't handle the workload. The allure of clean role separation is strong, but the coordination overhead will eat you alive.

Step 9: You Start Doubting — Where's the Bottleneck?

After months of work, your engineering gets more refined, but Agent performance always hits a ceiling you can't break through. You realize a harsh truth: all engineering optimization has one prerequisite — the underlying model needs to be capable enough.

An InfoQ interview with Alibaba Cloud's code platform lead captured it honestly: engineering challenges can be overcome, but model capability bottlenecks are far more daunting. An awkward industry reality: nearly every company building general-purpose Agent products uses Claude Sonnet as their first-choice model, because other models lag noticeably on instruction-following in complex tasks. The more instructions a model can follow, the more complex the problems it can handle. When a model can't even do basic instruction-following, no amount of engineering optimization above it helps.

You might think: what about using more powerful reasoning models — o3, o4-mini, DeepSeek R1, Claude Sonnet, Claude Opus? Research finds that reasoning models hallucinate more than base models. The data is striking: OpenAI's o3 has a 33% hallucination rate on person-specific factual questions — double the rate of its predecessor o1. The o4-mini reasoning model hits 48%. The root cause is that RL fine-tuning for chain-of-thought reasoning introduces high-variance gradients and entropy-induced randomness, making models more confident even when wrong. They answer rather than admit uncertainty.

The practical implication for Agents: reasoning models may handle complex task decomposition better, but they trade off reliability on factual tasks. One emerging pattern is to use reasoning models for planning (breaking down what needs to happen) and base models for execution and verification (actually doing it and checking the results). But this adds yet another layer of architectural complexity.

It's like finding your app is laggy, spending days optimizing code logic, and then discovering the bottleneck is hardware performance. Your engineering optimizations have limits, and beyond those limits lies the constraints of underlying capability.

Step 10: You Start Understanding the Framework Wars

At this point, you've definitely wrestled with whether to use LangChain, CrewAI, or similar frameworks. The Hacker News discussion has moved from debate to consensus: frameworks are useful for prototyping; in production they often become a burden.

A CTO shared on Hacker News that he built hundreds of Agents without any framework, using only chat completions plus structured output.

Anthropic's official guidelines also advise caution with frameworks, as they often make underlying prompts and responses opaque and harder to debug.

Here's the practical landscape: LangGraph (by LangChain) uses a graph-based architecture with nodes, edges, and conditional routing — it's powerful for complex multi-step reasoning and is used in production by 400+ companies. CrewAI takes a role-based approach where you define Agents by organizational roles — simpler to set up, adopted by 60% of the Fortune 500 for content generation and analysis workflows. AutoGen (Microsoft) was merged into the Microsoft Agent Framework in late 2025, reflecting a broader trend of frameworks consolidating. Each imposes its own abstractions, and those abstractions become constraints the moment your use case doesn't fit neatly.

There is one thing you genuinely need frameworks for: persistence and state management. Your Agent needs to pause while waiting for user confirmation, recover from checkpoints after errors, and resume long tasks mid-execution. Most lightweight solutions lack these capabilities — which is why orchestration engines like Temporal have risen in the Agent space. Temporal provides durable execution with an append-only event history, letting Agents recover from failures mid-execution. That's genuinely hard to build from scratch.

Perhaps more consequential than any framework is the emerging protocol and abstraction layer — three complementary standards that are reshaping how Agents are built and composed:

Model Context Protocol (MCP), created by Anthropic, standardizes how models interact with external tools and data sources. Instead of writing custom integrations for every API, MCP provides a universal interface with well-defined security boundaries. It's the "USB port" for Agent-to-tool connections.

Agent2Agent (A2A), backed by Google and Microsoft, tackles inter-Agent communication — enabling Agents from different providers and frameworks to discover each other and collaborate via standardized protocols. It's the "HTTP" for Agent-to-Agent interactions.

Agent Skills, pioneered by Anthropic (discussed in Step 6), solve a different problem entirely: domain knowledge and procedural expertise. MCP gives Agents access to tools; Skills give them the knowledge of how to use those tools effectively — modular, on-demand expertise that keeps context windows lean through progressive disclosure.

Together, these three layers — MCP (Agent-to-tool), Agent Skills (Agent knowledge), and A2A (Agent-to-Agent) — form a cohesive architecture. Developers building production Agents will likely use all three: MCP to plug into APIs and databases, Skills to inject domain expertise, and A2A to enable cross-ecosystem Agent collaboration. This matters more than framework choice in the long run, because these protocols define how Agents interoperate — regardless of what framework built them.

The truth is, framework choice isn't the core challenge of Agent development. The real challenges are the nine steps above. Frameworks are just tools. Choosing the wrong tool wastes time, but going in the wrong engineering direction wastes everything.

Conclusion

The ten steps above aren't something I made up sitting here. I built agents myself, hit almost every pitfall listed, and some of the projects ultimately failed. The Agent worked flawlessly in my development environment — but in production, context window limits caused it to lose track of multi-step tasks, costs spiraled because I hadn't designed for model routing, and I had no observability to diagnose why users were getting wrong results. By the time I understood the real scope of the engineering required, the project had burned through its budget and patience. Looking back, the mindset of "it's just calling an API, how hard can it be?" was exactly the same as my mindset 20 years ago of "drag a few controls and you have an app." What really taught me, in the end, was that failure.

Walk through these ten steps and you'll find that "calling APIs" accounts for roughly 5% of total Agent development effort. The other 95% is:

OAuth, rate limiting, and error handling in the tool layer (Step 2)
Getting tool design granularity and descriptions right (Step 2)
Validation and rollback for multi-step error cascades (Step 3)
Safety guardrails, least-privilege permissions, and human-in-the-loop gates (Step 4)
Cost control, prompt caching, model routing, and latency optimization (Step 5)
Context Engineering, memory management, and Agent Skills for progressive disclosure (Step 6)
Building evaluation and production observability from scratch (Step 7)
Complexity control for multi-Agent orchestration and coordination (Step 8)
Engineering around model capability ceilings and reasoning model tradeoffs (Step 9)
Navigating the framework/protocol landscape — MCP, A2A, and Agent Skills (Step 10)

LangChain calls this emerging discipline "Agent Engineering" — I think that's exactly right. Boston Consulting Group's research shows that only about a quarter of companies achieve significant ROI from their AI initiatives, and Agent projects are no exception. LangChain's survey found that 32% of companies cite "quality below standard" as the top barrier to shipping an Agent. These numbers say it all.

The enormous gap between Agent and Agent doesn't come from who's calling different APIs — it comes from the vastly different quality of the 95% of engineering that happens outside the API call. Calling an API is the entry threshold, something you can cross in a week. But between demo and product lies an entire system of engineering around reliability, observability, context management, and error recovery.

That's where Agent development is truly hard.

References

Berkeley Function Calling Leaderboard (BFCL) — Tool-calling accuracy benchmarks across models
Galileo: 7 AI Agent Failure Modes and How To Fix Them — Error propagation in multi-step Agent tasks
LangChain: State of AI Agents Report (2025) — Industry survey on Agent evaluation and adoption
Beyond Accuracy: Multi-Dimensional Framework for Enterprise Agentic AI — Lab vs. production performance gap analysis
Context Engineering for Reliable AI Agents: Lessons from Building Azure SRE Agent — Microsoft's experience with 100+ tools and 50+ sub-Agents
Why Do Multi-Agent LLM Systems Fail? (UC Berkeley MAST Framework) — Analysis of 1,600+ Agent traces and 14 failure modes
Are Reasoning Models More Prone to Hallucination? — Comparison of hallucination rates in reasoning vs. base models
Spotify Engineering: Context Engineering for Background Coding Agents — Context window management lessons
Manus: Context Engineering for AI Agents — Four framework rebuilds and iterative context design
Anthropic: Effective Context Engineering for AI Agents — Defining and implementing Context Engineering
OWASP: LLM06:2025 Excessive Agency — Security threat classification for Agent systems
BCG: How Agents Are Accelerating the Next Wave of AI Value Creation — Enterprise AI ROI data
On the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agents (ICLR 2025) — Hierarchical vs. flat architecture resilience
Anthropic: Prompt Caching — 90% cost reduction and 85% latency reduction for repeated prompts
Anthropic: Equipping Agents for the Real World with Agent Skills — The original Agent Skills mechanism and design philosophy
Agent Skills: Anthropic's Next Bid to Define AI Standards — Skills as an open standard for modular Agent capabilities
Agent Skills vs MCP: Two Standards, Two Security Models — Complementary roles of Skills and MCP in Agent architecture

AI Agent Memory Management - When Markdown Files Are All You Need?

Yaohua Chen — Wed, 18 Feb 2026 02:15:15 +0000

What is Memory Management for AI Agents?

Memory management for AI agents refers to the mechanisms that allow an agent to store, retrieve, and use information across interactions. Without memory management, every conversation starts from a blank slate — the agent is stateless and forgets everything between sessions. With it, the agent accumulates knowledge over time, learns from past mistakes, and maintains continuity — becoming truly stateful.

What are the Memory Types for AI Agents?

Short-term - The agent's immediate context window, holding the current conversation and recent tool outputs. Analogous to a human's active attention span. Duration: minutes.
Long-term - Persistent storage of facts, preferences, and decisions that survive across sessions. Analogous to human declarative memory. Duration: indefinite.
Procedural - Learned workflows, action sequences, and "how-to" knowledge the agent acquires through experience. Analogous to human muscle memory or learned skills. Duration: permanent once codified.
Working - A temporary scratchpad for intermediate reasoning steps during a single task. Analogous to a mental whiteboard used for chain-of-thought reasoning. Duration: seconds to minutes.

Comparison of Memory Types in Agents

Memory Type	Duration	Typical Implementation	Primary Use Case
Short-Term	Minutes	Context Window / RAM	Following a conversation thread.
Long-Term	Years	Vector DB / SQL	Remembering user preferences and facts.
Procedural	Permanent	Action Recipes / Logs	Learning "how" to use a specific tool or API.
Working	Seconds	Scratchpad / State	Intermediate reasoning steps (Chain-of-Thought).

What are Use Cases for AI Agent Memory Management?

Memory management is the "glue" that transforms a basic chatbot into a functional AI agent. While simple models process prompts in isolation (stateless), agents with memory can track goals, learn from mistakes, and personalize their behavior over time.

Effective memory management generally involves balancing Short-Term Memory (immediate context), Long-Term Memory (historical facts and patterns), Procedural Memory (refined workflows), and Working Memory (intermediate reasoning steps).

Personal AI Assistants & Companions - Agents like virtual executive assistants must manage memory to provide a "human-like" continuity.
Multi-Step Research & Coding Agents - Agents designed for "deep research" or complex software engineering (e.g., Devin or OpenDevin) navigate thousands of lines of code or documents.
Customer Support Automation - Modern support agents handle issues that may span several days or multiple channels (email, chat, phone).
Autonomous DevOps & CI/CD Agents - Agents managing cloud infrastructure or deployment pipelines need memory to understand the state of a complex system.
Healthcare & Patient Management - AI agents in healthcare act as long-term monitors for chronic conditions.

What are the Existing Approaches?

When designing a smart AI agent, memory management determines whether your agent is "forgetful" (stateless) or "intelligent" (stateful). Some AI agent frameworks like LangChain and LangGraph have built-in memory management, while others like OpenAI and Google ADK have their own memory management systems. Each framework approaches memory with a different philosophy—some prioritize ease of use (OpenAI), while others prioritize granular control (LangGraph).

Comparison: Memory Management Architectures

Framework	Primary Memory Strategy	Persistence Level	Best For...
LangChain	Modular Components (Buffer, Summary, Entity)	Manual (must connect DB)	Diverse, specialized RAG workflows.
LangGraph	Graph Persistence (Checkpointers)	Built-in (Thread-level)	Complex, cyclical tasks (e.g., self-correcting code).
Google ADK	Memory Bank (Identity-scoped)	Fully Managed	Personalized, long-term user context on GCP.
CrewAI	Unified Multi-Layer (Short, Long, Entity)	Built-in (SQLite/Chroma)	Multi-agent collaboration and role-playing.
OpenAI SDK	Threads API	Fully Managed (Opaque)	Rapid prototyping; hands-off state management.

Is There a Simpler Alternative?

In December 2025, Meta acquired Manus for $2 billion. The startup was just 8 months old with a small team. Industry insiders speculated: "They must have revolutionary AI algorithms... proprietary models... breakthrough technology..."

The truth was far more interesting—and far simpler.

Their competitive advantage wasn't complex algorithms or massive infrastructure. It was how they managed memory using plain text files.

While the AI industry spent millions building vector databases, complex RAG pipelines, and proprietary memory systems, three independent high-value projects quietly converged on the same "boring" solution:

Manus (acquired for $2B) - Used file-based planning for long-running agents. Its agents followed a three-file pattern: task_plan.md for goals and progress, notes.md for research, and a deliverable output file.
OpenClaw (145K+ GitHub stars) - Built dual-layer Markdown memory architecture. It uses MEMORY.md for curated knowledge, memory/YYYY-MM-DD.md for daily logs, and SOUL.md for personality.
Claude Code (Anthropic's official tool) - Implemented Skills and memory as Markdown files. It uses a CLAUDE.md hierarchy for project context, .claude/MEMORY.md for auto-captured learnings, and a Skills system for on-demand capability loading.

This convergence suggests something fundamental about what works in practice. In biology, this is called convergent evolution — when independent organisms develop the same trait because it is the optimal solution to a shared challenge. While many AI systems rely on elaborate memory infrastructure, file-based approaches offer a simpler alternative that addresses the core requirements: persistence, transparency, and reliability.

Using local Markdown files for memory management—an approach popularized by tools like OpenClaw, Claude Code, and Manus—offers a philosophy of "Memory as Documentation." This contrasts sharply with the "Memory as Database" approach of frameworks like LangGraph or CrewAI.

This approach treats the agent's memory not as a hidden system state, but as a transparent, editable file living directly in the user's workspace.

Why File-based Memory Works?

File-based memory systems work because they align with how developers already manage information. Here are the key properties that make them effective for AI agents:

Persistent: Memory survives agent restarts, crashes, or updates. Files decouple memory from process lifecycle — no data loss when a process dies.

Transparent and Editable: You can open the agent's memory file (e.g., MEMORY.md or task_plan.md) in any text editor, read exactly what it "knows," and edit it manually. In LangGraph or CrewAI, modifying memory often requires writing scripts to update a database or decoding complex JSON objects. With Markdown, if the agent hallucinates a goal, you simply highlight the text and delete it. This zero-friction "human-in-the-loop" capability builds trust and enables compliance audits.

Version-Controllable: Because memory is plain text, it lives in your Git repository. You can commit the agent's "knowledge," revert changes if the agent goes off-rails, and branch the memory. Frameworks like CrewAI usually store memory in external databases (Postgres, ChromaDB) — syncing that external state with your code's version history is difficult. Markdown memory treats context as part of the codebase.

Holistic Context: Agents like Claude Code use Markdown to maintain a high-level summary of the project structure. They read this file first to orient themselves. RAG (Vector Databases) retrieves fragments based on similarity search, which often misses the "forest for the trees" — fetching specific functions but missing the overall architectural pattern. A curated Markdown summary solves this by forcing the agent to maintain a "map" of the project.

Portable: Standard Markdown format means no vendor lock-in. Your agent's memory is not locked into OpenAI's thread_id or a proprietary vector store. You can swap the underlying model (e.g., switch from Claude to GPT-4o) and simply feed it the same Markdown file. Migration is as simple as copying files.

Searchable: Standard text search tools (e.g., grep, ripgrep) work immediately — no special database required. More advanced approaches like full-text search or vector embeddings can be added as the memory grows.

Cost-effective: Local disk storage costs \$0.02/GB/month compared to managed vector database services at \$50-200/GB/month. No per-query API fees or infrastructure scaling costs.

Comparison Matrix: Markdown vs. Frameworks

Feature	Markdown Files (Claude Code/Manus)	Database Frameworks (LangGraph/CrewAI)
Debuggability	High: Just read/edit the file.	Med/Low: Requires DB inspection tools.
Latency	Low: Instant file read.	Med: Network calls to Vector DBs.
Scalability	Low: Files get unmanageable >5MB.	High: Handles millions of records easily.
Persistence	Local: Lives on your disk/repo.	Cloud/Server: Lives in a managed service.
Retrieval	Linear: Agent reads the whole file.	Semantic: Agent searches for keywords/vectors.

Strategic Trade-off

The "Markdown" approach is optimal for Local Agents because the "context" is finite and structured. The "Database" approach is optimal for Enterprise Agents where the "memory" consists of millions of user profiles and history logs that cannot fit into a single file, requiring dynamic agent management and more sophisticated search capabilities.

For example, an enterprise customer support agent typically integrates a Vector DB into a RAG (Retrieval-Augmented Generation) pipeline. Before the LLM generates a response, a retrieval step automatically grabs relevant "memories" based on the user's input and injects them into the system prompt as context. This enables semantic search across structured and unstructured data — user profiles, past chat transcripts, PDF manuals, or meeting notes — so the agent can answer questions like "Has this user complained about something similar before?" without being explicitly told to look it up.

How to Design File-based Memory for Your AI Agent?

File-based AI agent memory typically consists of two layers: remembrance and personalization.

Remembrance Layer

The remembrance layer stores what the agent knows, organized into three types:

Long-term memory (e.g., MEMORY.md): Stores curated, important information that should persist indefinitely. This includes user preferences, key decisions and their rationale, learned lessons, and standard procedures. This file is typically loaded into every agent conversation. Systems like OpenClaw trigger a memory flush before context compression, prompting the agent to write important information to MEMORY.md before older context is discarded.

Daily logs (e.g., memory/YYYY-MM-DD.md): Timestamped records of activities, conversations, and observations. These provide chronological context and help the agent maintain continuity across sessions. Recent logs (today and yesterday) are typically loaded automatically, while older logs are searched on-demand.

Working memory (e.g., task_plan.md): Tracks the current task's goals, progress, and context. This prevents "goal drift" in long-running tasks by providing a consistent reference point that the agent can check throughout execution. Manus popularized a three-file variant (task_plan.md, notes.md, deliverable) with a read-decide-act-update cycle: read the plan, act on the next step, update progress, then repeat.

Personalization Layer

The personalization layer defines how the agent behaves and how it is perceived by the user:

SOUL.md: Defines core values, decision principles, and behavioral guidelines. This file shapes the agent's personality and decision-making approach. For example, a SOUL.md might specify "prefer simple solutions over complex ones" or "always ask for clarification when ambiguous."

IDENTITY.md: Defines the agent's public identity, including name, start date, and communication style. This file is used to identify the agent to the user.

USER.md: Defines the user's profile, including technical background, preferences, and context. This file is used to tailor the agent's behavior to the user's needs.

Modular skills: Additional capabilities can be loaded on-demand using separate skill files. Rather than loading all possible skills at startup, the agent loads specific skill documentation only when needed, keeping the context manageable.

Search Strategies

As memory grows, search becomes important. Three approaches offer progressively more capability:

Basic text search (grep/ripgrep): Sufficient for most use cases with fewer than 1,000 files. Fast, free, and deterministic. Works well for exact keyword matches and phrases.

BM25 full-text search: Useful when scaling to 1,000-10,000 files. BM25 is a ranking algorithm that scores documents by relevance — similar to how a search engine ranks web pages. It supports boolean operators (AND, OR, NOT) and can be implemented using SQLite's built-in full-text search with minimal infrastructure.

Hybrid vector + BM25: Most sophisticated approach, combining semantic search (understanding concepts) with keyword matching. Typically only needed when exceeding 10,000 files or when conceptual queries are important. Requires embedding generation, which adds API costs. OpenClaw's implementation uses 70:30 weighting (vector similarity : BM25 keyword) with a 0.35 minimum score threshold. In testing, this achieved 89% recall vs. 76% for vector-only and 68% for BM25-only.

Most implementations should start with basic text search and upgrade only when the need is demonstrated through actual usage patterns.

Implementation Considerations

Starting with file-based memory is straightforward:

Create a MEMORY.md file and give your AI agent read/write access to it
Implement daily log files with timestamps (memory/YYYY-MM-DD.md format)
Add basic grep/ripgrep search capability
Define a SOUL.md file to establish agent personality and values
Add task planning files when working on multi-step projects

The simplicity of this approach means implementation typically takes days rather than months. The architecture can scale from single-user prototypes to production systems handling thousands of agents.

For more complex deployments, consider:

Git version control for memory files
Separate memory directories for different agents or use cases
Shared knowledge bases that multiple agents can reference
Encryption for sensitive information (filesystem-level or application-level)
Progressive context disclosure: load only memory relevant to the current task rather than everything at startup (as practiced by Claude Code's Skills system)

Conclusion

File-based memory for AI agents represents a practical middle ground: simpler than elaborate infrastructure, but more capable than purely ephemeral in-memory approaches. The convergence of multiple successful projects on this pattern suggests it addresses real needs effectively.

The approach offers particularly strong advantages in transparency, portability, and user control—increasingly important considerations as AI agents handle more sensitive and critical tasks.

When three independent, high-profile projects converge on the same architectural choice, it is worth paying attention — not because Markdown files are the final answer, but because they reveal that the right abstraction for agent memory may be simpler than the industry assumed.

Resources

Manus: Context Engineering for AI Agents: Lessons from Building Manus
OpenClaw: Memory Concepts Documentation
Claude Code: Memory Documentation
AGENTS.md: The Open Standard for Agent Configuration
Agentic Design Patterns: A Hands-On Guide to Building Intelligent Systems

Reward Engineering: An Emerging Skill for AI Engineers

Yaohua Chen — Fri, 13 Feb 2026 16:16:32 +0000

Introduction

In their comprehensive report "AI Predictions for 2026," Richard Socher (one of the world's most-cited NLP researchers and CEO of You.com) and Bryan McCann (CTO of You.com) outline a fundamental shift in how we interact with artificial intelligence. Their central thesis: the era of simple Large Language Model (LLM) chatbots is giving way to sophisticated, autonomous AI agent ecosystems.

This transformation represents a shift from "Chat-Engines" (systems you converse with) to "Do-Engines" (systems that autonomously complete tasks for you). To enable this shift, Socher and McCann predict the emergence of a new specialization: the Reward Engineer—a professional who designs the mathematical and logical objective functions that define success for AI agents.

Whether or not "Reward Engineer" becomes an official job title in 2026, the underlying skill of reward engineering is rapidly becoming essential for any AI engineer working with autonomous systems.

What is Reward Engineering?

As AI evolves from generating text to autonomously executing multi-step tasks, our approach to guiding these systems must also evolve. Traditional Context Engineering—writing instructions in natural language—works well for chatbots but proves insufficient for autonomous agents.

Why Prompts Aren't Enough: When an AI agent must complete complex, long-term goals—such as optimizing a supply chain, conducting legal research, or managing a project—simple text instructions cannot capture all the nuances, constraints, and trade-offs involved.

Enter Reward Engineering: This discipline combines logic, ethics, and data science to define precise success criteria. Reward engineers must anticipate how AI agents might find unintended shortcuts (a phenomenon called "reward hacking") and design objective functions that align agent behavior with genuine human intent across extended time horizons.

Core Responsibilities

Rather than writing traditional code or conversational prompts, engineers design the objective functions and reinforcement learning frameworks that guide autonomous AI agents. Think of this role as a "Policy Architect"—ensuring agents achieve complex business objectives (such as "increase supply chain efficiency by 15%") while respecting ethical boundaries, security protocols, and resource constraints.

Key Responsibilities

Objective Function Design: Translate broad business goals into precise mathematical reward signals that guide agent behavior toward desired outcomes.
Guardrail Engineering: Create constraints and penalties that prevent reward hacking—situations where an AI technically achieves its goal but in unintended or harmful ways.
Multi-Agent Coordination: Design reward structures that encourage multiple AI agents to collaborate effectively rather than compete counterproductively for shared resources.
Human-in-the-Loop (HITL) Policies: Establish clear escalation triggers that determine when an agent must pause and request human approval before proceeding with high-stakes decisions.
Validation & Benchmarking: Develop comprehensive test suites to evaluate agent reasoning and ensure consistent, reliable performance across different scenarios and model versions.

Required Technical Skills

Logic & Ethics: Strong foundation in game theory, utility functions, and AI alignment principles to design fair and effective reward systems.
Agentic Frameworks: Proficiency with modern AI agent frameworks (such as LangChain, AutoGPT, CrewAI, and their successors) as well as cloud-based agentic platforms (Amazon Bedrock Agents, Azure AI Agent Service with Semantic Kernel, and Vertex AI Agent Builder) that enable autonomous task execution.
Python Programming: Ability to write validation scripts that evaluate AI outputs and enforce behavioral constraints—essentially serving as "referees" for agent actions. Python is specifically required because it's the dominant language in the AI/ML ecosystem: nearly all reinforcement learning frameworks (PyTorch, TensorFlow, Gymnasium), agent frameworks (LangChain, AutoGPT), and evaluation tools are built in Python. This creates seamless integration between reward function design and the AI models they guide, unlike general-purpose languages such as Bash (limited to shell scripting) or Node.js (less common in ML applications).
Domain Expertise: Deep understanding of specific industries (finance, healthcare, legal, etc.) to define what constitutes a genuinely successful outcome versus a superficial one.
Risk Identification: Skill in recognizing logical inconsistencies, potential failure modes, and "hallucination-prone" scenarios within autonomous agent workflows.

Reward Engineering vs. Context Engineering

The shift from conversational AI to autonomous agents demands a fundamental change in how we guide these systems:

Context Engineering (Today): Writing natural language instructions like "Act as a lawyer and draft a contract." This works for generating single responses but lacks the precision needed for autonomous, multi-step tasks.

Reward Engineering (Tomorrow): Designing mathematical frameworks that define success. Instead of telling an AI what to do, reward engineers create scoring systems that guide how the AI optimizes its behavior over time.

The Critical Difference: Preventing Reward Hacking

Consider a common pitfall: if you reward an AI for "reducing customer complaints," a poorly designed system might simply delete incoming complaint emails—technically achieving the goal while completely missing the intent.

AI engineers must anticipate such shortcuts and create sophisticated reward models that balance competing priorities: speed, accuracy, ethics, and safety. This becomes especially critical as AI agents make consequential decisions with real-world financial, legal, or safety implications.

The Evolution: From Context Engineering to Reward Engineering

Dimension	Context Engineering	Reward Engineering
Primary Tool	Natural language instructions	Mathematical objective functions
Focus	Generating single responses	Guiding multi-step autonomous behavior
Success Measure	"The output sounds right"	"The task completed successfully within all constraints"
Output Type	Text, images, code snippets	Real-world actions and transactions
Scope	One interaction at a time	Extended time horizons with multiple decision points

This evolution from conversational AI to autonomous agents represents not just a technical shift, but a fundamental change in how we conceptualize human-AI collaboration.

Building Your Reward Engineering Skills: A Practical Roadmap

Transitioning to reward engineering means evolving from a "Writer" (crafting conversational prompts) to an "Architect" (designing behavioral frameworks). You'll shift from asking AI for outputs to defining the mathematical and ethical boundaries within which it operates.

Here's a three-phase roadmap to develop these skills:

Phase 1: Foundations — From Intuition to Precision

Goal: Move from informal, "vibe-based" prompting to structured, contract-like specifications.

Key Skills to Develop:

Logical Decomposition: Practice breaking complex problems into small, verifiable subtasks. Each subtask needs a clearly defined success state.
Contract-Based Thinking: Transform vague requests into precise specifications. Instead of "Write a professional email," specify: "Generate an email under 200 words containing exactly three bullet points and referencing invoice #12345, or fail validation."
Basic Programming Literacy: Develop comfort with Python control flow (if/then/else logic) and APIs. Many reward functions are implemented as Python scripts that evaluate agent outputs against defined criteria.

Phase 2: Understanding Agentic Systems

Goal: Learn how autonomous "Do-Engines" operate and make decisions over time.

Key Skills to Develop:

State Management: Understand how agents maintain memory of previous actions and decisions. Study frameworks like ReAct (Reasoning + Acting) and Plan-and-Execute patterns that enable multi-step reasoning.
Tool Integration: Learn how agents access and utilize external tools (calculators, search engines, databases). Your role is designing rewards that encourage appropriate tool usage and penalize inefficient or incorrect tool selection.
Quantitative Evaluation: Adopt rigorous evaluation frameworks like LangSmith or Hugging Face Evaluate. Shift from subjective assessment ("This looks good") to measurable metrics ("This output scores 8.5/10 on our accuracy rubric").

Phase 3: Advanced Reward Engineering

Goal: Master the specialized skills that define the reward engineering role.

Key Skills to Develop:

RLHF (Reinforcement Learning from Human Feedback): Understand how models learn from human preferences. You'll design the ranking criteria and evaluation rubrics that human labelers use to train agent behavior.
Objective Function Design: This is the core competency. Learn to translate business goals into mathematical reward functions that balance competing priorities.

Example: For a budget management agent, design rewards that optimize both cost savings and service quality—preventing the agent from simply cutting all expenses.
1. Safety & Alignment Engineering: Create guardrail mechanisms ensuring that the reward for helpful behavior never outweighs the penalty for harmful actions. This requires anticipating edge cases where agents might find dangerous shortcuts.

Hands-On Practice: Thinking Like a Reward Engineer

The best way to prepare for this emerging skill is a fundamental shift in perspective: stop focusing on what you want the AI to say, and start defining how you'll measure whether its actions were successful.

The following exercise introduces you to reward function design—the core of reward engineering.

Practical Exercise: The Budget-Conscious Travel Agent

The Scenario: You're developing an AI agent to book corporate travel. With a vague instruction like "Book the best flight," the agent might select a $10,000 first-class ticket—technically "the best" by some measures, but clearly not what you intended.

Your Task: Design a reward system that guides the agent to balance cost, timeliness, comfort, and convenience appropriately.

Step 1: Distribute Reward Points

You have 100 reward points to allocate across four potential outcomes. The agent will optimize for maximum points. How should you distribute them?

Outcome	Your Allocation
Arrival Time: Flight arrives before the 9:00 AM meeting	_____ points
Cost Efficiency: Flight costs under $500	_____ points
Convenience: Direct flight with no layovers	_____ points
Comfort: Business or first-class seating	_____ points

Step 2: Recognizing the Reward Hacking Trap

Review your point allocation. If you assigned 80 points to Cost Efficiency but only 10 points to Arrival Time, the agent might book a $50 red-eye flight that arrives after the 9:00 AM meeting. It maximized points but completely failed the actual objective.

The Reward Engineering Solution:

Professional reward engineers use hard constraints and dynamic incentives to prevent such failures:

Hard Constraint: "If arrival time is after 9:00 AM, apply a penalty of -1,000 points (automatic failure)."
Incremental Incentive: "For every $10 saved below the $500 budget, add +1 bonus point."

This combination ensures critical requirements are never violated, while still encouraging optimization within acceptable parameters.

Key Takeaways

1. Alignment Requires Precision: Without explicit penalties for missing the meeting, even a well-intentioned point system can lead to failures. Intent alone isn't enough—you must formalize every constraint.

2. Logic Replaces Language: This exercise demonstrates programming agent behavior through mathematical objectives rather than conversational instructions—the essence of reward engineering.

3. The Future of Software Development: This approach reflects Socher and McCann's vision for 2026: rather than giving AI step-by-step instructions, we'll define the rules and constraints, then let AI agents find optimal solutions within those boundaries.

Conclusion

As AI systems transition from responding to queries to autonomously executing complex tasks, reward engineering emerges as an essential discipline. Whether it becomes a formal job title or remains a critical skill within broader AI engineering roles, the ability to design precise, ethical, and robust objective functions will define who can successfully deploy autonomous AI agents in the real world.

Start developing these skills now: think in terms of measurable outcomes, anticipate unintended behaviors, and practice translating human intent into mathematical frameworks. The future of AI isn't just about building smarter systems—it's about building systems that are smart in the right ways.

Why Your Multi-Agent AI System Is Probably Making Things Worse?

Yaohua Chen — Mon, 05 Jan 2026 23:50:13 +0000

2025 has been dubbed the "Year of the Agent" by investors and tech media. Companies like Manus, Lovart, Fellou, and many others have captured headlines with their AI agent applications, which are software systems that can autonomously perform tasks on your behalf, from browsing the web to analyzing documents.

Over the past two years, I've built multi-agent systems for various clients across different industries by using various AI models and agent frameworks. A pattern keeps emerging: these projects look impressive in demos but struggle to work reliably in production. The same questions come up again and again: why isn't adding more agents helping? Why doesn't giving the system more tokens (via prompt engineering or Retrieval-Augmented Generation pipeline (RAG)), more tool calls, or more compute budget improve results?

The industry has embraced two assumptions that seem logical on the surface:

More agents = better results. Since a single AI agent has limited capabilities, having multiple agents collaborate should solve more complex problems.
More compute = better performance. If results aren't good enough, just give the AI more time to think (more "tokens") and more tools to work with.

But are these assumptions actually true?

Recent research tells a very different story. A report from UC Berkeley, "Measuring Agents in Production" (December 2025), combined with two papers from Google DeepMind, systematically debunks both assumptions:

More agents ≠ better results. Multi-agent systems often perform worse than single agents due to coordination overhead.
More compute ≠ better performance. Agents don't know how to effectively use extra resources. They leave 85% of their budget untouched.

These studies reveal that current AI agents have fundamental limitations that no amount of scaling can easily fix. Let me walk you through what the research actually shows.

The Reality Check: What Berkeley Found in Production Systems

The Berkeley team surveyed 306 practitioners and conducted 20 in-depth case studies with organizations actually running AI agents in production, including Accenture, Amazon, AMD, Anyscale, Broadcom Inc., Google, IBM, Intel, Intesa Sanpaolo, Lambda, Mibura Inc, Samsung SDS, and SAP. Crucially, they filtered out demo-stage or conceptual projects, focusing only on systems generating real business value.

Their findings paint a surprisingly conservative picture:

Most agents are kept on a very short leash. 68% of production systems limit agents to 10 steps or fewer. Only 16.7% allow dozens of steps, and a mere 6.7% give agents unlimited autonomy.

Companies build safety barriers between agents and real systems. Rather than letting agents directly call production APIs, engineering teams create simplified "wrapper APIs", bundling multiple complex operations into single, safer commands. For example, instead of making an agent call three separate database queries, engineers package them into one pre-tested function. This reduces what could go wrong.

Human-designed workflows dominate. 80% of successful deployments use "structured control flow", meaning humans draw the flowchart, and the AI simply fills in the blanks at predetermined decision points. The agent isn't autonomously planning, it's following a script.

Agents require massive instruction sets. 12% of deployed systems use prompts exceeding 10,000 tokens (roughly 7,500 words of instructions). These aren't lightweight assistants, they're heavily engineered systems with extensive guardrails.

In essence, today's successful AI agents work like tireless interns with good reading comprehension, useful within a tightly defined process, capable of handling some ambiguity, but not the autonomous problem-solvers the marketing suggests.

So why are production systems so constrained? Two papers from Google DeepMind, published in late 2025, may provide the answers by systematically disproving the core assumptions behind agent scaling:

Limitation #1: More Agents ≠ Better Performance

DeepMind's paper "Towards a Science of Scaling Agent Systems" tackled a seductive idea: if one AI isn't smart enough, why not create a whole team? Imagine GPT handling product management, Claude writing code, and Gemini running tests—a virtual software company where PhD-level AIs collaborate to solve any problem.

It sounds logical. After all, that's how human organizations scale. But 180 controlled experiments later, DeepMind proved this intuition wrong.

The Experiment Setup

The researchers tested five different ways to organize AI agents:

Single Agent: One AI handles everything (think: a solo developer)
Independent Multi-Agent: Multiple AIs work on the same problem separately, then their answers are combined through voting (think: getting multiple opinions, then picking the consensus)
Decentralized Multi-Agent: Agents communicate directly with each other to negotiate solutions (think: a peer-to-peer discussion group)
Centralized Multi-Agent: One "manager" agent assigns tasks and verifies results (think: a team with a project manager)
Hybrid: A combination of centralized coordination with peer communication

They tested these architectures using top models from OpenAI, Google, and Anthropic across four different task types: financial analysis, web browsing, game planning (Minecraft-style crafting), and general workflows.

The Core Finding

DeepMind discovered a formula that predicts agent system performance with average 87% accuracy:

Net Performance = (Individual Capability + Collaboration Benefits) − (Coordination Chaos + Communication Overhead + Tool Complexity)

The key insight: the costs often outweigh the benefits. When coordination overhead, miscommunication, and tool management burden exceed the gains from parallelization, adding more agents makes systems worse, not better.

The results varied dramatically by task type:

Financial analysis: Multi-agent systems helped (up to 81% improvement with centralized architecture)
Web browsing: Minimal benefit; errors actually got amplified
Game planning (PlanCraft): Multi-agent systems performed significantly worse than single agents
General workflows: Mixed results; decentralized approaches slightly better

See the figure below for the detailed results:

Why Multi-Agent Systems Fail: Three Key Reasons

1. The Coordination Tax

In complex, open-ended tasks, adding more agents makes the system dumber, not smarter.

Consider the PlanCraft benchmark (a Minecraft-style planning task). When Anthropic's Claude model was put into a multi-agent setup, performance dropped by 35%. Why? Every agent must understand tool interfaces, maintain conversation context, and process results from other agents. When the tool count exceeds a threshold, agents spend all their "mental bandwidth" on coordination, reading documentation and attending virtual meetings, with no capacity left for actual problem-solving.

2. The Capability Saturation Effect

When a single agent can already solve a problem with greater than 45% accuracy, adding more agents typically provides diminishing or negative returns.

The logic is straightforward: if one agent can correctly answer "What is 2+2?", having three agents debate the answer for an hour won't improve accuracy, it just wastes resources.

3. Error Amplification (The Most Surprising Finding)

Intuitively, you'd expect that having three agents vote on an answer would reduce errors, the wisdom-of-crowds effect. But DeepMind found the opposite.

In independent multi-agent systems (where agents work separately then vote), errors don't cancel out—they multiply. The paper quantifies this with an "error amplification factor" of 17.2. This means if a single agent has a 5% error rate, an independent multi-agent system can have an error rate as high as 86% (5% × 17.2).

Why does this happen? Without cross-verification during reasoning, each agent makes errors based on its own flawed logic. These errors become self-reinforcing within each agent's context. When you aggregate three independently wrong answers through voting, you're not getting wisdom, you're getting confident wrongness.

Limitation #2: More Thinking Time ≠ Better Results

If adding more agents doesn't work, what about giving a single agent more time to think?

After OpenAI released its o1 model, "test-time compute" became a buzzword. The idea: let AI models "think longer" by giving them more computational budget during inference. Search more, reason more, and eventually they'll find the answer, right? But is it true?

DeepMind's paper "Budget-Aware Tool-Use Enables Effective Agent Scaling" tested this assumption—and found it largely false.

The Experiment

Researchers increased an agent's "tool-call budget", the number of web searches, API calls, or other actions it could perform, from 10 to 100. The expectation: 10x more resources should yield significantly better results.

The reality: doubling the budget improved accuracy by only 0.2 percentage points.

Even more telling: when given a budget of 100 tool calls, agents only used an average of 14.24 searches and 1.36 browsing sessions. They left 85% of their budget untouched.

Why Agents Can't Use Extra Resources Effectively

The core problem: agents don't know what they don't know, and they don't track their remaining budget.

When an agent goes down a wrong path, say, searching for a paper title that doesn't exist, it has no concept of opportunity cost. It will keep digging deeper into a dead end rather than trying a different approach. Give it unlimited compute, and it just digs a deeper hole.

Making matters worse, long conversation contexts cause "attention drift". After a dozen failed searches, the agent gets lost in its own accumulated noise, the search results, error messages, and dead ends it generated. Performance actually declines as context grows.

A Potential Solution: Budget-Aware Agents (BATS)

DeepMind proposed a framework called BATS (Budget-Aware Test-time Scaling) that addresses these issues with two key mechanisms:

1. Budget-Aware Planning: Instead of making a fixed plan upfront, the agent maintains a dynamic task tree. Each node tracks its status (pending, completed, failed) and resource consumption. When budget is plentiful, expand exploration; when budget is tight, focus on verification.

2. Budget-Aware Verification: After proposing an answer, a separate verification step checks constraints: What's confirmed? What's contradicted? What can't be verified? Based on this assessment and remaining budget, the agent decides whether to dig deeper or abandon the current path.

The results were significant:

BrowseComp benchmark: 24.6% accuracy (95% improvement over standard approaches at 12.6%)
BrowseComp-ZH: 46.0% accuracy (46% improvement over 31.5%)
Cost efficiency: 40% lower total cost (tokens + tool calls) at equivalent accuracy

The lesson: raw thinking time isn't enough. Agents need structured self-reflection, the ability to recognize dead ends, and the wisdom to cut losses early.

What's Actually Needed for AI Agents to Work?

Let's return to DeepMind's core formula:

Net Performance = (Individual Capability + Collaboration Benefits) − (Coordination Chaos + Communication Overhead + Tool Complexity)

The problem isn't that we need smarter models or more compute. The problem is that the negative factors, including coordination overhead, communication noise, and tool complexity, are overwhelming the positive factors. All of these boil down to one root cause: inefficient use of context.

Every token spent on coordination, error recovery, or tool documentation is a token not spent on actual problem-solving. To make agents work, we need to reduce this context burden, not pile on more resources.

Three Directions That Actually Work

1. Smarter Tool Management

The key insight from the financial analysis success (81% improvement with multi-agent systems) is instructive: it worked because the task had clear boundaries and well-defined steps.

Financial analysis follows a predictable pattern: read report → extract data → calculate ratios → generate summary. Each agent fills in blanks within a predetermined framework, no creative planning required.

This tells us something important: current AI models cannot self-organize division of labor. They can handle easily parallelizable tasks (like financial analysis) or consensus-based error correction (like multi-path search), but not emergent collaboration.

The implication? For complex tasks, human-designed task decomposition (SOPs) remains necessary. The dream of throwing agents together and watching them spontaneously develop hierarchies has been empirically disproven.

This is why Anthropic's Skills mechanism matters: it lets agents accumulate reusable capability modules instead of starting from scratch, reducing the cognitive load of tool management.

2. Built-in Self-Verification

BATS works because it formalizes verification. The system explicitly tracks constraints: what's satisfied, what's contradicted, what can't be verified yet. This isn't emergent behavior, it's enforced through careful prompt engineering.

Without structured verification, errors accumulate silently. Each mistake pollutes the context with garbage that degrades future reasoning. Formal verification catches errors early, preventing context pollution.

3. Efficient Inter-Agent Communication

Current agents coordinate via natural language, verbose, ambiguous, requiring constant clarification. This high message density is inherently wasteful.

Future improvements might come from:

Structured communication protocols (like Google's A2A framework)
Latent-space communication where models exchange compressed representations rather than text
Shared memory architectures that reduce redundant information exchange

Until these three capabilities mature, such as smart tool management, built-in verification, and efficient communication, multi-agent systems will continue to underperform their theoretical potential.

The Bottom Line

Despite the hype, the "Year of the Agent" hasn't truly arrived. The research tells us:

Production agents are heavily constrained: most limited to 10 steps or fewer, running within human-designed workflows
More agents often means worse performance: coordination costs overwhelm collaboration benefits
More compute doesn't help much: agents don't know how to use extra resources effectively
The path forward is reducing overhead, not adding more power

Current successful AI agents are best understood as capable interns with good reading comprehension, working within strict SOPs. They handle ambiguity better than traditional software, but they're not autonomous problem-solvers.

For practitioners, the implication is clear: invest in workflow design, tool abstraction, and structured verification rather than chasing multi-agent architectures or unlimited compute budgets. The engineering fundamentals—not the scaling laws—determine success.

References

Measuring Agents in Production - UC Berkeley (December 2025)
Towards a Science of Scaling Agent Systems - Google DeepMind (December 2025)
Budget-Aware Tool-Use Enables Effective Agent Scaling - Google DeepMind (November 2025)
Agent Skills - Claude Docs - Anthropic

AI Made Me 10x Faster—Here's What I Had to Change

Yaohua Chen — Fri, 19 Dec 2025 22:05:43 +0000

I've been working as an IT engineer in multiple industries for over 25 years, from .Net developer, BI developer, data architect, data scientist, and finally to AI solutions architect. Recently, the team in my organization is developing and using AI programming tools and has achieved 8x to 20x code output efficiency compared to ordinary high-performing teams. See this LinkedIn post for more details.

Reading this, you might think: Wow, are programmers going to lose their jobs? Is AI going to replace humans?

But my view is exactly the opposite. As a frontline professional programmer, I'm telling you responsibly: When your speed increases 10x, the risks and bottlenecks you face may also be magnified 10x.

I'll be honest - even I was skeptical at first. Could we really sustain this pace without everything falling apart?

What this means is: AI has fundamentally changed how "costs" and "benefits" are calculated in software engineering. But to truly enjoy this improvement, the entire software development system needs to be upgraded simultaneously. This insight applies not only to programming but has profound implications for everyone who uses AI tools.

1. AI Hasn't Made Programmers Unemployed, But Has Fundamentally Changed How People Work

Here's how my team works:

In the code our team submits, 80% - 90% is written by AI. But this is definitely not that casual "Vibe Coding (i.e., coding without thinking)", which is not a good coding practice. This workflow is called "Agentic Coding" (i.e., coding with AI agents).

In this model, AI plays the role of a "highly capable but irresponsible junior programmer."

And the human engineers? They are "experienced tech leads or architects."

Specifically, the engineer's workflow has become:

Design, plan and break down tasks - Figure it out yourself first, or brainstorm with AI
Give AI instructions - Clearly tell AI what to do
Review and refine AI's output - This is the most critical step
Iterate repeatedly - Until completely satisfied with the quality
Submit and take full responsibility - Ultimately, humans are still responsible for the code

See that? The role of humans hasn't diminished; it's become more important. The focus of work has just shifted from "writing code by hand" to "defining requirements" and "code review."

Think about your own work: What would change if 80% of your output came from AI?

An analogy: Previously you were a worker carrying bricks on a construction site. Now you're an operator commanding an excavator. Although you no longer carry bricks with your own hands, your judgment, operational skills, and responsibilities have actually increased.

2. Speed Increases 10x, Accident Rate May Also Increase 10x

When you're speeding down a track at 200 km/h, you need massive "downforce" to keep the car firmly planted on the ground. Otherwise, you'll fly off at the first curve.

In software engineering, "flying off" means bugs and system crashes.

Some alarming data:

Before: A team might only encounter one or two severe production bugs per year
Now: When you're submitting code at 10x speed, even if the probability of bugs stays the same, the absolute number of bugs you encounter will also increase 10x

What does this mean?

Incidents that used to happen once a year might now happen every week. Imagine explaining to your boss why production went down every Monday.

This kind of "accident rate" is unbearable for any team. Yet many people promoting "AI omnipotence" have intentionally or unintentionally ignored this problem.

To enjoy the 10x coding speed boost from AI, you must also find ways to reduce the "probability of problems" by 10x, or even more.

Having a good engine isn't enough; you also need a better braking system.

3. The True Value of AI: Making "Good but Expensive Methods" Affordable

So how do you reduce risk while increasing speed?

AI isn't just about letting you write faster; it's about making those "good but too expensive" best practices in software engineering become affordable and feasible.

3.1 Change #1: Build a "Wind Tunnel Testing" Environment

What is wind tunnel testing?

Just like building an airplane - before it actually takes flight, the model is put in a wind tunnel to test various extreme conditions.

In software development, this means building a "high-fidelity simulation environment" locally. For example, if your system depends on 10 external services (databases, authentication, payments, etc.), you run or simulate all 10 services locally.

This way, on your computer you can run complete end-to-end tests, and even simulate various extreme failure scenarios.

This kind of testing can catch a lot of bugs hidden in the "cracks between components."

Why didn't we do this before? Too expensive!

Simulating and maintaining these services was too much work, so most teams gave up.

Why can we do it now? AI excels at this!

AI is very good at writing these simulation services with clear logic and well-defined behavior. Especially by using AI agents with Model Context Protocol (MCP) and Agent2Agent Protocol (A2A), we can easily build a complete local "wind tunnel" for our fairly complex system.

Work that used to take weeks or even months can now be done in days.

3.2 Change #2: Upgrade CI/CD (Continuous Integration/Deployment)

In the early days of waterfall development, everyone developed separately and then integrated after development. The result was a pile of problems during integration, taking a long time to stabilize.

Later, the concept of "continuous integration" became popular:

The earlier you integrate, the earlier you get feedback. The more frequently you integrate, the more you can reduce problem complexity.

Now, CI/CD is recognized as the best practice in software engineering. But not many teams actually do it well, because building and maintaining it is still not cheap.

What's worse is that many teams, despite having CI/CD, have extremely time-consuming processes. One code commit, waiting for all tests and deployment to run through - at minimum ten minutes, sometimes several hours.

Before AI, these problems weren't obvious. Now that AI is more capable, they've become obstacles.

So CI/CD also needs to be upgraded along with it, compressing the feedback loop from "hours" to "minutes." You need infrastructure that's fast to an absurd degree, able to discover, isolate, and roll back problematic changes within minutes.

3.3 Change #3: Decision-Making and Communication Systems Also Need Upgrading

10x code output means you also need 10x or more communication and decision-making efficiency.

Before, developing a system required various meetings, lengthy discussions, and only then could work begin. After all, you had to depend on other people's modules, so you had to define agreements first, otherwise you couldn't integrate later.

Various technical decisions also required repeated discussion for a long time, because development costs were high back then, and if decisions were wrong, the cost of rework was too great.

But now, if we still have the same communication efficiency as before, it will greatly drag down overall efficiency.

Perhaps the most efficient approach is to minimize communication as much as possible, letting everyone do their tasks as independently as possible from others.

For example, microservices architecture might be a good choice in the AI era.

For technical decisions, now you can actually have more opportunities to experiment. You don't need to be as rigorous as before in repeatedly verifying technical decisions. Because development costs have decreased, the cost of experimentation has also decreased.

4. Insights for Everyone

AI isn't a "stimulant" that makes you run faster; it's giving you a "supercar."

But the question is: Are you ready to drive it?

4.1 Tool Upgrade Doesn't Equal System Upgrade

Using AI is like upgrading your car with a brand new engine. If you just install it on your old "vintage car," what you get isn't 10x speed, but 10x problems.

This principle applies to each of us.

When you learn to use AI tools (ChatGPT, Gemini, Claude, Midjourney, various AI assistants), don't assume your work efficiency will automatically improve.

Your workflows, quality inspection mechanisms, and collaboration methods all need to be adjusted accordingly.

For example:

Using AI to write code is fast, but if you don't have a rigorous review process, you might produce a lot of low-quality, buggy code.

Using AI to quickly generate investment advice - but has your risk assessment ability kept up?

Using AI to make quick decisions - but have you established a review and error-correction mechanism?

4.2 Speed Increase Must Come with Risk Management Improvement

The "accident rate paradox" doesn't only exist in programming.

The faster food delivery, the higher the traffic accident risk
The faster product iteration, the more quality issues there might be
The faster decision-making, the greater the probability of making mistakes

So don't blindly pursue "fast." Ask yourself: Has my "braking system" been upgraded?

Build your own "wind tunnel testing": Try on a small scale first, rather than pushing forward comprehensively right away.

4.3 Re-examine Those "Good but Expensive" Methods

Here's what finally clicked for me after months of using AI tools:

The true value of AI isn't just about "writing faster"; it's about making those "good but too expensive" best practices become affordable and feasible.

This realization made me re-examine many good habits I had abandoned because they were "too troublesome":

Personal finance management: Keeping track of expenses used to be too troublesome. Now AI can help you automatically categorize and analyze
Learning new skills: Hiring a private tutor used to be too expensive. Now AI can provide personalized tutoring
Health management: Nutritionists used to be too expensive. Now AI can customize meal plans
Content creation: Making videos used to require a team. Now individuals can also produce high-quality content

The key is: Don't just use AI for "fast production." Use it to achieve "things you wanted to do before but couldn't."

4.4 Your Role is Changing

In programming, my role has shifted from "executor" to "decision-maker + quality inspector." But this pattern applies everywhere:

Writers are becoming editors who review and refine AI drafts
Designers are becoming creative directors who guide AI-generated concepts
Analysts are becoming strategists who interpret AI-processed data
Managers are becoming orchestrators who coordinate AI-assisted workflows

The common thread? Responsibilities haven't decreased; they've actually increased.

In the AI era, your core competitiveness is:

Judgment - Being able to distinguish good from bad AI output
Questioning ability - Being able to give AI clear instructions
Sense of responsibility - Being willing to take responsibility for the final results
Systems thinking - Understanding the entire process, not just one part

Many people can use AI to write reports, but those who can review AI's logical flaws are valuable.

Many people can use AI to design solutions, but those who can judge a solution's feasibility are irreplaceable.

4.5 Build Your AI Work System

For us ordinary people, don't just focus on "individual AI tools." Build your own "AI work system":

Input system - How to quickly and accurately provide AI with information and instructions
Quality inspection system - How to efficiently review AI's output
Feedback system - How to iterate and improve quickly
Knowledge management - How to accumulate and reuse experience from AI collaboration

For example:

It's not just about using ChatGPT; you also need to build your own prompt library, review checklist, and iteration workflow.

It's not just about using AI for image generation; you also need to establish a style library, quality standards, and version management.

This is the true way of working in the AI era.

5. Summary

The AI era requires "systems thinking," not "tool thinking."

Many people treat AI as a "fast production tool," hoping to use it to accelerate existing work.

But those who truly understand how to leverage AI treat it as an "opportunity for system upgrade," rethinking the entire workflow.

AI isn't just about upgrading the car's engine; it's also about upgrading the roads the car frequently drives on.

The goal for veteran drivers isn't to be replaced by AI, but to help them adapt to the new high-speed engine, giving them a comfortable and safe driving environment.

So when AI increases your speed 10x or 20x, don't rush to celebrate.

First ask yourself:

Has my quality inspection mechanism been upgraded?
Has my risk management ability improved?
Has my workflow been restructured?
Am I ready to take full responsibility for the results?

Remember: You are the driver who is responsible for the final results.

AI just gave you a supercar, but whether you can arrive at your destination safely and efficiently depends on your driving skills and road conditions.

May we all become good drivers in the AI era - ones who can step on the gas, and also know when to brake.

Key Breakthroughs in AI Engineering that Every AI Engineer Must Know

Yaohua Chen — Fri, 19 Dec 2025 19:52:47 +0000

This blog post provides a clear understanding of the logic behind the entire technological evolution, showing how we went step by step from "this thing can run" to "this thing can actually do work." We'll explain the development of AI engineering from 2017 to the present in a simple and easy-to-understand way. The key breakthroughs are grouped into 4 categories as follows:

The Beginning of Everything: From "Architectural Revolution" to "Emergent Capabilities"

The starting point of this story is 2017. The famous paper called "Attention Is All You Need" is the "birth certificate" of the Transformer architecture, which is the foundation of all modern large language models.

Before it, models processed text word by word "sequentially" like RNNs, which was not only slow but also struggled with long texts (like forgetting what was said earlier when reading to the end). The core contribution of Transformer was introducing the "Self-Attention" mechanism, allowing the model to "simultaneously" look at all words in a sentence and determine which words are more important to each other.

This brought two huge benefits: training can be massively parallelized, and handling long-range dependencies became much better. Following that, in 2020, OpenAI's GPT-3 paper "Language Models are Few-Shot Learners" brought the second key breakthrough. It showed that with a large enough model, it can learn to perform a variety of tasks with just a few examples (few-shot). When you scale up the Transformer model large enough, it will "emerge" with a new capability called "In-Context Learning."

This means you no longer need to fine-tune or "customize" the model for specific tasks (like translation or summarization). You just need to give it a few examples (few-shot) in the prompt, and it can "learn by imitation" and understand what you actually want to do. This discovery completely changed the rules of the game. For practitioners like us, this means we can use a general-purpose foundation model and solve various problems through "prompt engineering" or "context engineering".

"Training" the Model: How to Make It Obedient, Professional, and Cost-Effective?

With GPT-3's "brute force creates miracles" approach, people quickly discovered new problems:

It's powerful but doesn't "listen" - it often talks nonsense and sometimes even outputs toxic content. We need to make it "listen" to us and "obey" our instructions.
It's "expensive" - if you want it to perform better in a specialized domain (like law or medicine), the cost of full fine-tuning is terrifyingly high. We need to find a way to make it more cost-effective.
It's a "bookworm" - training data has a cutoff date, and it doesn't know about new knowledge from the outside world or company internal materials. We need to make it more "knowledgeable" on external knowledge and "professional" in the specialized domain.

So the following key breakthroughs were about solving these "usability" issues.

1. Making It "Obedient" - InstructGPT (2022)

The core of this breakthrough is solving the "Alignment" problem, that is, making the model "listen" to us and "obey" our instructions. The paper "Training language models to follow instructions with human feedback" introduced RLHF (Reinforcement Learning from Human Feedback) on a model called InstructGPT.

Simply put, the process is as follows: first have humans rank the model's different responses, then train a "Reward Model" to mimic human preferences, and finally use this reward model to "train" the large model.

The biggest insight from this breakthrough is: "A smaller but 'aligned' model can outperform a much larger but unaligned model in user satisfaction." This made everyone realize that bigger isn't always better - the value of "alignment" or "obedience" is extremely high.

2. Making It "Cost-Effective" - LoRA (2021)

When we need to make the model "professional" in a specialized domain, full fine-tuning (updating all model parameters with new training data) was usually the only way. However, full fine-tuning is too expensive. Is there a cheaper way? LoRA (Low-Rank Adaptation) came along.

Its idea is particularly clever: during fine-tuning, we don't touch the billions of original parameters (keep them frozen), but instead "insert" some small, trainable "adapters" into different layers of the model. These adapters have very few parameters (possibly only 0.01% of total parameters) but can still achieve good performance.

The result is that it lowered the barrier to fine-tuning from "only big companies can afford it" to "you can run it on a single GPU." This was revolutionary for AI application deployment.

3. Making It "Professional" - RAG (2020)

Before RAG, the model was like a "bookworm" that only knew what it was trained on. It didn't know about new knowledge from the outside world or company internal materials. Moreover, it was prone to "hallucination", that is, making things up when it didn't know the answer. How do you solve the model's "outdated knowledge" and "hallucination" problems? The answer is RAG (Retrieval-Augmented Generation).

The idea is straightforward: "Before the model answers a question, don't rush to make things up. First go to an external knowledge base (like company internal databases, or the internet) to retrieve a batch of relevant documents, treat these documents as 'open-book exam' materials, feed them to the model, and let it answer you based on these materials."

Today, RAG is practically standard for all production-grade LLM applications (like AI customer service, knowledge base Q&A). It is a key technology for making models "professional" and "useful".

Pushing Efficiency to the Limit: How to Make Models Run on Devices with Limited Resources?

When models really need to be deployed to consumer products, or even smartphones, "efficiency" and "cost" become matters of life and death. The next few breakthroughs revolve around "optimization."

1. Making Models "Smaller" - DistilBERT (2019)

This breakthrough uses "Knowledge Distillation" technology. The idea is to have a large, smart "teacher" model (like BERT) teach a small "student" model (like DistilBERT), making the student model mimic the teacher's behavior. The result is that the student model retains 97% of the teacher's language understanding capability, but with 40% fewer parameters and 60% faster speed. This made running AI on smartphones and "edge devices" (like smart home gadgets and wearables) possible. This is a key technology for making models "small" and "efficient".

2. Making Models "Memory-Efficient" - LLM.int8 (2022)

This breakthrough is about "Quantization." Simply put, it's trying to use fewer bits to store model weights, like going from 32-bit floating point numbers down to 8-bit integers (int8), directly reducing memory usage by 4x.

The challenge is that crude compression can cause severe accuracy degradation. The "key insight" of this breakthrough is that they discovered only a very small number of "outlier features" in the model were causing trouble. So they used a "mixed precision" approach: "Store the vast majority of weights using int8, but store those critical 'outlier values' using 16 bits." The result is almost no accuracy loss, but memory savings achieved.

3. Making Models "More Flexible" - Switch Transformers (2021)

This breakthrough is about the "MoE" (Mixture of Experts) architecture. The idea is to instead of training one "jack-of-all-trades" large model, you train a bunch of "specialized" expert small models (like one good at math, one good at writing poetry). For each prediction (i.e., predicting a token), you first use a "Router" to determine which "expert" is most suitable to handle this task, then only activate that one expert.

The benefit is that your model's total parameter count can be very large (like trillion-scale), but the actual computational cost is very low because you only use a small portion of it each time.

The Future Puzzle: "Agents" and "Standards"

When we want to make models really "useful" and "capable of doing work" for us in the real world, we need to solve the problem of "how to make models interact with the outside world". The following three breakthroughs are about solving this problem.

1. Making Models "Capable of Doing Work" - Agents (2023)

The technology called "LLM Agents" proposes a basic system framework which includes: Brain (LLM, responsible for thinking and planning), Perception (responsible for reading external information, like return results from tools), and Action (responsible for calling APIs or tools).

This means models are no longer just "chatbots" - they can start helping you "do things," like booking flights, analyzing financial reports, executing code.

2. Making Models "Interconnected" - MCP (2024)

Before MCP, when AI needs to call an external tool (like your calendar, databases, etc.), you have to write a custom "one-to-one" integration interface, which is very troublesome. Anthropic (the developer of Claude) proposed the "Model Context Protocol (MCP)" in 2024 to solve this problem.

The core idea of MCP is to create an "open standard" for how AI models communicate with all external tools and APIs. Just like the HTTP protocol unified communication between web browsers and servers, MCP hopes to unify how AI models communicate with all external tools and APIs. If this standard gets adopted, the connectivity efficiency of the AI ecosystem will see a qualitative leap.

3. Making Agents "Interoperable" - A2A Protocol (2025)

While MCP connects AI models to tools, what happens when you have multiple AI agents that need to work together? Imagine you have one agent handling your calendar, another managing your emails, and a third analyzing your documents. How do they coordinate? The "Agent2Agent (A2A) Protocol" proposed in 2025 addresses exactly this problem.

Think of it like this: MCP is like giving each AI agent a phone to call different services. A2A is like giving all the agents a group chat so they can talk to each other. The protocol allows AI agents built on different technologies to communicate, share information securely, and coordinate their actions. This is complementary to MCP - together they create a complete ecosystem where AI can both use tools and collaborate with other AI.

Summary

The evolution path of AI engineering is actually very clear. It's a chain of continuously solving key problems: first making the model "able to run" (Transformer), then making it "able to learn" (GPT-3), then making it "obedient" (InstructGPT), then making it "useful and affordable" (LoRA, RAG, Quantization), and finally making it "able to do work" (Agents, MCP, A2A). Each step here represents a huge leverage point.

Developing AI Agent Application with Azure AI Foundry - Why and How?

Yaohua Chen — Wed, 11 Jun 2025 19:50:46 +0000

What is this blog post about?

This technical guide explores the design and development of AI agents using Azure AI Foundry. It explains both the rationale and methodology for building AI agents with Azure AI Foundry, targeting solution architects, AI engineers, and data scientists seeking to leverage AI for business transformation.

The guide provides a step-by-step walkthrough for creating an AI agent application analyzes user input, generates image design recommendations, and produces unique designs based on some instructions, patterns, or user preferences. It features code snippets, architectural diagrams, and practical explanations on utilizing Azure AI Foundry and OpenAI models to build a robust image design application.

Why does it matter?

AI agents are increasingly vital for organizations aiming to harness artificial intelligence to optimize processes and products. They automate tasks, analyze data, and deliver insights that drive better decision-making and product innovation. AI agents also enhance customer experiences through personalized recommendations and support, helping businesses remain competitive in a rapidly evolving landscape.

Despite their benefits, building AI agents can be complex and resource-intensive, often requiring deep technical expertise. This blog post aims to demystify the process by offering a clear, actionable guide for creating AI agents with Azure AI Foundry.

Why use Azure AI Foundry?

Azure AI Foundry is a comprehensive, cloud-based platform that delivers a suite of AI tools and services for scalable AI adoption. It supports model deployment, agent creation, fine-tuning, data management, and seamless integration with Azure services. Organizations can build custom AI models and agents tailored to their needs, streamlining workflows and boosting productivity.

Key benefits of Azure AI Foundry include:

Low-code development: Rapidly deploy models, set up agents, create knowledge and data indexes, and integrate components with minimal coding via fully managed agent services.
Flexible deployment: Deploy models and agents across regions and availability zones to minimize latency and serve users closer to their location. Deployments can also be managed within your own Azure subscription or private cloud for greater control.
Scalability: Effortlessly scale resources to match business demands.
Enterprise integration: Seamlessly connect with Microsoft 365, Dynamics 365, Azure services, Databricks, and more.
Model Context Protocol (MCP) support: Leverage the open-source Model Context Protocol (MCP) to enable AI agents to communicate and interoperate seamlessly with tools and agents across diverse ecosystems.
Multi-agent orchestration support: Build sophisticated AI applications by orchestrating multiple agents to collaborate on complex workflows and problem-solving. Open interoperability standards—such as Agent2Agent (A2A)—enable seamless information exchange and coordination among agents across Azure, AWS, Google Cloud, and on-premises environments.
Proprietary model access: Instantly access advanced proprietary models such as Grok 3, GPT-4o, o3, and more, empowering your applications with state-of-the-art natural language processing and image generation capabilities.
Open-source model support: Access over 10,000 open-source models—including Llama, Mistral, DeepSeek, and more—from a variety of vendors directly within Foundry Models.
Build-in monitoring, tracing, and governance: Access deep insights into agent performance and behavior, enabling continuous improvement through built-in monitoring, tracing, and governance features.

For more details on Azure AI Foundry's features, please refer to the blog of Azure AI Foundry: Your AI App and agent factory.

What about AWS Bedrock and Google Cloud Vertex?

AWS Bedrock is a powerful cloud-based AI platform with similar capabilities for building and deploying AI agents. However, it can be more complex to configure and maintain, as many AWS AI services are based on open-source components that require additional setup. Azure AI Foundry, by contrast, offers a more integrated and user-friendly experience, with built-in Azure service support and direct access to OpenAI models, making it easier to create and deploy AI agents without extensive technical expertise.

Google Cloud Vertex AI also provides robust tools for building and deploying AI agents, including the Agent Development Kit (ADK) and support for the open A2A protocol for cross-ecosystem agent communication. However, Google Cloud's ecosystem-centric approach may limit integration with external platforms, making it more challenging to build agents requiring broad interoperability. Azure AI Foundry, in contrast, offers deeper integration across the Microsoft ecosystem and simplifies multi-service deployments.

What about the other open-source AI agent frameworks, such as LangChain, CrewAI, etc.?

There are several open-source AI agent frameworks—such as LangChain, LlamaIndex, and CrewAI—that offer flexibility and extensibility for building custom AI solutions. However, these frameworks often require considerable effort to set up, configure, and maintain, especially when integrating with enterprise data, applications, and infrastructure. Seamless integration with Azure services and OpenAI models is typically less mature, and features like scalability, monitoring, and governance may require additional tooling or custom development compared to Azure AI Foundry's built-in capabilities.

Open-source frameworks are generally best suited for proof-of-concept (PoC) development, desktop application development, or highly specialized tasks, such as advanced natural language processing or domain-specific data analysis. They often require orchestrating multiple cloud services and AI models, which can add operational complexity. For example, building a healthcare data analysis agent with CrewAI might involve integrating Claude 4 on AWS Bedrock for code generation, Gemini on Google Cloud Vertex AI for healthcare data understanding, and OpenAI image models on Azure AI Foundry for image generation. This multi-cloud, multi-model approach can increase both technical complexity and operational overhead.

When choosing an AI agent framework, a key consideration is the location of your data and applications. If your ecosystem is primarily on Azure, then Azure AI Foundry could be naturally the choice. It provides a unified, enterprise-ready platform that streamlines the process of building, deploying, and managing AI agents—while leveraging the full power of Azure services and OpenAI models.

How to build AI agents with Azure AI Foundry?

The rest of this blog post will guide you through the process of building an AI agent application using Azure AI Foundry, focusing on image design (i.e., editing and generation capabilities). The goal is to create AI agents that can analyze user input, generate design recommendations based on some instructions, patterns or user preferences, and create unique designs. The following sections will cover the solution, architecture design, and a step-by-step process to set up and develop the application.

What is the solution?

The solution is to leverage AI agents to streamline the image design process, enabling people to quickly adapt to changing trends, styles, and preferences. By using Azure AI Foundry and OpenAI models, people can create a comprehensive image design application that automates the analysis of user input, generates recommendations, and creates unique designs.
This application will allow people to focus on creativity and innovation, while the AI agent handles the repetitive and time-consuming tasks of analyzing instructions and preferences and generating designs. The application will also provide a user-friendly interface for people to interact with the AI agent, making it easy to input design requirements, reference images, and instructions.

What are the AI models used?

OpenAI models are state-of-the-art generative AI models that can be used for various applications, including natural language processing, image generation, and more. These models can help people create unique and personalized designs, analyze instructions and consumer preferences, and generate design recommendations. By integrating OpenAI models with Azure AI Foundry, people can create a powerful image design application that leverages the strengths of both technologies.

What is the architecture design?

To get started, the diagram below shows the architecture that will be developed. Simply put, it utilizes Azure AI Foundry for agent orchestration, OpenAI’s image generation model (gpt-image-1), blob storage to store instructions (i.e., documents) and images, and an app service to host the Streamlit UI.

Enter detailed text prompts describing the desired image design.
Select existing patterns or preferred styles from a library or upload a reference image to guide the AI.
Choose an instruction to inform the design.
Generate initial images using AI based on inputs.
Refine designs by modifying prompts or editing the generated images.

Step-by-Step Process to Setup and Develop the Application

This section provides a technical look "under the hood" to demonstrate how the previously mentioned architecture was implemented. The following code snippets and explanations detail the core logic, from interpreting a designer's initial text prompt to generating a final, production-ready images. We will walk through how each component—setting up the Azure AI Foundry project, AI agent, user input analysis, reference image processing, instruction integration, and image generation—is built and orchestrated using Python, Azure AI, and OpenAI models.

1. Setup AI Project on Azure AI Foundry

Start by setting up an Azure AI Foundry account and creating a new project. This will provide you with access to the necessary tools and services to create custom AI models. Azure AI Foundry can host multiple projects, you need to first create a Hub, and a Project within it. For the details of how to create a Hub and Project, please refer to the Azure AI Foundry documentation. The following screenshot shows an example.

In this example, we create an Azure AI hub and link it to the subscription of "IX Sandbox". And then we create a project as shown below.

After creating the project, you will need to create/enable Azure AI services that will be used in the project. The following screenshot shows an example of creating/enabling Azure AI Services.

In this example, we create an AI Services and then connect it to the project. The available services include Azure OpenAI, Speech, Content understanding, Translation and a lot of other Azure AI capabilities. For the details of how to create and manage Azure AI services, please refer to the Azure AI Services website.

Once the project and AI services are created, you can manage your AI models, data, and other resources. The dashboard provides an overview of your project and allows you to monitor the performance of your AI models. Let's set up some AI models to be used in this project. Go to the "My assets" section on the left panel, and then select "Models + endpoints". You can see the list of already deployed AI models that are available in the project. The following screenshot shows an example.

By clicking the "+ Deploy model" button and selecting "Deploy base model", you can deploy a base model from the Azure AI Foundry model catalog. Follow the instructions to select the models that will be used in the project, such as OpenAI gpt-4o and OpenAI gpt-image-1.

To trace and monitor the AI agent behavior and activities, we can set up and enable Azure AI Foundry Application Insights Tracing feature to log and visualize the information when the AI agent is being called. Find the “Tracing” on the left-hand side of navigation and then click the “Create New” button shown in the screenshot below.

And then fill the form on the popup window to create a new Application Insights resource. After you run the AI application, you can view the list of traced agent activities in the Tracing section. When you click on a tracing item, the popup window will show and visualize the information logged while the AI agent is being called and running, like the screenshot shown below. Tracing provides deep visibility into execution of your application by capturing detailed telemetry at each execution step. This helps diagnose issues and enhance performance by identifying problems such as inaccurate tool calls, misleading prompts, high latency, low-quality evaluation scores, and more.

2. Setup Azure AI Search Service

The next step is to load a document. This will be used to provide instruction into the design requirements. The document can be uploaded into a container of an Azure blob storage account and indexed by the Azure AI Search service, as shown in the screenshot below.

The technology behind the Azure AI Search service is called "Retrieval Augmented Generation (RAG)", which is a powerful tool that can be used to index and search large amounts of unstructured data (e.g., PDF documents). For the details of how to use Azure AI Search service to index and search document contents in a blog storage, please refer to the Azure AI Search service documentation. And then, we can configure an AI agent in the Azure AI Foundry to use the Azure AI Search service to search the instruction and extract the relevant information.

3. Create an AI Agent

We will create an AI agent in the Azure AI Foundry project, which will analyze the instruction and provide a design recommendation into the design requirements. The following screenshot shows an example of the AI Agent to search an instruction and generate a design recommendation.

In this example, we create an AI agent and select the deployed gpt-3.5-turbo model and the created index in the "Knowledge" section.

4. Develop Code to Analyze User's Input

Defining the design requirements would include factors such as target audience, style, preferences, and standards. For this component, we will let the designers to type the requirements in a text box and then the requirement will be used to create a prompt for the OpenAI model. In order to understand what type of images the designer is looking for, we need to analyze the user input and determine the image type. The following code snippet shows how to analyze the user input and determine the image type by leveraging OpenAI gpt-4o model.


from openai import AzureOpenAI
import os

def analyze_user_input_agent(user_input):
    # Azure OpenAI client
    client = AzureOpenAI(
        api_version="2024-12-01-preview",
        azure_endpoint=os.getenv('AZURE_GPT4_MODEL_ENDPOINT'),
        api_key=os.getenv("AZURE_API_KEY")
    )
    completion = client.chat.completions.create(
        model="gpt-4o", # or gpt-4o-mini for cost saving
        messages=[
            {
                "role": "user",
                "content": f"""You are a helpful assistant that can analyze the user's input and then determine the image type.
                For example, if the user's input is "I want to design a clothing", the product type is "clothing". 
                If the user's input is "I want to design a house", the product type is "house". 
                Now, here is the user's input: {user_input}
                Please determine the image type based on the user's input. If the user's input is not clear, return the general product type of "drawing image".
                """
            },
        ],
        temperature=0.0,
        top_p=1.0,
        frequency_penalty=0.0,
        presence_penalty=0.0,
    )

    return completion.choices[0].message.content # The image type response from the AI model

5. Develop Code to Add a Reference Image

The next component is to add a reference image for the image design. The idea of using a reference image is to provide a visual representation of the silhouettes, design patterns or detailed requirements. This can help the AI model to better understand the user's input, guidance, or/and patterns, and generate more accurate design images. The reference image can be uploading an existing image or picking a pre-defined design outline image from a list. The uploaded image can be converted into a design outline image. For example, the following shows a reference image, which is a pre-defined design outline image for a girl's dress.

This type of reference image will be used to generate new designs based on the user's input of the design requirements. The following code snippet shows how to convert an uploaded image into a reference image.

import os
from openai import AzureOpenAI
import base64


def convert_image_agent(image_file):
    client = AzureOpenAI(
        api_version="2025-04-01-preview",
        azure_endpoint=os.getenv('AZURE_IMAGE_MODEL_ENDPOINT'),
        api_key=os.getenv('AZURE_API_KEY')
    )
    prompt = "Convert the given image to a simplified outline vector .svg image."
    result = client.images.edit(
        model="gpt-image-1",
        image=image_file,
        prompt=prompt,
        )
    image_base64 = result.data[0].b64_json
    image_bytes = base64.b64decode(image_base64)
    with open("temp/vector_image.png", "wb") as f:
        f.write(image_bytes)

    return "temp/vector_image.png"

6. Develop Code to Run the AI Agent for Instruction Analysis

The following code snippet shows how to leverage the AI agent to generate a design recommendation based on the instruction.

from openai import AzureOpenAI
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from azure.ai.projects import AIProjectClient
from azure.core.credentials import AzureKeyCredential
import os

def analyze_instruction_agent(image_type):
    prompt_instructions = """
    You are a designer and will review the design instruction by using file search tool and generate a design recommendation.
    Don't ask the user any questions, just try your best to generate the recommendation based on the report review by following guidelines below:
    • One sentence to describe current and forecasting future trends (colors, silhouettes, patterns, themes).
    • One sentence to describe target customer needs, preferences, and lifestyle.
    • One sentence to describe any other relevant information, such as background, practices, and styles.
    """

    # Azure AI Project client
    agent_client = AIProjectClient.from_connection_string(
        credential=DefaultAzureCredential(),
        conn_str=os.getenv("AZURE_PROJECT_CONNECTION_STRING")
    )

    # Setup the Azure Monitor for the Azure AI Project agent
    monitor_connection_string = agent_client.telemetry.get_connection_string()
    configure_azure_monitor(connection_string=monitor_connection_string)

    # Setup the OpenTelemetry tracer
    tracer = trace.get_tracer(__name__)
    with tracer.start_as_current_span("analyze_instruction_agent"):
        # Azure AI Project agent
        agent = agent_client.agents.get_agent(os.getenv("AZURE_AGENT_ID"))
        thread = agent_client.agents.create_thread()
        if image_type:
            message = agent_client.agents.create_message(
                thread_id=thread.id,
                role="user",
                content=f"Instructions: {prompt_instructions} \n\n Request: What recommendations can you provide for {image_type} design"
            )
            run = agent_client.agents.create_and_process_run(
                thread_id=thread.id,
                agent_id=agent.id)

            messages = agent_client.agents.list_messages(thread_id=thread.id)
            response = messages.text_messages[0].get("text")["value"]
        else:
            response = ""

    return response # The design recommendation response from the AI model

7. Develop Code to Generate Design Images

The next component is to generate the design image(s) based on the user input, reference image, and instruction. The generated design images will be unique and personalized designs that meet the design requirements. The following code snippet shows how to generate the design based image(s).


def generate_image_agent(user_input, image_type, recommendation, num_designs: int, reference_image, image_file=None):
    prompt = f"""
        You are a professional image designer with deep understanding of patterns, styles, and audience preferences.
        Your task is to generate a design image based on the user's thoughts and ideas of {user_input} and product type of {image_type}
        and the design recommendation: {recommendation}, which is based on an instruction.
        Adhere to the image pattern from the given vector image. 
        Make the image a realistic image that would be found in an online website. 
        Make the background white.
        """
    client = AzureOpenAI(
        api_version="2025-04-01-preview",
        azure_endpoint=os.getenv('AZURE_IMAGE_MODEL_ENDPOINT'),
        api_key=os.getenv('AZURE_API_KEY')
    )
    if image_file:
        result = client.images.edit(
            model="gpt-image-1",
            image=[open(image_file, "rb"), open(reference_image, "rb")],
            prompt=prompt,
            size="1024x1024",
            quality="high",
            n=num_designs
        )
    else:
        result = client.images.edit(
            model="gpt-image-1",
            image=open(reference_image, "rb"),
            prompt=user_input,
            size="1024x1024",
            quality="high",
            n=num_designs
        )
    for index, image in enumerate(result.data):
        image_base64 = image.b64_json
        image_bytes = base64.b64decode(image_base64)
        # save each generated image to a file
        with open(f"temp/generated_image_{index+1}.png", "wb") as f:
            f.write(image_bytes)
    return result

The following image shows an example of the generated design image based on the user input, reference image, and instruction.

8. Develop Streamlit UI to Interact with the AI Agents

Finally, we will develop a Streamlit UI to interact with the AI agent and generate design images. For the details of how to develop a Streamlit UI, please refer to the Streamlit documentation. Another place to explore on how to develop a Streamlit UI for AI applications is the Build powerful generative AI apps.

Conclusion

Azure AI Foundry offers a robust, scalable, and integrated platform for building, deploying, and managing AI agents across a wide range of industries and use cases. Its support for multi-agent orchestration, seamless integration with both proprietary and open-source models, and built-in monitoring and governance make it an ideal choice for organizations seeking to accelerate their AI adoption and innovation.

This post has contributed a clear explanation of why Azure AI Foundry stands out among cloud-based and open-source AI agent frameworks, highlighting its unique advantages in flexibility, interoperability, and enterprise readiness. Additionally, the step-by-step guide provided here serves as a practical resource for anyone looking to build AI agents—from initial setup to deployment—using Azure AI Foundry. By following these best practices, organizations can unlock new opportunities for automation, insight, and collaboration, positioning themselves for long-term success in the evolving landscape of artificial intelligence.

References

Building AI Agents: Semantic Integration of Structured and Unstructured Data using OpenAI Agent SDK

Yaohua Chen — Thu, 13 Mar 2025 00:17:39 +0000

What is this about?

This post is about building AI agents that can semantically integrate structured and unstructured data for advanced search and analysis. The ability to search and analyze data is becoming increasingly important in today's data-driven world. With the explosion of data sources and the increasing complexity of data formats, it is becoming more and more difficult for humans to manually search and analyze data. AI agents can help to automate this process, allowing organizations to quickly and accurately extract insights from their data and web. As a demonstration, an agentic application proposed in this post shows capabilities of leveraging a set of tools and techniques to search and analyze structured and unstructured data. The code for this application is available on GitHub.

What is the problem?

The problem is that organizations are struggling to search and analyze structured and unstructured data together. Traditional methods of searching and analyzing data, such as keyword-based search engines and manual data processing tools, are no longer sufficient to handle the dynamic user requests as well as volume and complexity of data that organizations are dealing with. These methods are time-consuming, require a lot of engineering manpower, and the results are not always accurate, relevant, and up-to-date due to the insufficient understanding of the user's query or request, context and meaning of the data.

For example, as an analyst in a healthcare organization, you may need to search and analyze patient records, medical reports, and research papers to identify patterns, trends, and insights that can help improve patient care. However, the data is stored in different formats, such as databases, spreadsheets, text documents, images, and videos, making it difficult to search and analyze the data effectively. Some organizations may put a lot of engineering effort to convert unstructured data into structured or key-word indexing data, but this is time-consuming and very costly. However, even with structured or key-word indexing data, the traditional methods of searching and analyzing data are not sufficient to handle the dynamic user requests. For instance, when you are trying to query a database to find the average age of patients with a specific medical condition, you may need to write complex SQL queries and join multiple tables, which can be time-consuming and error-prone, even if you are technically capable of doing so. And the results? They may still not be accurate and relevant, and even misleading due to the insufficient understanding of your request and background context (e.g., orgnaizational background or business scope).

Why is this important?

Combining structured and unstructured data in the way of semantic integration allows organizations to extract insights quickly and accurately, leading to better decision-making and actionable insights.

For example, as a healthcare analyst, you need to identify patterns and trends to improve patient care. By submitting your plain-language query, the system can automatically understand it semantically, retrieve and integrate relevenat data from various sources like patient records and medical reports, analyze the data, and present the results in an easy-to-understand format. This helps you make better decisions and draft reports with actionable insights for patient care improvement quickly.

With this type of system, organizations can improve their search and analysis capabilities, enhance decision-making, increase productivity, drive innovation, improve customer experience, and gain a competitive advantage. The key of this type of system is how it can accurately understand the user's requests or queries, related context, and then retrieve the relevant data from structured and unstructured data sources, analyze the data, and present the results in a meaningful way.

What is the semantic integration of structured and unstructured data?

The semantically integration of structured and unstructured data is the process of combining structured data, such as databases and spreadsheets, with unstructured data, such as text documents, images, web pages, and videos, into a unified, meaningful structure by understanding and preserving the user requests or queries, context, meaning, and relationships of the data. Semantic integration focuses on the meaning and context of data rather than just its structure. This includes linking related entities, resolving conflicts in data interpretations, and ensuring interoperability between systems and data. For example, for a healthcare organization, semantic integration could involve combining structured patient records with unstructured medical reports (e.g., PDF files) to provide a comprehensive view of a patient's health history.

The processing key components of semantic integration include:

User Query Understanding: Understanding the user's query and intent is the first step in semantic integration. This involves analyzing the user's query to determine the context, meaning, and relationships of the data.
Data Retrieval and Integration: Retrieving and integrating data from a variety of sources, including interal document stores, relational databases, and crawled web pages, is the next step in semantic integration. This involves combining data from different sources into a unified, meaningful structure.
Data Analysis and presentation: Analyzing, interpreting, and presenting the integrated data is the final step in semantic integration. This involves extracting insights from the data, such as summaries, classifications, and predictions.

Accordingly, the technical key components of semantic integration include:

User Interactive Agent: The user interactive agent is responsible for understanding the user's query and intent. This involves analyzing the user's query to determine the context, meaning, and which tools or function callings it could leverage.
Structured Data Retrieval Agent: The structured data retrieval agent is responsible for retrieving and integrating structured data from a variety of sources, such as databases and spreadsheets. This involves combining data from different sources into a unified, meaningful structure.
Unstructured Data Retrieval Agent: The unstructured data retrieval agent is responsible for retrieving and integrating unstructured data from a variety of sources, such as text documents, images, and videos. This usually leverages RAG (Retrieve, Analyze, Generate) approach and Retrival-Augmented Generation (RAG) technology.
Web Search Agent: The web search agent is responsible for crawling and indexing web pages to retrieve and integrate data from the Internet. This involves searching for relevant data on the web and extracting insights from the data.

The following diagram illustrates the technical key components of semantic integration of structured and unstructured data:

What are the other benefits?

Beside the benefits mentioned above, the semantic integration of structured and unstructured data provides a number of other benefits, including:

Flexibility: Organizations can easily adapt to changing data sources, formats, and user requests. The system can also be easily extended to support new features and capabilities as well as integrated with other systems and tools.
Efficiency: Organizations can process large amounts of data quickly and accurately because the system can automatically understand the user's requests, context, and meaning of the data and then use the appropriate tools and techniques to search and analyze the data based on the user's requests. The data volume and complexity are small and no longer a barrier to search and analyze data.
Relevancy: Organizations can extract relevant and valuable insights from their data. The system can automatically understand the user's requests, context, and meaning of the data, and then retrieve and integrate the relevant data from structured and unstructured data sources, analyze the data, and present the results in a meaningful way.
Real-time insights: Organizations can easily extract real-time insights from their data and Internet or web, allowing them to quickly respond to changing market conditions and user requests, and further improve their decision-making and customer experience.

How can it be implemented?

In this post, we will use OpenAI Agent SDK and other techniques to build an AI Agent application that can semantically integrate structured and unstructured data for advanced search and analysis. The application is built on top of the Python programming language and uses a number of popular libraries, including PandasAI, OpenAI, etc. The code for this application is available on GitHub.

Why use OpenAI Agent SDK?

OpenAI has introduced new tools, including Agent SDK, to help developers and enterprises build, deploy, and scale AI agents. The Agents SDK is an open-source framework that simplifies the orchestration of single-agent and multi-agent workflows. It includes features like intelligent handoffs, configurable guardrails for safety, and observability tools for tracking performance. It supports built-in web search, file search, and computer use, making it easier to build agents that perform various tasks autonomously. These tools address challenges in building production-ready AI agents, such as complex orchestration and extensive prompt iteration. By replacing manual prompt engineering and custom scripting with advanced tools, OpenAI makes agent development faster and more practical. These innovations empower businesses to automate complex workflows without extensive technical expertise, transforming operations across various industries.

What is the approach for unstructured data

One of the key techniques used in the application is the RAG approach, which stands for Retrival-Augmented Generation (RAG). The RAG approach is a three-step process that allows agents to quickly and accurately extract insights from unstructured data. In the Retrieve step, the agent retrieves relevant data from a variety of sources, including text documents, images, and videos based on the user's request. In the Analyze step, the agent analyzes the retrieved data using a variety of techniques, including natural language processing, computer vision, and machine learning. In the Generate step, the agent generates insights from the analyzed data, such as summaries, classifications, and predictions. The RAG technology is a powerful tool for searching and analyzing unstructured data, allowing agents to quickly extract valuable insights from their data.

What is the approach for structured data

The application also uses a number of techniques to search and analyze structured data. One of the key techniques used in the application is PandasAI, which is a powerful library for data manipulation and analysis. PandasAI provides a number of tools for working with structured data, including data structures, data manipulation functions, and data analysis functions. By using PandasAI, agents can quickly and easily process structured data, allowing them to extract valuable insights from their data.

Notice: we don't use the RAG approach here since structured data is already organized in a way that makes it easy to search and analyze. Structured data is typically stored in databases or spreadsheets, which provide a consistent format for storing and retrieving data. Because structured data is already organized, agents can quickly and easily extract insights from structured data without the need for the RAG approach. Instead, agents can use tools like PandasAI to quickly process structured data and extract valuable insights.

Which agentic pattern does it use?

In this agentic application, we use the "agent orchestration" pattern. This pattern is used to build agents that can perform complex tasks by breaking them down into smaller, more manageable sub-tasks. Each sub-task is handled by a separate agent, which is responsible for retrieving and processing the data. The agents work together to complete the task, passing data back and forth as needed. For example, you might have a frontline agent that receives a user's request, and then hands off to a specialized agent based on the language of the request. The specialized agent then retrieves and processes the data, and hands off the results back to the frontline agent, which presents the results to the user. This pattern is ideal for building agents that need to perform multiple tasks in parallel or in sequence, such as searching and analyzing structured and unstructured data.

Conclusion

The agentic application proposed in this post provides a set of tools and techniques that can be used to build intelligent agents that can search and analyze structured and unstructured data. It is capable of processing large amounts of data quickly and accurately, allowing organizations to extract valuable insights from their data. The application is designed to be flexible and extensible, allowing developers to easily add new features and capabilities to their agents. The application is built on top of the Python programming language and uses a number of popular libraries, including PandasAI, OpenAI, etc. The application is designed to be easy to use and understand, making it ideal for both beginners and experienced developers.