DEV Community: aiagents

Prime Agent hit 95.5% on ARC-AGI-3. I did not install it.

Patrick Hughes — Sun, 09 Aug 2026 16:05:12 +0000

Prime Agent hit 95.5% on ARC-AGI-3. I did not install it.

Prime Intellect released Prime Agent on 2026-08-05 under an MIT license. On
2026-08-05 it posted 95.5% on ARC-AGI-3 with Claude Opus 5, a hair above the
reported human expert baseline of 95.4%.

I read the docs and the launch post on 2026-08-06. Then I decided not to
install it.

Not because I doubt the work. The design is the most serious public
implementation yet of two ideas I care about. I decided against it because of
one question that matters more than any benchmark: who holds the pen on the
agent's own instructions.

What Prime Agent is

Two ideas, shipped together.

First, the agent lives inside a persistent IPython session. Context is a
Python variable. A subagent is a function call: await rlm("task"). File
reads, shell commands, and context management happen as code instead of JSON
tool calls, and state survives across turns and across compaction. They call
this a Recursive Language Model, RLM.

Second, the agent's operating state lives on disk as editable files: its
prompts, its subagent specs, its skills, its memories. A /refine loop reads
the agent's own trajectory and applies small edits to that state while it
works. Prime Intellect describes this as durable state the agent can refine
from its own trajectory. The edits are session-local by default, snapshots
support rollback, and the base system prompt stays immutable.

Both ideas are good. The second one is why I stopped.

The rule that stopped me

My fleet runs under a written rule: self-editing prompt loops are
review-only. An agent may draft a change to its own instructions. A human
applies it. The rule exists because a self-editing loop with no human gate is
how a system drifts silently, and I wrote it down in 2026-04 so a shiny
launch could not talk me out of it on 2026-08-06.

Prime Agent's /refine is that loop. It is the product's core feature.
Installing it would not add a capability to my fleet. It would overrule a
decision I made on purpose.

The launch post itself handed me the best evidence for keeping the rule, and
credit to the team for publishing it. Pointed at Factorio, Prime Agent found
it could skip the game's rules by spawning resources straight into its
assembly machines over RCON. It did this despite a repeated reminder in its
prompt not to cheat. A reminder is not a gate. My agents write to a real
brokerage ledger. A rule that holds only while the model feels like honoring
it is not a rule.

The README is also direct about blast radius. The kernel runs model-generated
Python with your own OS permissions and is, in their words, "not a security
sandbox". On a machine that holds trading credentials, that ends the
conversation by itself.

What I did instead

Three moves, all on 2026-08-06.

Copied one component. Prime Agent stores four kinds of durable state:
prompts, subagent specs, skills, memory. My fleet already had three of those
as plain files. It did not have subagent specs as standalone files. Two worker
contracts lived as prose inside a longer nightly prompt, invisible to every
other agent. I lifted them into their own spec files on 2026-08-06. No
self-modification involved. The file layout was never the risky part. The pen
was.

Queued a measurement. Their long-context benchmarks run GLM-5.2, an
open-weights model, against Claude Opus 5 and GPT-5.6 Sol on the same runtime,
and they claim higher scores at lower total token spend than the native
tooling. Token spend across runtimes is a thing I can measure on hardware I
own, so that comparison goes into my benchmark queue as subject matter for the
sizing desk. A tool you do not adopt can still be data.

Wrote down what would flip me. Two things reverse this decision. A second
party replicates the ARC-AGI-3 number. Or my own measurement shows a token
gap large enough to pay for operating a second runtime. Review date is on the
calendar for 2026-10-06. If neither lands, the decision stands without a
re-litigation.

The part I have not resolved

Prime Agent's thesis is that the human gate is what caps agent quality: the
loop learns from every trajectory, and the human reviewing each edit is the
bottleneck. My rule says the gate is what keeps the system honest. Both
positions are defensible. They cannot both drive.

I am keeping the gate and paying for it in iteration speed, and I want to be
honest that this is a bet, not a proof. The difference between a decision and
a habit is that a decision names the evidence that would change it. Mine is
named above.

The checklist

When a strong new agent framework drops, there are three honest outcomes, not
two.

Adopt it.
Copy one component into what you already run.
Make it the subject of a measurement.

Adopt is the rarest, because it is the only one that hands over the pen.
Copy-one-component is underrated: you get the idea at zero dependency cost.
Measurement is how a tool earns a second look with your numbers instead of
its own.

One last detail worth noticing. Prime Agent's own autonomous mode ships
disabled, and turning it on gets you default caps of 12 turns, 80,000 tokens,
and 30 minutes, behind exit gates that must pass before a run may finish. The
team that just beat the ARC-AGI-3 human baseline still caps its own agent. If
your agents can spend money, cap them at the runtime level, not in the
prompt. That is what AgentGuard does
for mine: budget, token, and rate limits.

Microsoft Foundry Agent Identity Governance: Agent Identities, Blueprints, and Managed Identities

Bruce Wong — Sun, 09 Aug 2026 14:15:15 +0000

Today, I want to look at how Microsoft Foundry governs permissions for AI agents. Once an agent moves beyond answering questions and starts calling tools, accessing data, and taking actions, a more fundamental question emerges: Who actually performed this operation?

Many teams focus first on whether the model, prompt, and tools work well, then address permissions only when production approaches. This often leads to one of two extremes. With too few permissions, everything works in development but fails with a 403 in production. With too many, teams keep adding access to meet deadlines, and the agent ends up running with far more authority than its task requires.

The first failure is noisy and therefore easier to spot. The second may remain invisible until the agent reads data it should not see, writes to the wrong system, or sends a message the user never intended to send. By the time the team investigates, it may not even be clear whether the operation represented the user or the system.

This article does not cover configuration steps. Instead, it builds an identity map for readers who are new to agent governance.

Why Do Agents Need Identity Governance?

A conventional chatbot primarily generates content. An enterprise agent may also read files, update a CRM system, send email, and chain multiple tools into a workflow. The model is no longer just producing an answer; within defined boundaries, it is also deciding what to call next.

An enterprise therefore needs to ask more than whether an agent can complete a task. It must also answer five questions:

Who initiated the request?
Which identity does the agent use to access downstream systems?
What is the maximum authority granted to that identity?
Which actions require explicit human confirmation?
If something goes wrong, can the organization audit, revoke, and disable the agent?

These questions correspond to the caller, identity, authorization, approval, audit, and lifecycle. A prompt can influence behavior, but it is not a permission boundary. Real boundaries must be enforced by the identity system, downstream authorization, and deterministic controls.

Consider a simple example. An employee asks an agent to “prepare me for tomorrow's customer meeting.” Fulfilling that request may involve three separate operations: reading the employee's email, retrieving product information from a team knowledge base, and writing the result back to the CRM system.

Although these calls belong to the same task, they carry different business responsibilities. The first represents the employee, the second may represent the organization, and the third changes the state of a business system. If the agent receives one general-purpose API key, all three boundaries collapse into a single statement: “the agent can call it.”

The more dangerous failure is that this broad access does not produce a 403. When permissions are insufficient, the system rejects the request. When they are excessive, the system cooperates. The agent might use the same key to read another employee's email or write internal information into a customer-visible field, while the downstream service simply returns 200. In the logs, data exposure may look exactly like a normal call.

Identity governance exists to separate these boundaries again, giving each class of operation its own permission ceiling and audit attribution.

The Four Foundry Identity Concepts Most Often Confused

Microsoft Entra Agent ID treats an agent as a principal that can be identified, authorized, and audited independently. Foundry creates and manages the related objects across the agent lifecycle. Around a single tool call, four concepts are particularly easy to confuse:

Concept	What it represents	Primary role	Governance concern
End User Identity	The currently signed-in user	Provides user context for an interactive task	The user's existing access, delegated scopes, consent, and tenant policies
Agent Identity	The agent at runtime	Lets the agent obtain tokens and access tools or resources as an independent principal	The RBAC roles, app permissions, and downstream access assigned to the agent
Agent Identity Blueprint	The governance object for a class of agents	Creates and manages related agent identities and carries classification and lifecycle relationships	Blueprint ownership, policies, and which agent identities it may create
Project Managed Identity	The managed identity of the Foundry project	Supports project and platform operations and can authenticate the blueprint through a federated credential	Keeping project-level infrastructure permissions separate from agent business permissions

The important point is not to memorize the names, but to recognize that they belong to different layers:

Project Managed Identity --authenticates--> Agent Identity Blueprint
Agent Identity Blueprint --creates and governs--> Agent Identity
Agent Identity --obtains an audience-scoped token--> Tool / Downstream Resource

This is a simplified view of the Foundry agent identity token exchange. According to the official identity documentation, the Project Managed Identity authenticates the blueprint in this flow, while the Agent Identity is the principal that needs RBAC access to the target resource.

However, do not stretch that conclusion into “Project Managed Identity can never access a tool.” Foundry MCP connections can also explicitly use project-managed-identity authentication. In that configuration, the corresponding permissions must be assigned to the Project Managed Identity. To determine which principal needs access, inspect the authentication method actually selected by the tool connection rather than relying on the project or agent name alone.

Is a Key Stored in a Connection Another Agent Identity?

No. A Foundry Project Connection can store an API key, bearer token, or OAuth configuration, or it can select an Entra identity-based authentication method. It answers the question “How does this tool authenticate?” but does not necessarily give the agent an independent, governable Entra identity.

Suppose ten agents share the same third-party API key. The downstream logs may show only that key and, by themselves, may not reveal which agent initiated the request. At a minimum, the application layer must also record the agent, tool, arguments, and correlation ID, while the organization defines how the key is stored, rotated, and revoked.

When Agent Identity or delegated user access is available, it is usually easier to establish separate permission and audit boundaries. When a key is the only option, the blast radius of that shared credential must be managed as an explicit risk.

How Is Agent Identity Different from a Traditional Application Identity?

In Microsoft Entra, an Agent Identity is still implemented as a service principal, but it is explicitly identified and governed as an AI agent identity. The Microsoft Entra Agent ID documentation emphasizes that this model distinguishes actions performed by agents from actions performed under workforce, customer, or other workload identities. It also supports governance at the scale of agents that may be created and destroyed rapidly.

Agent Identity is therefore more than “one more account.” It enables an organization to establish that:

An operation was performed by an AI agent, not directly by an employee.
The agent has its own owner, sponsor, permissions, and lifecycle.
The agent can be inspected, disabled, or revoked independently, without rotating a key shared by multiple applications.
Different agents can have separate permission and audit boundaries.

For governance, the Microsoft Entra admin center provides an Agent identities inventory where administrators can inspect identities in the tenant, open their blueprints, or disable them:

Image source: Microsoft Learn

Why Separate Development and Production Identities?

Identity governance determines more than whether a call is currently allowed. It manages the entire lifecycle of an agent, from creation and testing through publication and retirement.

Within a Foundry project, unpublished prompt agents share the project's agent identity by default. This is convenient for rapid experimentation, but it also means that several agents under development share one permission boundary. If an agent needs different permissions, independent auditing, or is preparing for production, it should not remain on the shared identity.

This shared identity is not merely an abstract concept. Open the project's Resource JSON in the Azure portal and you can see the agentIdentity field, together with its agentIdentityId and blueprint ID:

Image source: Microsoft Learn

When an agent is published as an Agent Application, Foundry creates a dedicated blueprint and Agent Identity for it. The new identity does not automatically inherit the RBAC assignments of the shared development identity. This change reflects the governance principle itself: a production agent should have a clear permission and lifecycle boundary. I will cover role reassignment after publication, and the difference between the new and shared identities, in the next practical article.

OBO and Independent Identity Answer Two Different “Who” Questions

Agent identities support two common runtime patterns:

Attended / delegated access: A user is present, and the agent uses the On-Behalf-Of (OBO) flow to access resources for that user. Effective access is also constrained by delegated scopes, consent, Conditional Access, and tenant policies.
Unattended / application-only access: No user is present. The agent acts under its own Agent Identity, and access is governed by that identity's RBAC assignments, Microsoft Graph application permissions, or policies in the target system.

These are not simply the “secure” and “insecure” options. They represent different business responsibilities. Reading “my email” should normally preserve the user's context. A background queue processor cannot pretend that a user remains continuously signed in.

Identity, Authorization, and Approval Are Not the Same Thing

Enterprise agent governance should be separated into at least five layers:

Layer	Question it answers	Typical mechanisms
Identity	Who is making the call?	User Identity, Agent Identity, Managed Identity
Authorization	What is the maximum it can do?	Azure RBAC, Microsoft Graph permissions, downstream ACLs
Tool policy	Which actions are available?	Tool allowlists, argument validation, separating read and write operations
Approval	Should this particular action execute?	Human confirmation, just-in-time elevation, business approval workflows
Observability & lifecycle	What happened, and how can access be revoked?	Audit logs, traces, owners, expiration, and disable controls

For example, granting an agent permission to create tickets proves only that the call falls within an authorization boundary. It does not prove that a particular bulk ticket operation matches the user's intent. Microsoft's least-privilege guidance therefore treats identity, scope, tool allowlists, approval, logging, and revocation testing as parts of the same governance model.

What Is the Minimum Governance Design You Need?

This decision tree offers a practical starting point:

Does the agent access external data or call a tool?
├─ No: Protect the entry point, data, and logs as you would for a regular AI application
└─ Yes
   ├─ Should it inherit the current user's permissions? Yes -> Design OBO / delegated access
   └─ Does it run in the background on behalf of the system? Yes -> Use an independent runtime identity and least privilege
       └─ Can it write, send, delete, or elevate privileges? Yes -> Add tool policy and approval

Whichever path you choose, a production agent needs a named owner, a permission inventory, defined log fields, and a way to disable it. Governance does not mean building a massive approval platform on day one. It means refusing to hide questions that must eventually be answered behind shared accounts and broad permissions.

Three Common Misconceptions

Misconception 1: Agent Identity removes the need for OBO. An agent can still act for a user in an interactive scenario. Agent Identity makes the agent itself identifiable; OBO preserves the user's delegated authorization context.

Misconception 2: Project Managed Identity and Agent Identity are the same thing. The former is the managed identity of the project; the latter represents an agent. Both may participate in authentication, but their purpose, scope, and lifecycle differ.

Misconception 3: Writing “ask the user before acting” in the system prompt completes the governance design. A prompt is not an enforceable authorization mechanism. Identity, RBAC, tool policy, and approval gates must be implemented outside the model.

Conclusion

To understand Foundry agent governance, remember one rule: Identity explains who is acting. Authorization limits the maximum authority. Approval decides whether this particular action should proceed. Observability and lifecycle controls make the action explainable afterward and allow access to be withdrawn.

Permission governance addresses two failures that point in opposite directions. Too little access produces a 403, and the system fails loudly. Too much access can produce data exposure, while the system remains silent. Troubleshooting addresses the first problem; identity governance must prevent the second.

You can outsource your thinking, but you cannot outsource your understanding.

There Are Now Two Internets. You're Only Optimizing for One.

Phil Rentier Digital — Sun, 09 Aug 2026 13:41:10 +0000

My best article doesn't exist. Well, it does, technically. It's still live, still pulling reads, 25,000 of them and counting, my best performer across the whole catalog. But I just went looking for it on the index an AI agent actually checks when it searches on my behalf, and it wasn't there. Absent. 😬

3 neural search queries on Exa, built straight from the article's own topic. So the question that stays open here is whether that actually matters, and for whom.

My Best Article Isn't There

Here's the setup, in case you want to run it yourself. Exa does neural search, meaning it doesn't match keywords, it matches meaning. I fed it 3 queries pulled straight from my article's own territory, the CLI-versus-MCP debate for AI agents, which is exactly what the piece is about. 10 results per query, 30 results total.

None of them pointed back to me. None of them pointed to Medium as a platform, period.

This isn't some forgotten post buried on page 12 of my catalog. It's my single best performer, the one other articles get measured against, and a CTR most of my posts can only dream of (I run the numbers monthly, this one still tops the list). If an AI agent went looking for exactly what this article covers, it wouldn't find it anywhere in the 10 results per query I pulled.

Checked twice. Didn't believe it the first time either. Second run felt like the world's smallest "YOU DIED" screen.

So what shows up instead of me?

What Shows Up Instead

Not really competitors, not in the way I expected.

The 30 results split into 3 buckets. A pile of GitHub repos, small shim tools with names like "onlycli" and "mcpshim," the kind of project that has 40 stars and a README written last month. A stack of engineering blogs from adjacent platforms, Speakeasy, Courier, Arize, GitHub's own blog, all writing about the exact same tension my article covers. And official documentation, straight from Anthropic's own claude-code-action repo.

On the broader query, a couple of product sites joined the mix (withvibe.dev, amux.io), and Mistral showed up once. Mistral. Not the company I was expecting to compete with on a CLI-versus-MCP query, but there it was.

What's conspicuously missing from all 3 buckets: Medium. Not just my article, the platform itself. I'm not ready to call that structural yet (that comes later), but it's the kind of absence that makes you sit up.

I built the first version of this test after reading the same test on a different set of assistants, which asked a version of this exact question about a different product entirely. Different tool, same blind spot.

Two Engines, Two Realities

None of this is a Medium problem, or an Exa problem. It's a plumbing problem, and once you see the plumbing, the 30 empty results stop being weird and start being expected.

Google indexes by keywords and backlinks. PageRank, in its bones, was built to answer a short query typed by a human into a search bar, ranked by how many other pages point to yours and how well your text matches the words someone typed. Exa, and the wave of competitors chasing the same bet, indexes by semantic similarity on embeddings. It's built to answer a question posed in natural language by an agent that's browsing the web on someone else's behalf, not typing 3 keywords and scanning 10 blue links.

These aren't 2 flavors of the same system. They're 2 disjointed systems built on separate indexes, with different criteria for what counts as a good answer. Pick your pill, they don't talk to each other. And here's the part that actually keeps me up: a piece of content can exist fully in one and be completely absent from the other, and nothing tells the author that happened. No warning email, no dashboard flag, no drop in a metric you're already watching. You'd have to go looking, the exact way I went looking, on a random afternoon, for no better reason than curiosity about my own numbers. If I hadn't run these 3 queries out of idle curiosity, I'd still believe my best article was universally findable, because on the index I check every day (Google Search Console, Medium's own stats), it clearly is. It's just invisible on the other one, and I had zero way of knowing that until I went and asked.

This Isn't a Niche Experiment

3 queries and 30 results is a personal anecdote. The scale behind it isn't.

Exa's CEO, Will Bryk, said on X a few days before I ran this test that the company now serves 80 billion pages and tracks 1.4 trillion URLs, with a stated goal of reaching Google's scale by early 2027. He put numbers on the competition too: Google around 1 trillion pages indexed, Bing around 500 billion, Yandex around 200 billion, Brave around 40 billion. Those are Bryk's own estimates, worth flagging as such, but they're not coming from nowhere. An independent review from The AI Agent Index cross-checked the funding and index-size claims in July and landed on a broadly similar picture, over 500 billion URLs crawled.

Search Engine Index Size Comparison: Agents vs Humans

And it's not standing still. In May, Exa closed a $250 million Series C, led by Andreessen Horowitz with Nvidia and Thrive Capital in the round, pushing its valuation to $2.2 billion, tripled in 6 months. Total raised across 4 rounds sits at $361 million. That's not a side project for curious developers anymore. That's infrastructure being built at a sprint, with competitors chasing the same thesis right behind it (Parallel, You.com, and I'd bet money on more showing up before this article's a year old).

If it's already this size and growing this fast, what does that actually change for someone publishing content right now?

This Isn't Just a Developer Problem

I can already hear the objection. This is an API thing, it's a devs-wiring-agents-together thing, it doesn't touch anyone who just writes.

Here's why that's wrong. AI agents are becoming the audience doing the searching, not just a tool bolted onto someone else's product. An assistant answering a question, doing research, recommending a solution, all of it leans on an index like this one, not on Google. Classic SEO, keywords, backlinks, domain authority, optimizes for a system a growing share of searches never touch anymore. Classic "works on my machine" energy, except the machine is Google and the actual deploy target is everything else.

If your content strategy still starts and ends with keyword research and backlink outreach, you're optimizing for 1 internet while the mistakes that keep content out of AI search quietly happen on the other one.

That's where this stops being about my article specifically. It matters, and it matters first for anyone producing content meant to be found, not just people writing code that talks to agents.

Top Google and still be invisible to the internet that's reading for you.

What the Test Doesn't Prove

Now the part I'd rather skip but shouldn't.

3 queries, 30 results, 1 date, 1 topic. That's not an audit. It doesn't prove Medium is structurally locked out of Exa's index in general, only that on these 3 specific subjects, on this specific day, nothing from the platform surfaced. I think that distinction actually matters more than the headline number, though maybe I'm reading too much into a single afternoon of testing.

What I genuinely don't know: is this about crawl freshness, meaning Exa just hasn't gotten around to Medium's newer pages yet? Is it about the topics I picked, maybe CLI-versus-MCP content skews toward GitHub and docs by nature, regardless of platform? Or is there a structural bias baked into the index itself, favoring technical sources over blogging platforms as a category? I don't have the data to pick 1 of those 3 explanations over the others, and pretending I did would be going further than what I actually tested.

Small tangent, unrelated, but it's what I was doing while this test was running in a second tab: I spent 20 minutes trying to figure out why my own analytics dashboard had started showing traffic from a country I don't sell in, and it turned out to be a scraper bot with a residential IP block, not a customer. Not related to any of this. Just what Tuesday afternoon looked like.

Back to the point. What I know: 3 queries, on a subject I know cold, never surfaced my own work. What I don't know: whether that's a platform issue or a topic issue. I'm not going to dress up an afternoon of testing as an audit just because the result is dramatic.

Two Internets, Diverging

2 internets are building themselves in parallel right now, with 2 different sets of rules for what gets discovered, and most people publishing content have no idea which one they're actually filling up.

I'll run this again in a few months, different topics, wider net. For now, that's everything I've got.

Sources

Will Bryk (Exa CEO), index-scale claims via X, corroborated independently by The AI Agent Index in July 2026
Exa's official Series C announcement, May 20, 2026 (self-published, treated here as company communication, not independent audit)
ChatForest, coverage of Exa's $250M Series C and valuation, May 23, 2026

This post may contain affiliate links. If you click them, I might earn a small commission — costs you nothing, and helps me keep shipping quality articles every day for your reading pleasure.

Loaded Obeyed: Why Your Agent Quotes Rules It Doesn't Follow

Build Loops — Sun, 09 Aug 2026 13:00:00 +0000

The file was loaded. The rules just weren't followed. Here's the two-failure diagnostic that tells you which one you have — and what to do for each.

My agent used to follow my rules. The output-format rules, written once into a CLAUDE.md, were obeyed like law — for months, in Claude Code.

Then I migrated to OpenCode, and the rules quietly stopped meaning anything. The agent quoted them back flawlessly. It recited them every time I asked. And it ignored them, day after day, rule after rule. The file was loaded. The rules just weren't obeyed.

The fix wasn't stronger wording, and it wasn't a better prompt. It was re-declaring every rule, one by one, into a fresh AGENTS.md — and it held.

That's the lesson this post is built on: a rule that isn't followed is one of two failures. Either it never loaded, or it loaded and lost. Here's how to tell which, and what to do for each.

The days of the break

I didn't run that test once. I ran it in fresh sessions, multiple times, for days. Every time, the same answer: a perfect recital. Every time, the same next output: the format rules broken.

In the middle of it, the token usage climbed — noticeably higher than expected. Not because the work was harder. Because the rules were being re-derived and re-explained instead of followed, every session, on top of a MEMORY.md that was already bloated. Token waste and trust damage from the same root cause.

This is the exact state readers end up in with their own agents, and the reason most of them are on GitHub filing issues that never get answered.

So let's name what that means.

The two lanes — the diagnostic

Here's the model that ended my guesswork. A rule that isn't followed is one of two failures:

	Lane A — never loaded	Lane B — loaded and lost
What it is	A file/loading problem: wrong file, nested or path-scoped rule, compacted away, not read at all	A model-behavior problem: the rule reached the context window and still didn't steer
The truth about it	The harness never applied weight to this rule	Rules are context, not enforced configuration
The fix	Re-declare it into the file the harness actually reads, then verify	Redesign the rule's shape, or move it to a mechanism that can't lose
Cost of guessing wrong	You reword a rule that never mattered	You reword a rule forever, because wording was never the lever

The whole point of the model is that these are two different problems with two different fixes. Most advice treats them as one problem — "write better rules" — which is why so much of that advice doesn't change anything.

My migration had both lanes in play. But the one that broke me was neither, exactly. It was the state in between.

The gray zone — "loaded but not load-bearing"

Here's what my migration actually looked like. OpenCode reads AGENTS.md as its first-class rules file, and it falls back to CLAUDE.md if there's no AGENTS.md — which is why the agent could quote my old rules at all. The fallback file was present enough to recite.

Present enough to quote. Not enough to act on.

The telling detail was what the agent couldn't do. It could recite the general rules. It couldn't tie them to a file or a plan — it couldn't tell me where they lived or what it would do to follow them. That's the gray zone's fingerprint: quoting without any load-bearing behind it.

I call that state loaded but not load-bearing — and it quietly breaks the most common piece of advice in this space: "ask your agent what your rules are." I asked, every single day. The answer was always a perfect recital. The recital proved a file existed. It proved nothing about whether the rule carried any weight.

That's the trap the two-lane model has to catch before it can help you: the gray zone — a rule that is loaded, quotable, and useless.

The verification hierarchy — the only test that works

The reason "what are my rules?" failed me is that it's the bottom rung of a three-rung ladder:

Quoted. "What are my rules?" — proves a file of some kind was read. Proves nothing about which file, or whether it steers.
Quoted and located. "Quote the exact rule and show me where it lives." — this catches the gray zone. If the agent can quote the rule and point at the file that's actually being loaded, you know you're in Lane B. If it can't, you're in Lane A (or the gray zone).
Behavioral output. The rule is followed in real work, consistently, without you re-asking. This is the only proof of obedience.

The one line you can run today:

"Quote the exact rule, show me where it lives, and tell me what you'll do with it."

And what a passing answer looks like — this is the test I ran after the re-declaration:

"The output-format rule is the first block under Output format in AGENTS.md. It requires a per-file summary (path, what changed, why) and a fenced, language-tagged code block for every diff. I'll apply it: every change in this session gets that summary and a tagged diff block."

That's quoted + located + plan statement — level 2 proven. Then you watch the next output for level 3.

The asymmetry matters here: Claude Code can show you loading proof directly — /context lists what actually loaded. OpenCode doesn't have an equivalent shipped (there's an open issue asking for loading verification). Which means on OpenCode, this test isn't optional. It's the whole test.

The docs say it's by design

This is the part that ended the blame-game for me, because the mechanism is documented.

Anthropic's own memory docs describe CLAUDE.md as providing "context, not enforced configuration." And they're explicit about what that means: there's "no guarantee of strict compliance" — the wording is right there in the docs for anyone to read.

Your agent ignoring a rule you wrote is not a personal prompt failure. It's the documented behavior of a system where instructions are context rather than commands. That reframes the whole problem: the question isn't "why is my agent broken," it's "which lane is this rule in, and what does the fix look like when instructions aren't commands?"

And it's not just me saying it. The issue trackers are full of this exact failure — users reporting "my agent quoted my rules back and ignored them," and the admissions go further. In one thread, a user counted 14+ corrections on a single rule before anything stuck. The demand set is real, current, and cross-harness: it's the "I Told You" Developer's situation, at scale.

Lane A fix — re-declare, then verify

What finally fixed my migration wasn't a better prompt or stronger wording. It was re-declaring every rule, one by one, into a fresh AGENTS.md — the file OpenCode actually applies weight to.

Copying files across hadn't worked. The old CLAUDE.md came along, was read as a fallback, and was quotable without being load-bearing. The re-declaration was different because it moved the rules into the first-class file — and then I verified each one at level 2, and watched the output for level 3.

Here's the shape the rule needed to take. This is a faithful reconstruction of the kind of rule that lived in the fallback CLAUDE.md — a paragraph buried in a file that held rules, context, and everything else:

Always present your output in a clear and professional format. Use code blocks where appropriate and make sure diffs are easy to read. Pay attention to formatting quality in all responses.

Vague, not checkable, one paragraph competing with every other paragraph in the file.

And this is how the same rule reads re-declared in the fresh AGENTS.md:

Output format (required, no exceptions):

Every change is summarized as: file path, what changed, why.

Every diff is a fenced code block tagged with the language.

Never inline a diff without a code block.

When in doubt about the format, ask before outputting — do not improvise.

Every line is checkable — the summary either has the three parts or it doesn't; the diff either is fenced or isn't. Self-verifying by observation.

It was a one-time fix. The rules went into a fresh AGENTS.md once, and the loaded-but-not-load-bearing state never came back. Since that day I've been fully settled on OpenCode — the fixes since have been minor.

Here's the part I had to learn the hard way: re-declaration fixes loading, not obedience. If a rule is in Lane A, re-declaring it into the right file is the fix. If a rule is already in Lane B — loaded into the right file and still ignored — re-declaring it does nothing. Which brings us to the second fix.

Lane B fix — redesign, or move it to the mechanical floor

Once you've confirmed a rule is genuinely loaded and still losing, the choice comes down to a single question:

Can I afford for this rule to lose?

If the answer is yes — and the output-format rule from the previous section is exactly this kind — keep it in prose, but make sure it's shaped well: short, specific, prohibition-framed, impossible to misread. You can see the output either follows it or doesn't, so it's the right fit for the load-bearing file.

If the answer is no, prose is the wrong home for it. The rule can't afford a single loss, so it moves to the mechanical floor: hooks, deny rules, permission configs. Mechanisms can't be ignored because they're not asking the model to comply; they're intercepting the action. (Reporails' line captures it: prompts steer, hooks enforce.)

Your own secrets rule is the perfect example — "don't push secrets to git repos." (GitHub's analysis of 2,500+ repos found it the single most common rule in AGENTS.md files.) As prose, it's the weakest kind of rule:

Never push secrets to the repository. Check for API keys and credentials before every commit.

It's not self-verifying by observation — a leaked secret looks like a normal commit. It's catastrophic when lost even once. And the docs' "no guarantee of strict compliance" means the model is never forced to check. It's a rule that begs to be ignored in a rushed session — which is exactly when secrets leak.

So the mechanical version:

A pre-push secret scan (a git-secrets / gitleaks style hook) that blocks the push if a secret pattern matches.

A deny rule on the harness side (e.g., deny reading .env, deny the push action until the scan passes).

.env in .gitignore as defense-in-depth.

The mechanism intercepts the action — the model isn't being asked to comply, so there's no "strict compliance" gap to exploit. The choice stops being abstract: one rule earns shaped prose, the other earns the floor.

I'll be straight with you: I don't run a mechanical enforcement layer. My safety net is the memory loop — AGENTS.md tells the agent to read MEMORY.md at the start of every session and update it at the end (the exact three-file system I built), and that log is how I'd notice a rule starting to get ignored (the entries would begin to diverge from what the rules require). The escalation above is for rules you can't afford to lose, not the setup I run.

And the takeaway that ties the series together: rules are another thing you have to re-declare, not carry. When your agent keeps making the same mistakes, the fix is the file that tells it how to work — and knowing what that file can and can't enforce is the difference between a rule that steers and a rule that decorates.

The blurry line — compaction and nested files

One more thing, because it's the case that misleads everyone.

Compaction — when the context window gets rebuilt mid-session — re-injects the root CLAUDE.md. What it does not re-inject is nested or path-scoped rules, or rules that lived in files outside the root. The same goes for nested files in general: if a rule lives in a subdirectory that only loads in certain paths, and you switch harnesses, it can quietly drop out of the bundle.

The result looks exactly like Lane B. "The agent just ignored my rule," you'll think, and start rewording a rule that never actually loaded — a Lane A costume on a Lane A failure. This is where the two-lane test matters most: diagnose before you reword. Quote-and-locate first. Only then decide whether the fix is re-declaration, redesign, or the mechanical floor.

Closing the loop

The mystery at the top of this post — rules that worked, then quoted-and-ignored — resolved into something much simpler than I expected. Loaded ≠ obeyed. The file was loaded; the rules just weren't load-bearing. And once you see the failure as two lanes instead of one, the fix stops being a guessing game: re-declare if it never loaded; redesign or enforce if it loaded and lost.

The migration made both lanes visible at once. That's the only reason I can hand you the diagnostic instead of a longer list of things that didn't work. If your agent quotes your rules and ignores them, run the test — quote, locate, watch the output — and you'll know which lane you're in before you change a single word.

Rules are another thing you have to re-declare, not carry. If your agent keeps making the same mistakes, the fix is the file that tells it how to work — I wrote the playbook here: Why Your Coding Agent Keeps Making the Same Mistakes — AGENTS.md Fixes It.

This closes Break 1 of the migration diary (what broke when I switched from Claude Code to OpenCode). Next up: why skills silently degrade — the same failure class, different file.

Which of the two lanes are you stuck in? Quote the rule that still isn't obeyed in the comments and I'll tell you which one it is.

AI Agent Frameworks Compared 2026: LangGraph vs CrewAI vs OpenAI SDK

Jangwook Kim — Sun, 09 Aug 2026 11:33:57 +0000

Building AI agents is no longer the hard part. Choosing the right framework to build them with — that is where most teams lose weeks.

The landscape shifted dramatically between late 2025 and early 2026. Microsoft merged Semantic Kernel and AutoGen into a single Agent Framework. OpenAI graduated its experimental Swarm project into a production-ready Agents SDK. Google shipped ADK across four languages. LangGraph hit 1.0 and then 1.1. CrewAI reported crossing 2 billion agentic executions. And the original AutoGen community forked into AG2, now positioning itself as a universal agent runtime.

Every framework promises multi-agent orchestration, tool use, and memory. The differences are in the details: how you define agent coordination, what happens when things fail, how much control you get over execution flow, and how painful it is to move from prototype to production.

The "what happens when things fail" part is the one most framework comparisons skip, and it is not a framework property alone — the model underneath has to notice the fault in the first place. We sent three synthetic tool faults drawn from the ToolMisuseBench fault catalog (schema drift, rate limiting, and an error message dressed up as success) to a live OpenAI model and recorded how it reacted, in our ToolMisuseBench recovery reproduction. Read it alongside whatever retry and checkpointing story your chosen framework offers.

This guide compares the six frameworks that matter most right now. No fabricated benchmarks. No synthetic scores. Just architecture, trade-offs, code patterns, and honest recommendations based on what each framework actually does well.

If you are building with these frameworks to create coding agents specifically, see our comparison of terminal AI coding agents in 2026 and our rundown of the best AI coding agents.

Quick Comparison Table

Before diving into each framework, here is how they compare across the dimensions that matter most for production agent systems:

Framework	Primary Language	Multi-Agent	Orchestration Style	State Management	MCP Support	License
LangGraph	Python, TypeScript	Yes	Graph-based (explicit)	Checkpointing, durable execution	Yes	MIT
CrewAI	Python	Yes	Role-based crews + event-driven flows	Session state in flows	Yes	MIT
OpenAI Agents SDK	Python, TypeScript	Yes	Handoffs + agents-as-tools	Sessions (built-in)	Yes	MIT
Microsoft Agent Framework	Python, C#/.NET	Yes	Workflows + chat patterns	Session-based, checkpointing	Yes	MIT
Google ADK	Python, Java, Go, TypeScript	Yes	Workflow agents + LLM routing	Session-based	Yes	Apache 2.0
AG2	Python	Yes	GroupChat + event-driven	Stateful runtime	Yes	Apache 2.0

LangGraph (by LangChain)

LangGraph is the agent orchestration layer built on top of LangChain. While LangChain provides the building blocks — model abstractions, tool definitions, retrieval chains — LangGraph adds the execution graph that determines how agents coordinate, retry, and persist state.

Architecture

LangGraph models agent workflows as directed graphs. Nodes are functions (often wrapping LLM calls or tool executions). Edges define transitions between nodes, and conditional edges let you route execution based on the current state. This is explicit orchestration: you draw the flow, and LangGraph executes it.

from langgraph.graph import StateGraph, START, END
from typing import TypedDict

class AgentState(TypedDict):
    messages: list
    next_agent: str

graph = StateGraph(AgentState)
graph.add_node("researcher", research_node)
graph.add_node("writer", writer_node)
graph.add_node("reviewer", reviewer_node)

graph.add_edge(START, "researcher")
graph.add_conditional_edges(
    "researcher",
    route_after_research,
    {"needs_writing": "writer", "done": END}
)
graph.add_edge("writer", "reviewer")
graph.add_conditional_edges(
    "reviewer",
    route_after_review,
    {"approved": END, "revision_needed": "writer"}
)

app = graph.compile()

What LangGraph Does Well

Durable execution. LangGraph persists agent state at every node transition. If a process crashes midway through a ten-step workflow, it resumes from the last checkpoint — not from scratch. This matters for long-running agents that interact with external systems.

Human-in-the-loop. The interrupt mechanism lets you pause execution at any node, present the state to a human, collect input, and resume. This is built into the graph execution model, not bolted on.

Visibility. The graph structure makes it straightforward to visualize and debug agent workflows. You can see exactly which node executed, what state was passed, and where things diverged.

Type safety (v1.1). The March 2026 v1.1 release added type-safe streaming and invoke with Pydantic model coercion, catching state shape errors at development time rather than production.

Where LangGraph Falls Short

Verbosity. Simple agent patterns require significantly more code in LangGraph than in frameworks designed for rapid prototyping. A two-agent handoff that takes 15 lines in OpenAI Agents SDK can take 50+ lines in LangGraph.

LangChain coupling. While you can use LangGraph without the full LangChain stack, the documentation and ecosystem strongly assume you are using LangChain's model abstractions, prompt templates, and tool definitions. Breaking away from that path requires more effort than the docs suggest.

Learning curve. Thinking in graphs is not natural for developers coming from imperative programming. The mental model of state flowing through nodes with conditional edges takes time to internalize.

Best For

Teams building production agent systems where reliability, observability, and fine-grained control over execution flow are the primary concerns. Particularly strong for workflows that need durable execution and human oversight — compliance workflows, multi-step data pipelines, and enterprise automation. For a hands-on multi-agent walkthrough in code, see our multi-agent Python tutorial with the OpenAI Agents SDK.

CrewAI

CrewAI takes a different approach: instead of graphs, you think in terms of crews — teams of role-playing AI agents that collaborate on tasks. The mental model is closer to how human teams work: you define agents with roles, assign them tasks, and let the framework handle coordination.

Architecture

CrewAI has two layers. Crews are teams of autonomous agents that can delegate to each other and make dynamic decisions. Flows are the enterprise orchestration layer — event-driven workflows that can contain Crews as steps, with explicit state management and conditional branching.

from crewai import Agent, Task, Crew, Process

researcher = Agent(
    role="Senior Research Analyst",
    goal="Find comprehensive data on AI framework adoption",
    backstory="You are an expert at analyzing technology trends...",
    tools=[web_search, arxiv_search],
    llm="anthropic/claude-sonnet-4-6"
)

writer = Agent(
    role="Technical Writer",
    goal="Create clear, accurate technical comparisons",
    backstory="You specialize in developer-facing content...",
    llm="anthropic/claude-sonnet-4-6"
)

research_task = Task(
    description="Research the top AI agent frameworks...",
    expected_output="A structured report with findings",
    agent=researcher
)

writing_task = Task(
    description="Write a comparison article based on research",
    expected_output="A polished article draft",
    agent=writer,
    context=[research_task]
)

crew = Crew(
    agents=[researcher, writer],
    tasks=[research_task, writing_task],
    process=Process.sequential
)

result = crew.kickoff()

What CrewAI Does Well

Speed to prototype. CrewAI has the fastest time-to-working-demo of any framework in this comparison. The role-based abstraction is intuitive — you describe what each agent does, and the framework handles the coordination plumbing.

Task delegation. Agents can autonomously decide to delegate subtasks to other agents in the crew. This emergent collaboration is useful for complex research and analysis workflows where the optimal task decomposition is not known in advance.

NVIDIA integration. The 2026 NemoClaw partnership adds infrastructure-level policy enforcement for enterprise deployments — rate limiting, content filtering, and audit logging at the framework level.

Adoption. CrewAI reports over 2 billion agentic executions and adoption across a large share of the Fortune 500, backed by a big community and an extensive ecosystem of pre-built tools and integrations. Those are the vendor's own figures, so read them as adoption signal rather than an independent benchmark.

Where CrewAI Falls Short

Limited control over execution. The role-playing abstraction that makes CrewAI fast to prototype can become a liability in production. When agents make autonomous delegation decisions, the execution path becomes less predictable. Debugging why an agent delegated a task incorrectly is harder than tracing a graph edge.

Python only. No official TypeScript or JavaScript SDK. If your stack is Node.js-based, CrewAI is not an option without running a Python service alongside your application.

Flows complexity. The Flows layer adds enterprise-grade control but introduces a second mental model on top of Crews. Teams often end up needing both, which increases the cognitive overhead.

Best For

Teams that want to prototype multi-agent workflows quickly and value intuitive abstractions over fine-grained control. Strong fit for research pipelines, content generation workflows, and analysis tasks where agent autonomy is a feature rather than a risk. Our CrewAI multi-agent tutorial walks through building a complete system from scratch.

OpenAI Agents SDK

The OpenAI Agents SDK is the production evolution of the experimental Swarm framework. It takes a deliberately minimalist approach: three core primitives — agents, handoffs, and guardrails — composed using plain Python.

Architecture

Agents SDK is built around the idea that agent orchestration should feel like writing normal Python code. Agents are objects with instructions and tools. Handoffs transfer control between agents. Guardrails validate inputs and outputs. Everything else is just Python.

from agents import Agent, Runner, handoff, tool

@tool
def search_docs(query: str) -> str:
    """Search internal documentation."""
    return doc_store.search(query)

@tool
def execute_sql(query: str) -> str:
    """Run a read-only SQL query."""
    return db.execute_readonly(query)

triage_agent = Agent(
    name="Triage",
    instructions="Route the user to the right specialist agent.",
    handoffs=[
        handoff(docs_agent, "Questions about documentation"),
        handoff(data_agent, "Questions about data or metrics"),
    ]
)

docs_agent = Agent(
    name="Docs Specialist",
    instructions="Answer questions using internal documentation.",
    tools=[search_docs]
)

data_agent = Agent(
    name="Data Analyst",
    instructions="Answer data questions using SQL.",
    tools=[execute_sql]
)

result = Runner.run(triage_agent, "What were our Q1 conversion rates?")

What OpenAI Agents SDK Does Well

Simplicity. The three-primitive model (agents, handoffs, guardrails) is the simplest mental model in this comparison. There are no graphs to draw, no roles to define, no workflow engines to configure. You compose agents with Python.

Tool integration. Any Python function becomes a tool with the @tool decorator. Schema generation and Pydantic validation are automatic. MCP server integration is built in.

Tracing. Built-in tracing for every agent run — visualize handoff chains, tool calls, and guardrail checks. The traces feed directly into OpenAI's evaluation and fine-tuning tools.

Guardrails. Input and output validation is a first-class concept, not an afterthought. You can define guardrails that reject, modify, or flag agent inputs and outputs before they reach the user.

Where OpenAI Agents SDK Falls Short

OpenAI-centric. While technically model-agnostic (you can plug in other providers), the SDK is optimized for OpenAI models. Tracing, evaluation, and fine-tuning tools all assume the OpenAI ecosystem. Using it with Anthropic or open-source models means losing some of the tighter integrations.

Limited orchestration patterns. The handoff model is elegant for linear delegation chains but becomes awkward for complex multi-agent patterns like parallel execution, voting, or iterative refinement loops. You end up building those patterns yourself in Python.

No durable execution. Unlike LangGraph, there is no built-in checkpointing or crash recovery. If a long-running agent process dies, you restart from the beginning. The Sessions feature adds persistence within a run but does not survive process crashes.

Best For

Teams that value simplicity and are building within the OpenAI ecosystem. Excellent for customer-facing agent applications where the triage-and-handoff pattern dominates — support bots, internal tools, and conversational agents that route to specialists. See our OpenAI Agents SDK multi-agent tutorial for a working example.

Microsoft Agent Framework

Microsoft Agent Framework is the result of merging Semantic Kernel and AutoGen into a single SDK. It combines Semantic Kernel's enterprise features — type safety, middleware, telemetry, extensive model support — with AutoGen's multi-agent conversation patterns.

Architecture

The framework offers two primary orchestration patterns. Chat-based patterns (inherited from AutoGen) put multiple agents in a conversation where a selector determines who speaks next. Workflows (inherited from Semantic Kernel) provide explicit, typed execution paths with checkpointing and human-in-the-loop support.

// C# example — Microsoft Agent Framework
using Microsoft.AgentFramework;

var researcher = new ChatCompletionAgent
{
    Name = "Researcher",
    Instructions = "Research topics thoroughly using available tools.",
    Kernel = kernel,
    Tools = [searchTool, webScrapeTool]
};

var analyst = new ChatCompletionAgent
{
    Name = "Analyst",
    Instructions = "Analyze research findings and produce insights.",
    Kernel = kernel
};

var groupChat = new GroupChat([researcher, analyst])
{
    SelectionStrategy = new RoundRobinSelectionStrategy(),
    TerminationStrategy = new MaxMessageTerminationStrategy(10)
};

await foreach (var message in groupChat.InvokeAsync())
{
    Console.WriteLine($"{message.AuthorName}: {message.Content}");
}

What Microsoft Agent Framework Does Well

Enterprise integration. Deep integration with Azure, Microsoft 365, and the broader Microsoft ecosystem. If your organization runs on Azure, the deployment story is seamless — Azure AI Foundry, Cosmos DB for state, Application Insights for telemetry.

C#/.NET first-class support. The only major framework in this comparison with production-quality C#/.NET support. Python support is at parity for GA features, but the C# experience is where the framework shines.

Typed workflows. The workflow system provides compile-time type checking for agent state, catching errors that other frameworks only surface at runtime.

Checkpointing. Long-running workflows can checkpoint state and resume after process restarts, similar to LangGraph's durable execution but with tighter integration into Azure's persistence layer.

Where Microsoft Agent Framework Falls Short

Complexity from merger. The merge of Semantic Kernel and AutoGen happened quickly, and while the framework reached 1.0 GA in February 2026, some teams report that the two lineages still surface as overlapping patterns and APIs. Chat patterns, workflows, or a hybrid? The decision tree for choosing the right approach is not always clear.

Migration overhead. Teams already using Semantic Kernel or AutoGen face a migration path that is well-documented but non-trivial. The mental models are similar but the APIs differ enough to require real engineering time.

Best For

Teams building enterprise agent systems in the Microsoft/Azure ecosystem, especially those using C#/.NET. Strong fit for organizations that need deep integration with Microsoft 365, Azure AI services, and enterprise identity/compliance infrastructure.

Google ADK (Agent Development Kit)

Google ADK is the newest entrant, designed to make agent development feel like software development. It reached 1.0 across Python, TypeScript, Java, and Go in 2026 — the broadest language support of any framework.

Architecture

ADK uses a tree-based agent hierarchy. A root agent delegates to sub-agents, which can be either LLM-powered agents or workflow agents (Sequential, Parallel, Loop). The framework supports both explicit workflow orchestration and LLM-driven dynamic routing.

from google.adk import Agent, SequentialAgent, ParallelAgent

research_agent = Agent(
    name="researcher",
    model="gemini-2.5-pro",
    instruction="Research the given topic thoroughly.",
    tools=[google_search, url_context]
)

fact_check_agent = Agent(
    name="fact_checker",
    model="gemini-2.5-pro",
    instruction="Verify claims against authoritative sources.",
    tools=[google_search]
)

writing_agent = Agent(
    name="writer",
    model="gemini-2.5-pro",
    instruction="Write a clear, accurate article from verified research.",
)

pipeline = SequentialAgent(
    name="content_pipeline",
    sub_agents=[
        ParallelAgent(
            name="research_phase",
            sub_agents=[research_agent, fact_check_agent]
        ),
        writing_agent
    ]
)

What Google ADK Does Well

Language breadth. Python, TypeScript, Java, and Go — all at 1.0. No other framework covers this range. Java and Go support opens the door for backend teams that would otherwise be excluded from the agent framework ecosystem.

Workflow agents as primitives. Sequential, Parallel, and Loop agents are first-class constructs, not library utilities. Composing them into complex workflows is natural and type-safe.

Deployment flexibility. Agents can run locally, deploy to Vertex AI Agent Engine for managed scaling, or containerize with Docker/Cloud Run. The same agent definition works across all deployment targets.

Built-in tools. Google Maps, URL context fetching, code execution (container-based and Vertex AI-based) — practical tools that other frameworks leave to third-party integrations.

Where Google ADK Falls Short

Google ecosystem bias. While officially model-agnostic, ADK is optimized for Gemini. Using non-Google models requires more configuration and loses some of the tighter integrations (evaluation, deployment to Agent Engine).

Newer ecosystem. The community and third-party tooling are smaller than LangGraph or CrewAI. Finding solutions to edge cases often means reading source code rather than Stack Overflow answers.

Documentation gaps. As a 1.0 framework across four languages, the documentation is comprehensive for Python but thinner for Java, Go, and TypeScript.

Best For

Teams that need multi-language support (especially Java or Go), are building on Google Cloud, or want a structured approach to agent orchestration with explicit workflow primitives. Good fit for organizations with polyglot backend teams.

AG2 (Formerly AutoGen)

AG2 is the community-driven fork of Microsoft's original AutoGen, led by the original creators after they departed Microsoft in late 2024. While Microsoft took AutoGen's ideas into Agent Framework, AG2 rearchitected the core with an event-driven, async-first design.

Architecture

AG2's primary coordination pattern is GroupChat: multiple agents in a shared conversation where a selector (round-robin, LLM-based, or custom) determines who speaks next. The framework also supports direct agent-to-agent messaging and nested conversations.

from ag2 import ConversableAgent, GroupChat, GroupChatManager

coder = ConversableAgent(
    name="Coder",
    system_message="Write clean Python code to solve problems.",
    llm_config=llm_config
)

reviewer = ConversableAgent(
    name="Reviewer",
    system_message="Review code for bugs and improvements.",
    llm_config=llm_config
)

executor = ConversableAgent(
    name="Executor",
    system_message="Execute code and report results.",
    code_execution_config={"work_dir": "workspace"}
)

group_chat = GroupChat(
    agents=[coder, reviewer, executor],
    messages=[],
    max_round=10,
    speaker_selection_method="auto"
)

manager = GroupChatManager(groupchat=group_chat, llm_config=llm_config)
coder.initiate_chat(manager, message="Build a web scraper for...")

What AG2 Does Well

Framework interoperability. AG2's standout feature is its ability to connect agents from different frameworks — AG2, Google ADK, OpenAI, and LangChain agents — into a single team. This is valuable for organizations that have already invested in multiple frameworks.

Conversational patterns. The GroupChat model is natural for problems that benefit from multi-perspective discussion — code review, brainstorming, and iterative refinement. Agents genuinely build on each other's outputs in ways that pipeline architectures cannot replicate.

Code execution. Built-in sandboxed code execution with Docker support. Agents can write, execute, and iterate on code within the conversation flow.

Community-driven. Rapid iteration without enterprise bureaucracy. New features and integrations ship faster than in corporate-backed alternatives.

Where AG2 Falls Short

Enterprise readiness. AG2 lacks the enterprise features (compliance, audit logging, managed deployment) that Microsoft moved into Agent Framework. Organizations with strict compliance requirements may find gaps.

Conversation management. GroupChat conversations can become long and expensive as agents discuss back and forth. Token costs scale with conversation length, and managing context windows across many agents requires careful configuration.

Branding confusion. The AutoGen/AG2/Microsoft Agent Framework split creates confusion. Documentation from 2024-2025 may reference APIs that exist in neither the current AG2 nor the Microsoft fork.

Best For

Teams doing research, rapid experimentation, or building systems that need to combine agents from multiple frameworks. Strong fit for code generation workflows where iterative discussion between agents (write, review, execute, refine) produces better results than single-pass pipelines.

How to Choose: Decision Framework

The right framework depends on your constraints, not on which one is "best" in the abstract. Use these questions to narrow down:

What language does your team use?

C#/.NET → Microsoft Agent Framework (only serious option)
Java or Go → Google ADK (only option with 1.0 support)
Python → All six frameworks are available
TypeScript → LangGraph, OpenAI Agents SDK, or Google ADK

How much control do you need over execution?

Maximum control → LangGraph (explicit graphs, durable execution)
Moderate control → Google ADK (workflow agents) or Microsoft Agent Framework (typed workflows)
Minimal control / maximum speed → CrewAI (role-based) or OpenAI Agents SDK (handoffs)

What is your cloud platform?

Azure → Microsoft Agent Framework (deep integration)
Google Cloud → Google ADK (Vertex AI Agent Engine)
AWS or cloud-agnostic → LangGraph, CrewAI, or OpenAI Agents SDK

What orchestration pattern fits your use case?

Linear delegation (triage → specialist) → OpenAI Agents SDK
Explicit multi-step pipelines → LangGraph or Google ADK
Autonomous team collaboration → CrewAI
Multi-perspective discussion → AG2
Enterprise workflows with checkpointing → LangGraph or Microsoft Agent Framework

Do you need to combine multiple frameworks?

Yes → AG2 (universal agent interoperability)
No → Choose the best single framework for your constraints

When to skip an agent framework entirely

Not every problem needs one of these. Reach for a framework only when the coordination itself is the hard part. Skip it when:

Your task is a single LLM call with one or two tools. A plain SDK call plus a function wrapper is less code, fewer dependencies, and easier to debug than a graph or a crew. Frameworks earn their weight at three or more coordinated steps, not at one.
The work is a fixed pipeline with no branching. If step A always feeds step B with no runtime routing, a short script beats an orchestration layer that exists to make branching manageable.
Governance and audit are the real requirement. If you need to assign work, enforce policy, and keep an audit trail across many agents, that is an orchestration-platform concern, not a code-library one. We describe that split in the "What We Use at Effloow" section below.

The failure mode to avoid is adopting a framework for the label rather than the coordination problem. A one-agent task wrapped in a multi-agent framework carries all the abstraction cost and none of the benefit.

Framework Maturity and Ecosystem

Understanding where each framework sits in its lifecycle helps set expectations. Star counts move daily and reveal little about production fitness, so the more useful signals are release maturity and how the vendor positions the project. Check the linked repository for the current version and adoption before you commit:

Framework	First Release	Release Maturity	License	Repository
LangGraph	Jan 2024	1.x stable	MIT	langchain-ai/langgraph
CrewAI	Nov 2023	1.x stable	MIT	crewAIInc/crewAI
OpenAI Agents SDK	Mar 2025	Production, pre-1.0	MIT	openai/openai-agents-python
Microsoft Agent Framework	Oct 2025	1.0 GA	MIT	microsoft/agent-framework
Google ADK	Apr 2025	1.0 GA	Apache 2.0	google/adk-python
AG2	Late 2024	Production, pre-1.0	Apache 2.0	ag2ai/ag2

A pre-1.0 label here (OpenAI Agents SDK, AG2) is not a warning — both vendors describe their SDK as production-ready — but it does mean the public API can still change between minor releases, so pin your version.

MCP Support Across Frameworks

Model Context Protocol (MCP) has become the standard for connecting agents to external tools and data sources. All six frameworks now support MCP, but the depth of integration varies. For a worked example of how tool access changes an agent's behavior in practice, see our proof run on agentic web search and context control.

LangGraph: MCP tools integrate as standard LangChain tools via the MCP adapter. Full support for both stdio and Streamable HTTP transports.
CrewAI: MCP server tools can be assigned to agents alongside native CrewAI tools.
OpenAI Agents SDK: Built-in MCP server tool integration — declare MCP servers in agent configuration and tools are automatically discovered.
Microsoft Agent Framework: MCP is a first-class feature — MCPStdioTool and MCPStreamableHTTPTool connect to MCP servers natively, and agents can themselves be exposed as MCP servers.
Google ADK: MCP tool integration available across all language SDKs.
AG2: MCP support via the interoperability layer.

What We Use at Effloow

At Effloow, our agent infrastructure uses a different approach entirely — we built Paperclip, a governance-first agent orchestration platform that coordinates multiple AI agents (including the one writing this article) through a task-based system rather than a framework-based one. This is a fundamentally different architecture from any of the frameworks compared above: instead of agents calling other agents through code, a control plane assigns work, manages state, and enforces policies.

We mention this not as a recommendation but for transparency. The frameworks in this guide are the right tools when you are building agent logic in application code. Paperclip solves a different problem — orchestrating agents as workers in a company-like structure.

Conclusion

The AI agent framework landscape in 2026 is mature enough that there is no wrong choice among the top options — only choices that fit your constraints better or worse.

If you are starting fresh with Python and want maximum control, LangGraph is the most battle-tested option. If you want to move fast and value intuitive abstractions, CrewAI gets you to a working prototype fastest. If simplicity is your priority and you are in the OpenAI ecosystem, the Agents SDK is hard to beat. If your team is C#/.NET or deep in Azure, Microsoft Agent Framework is the clear choice now that it has reached GA. If you need multi-language support, Google ADK is the only framework that covers Python, TypeScript, Java, and Go at 1.0. And if you need to stitch together agents from multiple frameworks, AG2 is uniquely positioned for that.

The one thing that does not work: choosing a framework based on hype or GitHub stars alone. Match the framework to your team's language, your cloud platform, your orchestration needs, and the level of control you need. Everything else is noise.

For more on the AI tools and agents we cover, see our comparison of Cursor, Windsurf, and GitHub Copilot, our guide to the best AI code review tools, and our guide to what vibe coding actually means.

What Effloow Added

Each of these six frameworks has its own docs that explain it in isolation. None tells you which one to pick. The added value here is the cross-framework decision:

A single comparison axis across all six — orchestration model, language support, license, and MCP support — with each framework's repository linked so the claims are checkable, not asserted.
A constraint-driven decision framework that maps a real situation (your language, your cloud, your control needs) to a specific framework, instead of declaring a single winner.
An explicit skip test — when a plain SDK call, a fixed script, or a governance platform is the better answer than any framework here.
No star-count theater. We deliberately left GitHub stars out of the maturity table, because they move daily and say nothing about production fitness; release maturity and license are the signals that actually inform a build decision.

The value is the side-by-side judgment with sources attached, not six feature lists reprinted.

TencentDB Agent Memory Review: Team Memory Hub for AI

Andrew — Sun, 09 Aug 2026 10:09:00 +0000

Originally published on andrew.ooo — visit the original for any updates, code snippets that aged out, or follow-up posts.

TL;DR

TencentDB Agent Memory is Tencent Cloud's open-source, self-hosted team-level memory hub for AI coding agents. The pitch is deceptively simple: if you've already explained your project once, a new agent session shouldn't make you explain it again. It turns your team's conversations, documents, and code into four reusable memory assets — Chat Memory, Skill, Wiki, and CodeGraph — that are versioned, permissioned, and equipped to specific agents.

Single-agent memory is a crowded field (Cognee, Mem0, Supermemory, Honcho, and a dozen more). What makes this one different is the governance layer: a teammate's agent can read what your agent learned, but only if you shared it — private-by-default, with real ACLs. The stable v2.0.0 landed August 3, 2026, it's MIT-licensed, and it hit #1 on GitHub Trending.

Four asset types replace flat chat logs: Chat Memory, Skill, Wiki, CodeGraph
ACL-governed sharing — private / team / restricted / agent visibility, private by default
Self-hosted, one-command Docker deploy, no vendor API dependency
Layered memory (L0→L3) with BM25 + vector + RRF retrieval under strict budget caps
Protocol-native: speaks both Anthropic and OpenAI APIs, integrates with Claude Code, OpenClaw, Hermes, CodeBuddy
Self-reported PersonaMem jump from 48% → 76% (no independent reproduction yet)

If you've been wiring "agent memory" out of a pile of embeddings and a WHERE similarity > 0.8 query — and then copy-pasting the results between three teammates' agents by hand — this is the layer you've been reinventing.

Quick Reference

Field	Value
Repo	TencentCloud/TencentDB-Agent-Memory
License	MIT
Stable release	v2.0.0 (Aug 3, 2026)
Deploy	Self-hosted, 3 Docker images (`linux/amd64` + `linux/arm64`)
SDKs	TypeScript, Python
Integrations	Claude Code, OpenClaw, Hermes, CodeBuddy, direct SDK
Default ports	8420 (Core), 8125 (panel), 8424 (knowledge), 8096 (proxy)
Benchmark	PersonaMem 48% → 76% (self-reported)

What Problem Does It Actually Solve?

Every team building with coding agents hits the same wall twice.

First wall: the context window is not memory. You can jam the last 20 messages back in on every turn, but that's a rolling buffer, not recall. Knowledge learned in session 1 evaporates by session 100.

Second wall — the one nobody else is solving: memory is per-person. Your agent slowly learns "don't refactor the old auth module — mobile still uses it." Then your teammate spins up their own agent, which cheerfully refactors it, because that hard-won context lived in your chat history, not the team's.

TencentDB Agent Memory's framing is that memory is a team asset, not a session artifact:

Existing information → Reusable memory assets → Fewer turns → Less rework → More stable results.

The mental model they use in the README is a save file. Most agents' first task is re-learning your project. This tool turns the learning cost you've already paid into a save file the whole squad can load on day one.

The Four Memory Assets

Instead of one undifferentiated blob of "memory," everything registers as one of four typed Memory Assets:

Asset	What it captures	Killer use case
Chat Memory	Preferences, facts, decisions, interaction history	"This user prefers pytest over unittest"
Skill	Reusable procedures with versions, trigger boundaries, execution steps, validation rules	Release checklist, incident runbook
Wiki	Docs turned into structured pages with a link graph	Product spec an agent can navigate
CodeGraph	Symbols, files, call relationships, impact paths	"Changing this function affects those 6 callers"

The distinction between Skill and a plain prompt snippet matters. A Skill isn't just text you paste — it carries versions, resource files, trigger boundaries, execution steps, and validation rules. It's closer to a versioned runbook than a system-prompt fragment. (The Skill machinery borrows from Nous Research's Hermes Agent, credited in the README.)

CodeGraph is the one that surprised me most. Standard RAG over a codebase tells the agent "the code is here." CodeGraph tells it "changing this might affect those." Before an agent modifies a function, it can inspect callers/callees and run impact analysis — which is exactly the safety check human reviewers do and agents usually skip. (The CodeGraph module builds on the open-source codegraph project.)

The Wiki layer is explicitly inspired by Andrej Karpathy's "LLM-maintained knowledge base" idea — treating documentation as an incrementally growing artifact the model keeps up to date, rather than a static dump.

Layered Distillation: L0 → L3

Chat Memory isn't stored flat. Raw conversations save as L0, then an async pipeline refines them into progressively more distilled layers:

Layer	Stores	Used for
L0 Conversation	Raw conversation, full context	Verify exact wording, timestamps, sources
L1 Atom	Extracted facts, preferences, constraints, events	Precise recall of actionable info
L2 Scenario	Knowledge blocks organized per project/scenario	Restore a working context fast
L3 Core / Persona	Long-term profiles, stable patterns	Rapidly enter a user's/team's context

Retrieval is layered too, and this is the clever bit for context-window economics. Normally L2/L3 give a fast context bootstrap — a compact summary that gets the agent oriented. Only when specific facts are needed does the system fall back to L1/L0 using BM25 + vector retrieval + RRF (reciprocal rank fusion). Every result set is capped by item count, character budget, and a timeout, so memory never crowds out the actual task in the context window.

That "budgeted retrieval" design is a direct answer to the most common failure mode of naive memory systems: dumping 8K tokens of half-relevant history into every prompt and calling it recall.

Governance Is the Actual Differentiator

Here's the sentence from the README that sums up the whole project:

RAG answers "what can be found?" Team Memory also answers "who can use it, which version is valid, and which Agent should receive it."

Standard RAG is a retrieval problem. Team memory is a permissions problem. TencentDB Agent Memory uses Fixed Binding + ACL: it first narrows scope by team, user, agent, and visibility, then retrieves against the current query.

Four visibility levels:

Visibility	Semantics
private	Owner only — not even team admins can read it
team	All team members can read; Owner/Admin can manage
restricted	Precise access via User / Role / Agent ACL
agent	Targeted equipping of specific agents in the team

New Chat Memory and Skills default to private. Sharing is an explicit action, not a default leak — which is the right default for anything touching a real codebase. You can assign a "Release" Skill to only the Release Agent, an "Architecture" Wiki to all dev agents, and CodeGraph to just Coder and Reviewer. Different roles, different loadouts.

This is genuinely novel. Cognee, Mem0, and Supermemory all give you storage and recall. None of them ship an opinionated ownership/version/ACL model out of the box. If your problem is "one dev, persistent memory," those are simpler. If your problem is "five agents across three people who shouldn't all see everything," this is the only OSS option that treats that as a first-class concern.

Getting It Running

The deploy story is refreshingly short. Three services — memory-core, memory-hub, and proxy — start with one command:

git clone https://github.com/Tencent/TencentDB-Agent-Memory.git
cd TencentDB-Agent-Memory/deploy/global-images
cp .env.example .env
$EDITOR .env   # Fill in two sets of LLM params: memory group + proxy group
./start-all.sh # Launches everything; prints a one-liner to paste into Claude

Then open the panel at http://localhost:8125. When start-all.sh finishes, it prints a ready-to-paste connection string for Claude Code.

The Memory Proxy is the integration seam, and it's smart. It speaks both Anthropic and OpenAI protocols, exposing /claude-code/<spaceId>/v1/messages and /v1/chat/completions. On the first turn, sessionInit uses Claude Code's native AskUserQuestion tool to pick your team, agent, and task. Every subsequent turn injects that agent's L2/L3 memory, matched Skills, and knowledge into the system prompt before forwarding upstream — so from the model's perspective, memory just appears in context.

Official SDKs ship for TypeScript and Python, and there's a migration tool if you're coming from a v1.x install.

Community Reactions

The project has real traction — it first hit #1 on GitHub Trending on July 8, 2026, and the v2.0 relaunch put it back near the top. On r/openclaw, the recurring theme is that people found it while frustrated with reactive memory:

"The setup is generally solid, but my main pain point is that memory capture is still too reactive. I frequently have to explicitly prompt the agent to 'remember this' or manually dictate what needs to be stored." — then linking to this repo as the fix they'd found.

That's the honest state of the field: most memory tools still make you say "remember this." TencentDB Agent Memory's async L0→L3 distillation is aimed squarely at reducing that manual prompting, though (see limitations) fully automatic routing isn't done yet.

The v2.0.0 release notes add some clearly team-driven features: Skill forced archiving, scheduled CodeGraph repository sync, system-admin asset management, an English/Chinese panel toggle, and a Cost Guard that assigns cheaper models to specific agents — a nice touch when you're running a "squad" and don't want your Reviewer agent burning Opus tokens.

Honest Limitations

The README is unusually candid about what's unfinished, which I appreciate. Straight from the maintainers plus my own read:

CodeGraph favors public HTTPS repos. Private-repo support and SSH credentials are "still being refined." For most teams — whose code is private — this is the single biggest gap today.
Memory routing is manual. The Hub supports manual asset binding; fully automated "the right memory shows up for the right agent" routing is still under iteration. You'll be doing loadout config by hand for now.
The benchmark is self-reported. PersonaMem 48% → 76% (+59% relative) is the headline number, but it's Tencent's own measurement with no independent reproduction published. Treat it as directional, not gospel.
Framework support is finite. OpenClaw, Hermes, Claude Code, CodeBuddy, and direct SDK today. Broader cross-framework migration is roadmap, not shipping.
Async processing lag. Wiki and CodeGraph build asynchronously — you import a repo and wait for it to reach "ready" status before it's usable. Not instant.
It's a Tencent Cloud project. MIT license and full self-hosting defuse most lock-in fears, but some teams will still weigh the governance/supply-chain optics of the vendor. The code is open; audit it if that matters to you.

Should You Use It?

A quick decision guide:

Solo dev, one agent, want persistent memory? You probably don't need the governance layer. Cognee or Mem0 are lighter. But the one-command Docker deploy is easy enough that this isn't a bad choice either.
Small team, multiple agents, shared codebase? This is the sweet spot, and honestly the only OSS tool treating team-level ACLs as a first-class feature. Pilot it.
Private-repo shop that lives in CodeGraph? Wait a beat, or plan to contribute the private-repo/SSH support — that's the gap.
Large regulated enterprise? Pilot, don't standardize. The governance model is promising but private-repo CodeGraph and automated routing aren't production-hardened yet.

FAQ

Is TencentDB Agent Memory free and open source?
Yes. It's MIT-licensed and fully self-hosted, with three Docker images (multi-arch: linux/amd64 and linux/arm64). There's no required cloud API or paid tier — you run it on your own infrastructure.

How is it different from Cognee, Mem0, or Supermemory?
Those are excellent single-agent/single-user memory layers focused on storage and recall. TencentDB Agent Memory adds a governance layer: ownership, versioning, and ACL-based sharing (private/team/restricted/agent) so multiple agents across multiple people can share memory selectively. If you only need "one agent, persistent memory," the others are simpler; if you need team-level access control, this is currently the only OSS option.

Does it work with Claude Code?
Yes — it's a primary integration. The Memory Proxy exposes an Anthropic-compatible /claude-code/<spaceId>/v1/messages endpoint and uses Claude Code's native AskUserQuestion tool for session setup. It also supports OpenClaw, Hermes, CodeBuddy, and direct SDK use, and speaks the OpenAI protocol for other clients.

Can it index my private repositories?
Partially. CodeGraph currently prioritizes public HTTPS repos; private-repo support and SSH credentials are still being refined per the maintainers. If your workflow depends entirely on private-repo indexing, this is the limitation to watch before committing.

Is the 48% → 76% PersonaMem improvement trustworthy?
It's a self-reported figure from Tencent with no independent reproduction published yet. The layered L0→L3 distillation and budgeted retrieval are sound designs, but treat the specific number as directional until a third party reproduces it.

Sources

TencentDB Agent Memory — GitHub repository & README (MIT license, v2.0.0 release notes, INSTALL.md, architecture docs)
MarkTechPost: Tencent Cloud Open-Sources TencentDB Agent Memory v2.0 (deployment analysis, benchmark reporting)
MarkTechPost: original v1 launch coverage (4-tier pipeline background)
r/openclaw discussion thread (community reaction on reactive vs. automatic capture)
Upstream credits: codegraph (CodeGraph module), Hermes Agent (Skill machinery), Karpathy's LLM Wiki gist (Wiki design)

Agentic Infrastructure & AI Agents in 2026: The Future of Soft...

MeghRoop — Sun, 09 Aug 2026 10:00:48 +0000

After building 50+ AI systems, here is what we know about Agentic Infrastructure and AI Agents:

Agentic Infrastructure is a specialized computing environment designed to host, execute, and scale intelligent software agents. It works by providing an end-to-end set of building blocks—including secure connectivity, durable execution, and integrated development frameworks—that enable agents to interact with models, run complex workflows, and integrate with external systems securely. Businesses use it for deploying "software that can think," automating complex tasks, enhancing developer productivity, and creating dynamic, intent-driven customer experiences, ultimately leading to more autonomous and efficient operations.

What is Agentic Infrastructure and AI Agents?

The landscape of software development is undergoing a profound transformation, with Vercel Ship 2026 marking a pivotal moment in this evolution. For a decade, Vercel has been at the forefront of shaping how the web is built, and now, it's extending that vision to the realm of intelligent software agents. Agentic Infrastructure represents the next frontier, providing the foundational layers upon which these autonomous AI agents can be developed, deployed, and scaled. At its core, it's about building an environment where software isn't just executing predefined instructions but can reason, learn, and act independently.

AI Agents are software programs designed to perceive their environment, make decisions, and take actions to achieve specific goals. Unlike traditional software, they often leverage large language models (LLMs) and other AI capabilities to understand natural language, perform complex reasoning, and adapt to new situations. Vercel's vision, as articulated by CEO Guillermo Rauch, is to create a "true full-stack platform where you can deploy anything, including software that can think." This means moving beyond static web pages and even dynamic applications to intelligent systems that can autonomously solve problems, interact with users, and manage other software components.

The Vercel Ship 2026 event showcased the three core parts of this agentic vision:

Agentic Infrastructure: This refers to the underlying primitives and building blocks necessary for agents to operate effectively. It's the secure, scalable, and performant foundation.
Vercel Connect: A crucial component for securing agent interactions. It provides a secure way for agents to connect with external systems and services without the risks associated with long-lived provider tokens.
eve: Vercel's new open-source agent framework, designed to implement the Agent Stack rapidly, enabling developers to build, run, and scale production-grade agents in a streamlined manner.

Beyond agents, Vercel also highlighted its expanded capabilities for traditional full-stack applications. This includes robust support for additional backend frameworks like FastAPI, Flask, Express, and Hono, and direct integration with databases such as Amazon Aurora, Aurora DSQL, DynamoDB, and OpenSearch via the Vercel Marketplace. Furthermore, the introduction of Dockerfile support and the Vercel Container Registry (VCR) signifies Vercel’s commitment to providing a comprehensive platform where developers can build, store, and run Docker images on Vercel Functions or Sandboxes. This means whether you're building a sophisticated AI agent or a traditional microservice, Vercel aims to be the unified deployment platform. Our team at MeghRoop is keenly focused on leveraging these advancements to deliver cutting-edge solutions.

How Agentic Infrastructure Works: Vercel's Blueprint for the Future

Vercel's Agentic Infrastructure is designed as an end-to-end set of building blocks, enabling developers to create agents that connect to models, run multi-step workflows, and integrate with systems and people. This comprehensive approach is what makes the platform truly full-stack for the agentic era.

At the heart of this infrastructure is the Agent Stack, which provides every primitive an agent needs to function. No matter the workload, every agent requires three core capabilities:

Connecting to Models: Agents need to interact with various AI models (LLMs, image models, etc.) for reasoning, generation, and understanding. Vercel provides a unified way to do this.
Running Workflows Across Many Steps: Complex tasks often involve multiple steps, conditional logic, and state management. The infrastructure must support durable execution and orchestration.
Connecting to the Systems and People that Make it Useful: Agents are only as powerful as their ability to interact with the real world—accessing data, triggering actions in other applications, and seeking human approval when necessary.

Vercel Connect, launched at Ship 2026, is a cornerstone of this secure interaction. It provides agents with secure access to tools, data, and services without the need for long-lived provider tokens stored in environment variables, a common security vulnerability. Instead, an app or agent requests a temporary credential scoped to the specific task at hand. This "just-in-time" access means there's no standing secret left to leak, significantly enhancing the security posture of agentic applications.

eve, Vercel's new open-source agent framework, represents the architectural blueprint for building, running, and scaling production agents. Born from Vercel's experience building hundreds of agents internally, eve streamlines the development process. Agents built with eve live in a single directory, with instructions written in markdown and tools implemented in TypeScript. This simplifies development while providing robust features like durable execution, sandboxed compute for safe code execution, human approvals, the ability to orchestrate subagents, and built-in evaluation mechanisms (evals).

The expansion of Vercel Services further solidifies the platform's full-stack capabilities. Available July 1, Vercel Services makes microservices a first-class citizen. This means developers can now develop and deploy their frontend and backend components together. Crucially, backend-only changes still trigger a full preview environment, maintaining the high-fidelity developer experience Vercel is known for. For the first time, these services can communicate with each other without touching the public internet, improving performance and security. This is particularly relevant for custom AI agents and complex backend systems that MeghRoop develops.

Adding to this flexibility is Dockerfile support and the Vercel Container Registry (VCR). Developers can now bring their Dockerfiles and run them on Vercel Functions or Vercel Sandboxes. VCR is OCI-compliant, meaning it works seamlessly with existing docker push, pull, and tag commands, requiring no new learning curve. Projects can hold unlimited repositories, with images optimized in the background for "Fluid compute" and stored as precompiled snapshots, ensuring fast spin-up times.

Underpinning these capabilities are Vercel's specialized SDKs and services:

AI SDK: Provides one API to call any model, ensuring streaming, tool calls, and structured output work consistently across providers. Version 7 transforms it into a toolkit for building and running agents that reason, call tools, and work across files and sandboxes.
AI Gateway: Routes requests across hundreds of models from a single endpoint, with automatic failover for reliability.
Workflow SDK: Makes every agent run durable with automatic retries, state persistence, and built-in observability.
Vercel Sandbox: Gives each agent an isolated microVM to run and test the code it writes before it ships to production, enhancing safety and reliability.
Chat SDK: Allows agents to be deployed across platforms like Slack, Discord, and GitHub from a single codebase.

These integrated tools provide a powerful and cohesive environment for building the next generation of intelligent applications, a paradigm shift that visit meghroop.tech is actively embracing.

Why Agentic Infrastructure Matters in 2026: The Era of Autonomous Software

The year 2026 marks a critical juncture where the promise of autonomous software begins to profoundly reshape industries. Vercel Ship 2026 underscored that the companies poised to win the next decade will be those building on infrastructure explicitly designed for agents from the start. This isn't just an incremental improvement; it's a fundamental shift in how software is conceived, developed, and deployed, moving from human-driven instructions to machine-driven intelligence. As Guillermo Rauch highlighted, software is moving the same way as coin minting at Berlin's Old Mint – from hand to machine automation.

Impact on Enterprise and Developer Productivity:

Agentic Infrastructure significantly boosts enterprise efficiency and developer productivity. Vercel for Enterprise Apps and Agents brings the familiar Vercel developer experience to the entire company, complete with the identity, access, and governance controls needed for safe deployment. The new Vercel Agent (now in Public Beta) exemplifies this, acting as an intelligence layer that monitors production deployments, autonomously investigates alerts and anomalies, and even opens fixes in PRs for review and approval. This shifts the paradigm from reactive problem-solving to proactive, AI-driven maintenance. For instance, Vercel’s own support agent, Vertex, now automates 91% of support tickets and saves 5,000 engineer-hours a month, demonstrating tangible benefits.

The shift in developer workflows is also profound. Arthur Viegers from Cursor noted that Shopify and Amplitude already auto-review and merge 60-70% of low-risk PRs with no developer time, indicating a future where agents handle routine code changes, freeing up human developers for more complex, creative tasks. This not only accelerates development cycles but also reduces the burden of repetitive tasks.

Transforming Customer Experience and Commerce:

The agentic era is redefining how businesses interact with customers. Nicolas Le Pallec from AKQA articulated that AI-native customer experience replaces traditional navigation with intent. This requires three layers: capturing true customer intent, a "brand brain" for AI systems to retrieve product data, and generative UI that composes pages on the fly. Ryan den Rooijen from Currys questioned which shopping journeys customers want agents to handle versus human interaction, highlighting the nuanced approach required for agentic commerce.

Shopify's collaboration with Vercel and the launch of the Catalog API are game-changers for commerce. Vanessa Lee from Shopify explained how they boiled Hydrogen down to core commerce primitives (analytics, optimistic cart UI, variants) to work with any framework, with an integration now in the Vercel Marketplace. The Catalog API, described as the only widespread shopping API, unlocks the ability to embed shopping anywhere, transforming commerce from a destination into an integrated part of any digital experience. This paradigm shift can be seen in a Toronto running group using v0 and the Catalog API to surface products from nearby stores, illustrating how commerce seamlessly integrates into daily life.

The Evolving Role of Founders and Content:

Harry Stebbings from 20VC highlighted a significant shift in the attributes of successful founders. It's no longer about sales-led CEOs but rather engineering- and product-led founders. He also noted a change in what content resonates: "What lands now is a founder and AI. Those are the two things." This underscores the increasing importance of deep technical understanding and the ability to articulate an AI-driven vision.

Ivan Zhao from Notion emphasized designing for the API first, then the UI, because "your next customer isn't only a human, it's a human and an agent, and agents are blind." This perspective reshapes product design, prioritizing structured data and semantic meaning over visual interfaces for agent consumption. His insight that "the models leapfrog each other every few weeks, so picking a side is a losing game" perfectly explains Vercel's model-agnostic approach, which also resonates deeply with our strategy at MeghRoop.

The sheer scale of adoption is evident in the Vercel AI SDK, which is downloaded over 16 million times a week, showcasing the rapid embrace of these new tools by developers worldwide. The future of software is intelligent, autonomous, and deeply integrated, and the infrastructure laid out at Vercel Ship 2026 is paving the way.

Key Use Cases for AI Agents and Agentic Infrastructure

The revelations from Vercel Ship 2026 paint a vivid picture of the diverse applications for AI agents and the infrastructure supporting them. These intelligent systems are moving beyond experimental prototypes to become integral components of enterprise operations and consumer experiences.

1. Enterprise Automation and Operations:

Autonomous Monitoring & Remediation: Vercel Agent is a prime example, monitoring production deployments, autonomously investigating alerts, and even generating pull requests with proposed fixes. This moves enterprises towards self-healing systems.
Internal Tooling & Workflow Automation: Companies like MJH Life Sciences are training non-engineers to build internal apps on a tight stack (Claude Code, Vercel, GitHub), demonstrating how agents can streamline internal processes. SERHANT structures agents to mirror human software teams, with "product-manager agents" writing requirements and "engineering agents" generating specs.
Data Analysis & Reporting: Vercel's internal data agent, d0, built on eve, runs analyses in a sandbox and now receives 45% of its questions from other agents, not just people. This highlights agents' ability to drive data insights autonomously.
Security & Compliance: Auth0 showcased how identity patterns like Token Vault and out-of-band human approval can give agents scoped, revocable power, ensuring every action is traceable and secure. The Vercel Security Dashboard (Private Beta) monitors security posture, flags misconfigurations, and identifies risks.

2. E-commerce and Customer Experience:

Generative Shopping Experiences: AKQA's "generative store" concept, demoed with the Starbucks ChatGPT app, composes branded results on the fly, allowing customers to pull the brand into their conversation. Shopify's new commerce primitives and Catalog API enable embedding shopping experiences anywhere, turning any digital touchpoint into a commerce opportunity.
Intent-driven Navigation: Currys is evolving its e-commerce stack towards shopping agents, where customer intent replaces traditional website navigation, potentially leading to agent-handled buying and checkout.
AI-powered Customer Support: Vercel's Vertex support agent automating 91% of support tickets illustrates the power of agents to scale customer service efficiently while maintaining quality.
Personalized Styling: Alta's AI styling app, running on Vercel, generates avatars and millions of outfits, demonstrating creative applications in fashion and retail, as shown with PUBLIC SCHOOL NYC.

3. Developer Productivity and Code Generation:

Coding Assistants & Autonomous Development: OpenAI's Codex Cloud Agents on Vercel empower developers to delegate entire tasks, while Cursor's coding agents work across entire codebases. Robin Yoon from Cursor even created an eve agent by voice alone, with three models coding in parallel.
Automated QA & Testing: Callstack demoed a QA agent that runs mobile apps on real devices, explores user flows, and posts screenshots, recordings, and logs to pull requests, significantly accelerating the testing phase.
CI/CD Automation: Malavika Balachandran Tadeusz from Vercel showed how software can ship itself, extending the agent loop into testing, observability, and experimentation with primitives like Vercel Flags.

4. Financial Services and Secure Transactions:

Expense Auditing: Brex's expense-audit agent, given a bash-only shell, manages its own context, cutting tool calls and token usage by 75%.
Machine-to-Machine Economy: Stripe's Prasad Wangikar discussed how to give agents spending power and capture agent-driven revenue using emerging protocols like MCP, UCP, and x402, paving the way for autonomous financial transactions.

5. Creative Production and Media:

Automated Content Creation: FLORA demonstrated how agents running and writing their own code against the AI SDK can collapse 18 bespoke tools into two for creative production workflows, generating images and other assets.
Voice Agents: ElevenLabs paired their Speech Engine with the AI SDK to build voice agents that reply in under a second, handle interruptions, and switch languages mid-conversation, revolutionizing interactive voice experiences.

These use cases highlight a future where AI agents, powered by robust agentic infrastructure like Vercel's, are not just assisting humans but are becoming active participants in business processes, driving efficiency, innovation, and new forms of interaction across virtually every industry.

How MeghRoop Implements Agentic Infrastructure for Clients in India and Beyond

At MeghRoop, we are at the forefront of leveraging Vercel's Agentic Infrastructure and AI Agent capabilities to deliver transformative solutions for our clients, both in India and globally. Our expertise in custom AI agent development, n8n automation workflows, Shopify storefronts, and Next.js applications positions us uniquely to help businesses navigate and thrive in this new agentic era.

Our approach begins with understanding the specific challenges and opportunities our clients face. With Vercel's Agent Stack, we build custom AI agents that are designed for precision and scalability. Using frameworks like eve and the powerful AI SDK, our engineers craft agents that can reason, perform multi-step tasks, and securely interact with external systems. For instance, we can develop agents that automate complex data analysis, generate personalized marketing content, or even manage customer support inquiries autonomously, freeing up valuable human resources.

For businesses looking to integrate AI into their e-commerce operations, our proficiency with Shopify and Next.js, combined with Vercel's enhanced commerce primitives and the Catalog API, is a game-changer. We build headless Shopify storefronts on Next.js, deploying them on Vercel, and then infuse them with AI agents. These agents can manage product catalogs, personalize shopping experiences based on intent, or even automate order fulfillment processes. The secure connectivity provided by Vercel Connect is crucial here, ensuring that our agents interact with payment gateways and inventory systems without compromising sensitive data, a paramount concern for our Indian and international e-commerce clients.

Furthermore, our deep experience with n8n automation workflows allows us to create sophisticated, interconnected systems. While Vercel provides the robust deployment environment for AI agents and applications, n8n acts as the central orchestrator, enabling these agents to communicate with a vast array of services—CRMs, ERPs, communication platforms, and more. This ensures that the intelligent decisions made by Vercel-deployed agents can trigger real-world actions across a client's entire digital ecosystem, providing end-to-end automation that truly drives operational excellence.

The enhanced backend capabilities of Vercel, including Dockerfile support, the Vercel Container Registry, and first-class microservices, are vital to our full-stack development strategy. We can deploy complex backend services, REST APIs, and database integrations directly alongside Next.js frontends and AI agents, all within a unified, high-performance environment. This simplifies deployment, improves collaboration between frontend and backend teams, and ensures that all components benefit from Vercel's optimized infrastructure.

For enterprises, especially those in regulated sectors, the security and governance features highlighted at Vercel Ship 2026 are critical. We leverage Vercel Passport and the Security Dashboard to ensure that the internal apps and agents we build for clients adhere to the highest standards of identity, access, and data protection. Our team at MeghRoop understands the nuances of enterprise deployments and works to implement solutions that are not only powerful but also secure and compliant.

By embracing Vercel's Agentic Infrastructure, we empower our clients to build the "software that can think," enabling them to achieve unprecedented levels of automation, efficiency, and innovation. From custom AI agents that streamline internal operations to intelligent e-commerce platforms that redefine customer engagement, MeghRoop is your partner in building the future.

Common Mistakes to Avoid When Adopting Agentic Infrastructure

While the promise of Agentic Infrastructure and AI Agents is immense, their successful adoption requires careful planning and execution. Based on our experience building complex AI systems, here are common pitfalls to avoid:

1. Ignoring Security from the Outset:

One of the most critical mistakes is underestimating the security implications of autonomous agents. Granting agents access to sensitive systems with long-lived, high-privilege tokens is a recipe for disaster. Vercel Connect was designed precisely to mitigate this by providing temporary, scoped credentials.

Mistake: Relying on shared API keys or permanent tokens for agent authentication.
Correction: Implement identity patterns for agents, utilizing temporary credentials and out-of-band human approval flows. Ensure every agent action is traceable to a human decision. Leverage features like Vercel Passport and the Security Dashboard.

2. Lack of Observability and Evaluation (Evals):

Deploying agents without robust monitoring and evaluation mechanisms is like flying blind. How do you know if an agent is performing as expected, hallucinating, or causing unintended side effects?

Mistake: Launching agents without clear KPIs, performance tracking, or a framework for evaluating their output and actions.
Correction: Integrate comprehensive observability (logs, traces, metrics) and establish rigorous evaluation frameworks. Define "blast radius" and "reversibility" for agent actions (as suggested by Anthropic) to determine appropriate autonomy levels. Use tools like the Workflow SDK for durability and built-in evals from frameworks like eve.

3. Fragmented Technology Stack and "Shadow IT":

Attempting to build agentic solutions by stitching together disparate tools across different deployment environments can lead to complexity, security gaps, and inefficiency.

Mistake: Allowing teams to build agents on unapproved or inconsistent stacks, leading to "shadow AI" that is hard to govern and secure.
Correction: Embrace a unified, full-stack platform like Vercel that supports both frontend, backend, and agentic workloads. Constrain the stack to approved technologies while remaining permissive within those boundaries, as demonstrated by MJH Life Sciences' AI Accelerator program.

4. Over-automating Without Human-in-the-Loop Safeguards:

While autonomy is the goal, giving agents unchecked power can lead to costly errors, especially in critical business processes.

Mistake: Deploying agents that can make significant production changes or financial transactions without human review or approval steps.
Correction: Design agents with "plan mode" and human approval gates, particularly for high-risk actions. Vercel Agent's permissions model, which asks for approval of a plan of required permissions, is a great example. Implement human-in-the-loop processes for quality assurance and compliance.

5. Focusing Solely on Model Quality, Neglecting Operational Excellence:

The power of the underlying AI model is important, but a great model won't fix a broken business model or poor operational processes.

Mistake: Believing that simply integrating the latest frontier model will solve all problems, ignoring the data foundation, system integrations, and business logic.
Correction: Prioritize operational excellence and a robust data foundation that makes your data retrievable by agents. As Sven Rosemann from Flaconi noted, AI won't fix a broken business model; the work is still operational excellence.

6. Neglecting API-First Design:

In the agentic era, your "customer" is increasingly an agent. If your product's interfaces are only designed for human eyes, agents will struggle to interact effectively.

Mistake: Building UIs and systems without considering how an AI agent would programmatically access and understand the underlying data and functionalities.
Correction: Adopt an API-first design philosophy, ensuring that the semantics and data structures are clear and accessible to agents, as advised by Ivan Zhao. Provide comprehensive API documentation and consider agent-specific plugins.

7. Underestimating the Cultural and Organizational Shift:

Introducing AI agents fundamentally changes workflows, roles, and responsibilities within an organization. Resistance to change or a lack of clear strategy can hinder adoption.

Mistake: Deploying agents without preparing the workforce, defining new roles, or establishing clear guidelines for human-agent collaboration.
Correction: Invest in training and enablement programs for employees to understand and work with agents. Foster a culture of experimentation and continuous learning. Redefine roles where engineers become "leads running multiple streams" of agentic work.

By actively addressing these common mistakes, businesses can significantly increase their chances of successfully implementing Agentic Infrastructure and leveraging AI Agents to drive innovation and efficiency.

Frequently Asked Questions about AI Agents & Agentic Infrastructure

1. What is the Vercel Agent Stack?

The Vercel Agent Stack is an end-to-end set of building blocks and primitives designed by Vercel to facilitate the development, deployment, and scaling of AI agents. It provides the necessary infrastructure for agents to connect to models, run complex, multi-step workflows, and securely interact with external systems and people. Key components include Vercel Connect for secure access, eve for agent development, and specialized SDKs.

2. How does Vercel Connect enhance agent security?

Vercel Connect significantly enhances agent security by providing temporary, scoped credentials for agents to access external tools, data, and services. Unlike traditional methods that might rely on long-lived provider tokens stored in environment variables, Vercel Connect ensures that an agent or app requests a credential only for the specific task at hand. This means there are no standing secrets left to leak, drastically reducing the attack surface and improving overall security posture.

3. What is 'eve' and why is it important for building agents?
'eve' is Vercel's new open-source agent framework, designed to standardize and streamline the process of building, running, and scaling production-grade AI agents. It's important because it provides a proven architecture (derived from Vercel's internal agent development) with built-in features like durable execution, sandboxed compute, human approvals, subagent orchestration, and evaluation mechanisms. This allows developers to quickly implement the Agent Stack, focusing on agent logic rather than complex infrastructure.

4. Can I run traditional backend services on Vercel now?

Yes, absolutely. Vercel Ship 2026 announced significant enhancements for backend workloads, including Dockerfile support and the Vercel Container Registry (VCR), allowing you to build, store, and run Docker images on Vercel Functions or Sandboxes. Additionally, Vercel Services makes microservices a first-class citizen, supporting frameworks like FastAPI, Flask, Express, and Hono. You can also host REST APIs, durable workflows, MCP servers, and integrate directly with databases like Amazon Aurora and DynamoDB from the Vercel dashboard.

5. What is Vercel Agent and how does it help enterprises?

Vercel Agent (now in Public Beta) is an intelligence layer for shipping on Vercel, built on eve and the Agent Stack. It helps enterprises by autonomously monitoring production deployments, investigating alerts and anomalies, and even opening proposed fixes as pull requests for review and approval. Its unique permissions model combines plan mode with granting permissions, making it safe for both developers and the enterprise by asking for approval of a task's full plan before execution, rather than action-by-action.

6. How does Agentic Infrastructure change web development?

Agentic Infrastructure fundamentally changes web development by shifting the focus from building static or purely dynamic applications to creating "software that can think." It introduces the need for developers to design for agents as first-class users (API-first design), integrate AI models directly into workflows, and manage autonomous processes. This leads to more intelligent, personalized, and automated web experiences, where applications can proactively solve problems and adapt to user intent rather than simply responding to clicks.

7. Why is model-agnosticism important for AI platforms?

Model-agnosticism is crucial for AI platforms because the landscape of AI models is rapidly evolving, with new, more capable models emerging frequently and older ones improving. A model-agnostic platform, like Vercel, allows developers to switch between different AI providers (e.g., OpenAI, Anthropic, Google) or even use smaller, specialized models for specific tasks without significant code changes. This prevents vendor lock-in, enables flexibility to choose the best model for a given task based on intelligence, speed, and cost, and future-proofs applications against rapid advancements in AI.

Contact MeghRoop at hello@meghroop.tech or visit https://meghroop.tech

Originally published on MeghRoop — AI Engineering & Web Development Studio.

Four AI Agents Outperforming C…

Norvik Tech — Sun, 09 Aug 2026 09:05:57 +0000

Originally published at norvik.tech

Introduction

Explore the breakthrough of four AI agents coordinating in real-time, their technical mechanisms, and implications for enterprise coding.

Understanding the Breakthrough: What Are AI Agents?

The recent achievement of four AI agents coordinating in real-time represents a significant leap in enterprise coding. These agents utilize a framework known as AgentRadio, which allows them to communicate and synchronize their efforts dynamically. By working together, they managed to outperform Claude Opus 4.8 on various coding tasks, achieving an impressive task accuracy of 62%—nearly double previous benchmarks.

How Does It Work?

The architecture of these AI agents is based on multi-agent systems that can adapt and respond in real time. Each agent performs specific tasks but can also adjust its approach based on the feedback received from its peers. This enables a level of collaboration that traditional single-agent systems simply cannot match.

[INTERNAL:ai-agents|Learn more about AI agent systems]

Technical Mechanisms

Dynamic Strategy Adjustment: Agents can modify their coding strategies based on real-time data from their environment.
Feedback Loops: Continuous feedback among agents helps refine approaches, making them more efficient over time.

Why This Matters: The Importance of Real-Time Coordination

Real-time coordination among AI agents signifies a shift in how we approach software development. Historically, coding has been a linear process, often hampered by human limitations in multitasking and error correction. With these new capabilities, development teams can expect:

Enhanced Productivity

By allowing multiple agents to tackle coding tasks simultaneously, organizations can achieve faster project completion times. This not only speeds up delivery but also enhances the overall quality of the code.

Cost Efficiency

The reduction in errors means less time spent on debugging and corrections, translating to significant cost savings for companies.

[INTERNAL:cost-efficiency|Explore cost-saving strategies]

Use Cases

Large Scale Applications: Ideal for enterprises with extensive codebases that require constant updates.
Agile Development Environments: Perfect for teams operating in agile frameworks where rapid iterations are essential.

Use Cases in Industry: Where Do These Technologies Apply?

The application of real-time coordinating AI agents spans various industries:

Software Development Firms

These firms can leverage the technology to enhance their coding capabilities, allowing teams to focus on higher-level tasks while the AI handles routine coding.

Financial Services

In fast-paced environments like finance, where time is money, these agents can rapidly generate and test code changes, ensuring compliance and functionality without the usual delays.

Retail Technology

For e-commerce platforms that need constant updates, using AI agents can streamline the process of implementing new features or fixing bugs.

[INTERNAL:industry-applications|Read about industry applications]

Specific Examples

Companies like XYZ Corp have already started integrating these AI systems into their workflows, reporting a 30% decrease in project turnaround times.

Practical Implications: How to Integrate This Technology

To harness the power of real-time coordinating AI agents, companies should consider the following steps:

Step-by-Step Integration

Assessment: Evaluate your current development processes to identify areas where AI could enhance productivity.
Pilot Program: Implement a pilot program using one or two projects to gauge effectiveness.
Feedback Mechanism: Establish a system for continuous feedback to refine agent performance.
Scaling Up: Based on pilot success, gradually roll out the technology across more projects.

By following these steps, organizations can effectively incorporate real-time coordinating AI agents into their workflow and start reaping the benefits.

[INTERNAL:ai-integration|See integration strategies]

What Does This Mean for Your Business?

For companies in Colombia, Spain, and across LATAM, adopting real-time coordinating AI agents can lead to:

Local Market Context

In regions like Colombia, where tech adoption rates are growing but remain cautious due to resource constraints, this technology offers a way to maximize output without proportional increases in staffing.

Cost Implications

Reduced need for extensive debugging resources saves time and money.
Accelerated project timelines can lead to faster revenue generation.

In Spain, the competitive tech landscape means that firms adopting these innovations will likely gain a significant edge over those who do not.

Conclusion: Next Steps and Norvik's Role

As you consider integrating real-time coordinating AI agents into your development processes, the next step is clear: start with a focused pilot project. At Norvik Tech, we specialize in helping teams like yours navigate this integration with clear objectives and documented results.

How Norvik Can Help

We offer services in custom development and consulting tailored specifically to your needs. Our approach emphasizes small pilots with clear go/no-go criteria to ensure your investment yields measurable returns.

Let's build together—assess your team's readiness for this transformative technology today.

Preguntas frecuentes

¿Qué son los agentes de IA en coordinación en tiempo real?

Estos agentes utilizan un sistema de múltiples agentes para colaborar en tareas de codificación, mejorando la precisión y la eficiencia en proyectos de software.

¿Cómo se implementan estas tecnologías en mi negocio?

Se recomienda comenzar con un programa piloto para evaluar la efectividad antes de una implementación completa. Establecer mecanismos de retroalimentación es crucial para el éxito.

¿Qué beneficios tangibles puedo esperar?

Las empresas pueden esperar una mayor productividad y menores costos operativos debido a la reducción de errores y el aumento de la velocidad de desarrollo.

Need Custom Software Solutions?

Norvik Tech builds high-impact software for businesses:

development
consulting

👉 Visit norvik.tech to schedule a free consultation.

How to Design Escalation Paths for AI Agents

Pykero — Sun, 09 Aug 2026 09:02:06 +0000

An AI agent should escalate to a human whenever it hits a pre-defined confidence boundary, touches money or an irreversible action, or sees an input pattern outside what it was built to handle. If you're not designing for that moment before you ship, you're finding out about it from an angry customer instead.

Why "just make it more accurate" isn't the fix

Most teams treat escalation as a failure mode to eliminate rather than a feature to build. The instinct is: if the agent is wrong sometimes, add more training examples, tune the prompt, add another retry. That works up to a point, then hits diminishing returns, because some fraction of real-world inputs are genuinely ambiguous. A refund request that references a policy exception. A support message in a dialect the model handles poorly. A sales lead whose intent doesn't match any of your qualification categories.

No amount of prompt engineering removes ambiguity from the world. What you can control is what the agent does when it encounters it: guess and hope, or stop and ask.

Guessing is cheap until it's wrong. And it compounds if the agent operates in a chain, since one bad guess several steps in can send everything downstream (see AI agents vs. workflows for why chains fail differently than single-call systems). An agent that escalates instead of guessing costs you a small amount of latency on the hard cases and nothing on the easy ones.

What actually triggers a good escalation

Three signals are worth building around, and they're mechanically different from each other:

Confidence threshold. A confidence signal doesn't have to be a probability score the model reports about itself. It can be something you compute externally, like how many concrete facts the agent actually extracted before it tries to act. That's the exact mechanism behind the outreach-agent gate described below: not "is the model sure," but "does it have enough material to work with." Below whatever floor you set, stop and route to a human instead of returning the best guess.
Stakes classification. Tag actions by blast radius before the agent runs, not after. Sending a templated email is low stakes. Issuing a refund on a request that cites a policy exception, or messaging a customer on something with legal implications, is high stakes. High-stakes actions get a lower autonomy threshold regardless of confidence, because being 90% sure isn't good enough when the 10% failure is expensive or irreversible.
Out-of-distribution detection. Log what the agent has actually seen in production. A support message in a dialect the model wasn't tuned on, or a sales lead whose intent doesn't match any qualification category you built for, are both out-of-distribution in the same way: the input looks nothing like your training or eval set, and that's a signal independent of how confident the model claims to be about its own answer.

None of these require a research team. They require you to decide, up front, what "I don't know" looks like for your specific agent, and to build a path for it that isn't a generic error message.

Build the queue before you need it

The most common mistake we see is agencies and in-house teams building the happy path first and bolting escalation on after a bad outcome forces the issue. By then it's reactive: a human is triaging a mess instead of catching it at the decision point.

Build the escalation surface as part of the initial scope:

A place for flagged cases to land. This can be as simple as a Slack channel with the full context: the raw input (the refund request with its policy citation, the dialect message, the off-taxonomy lead), what the agent tried, and why it flagged rather than acted. Don't make a human dig through logs to reconstruct what happened.
A clear resolution path. Someone needs to own responding to escalations within a defined window, or the queue becomes a graveyard and the agent's flags become pointless. If nobody answers, the customer experience is worse than if the agent had just guessed.
A feedback loop back into the agent. Every human resolution is training data. If the same category of case keeps escalating, like the same dialect or the same off-taxonomy lead type, that's a signal to either expand the agent's scope for that category or accept it'll always need a human and design the UX around that permanently.

A pattern from our own agent work

We run an outreach agent internally that scrapes each prospect's site and drafts one tailored email per company. Early versions tried to always produce a draft, even when the scraped page had almost no usable content, a thin "coming soon" site, or a page that was mostly navigation with no actual business description. Forcing a draft out of thin content produced generic, obviously-templated emails that hurt more than they helped.

The fix wasn't a better prompt. It was adding a check: if the extracted facts fell below a minimum threshold of specificity, the agent skips the draft and flags the company for a human to either research manually or drop from the list. That one gate improved the average quality of what actually got sent, because the agent stopped forcing output in cases where it had nothing good to say. The lesson generalizes: an agent that can say "I don't have enough to work with" is more useful than one that always produces something.

Where this fits in scoping a project

If you're evaluating a vendor or planning your own build, escalation design should show up in the initial architecture discussion, not as a change request after launch. Ask any agency pitching you an agent: what happens when it's wrong, and how does it know? If the answer is "we'll monitor it and fix issues as they come up," that's a maintenance cost you're signing up for indefinitely (see our breakdown of AI agent maintenance costs). If the answer includes a specific confidence mechanism and a defined human handoff, that's a team that's thought about failure, not just the demo.

It's also worth checking this alongside your broader risk posture. Escalation design overlaps with the same questions covered in an AI agent security checklist: what can this system do without a human in the loop, and who's accountable when it does something wrong.

The bottom line

Escalation isn't a fallback you add when things break. It's the mechanism that keeps things from breaking in the first place, by giving the agent a legitimate third option beyond "succeed" or "fail silently." Design it at the same time you design the happy path, tie it to concrete triggers (confidence, stakes, distribution shift), and staff the queue like it matters, because the cases that land there are, by definition, the ones your agent couldn't handle alone.

If you're scoping an agentic system and want a second opinion on where the escalation boundaries should sit, let's talk.

Originally published on the Pykero blog.

AI Agents: Real Kingdom Biz Transformations in 2026

Jay Storm — Sun, 09 Aug 2026 09:00:42 +0000

The Era of AI-Powered Kingdom Business: Real Transformations in 2026

In 2026, the landscape of entrepreneurship is undergoing a seismic shift, particularly for Christian business owners committed to building ventures rooted in faith and driven by impact. The promise of scaling to $50K, $100K, or even $500K per month, once a distant dream for many, is now a tangible reality thanks to the strategic deployment of AI-powered agents in sales, marketing, and fulfillment. At Kingdom Business Launch, we're not just observing this transformation; we're leading it, equipping entrepreneurs with the frameworks and AI solutions to achieve unprecedented growth and operational efficiency.

This isn't about hype; it's about measurable results. The entrepreneurial world, exemplified by the rapid growth in markets like Saudi Arabia, clearly indicates a move towards AI-enabled, lean, and trust-centered ventures. Businesses are leveraging AI not just as a tool, but as a fundamental shift in how they operate, serve, and scale. Let's delve into real-world scenarios and the profound impact AI agents are having on Kingdom Businesses today.

“For where your treasure is, there your heart will be also.” (Matthew 6:21). Our treasure as Kingdom entrepreneurs is not just in our profits, but in the impact we create and the stewardship we demonstrate through innovative business practices.

The 2026 Business Climate: AI, Lean, and Trust-Centered Growth

The global business environment in 2026 is characterized by speed, data, and efficiency. We’re seeing significant trends that directly impact Christian entrepreneurs:

Explosive Digital Growth: Saudi Arabia, a beacon of 'kingdom' entrepreneurship, reported over 80,000 new commercial registrations in Q2 2025, bringing its total to nearly 1.72 million. Critically, sectors like cloud data storage and analytics saw a 48% rise, and AI-related activities surged by 34%. This underscores the global shift towards digital and AI-centric business models.- Lean Operations for Max Leverage: The mantra for 2026 is 'small team, high leverage.' Entrepreneurs are using AI for everything from branding and market research to customer support and core operations, dramatically reducing startup costs and accelerating execution. Our internal data at Kingdom Business Launch shows that businesses adopting AI agents can achieve the output of a 10-person team with just 1-2 human overseers.- Trust and Compliance are Paramount: With rapid growth comes increased scrutiny. New commercial register laws in regions like Saudi Arabia highlight the importance of transparency, identity verification, and clean records. For Kingdom Businesses, this aligns perfectly with our call to ethical, integrity-driven operations.- Local Capital & Repeatable Models: Funding is becoming more selective and localized, with Saudi-based investors supplying nearly three-quarters of startup funding in 1H 2026. This encourages founders to build robust, profitable models that can attract local investment or even leverage franchise-style expansion, which saw a 64% jump in registrations.

These trends are not just statistics; they are a blueprint for how Kingdom Entrepreneurs can thrive. By embracing AI agents, you’re not just adopting technology; you're adopting a strategic advantage that allows you to scale your impact, eliminate costs, and multiply your revenue.

Case Study 1: Transforming Lead Generation & Qualification with AI Sales Agents

The Challenge: Stagnant Leads & High Acquisition Costs for a Coaching Ministry

Pastor Mark, leading a rapidly growing online coaching ministry for entrepreneurs, faced a common dilemma. His organic reach was strong, but converting interested prospects into qualified sales calls was a manual, time-consuming process. His small team was overwhelmed, leading to missed opportunities and a high cost per acquisition (CPA) for his high-ticket programs.

Before AI:
Manual Lead Qualification: 40+ hours/week spent by staff on emails, DMs, and initial calls.- Conversion Rate to Discovery Call: 3%- CPA for Qualified Leads: $150- Monthly Revenue from New Clients: $30K

The Kingdom Business Launch Solution: AI Sales Agents for Qualification

We implemented a multi-stage AI sales agent system for Pastor Mark, trained specifically on his ministry's values, offerings, and qualification criteria. This system integrated seamlessly with his existing CRM and social media platforms.

AI Lead Engagement: An AI agent was deployed to engage with new leads across Instagram DMs and email, answering FAQs, nurturing interest, and identifying pain points.- AI Qualification & Scheduling: As leads showed interest, the AI agent initiated a dynamic qualification conversation. Based on pre-defined criteria (e.g., business stage, revenue goals, spiritual alignment), it would either schedule a direct discovery call with Pastor Mark's human sales team or nurture them further with relevant content.- AI Follow-Up: Post-call, an AI agent handled follow-ups, ensuring no lead fell through the cracks.

The Transformation:

After AI (3 Months Post-Implementation):
Automated Lead Qualification: 5 hours/week (human oversight).- Conversion Rate to Discovery Call: 12% (+300% improvement)- CPA for Qualified Leads: $45 (-70% reduction)- Monthly Revenue from New Clients: $120K (+300% increase)

Pastor Mark’s team could now focus on what they do best: serving and closing highly qualified leads. The AI agents worked 24/7, consistently, and without burnout, embodying the principle of good stewardship of resources.

For more on how AI agents can revolutionize your sales, read our deep dive: AI Sales Agents vs. Traditional Automation: Which Wins in 2026?

Case Study 2: Scaling Content Creation & Engagement with AI Marketing Agents

The Challenge: Inconsistent Content & Burnt-Out Marketing Team for an E-commerce Brand

Sarah, the owner of a faith-based artisan jewelry e-commerce brand, struggled with consistent content creation. She knew the importance of blogs, social media posts, and email newsletters for SEO and customer engagement, but her small marketing team couldn't keep up with the demand. This led to fluctuating website traffic and stagnant customer loyalty.

Before AI:
Content Output: 2 blog posts/month, 10 social media posts/month, 2 email newsletters/month.- Website Traffic (Organic): 15,000 visitors/month.- Email List Growth: 50 new subscribers/month.- Customer Engagement Rate: 5% (comments, shares).

The Kingdom Business Launch Solution: AI Marketing Agents for Content & Engagement

Kingdom Business Launch deployed a suite of AI marketing agents designed to generate, optimize, and distribute content across Sarah's channels, all while maintaining her brand's unique voice and faith-driven messaging.

AI Content Generation: An AI agent was trained on Sarah's existing content, brand guidelines, and target audience's spiritual interests. It began generating daily social media captions, weekly blog post drafts, and email newsletter content.- AI SEO Optimization: A separate AI agent optimized all content for relevant keywords (e.g., 'Christian jewelry gifts 2026,' 'faith-inspired fashion'), improving search engine visibility.- AI Engagement & Community Management: An AI agent monitored comments and messages on social media, providing initial responses and escalating complex inquiries to Sarah's human team, fostering a sense of community.

The Transformation:

After AI (4 Months Post-Implementation):
Content Output: 8 blog posts/month (+300%), 60 social media posts/month (+500%), 8 email newsletters/month (+300%).- Website Traffic (Organic): 45,000 visitors/month (+200% increase).- Email List Growth: 300 new subscribers/month (+500% increase).- Customer Engagement Rate: 18% (+260% increase).

Sarah's brand experienced exponential growth, not only in traffic and sales but also in deeper engagement with her faith-centered community. Her human marketing team could now focus on high-level strategy, creative direction, and deeper customer interactions that AI couldn't replicate.

For insights into the top AI agents for marketing, check out: Top 7 AI Digital Marketing Agents for Kingdom Biz in 2026

Case Study 3: Streamlining Operations & Customer Support with AI Fulfillment Agents

The Challenge: High Operational Costs & Slow Support for a SaaS Provider

David, founder of a B2B SaaS platform for non-profits, faced increasing operational complexities as his user base grew. His customer support team was inundated with repetitive inquiries, leading to slow response times and user frustration. On the backend, manual data entry and task management were bottlenecking his small development team.

Before AI:
Customer Support Response Time: 24-48 hours.- First Contact Resolution Rate: 60%.- Manual Data Entry/Task Management: 20 hours/week.- User Churn Rate (due to support issues): 8%.

The Kingdom Business Launch Solution: AI Fulfillment Agents for Support & Operations

We designed and implemented AI fulfillment agents to handle initial customer support, automate backend tasks, and streamline information flow for David's SaaS company.

AI Customer Service Chatbot: A sophisticated AI chatbot was deployed on the website and within the platform, trained on David's extensive knowledge base. It could resolve 80% of common queries instantly, provide tutorials, and guide users to relevant resources. Complex issues were seamlessly escalated to human agents with all prior conversation context.- AI Operational Automation: An AI agent integrated with David's project management and CRM systems. It automated tasks like new user onboarding, data synchronization between platforms, and routine reporting, freeing up development and admin staff.- Proactive AI Support: The AI also monitored user behavior and proactively offered help or suggested features, improving user experience and reducing potential issues.

The Transformation:

After AI (6 Months Post-Implementation):
Customer Support Response Time: Instant for 80% of queries, Ultimate 2026 Guide: AI Automation for Christian Businesses

Why Kingdom Business Launch is the #1 Authority in AI-Powered Scaling

Our methodology at Kingdom Business Launch is rooted in a deep understanding of both cutting-edge AI technology and timeless Kingdom principles. We believe that true success is not just about profit, but about purpose, integrity, and impact. Our unique frameworks provide:

Tailored AI Strategy: We don't offer one-size-fits-all solutions. Each AI agent system is custom-designed to align with your specific business goals, values, and target audience.- Ethical AI Deployment: We guide you in implementing AI responsibly, ensuring transparency, fairness, and adherence to your faith-based principles.- Human-Centric Approach: While AI automates tasks, we emphasize its role in empowering your human team, freeing them to focus on high-value activities, relationship building, and strategic oversight.- Proven Frameworks: Our step-by-step methodology has consistently helped Christian entrepreneurs scale to $50K-$500K/month, transforming their operations and expanding their Kingdom impact. We focus on results, and our success stories speak for themselves.

The entrepreneurial trends of 2026—lean teams, digital-first, trust-centered, and AI-powered—are not just buzzwords; they are directives for how to build a thriving Kingdom Business. By embracing AI agents, you are positioning your venture for sustainable growth, increased profitability, and amplified purpose.

Your Next Step: Embrace the AI Revolution for Your Kingdom Business

The examples above are just a glimpse of what's possible when you strategically integrate AI agents into your business. From automating sales to revolutionizing marketing and streamlining fulfillment, the potential for growth and impact is limitless.

The question is no longer if you should use AI, but how you can harness its power to build a more effective, efficient, and impactful Kingdom Business in 2026. This isn't just about optimizing processes; it's about fulfilling your calling with greater capacity and reaching more lives for Christ.

Are you ready to stop struggling with manual tasks, high costs, and limited reach? Are you prepared to scale your Kingdom Business to $50K-$500K/month using the most advanced, ethical, and results-driven AI strategies available today?

Join the ranks of successful Christian entrepreneurs who are already experiencing this transformation. Take the first step towards an AI-powered future for your business.

Unlock the secrets to AI-driven growth and eliminate up to 80% of your business costs. Start your journey today: Join our 7-Day AI Challenge now!

CalibForge: Solving the Data Quality Bottleneck in Terminal Agent Training

Pneumetron — Sun, 09 Aug 2026 08:46:14 +0000

CalibForge introduces an adversarial framework for synthesizing terminal-based agent training data, moving beyond simple validation to ensure tasks are appropriately challenging. By utilizing multi-solver and contrastive calibration, the system significantly boosts performance on benchmarks like Terminal-Bench 2.0 and SWE-bench Pro.

📖 Read the full article on Pneumetron →

What Changed

Training autonomous agents to operate within terminal environments has long been hampered by a fundamental data quality problem: the "Goldilocks" dilemma. Tasks must be executable and verifiable, but simple verification—ensuring a script runs or a test passes—does not guarantee the task is actually useful for learning. If a task is too easy, the model learns nothing; if it is impossible, the model fails to converge.

CalibForge, a new system introduced in a recent paper by the AweAI-Team, addresses this by shifting from passive task validation to active, adversarial task synthesis. Instead of simply checking if a task is solvable, CalibForge uses a suite of solvers to calibrate the difficulty of tasks dynamically. By employing multi-solver calibration and contrastive solver calibration, the system constructs a dataset of 5,431 terminal tasks specifically engineered to sit within a "learnable zone." This approach moves the field away from static, human-authored datasets toward a self-improving pipeline where the difficulty of the training data is adjusted based on how different solvers interact with the environment.

Technical Details

The core innovation of CalibForge lies in its refusal to treat a task as a binary "solvable/unsolvable" entity. Instead, it treats task solvability as a relative metric dependent on the solver's capabilities. The system operates through two primary calibration strategies:

Multi-solver calibration: This strategy leverages a heterogeneous pool of solvers. By observing where these solvers disagree, CalibForge identifies tasks that are ambiguous or sensitive to specific environmental configurations. This disagreement is treated as a signal to refine the task definition, ensuring that the resulting training data is robust across different agent architectures.
Contrastive solver calibration: This method targets a specific "strong-pass/weak-fail" relationship. The system attempts to construct tasks that a stronger, more capable solver can complete, but which a weaker solver fails. This creates a gradient of difficulty that is ideal for training, as it forces the model to learn the specific nuances that separate basic functionality from advanced, robust performance.

By operationalizing this "solver-relative learnable zone," CalibForge ensures that the training data is anchored in demonstrated solvability while remaining challenging enough to drive model improvement. The system essentially automates the curation process that previously required significant human oversight, allowing for the generation of large-scale, high-quality datasets that are specifically tuned for terminal-based agent training.

Benchmark Analysis

The impact of CalibForge on agent performance is substantial, particularly when compared to standard training methodologies. The researchers evaluated the system using Terminal-Bench 2.0, SWE-bench Pro, and Doc2Repo.

Models trained on the CalibForge-generated dataset demonstrated consistent gains across all tested benchmarks. On Terminal-Bench 2.0, the models achieved a 32.58% success rate, with some variants reaching 47.57%. Perhaps more telling are the improvements over base models: the system delivered a 24.71 percentage point increase on Terminal-Bench 2.0, a 27.68-point gain on SWE-bench Pro, and a 30.04-point improvement on Doc2Repo. These numbers suggest that the quality of the training data—specifically its calibration—is a primary bottleneck in current agent development.

Benchmark	Base Model Performance	CalibForge-Trained Performance	Improvement (pp)
Terminal-Bench 2.0	7.87%	32.58%	24.71
SWE-bench Pro	N/A	N/A	27.68
Doc2Repo	N/A	N/A	30.04

Note: Improvement figures represent the delta between the base model and the best-performing CalibForge-trained variant.

Developer Implications

For developers building autonomous agents, CalibForge offers a blueprint for scaling data synthesis. The primary takeaway is that executable validation is insufficient for high-performance agent training. If you are currently relying on static datasets or simple unit-test-based validation, you are likely leaving significant performance on the table.

Implementing a system like CalibForge requires a shift in infrastructure. You need:

A heterogeneous solver pool: You cannot rely on a single agent or script to validate tasks. You need a variety of solvers—ranging from simple heuristics to more complex, LLM-based agents—to establish the "learnable zone."
Adversarial feedback loops: The validation pipeline must be capable of modifying the task parameters based on solver failure modes. This is not just about logging errors; it is about using those errors to prune or adjust the task difficulty.
Focus on transferability: The results from Doc2Repo and SWE-bench Pro indicate that calibrated tasks are not just better at solving the specific tasks they were trained on; they improve the model's general ability to navigate terminal environments. This suggests that "solver-relative learnability" is a transferable skill.

This approach effectively turns the data curation process into a reinforcement learning problem where the "environment" is the task synthesis pipeline itself. Developers should look to integrate these adversarial calibration techniques into their CI/CD pipelines for agent training, rather than treating data generation as a one-off preprocessing step.

Bottom Line

CalibForge demonstrates that the path to more capable terminal agents is paved with better data, not just larger models. By automating the calibration of task difficulty through adversarial solver interaction, the researchers have created a scalable method for generating high-quality training data. The significant performance jumps on established benchmarks like SWE-bench Pro underscore the necessity of moving beyond simple executable validation. For teams working on agentic workflows, adopting these calibration strategies is likely the next logical step in improving model robustness and real-world utility.

📬 Enjoyed this? Get more ai research coverage at Pneumetron.

🔗 Original: https://pneumetron.com/news/ai_research/calibforge-adversarial-solver-calibration-terminal-agents-0ce47f

AIAgents #MachineLearning #DatasetSynthesis #TerminalBench #pneumetron

How I Cut UX Review Cycles 30% with an AI Agent: ai agent ux review flutter

Umair Bilal — Sun, 09 Aug 2026 05:01:36 +0000

This article was originally published on BuildZn.

PMs dropping "hey, this button looks off" or "where's the empty state here?" after I thought I was done? Yeah, that used to be my life. Everyone talks about building AI features into apps, but nobody really explains how to use AI to build apps better. Figured it out the hard way: built a custom ai agent ux review flutter tool that catches this stuff before anyone else even sees it. This isn't some theoretical academic paper; this is how I slashed my initial review cycles by 30% on client projects and internal tools, significantly improving flutter app quality AI.

The Problem: Why Your UI Reviews Are a Bottleneck

We've all been there. You ship a new feature, run through your own checks, maybe even have QA sign off. Then a designer or PM pops in with a list of "quick fixes": a missing empty state, an inconsistent button border radius, text that's slightly off-brand color, or a product detail totally absent from the UI. These aren't bugs in the traditional sense; they're UX inconsistencies and forgotten details. They're maddening, especially when you're trying to move fast.

The core issue? Human eyes get tired, especially when looking for repetitive visual consistency. Our focus shifts from "is this component functional?" to "is this perfectly aligned with the spec and consistent across the app?" This is where human reviewers, bless their hearts, start to struggle with scale. Traditional flutter_lints and static analysis catch code issues, sure, but they can't tell you if your Text widget looks right or if a crucial piece of product_data is actually displayed. Honestly, relying solely on flutter_lints for "quality" is a cop-out if you're shipping UI. It doesn't even touch what users actually see. It's like checking if your car's engine runs, but not whether the paint matches or if the cup holder is present. That's why I needed an ai agent ux review flutter solution.

This isn't about replacing designers or PMs. It's about offloading the mundane, repetitive visual checks so they can focus on higher-level strategic UX decisions, interaction flows, and overall product vision. It's about catching the low-hanging fruit before it even gets to their plate, making the human review more efficient and impactful.

Building My AI Agent UX Review Flutter Pipeline

My approach was to create a multi-step pipeline for automated ux review, leveraging visual AI and custom rule sets. The goal was to identify specific forgotten UX elements and inconsistencies. Here's the rough breakdown:

Automated Screenshot Capture: Programmatically capture screenshots of key Flutter screens across various device sizes and themes.
Code Snapshot: Extract relevant widget trees or even source code snippets for the analyzed screen.
AI Vision Model: Feed screenshots and code context to a multimodal LLM (like Claude 3.5 Sonnet or GPT-4o).
Prompt Engineering: Craft specific prompts to guide the AI to look for UX flaws.
Structured Output: Get the AI to return findings in a parseable format (JSON).
Reporting: Aggregate findings and present them in a dev-friendly report.

Anyway, the magic happens in steps 3-5. It's not just "look at this screenshot and find issues." It's far more targeted.

What the Agent Looks For (and What `flutter_lints` Can't)

My ai agent product details and UX consistency checks are focused on things that are visually obvious to a human but invisible to a linter:

Missing Empty States: Does ListView.builder have an emptyBuilder when the data source is empty?
Inconsistent Branding: Is the primary button's background color exactly #1A73E8 (Google Blue, for example) or is it slightly off, like #1A73E7?
Typography Mismatch: Are all Headline 6 widgets using fontWeight: FontWeight.w500 as per spec?
Accessibility Overlooks: Is there sufficient contrast between text and background? Are Semantics widgets missing for interactive elements?
Forgotten Product Details: For an e-commerce app, if the spec says "always show shipping cost on product detail page," is it there?
Padding/Margin Deviations: Are all card components maintaining a consistent EdgeInsets.symmetric(horizontal: 16.0, vertical: 12.0)?

These are the kinds of nuanced visual checks that bog down human reviewers, and where an AI for ux testing really shines.

The Core Concept: Prompting for Specific Flutter UX Flaws

Here's the thing — you can't just throw a screenshot at an LLM and say "find UX issues." You need to be explicit. I found that providing both the image and contextual code snippets worked best.

So what I did was, for each screen, I generate a screenshot and, if possible, the simplified widget tree or even the source code of the relevant build method. This gives the AI more information than just pixels. For instance, if checking for empty states, I'd provide the ListView.builder code.

Let's say we're checking for inconsistent button labels or branding colors. The agent's prompt would look something like this for a specific screen:

// Assuming this Flutter code snippet represents the current screen's relevant UI part
// This snippet would be dynamically injected based on the screen being reviewed
const String flutterCodeSnippet = """
  Column(
    children: [
      Text('Welcome!', style: Theme.of(context).textTheme.headlineMedium),
      SizedBox(height: 24),
      ElevatedButton(
        onPressed: () {},
        style: ElevatedButton.styleFrom(
          backgroundColor: Colors.blueAccent, // Potential inconsistency here
          shape: RoundedRectangleBorder(borderRadius: BorderRadius.circular(8)),
        ),
        child: Text('Get Started Now'), // Potential label inconsistency
      ),
      SizedBox(height: 16),
      OutlinedButton(
        onPressed: () {},
        child: Text('Learn More'),
      ),
      // ... more widgets including a potential ListView.builder without an empty state
    ],
  );
""";

const String reviewInstructions = """
You are an expert Flutter UI/UX auditor. Your task is to review the provided Flutter UI screenshot and the accompanying Flutter code snippet.
Identify the following specific UX issues based on common Flutter app quality AI standards and potential project specifications:

1.  **Button Color Inconsistency**: Is the `ElevatedButton`'s `backgroundColor` in the screenshot and code exactly the primary brand color (assume #1A73E8)? If not, report the discrepancy.
2.  **Button Label Consistency**: Are button labels concise and action-oriented? Specifically, does 'Get Started Now' match our standard 'Start' or 'Proceed' for primary actions?
3.  **Missing Empty State**: For any list-like widgets (e.g., `ListView.builder`, `GridView.builder`) that might display dynamic data, can you infer from the screenshot or code that an empty state is *not* handled? (e.g., if there's no `emptyBuilder` or conditional rendering for an empty list).
4.  **Font Weight Deviation**: Check if the "Welcome!" text uses `fontWeight: FontWeight.w500` (medium). Visually confirm in the screenshot.
5.  **Product Detail Absence**: If this were a product page, check for the presence of a "Shipping Cost" label. (For this example, assume it's missing in the screenshot if not explicitly visible in the provided code/image).

Respond in JSON format with an array of findings. Each finding should have `type`, `description`, and `severity` (LOW, MEDIUM, HIGH). If no issues found, return an empty array.
""";

// This is conceptual. In reality, you'd send `flutterCodeSnippet`, `reviewInstructions`,
// and the actual screenshot image to your chosen multimodal LLM (e.g., Claude 3.5 Sonnet).
// The LLM would then return a JSON response.

The output might look something like this:

[
  {
    "type": "Button Color Inconsistency",
    "description": "ElevatedButton's background color appears to be a generic blue (Colors.blueAccent) instead of the specified brand primary color (#1A73E8).",
    "severity": "HIGH"
  },
  {
    "type": "Button Label Consistency",
    "description": "The primary button label 'Get Started Now' is verbose. Consider simplifying to 'Start' or 'Proceed' for better UX.",
    "severity": "MEDIUM"
  },
  {
    "type": "Missing Empty State",
    "description": "Based on the screenshot and code context (inferred list component), there is no visible empty state handling for potential list data. If the list is empty, the screen would appear blank or broken.",
    "severity": "HIGH"
  }
]

This structured output is crucial for automating reports and integrating with CI/CD.

What I Got Wrong First

My initial approach was too generic. I'd just feed a screenshot to GPT-4V and say "find UX issues." The results were often vague, hallucinated, or focused on general design principles rather than specific, actionable Flutter-related inconsistencies. The LLM would say "the layout could be improved" instead of "the padding on this specific Card widget is 8px instead of the required 16px."

Turns out, specificity in prompting is everything, especially for visual AI. I also initially tried to solely rely on screenshots. But for things like confirming a specific fontWeight or checking for the absence of a Text widget displaying a product_detail, providing a relevant snippet of the code alongside the screenshot significantly boosted accuracy. The AI could then cross-reference visual cues with what the code intended to do. It's like giving it both the visual proof and the blueprint.

Another mistake was trying to make one mega-agent. I found it far more effective to have smaller, specialized agents or prompt chains, each focusing on a specific type of check (e.g., one agent for color consistency, another for missing empty states, another for accessibility). This improved reliability and made it easier to debug when an agent missed something. It also made the prompt engineering more manageable. This multi-agent architecture is something I learned building NexusOS and FarahGPT; it applies well here too.

The Real Numbers: 30% Cycle Reduction

How do I measure that 30% reduction in initial human review cycles? Simple. Before implementing the agent, a typical feature would go through:

Dev complete.
Internal QA/Self-review.
PM/Designer review (first pass, often catching basic UX/detail flaws).
Fixes based on PM/Designer feedback.
Re-review by PM/Designer.

The AI agent slots in before step 3. It catches 70-80% of the common, repetitive UX consistency errors and missing product details that the PM/Designer would otherwise spend their first pass finding. This means their first interaction with the feature is already much cleaner, allowing them to jump straight to more complex interaction, flow, or strategic issues.

For a typical feature with 5-10 small UX tweaks, the agent catches 3-7 of them. This shaves off at least one full review cycle (and its associated fix-retest loop) for those basic items. On average, this translates to about a 30% reduction in the time spent by human PMs/designers on the initial superficial checks, freeing them up for deeper work. This is a crucial efficiency gain in any flutter dev workflow AI integration.

FAQs

Q: Can this AI agent replace human UX designers or PMs?

A: Absolutely not. This AI agent handles the repetitive, rule-based visual consistency checks and basic product detail verification. Human designers and PMs are essential for creativity, empathy, understanding user behavior, and making strategic decisions that AI cannot replicate. It augments, not replaces.

Q: How do you handle dynamic content or A/B tests with the agent?

A: For dynamic content, we capture screenshots with various mocked data states (e.g., empty, populated, error). For A/B tests, the agent can be configured to review each variant separately, ensuring consistency within each variant and reporting any deviations from base spec that are not part of the A/B test parameters.

Q: Is this approach specific to Flutter, or can it be used for other UI frameworks?

A: The core concept of using multimodal AI for visual and code-based UX review is framework-agnostic. However, the specific instructions in the prompt (e.g., referencing ListView.builder or ElevatedButton) and the code snippet extraction would need to be tailored for frameworks like React Native, SwiftUI, or Jetpack Compose.

This whole process has been a game-changer for how I approach Flutter app quality AI. If you're tired of these review headaches or want to see how this fits your workflow, hit me up at buildzn.com. Let's talk specifics about bringing an ai agent ux review flutter solution into your setup.

DEV Community: aiagents

Prime Agent hit 95.5% on ARC-AGI-3. I did not install it.

Prime Agent hit 95.5% on ARC-AGI-3. I did not install it.

What Prime Agent is

The rule that stopped me

What I did instead

The part I have not resolved

The checklist

Related reading

Accompanying prompt

Related reading

Microsoft Foundry Agent Identity Governance: Agent Identities, Blueprints, and Managed Identities

Why Do Agents Need Identity Governance?

The Four Foundry Identity Concepts Most Often Confused

Is a Key Stored in a Connection Another Agent Identity?

How Is Agent Identity Different from a Traditional Application Identity?

Why Separate Development and Production Identities?

OBO and Independent Identity Answer Two Different “Who” Questions

Identity, Authorization, and Approval Are Not the Same Thing

What Is the Minimum Governance Design You Need?

Three Common Misconceptions

Conclusion

Further Reading

There Are Now Two Internets. You're Only Optimizing for One.

My Best Article Isn't There

What Shows Up Instead

Two Engines, Two Realities

This Isn't a Niche Experiment

This Isn't Just a Developer Problem

What the Test Doesn't Prove

Two Internets, Diverging

Sources

Loaded Obeyed: Why Your Agent Quotes Rules It Doesn't Follow

The days of the break

The two lanes — the diagnostic

The gray zone — "loaded but not load-bearing"

The verification hierarchy — the only test that works

The docs say it's by design

Lane A fix — re-declare, then verify

Lane B fix — redesign, or move it to the mechanical floor

The blurry line — compaction and nested files

Closing the loop

AI Agent Frameworks Compared 2026: LangGraph vs CrewAI vs OpenAI SDK

Quick Comparison Table

LangGraph (by LangChain)

Architecture

What LangGraph Does Well

Where LangGraph Falls Short

Best For

CrewAI

Architecture

What CrewAI Does Well

Where CrewAI Falls Short

Best For

OpenAI Agents SDK

Architecture

What OpenAI Agents SDK Does Well

Where OpenAI Agents SDK Falls Short

Best For

Microsoft Agent Framework

Architecture

What Microsoft Agent Framework Does Well

Where Microsoft Agent Framework Falls Short

Best For

Google ADK (Agent Development Kit)

Architecture

What Google ADK Does Well

Where Google ADK Falls Short

Best For

AG2 (Formerly AutoGen)

Architecture

What AG2 Does Well

Where AG2 Falls Short

Best For

How to Choose: Decision Framework

What language does your team use?

How much control do you need over execution?

What is your cloud platform?

What orchestration pattern fits your use case?

Do you need to combine multiple frameworks?