DEV Community: Jangwook Kim

Cloudflare Agents Week 2026 — When AI Agents Become Cloud Customers

Jangwook Kim — Fri, 15 May 2026 06:43:30 +0000

This time last year, every AI agent infrastructure conversation started with Kubernetes + LangGraph. Cloudflare's April Agents Week presented a different picture. Agents don't just call APIs — they create Cloudflare accounts, register domains, and deploy code on their own. The phrase "agents as cloud customers" sounds like marketing fluff, but this time they actually built it.

Here's my read on what matters, what doesn't, and where I'm skeptical.

What Agents Week Was

Cloudflare declared April 2026 "agents week" and shipped announcements every day — 20+ new features and GA transitions by the end of it. The overall impression is a company-wide bet that agents will be the primary actors on the internet, and they rebuilt infrastructure accordingly across compute, storage, networking, and security.

I'm focusing on the items that actually affect how you write and deploy agent code.

The Most Provocative Announcement — Agents That Create Their Own Accounts

My honest reaction when I first read this: "is this real?" The mechanics: once a user accepts Cloudflare's terms of service once, agents can autonomously create a Cloudflare account, start a paid subscription, register a domain, get an API token, and deploy code. Stripe partnership handles payment tokenization; OAuth + OIDC authenticate the agent as a trusted actor.

The implication is significant. Until now, agents worked within infrastructure that humans provisioned. Now agents can be the entity that provisions the infrastructure itself. If you're building a SaaS product, "agent handles new customer onboarding end-to-end" becomes a real architectural option.

That said, I have two concerns I can't shake. First, an agent connected to live billing requires airtight cost controls. Cloudflare's new task_budget concept seems designed for exactly this, but real-world examples of the two working together are scarce. Second, the legal accountability picture is murky. If an agent registers the wrong domain or incurs unexpected charges, who owns that? User consent to ToS exists, but the specific liability hasn't been tested.

Three Announcements Worth Your Attention

Past the headline, here are the things I'd actually build with.

Sandboxes GA: Nine months from beta (June 2025) to general availability. Each sandbox is an isolated Linux environment — real shell, real filesystem, background processes — that spins up on demand and, critically, picks up exactly where it left off after interruption. Sub-millisecond start times mean a code-generation agent can write, execute, observe output, and iterate in tight loops.

Compared to setting up a separate code execution environment alongside LangGraph or CrewAI, Sandboxes shifts the question from "how do I configure the execution environment" to "which infrastructure layer do I trust to manage it." Those are meaningfully different decisions.

Artifacts: Git-compatible versioned storage for agents. Create tens of millions of repos, fork from any remote, access with standard Git clients. Moved from private beta to public beta in early May. The practical use case: agents that produce code outputs now have a permanent home for those outputs, survives context resets, accessible from outside Cloudflare's stack.

Dynamic Workers: Isolated runtime for AI-generated code. Millisecond spin-up, scales to millions of concurrent executions. Enables the generate-execute-observe loop agents need without managing container infrastructure. Still feels early but the concept is right.

I Actually Installed the SDK

Theory aside, I ran through the setup myself.

mkdir cloudflare-agent-demo && cd cloudflare-agent-demo
npm init -y
npm install @cloudflare/agents

Clean install. @cloudflare/agents@0.0.16 exports Agent, AIChatAgent, and routeAgentRequest as the main surfaces.

Here's a minimal but representative agent:

// src/index.ts
import { Agent, routeAgentRequest } from "@cloudflare/agents";

interface TaskState {
  processedCount: number;
  lastHeartbeat: string;
}

interface Env {
  TASK_AGENT: DurableObjectNamespace<TaskAgent>;
}

export class TaskAgent extends Agent<Env, TaskState> {
  async onStart() {
    this.setState({ processedCount: 0, lastHeartbeat: new Date().toISOString() });
    // Built-in cron scheduling — no external scheduler needed
    await this.schedule("0 * * * *", "heartbeat", {});
  }

  async heartbeat() {
    const count = this.sql<{ n: number }>`SELECT COUNT(*) as n FROM tasks`;
    this.setState({
      processedCount: count[0]?.n ?? 0,
      lastHeartbeat: new Date().toISOString()
    });
  }

  async onRequest(request: Request): Promise<Response> {
    return Response.json({ state: this.state });
  }

  // Agents receive email directly
  async onEmail(email: ForwardableEmailMessage) {
    this.sql`
      INSERT INTO tasks (id, content, created_at)
      VALUES (${crypto.randomUUID()}, ${email.from}, ${Date.now()})
    `;
  }
}

export default {
  fetch: async (req: Request, env: Env): Promise<Response> => {
    const routed = await routeAgentRequest(req, env);
    return routed ?? new Response("OK", { status: 200 });
  }
};

wrangler dev starts immediately, no Cloudflare account needed for local work:

⛅️ wrangler 4.91.0
Your Worker has access to the following bindings:
  env.TASK_AGENT (TaskAgent)   Durable Object   local

⎔ Starting local server...
[wrangler:info] Ready on http://localhost:9998
[wrangler:info] GET / 200 OK (7ms)

One important caveat: @cloudflare/agents is Workers runtime-only. Trying to run it with standard Node.js throws ERR_UNSUPPORTED_ESM_URL_SCHEME because of the cloudflare: protocol imports. You need Wrangler. If you're used to SDKs like the Claude Agent SDK that run anywhere in Python or Node, this is an adjustment.

Architecture Choices Worth Understanding

A few design decisions in the SDK that reflect Cloudflare's broader approach:

Embedded SQLite: Declare new_sqlite_classes in wrangler.toml and every Agent instance gets its own SQLite. No external database configuration. Query with this.sql. The Durable Object isolation model gives you natural multi-tenancy — each agent instance has independent data. Sounds wasteful but it's actually clean for state isolation.

In-process scheduling: Register cron jobs directly from agent code. No external cron service. Wraps the Durable Object alarm API, which keeps scheduling and state management co-located. High cohesion, lower operational surface.

Email handler: onEmail lets agents receive email directly via Workers Email Routing. An agent that turns email into tasks is straightforward to write.

The way Dapr Agents handles state and messaging through Kubernetes sidecar patterns contrasts interestingly here. Cloudflare's model is more code-centric; Dapr is more infrastructure-centric. Both have legitimate use cases.

Where I'm Skeptical

I'll be direct about the rough edges.

Vendor lock-in is significant. The cloudflare:workers runtime dependency means your agent code doesn't run outside Cloudflare's stack. Migrating to a different platform later means substantial rewrites. Containerized approaches like running MCP servers on Kubernetes don't have this problem — you trade operational simplicity now for portability.

Multi-agent orchestration is thin. The single-agent story is compelling. But the SDK-level support for complex multi-agent coordination — handoffs, shared memory, hierarchical orchestration — is limited. Project Think is meant to address this but it's early. If your use case involves agents coordinating at scale, you'll need to build significant structure yourself.

SDK maturity. @cloudflare/agents@0.0.16 is pre-1.0. The API surface will change. For production use, you're accepting that risk.

My Take on When to Use This

Cloudflare is the right infrastructure choice when: response latency at the edge matters for your agents, your team already operates Cloudflare Workers, you want to minimize infrastructure management and focus on agent logic, or your architecture involves many independent agents each owning their own state.

It's not the right choice when: you need complex multi-agent orchestration and you're already invested in LangGraph, you're locked to AWS or GCP infrastructure, or your agents need to run in Python or standard Node.js environments.

The overall direction from Agents Week is coherent. Cloudflare is positioning itself as the infrastructure layer for the agent era — what Kubernetes became for containers. The SDK being at v0 means production adoption should be cautious, but the design thinking is consistent. Worth running through the setup and forming your own opinion.

Signed Agents: Cryptographic Identity for Agent Traffic

One announcement that got less coverage but caught my attention: Signed Agents. The idea is that HTTP requests made by agents carry a cryptographic signature proving their origin — "this was sent by an agent, not a human."

Right now there's no standard way to distinguish agent traffic from human traffic on the internet. User-Agent strings and IP patterns are guesses at best. Signed Agents gives servers a verifiable signal: they can check the signature and apply agent-specific rate limits, billing, or access controls. It's an early-stage primitive but it's the right one to build. Once agents are common enough to treat as distinct traffic types, having cryptographic identity for them becomes infrastructure rather than a feature.

Email Service Public Beta

Workers Email Service graduated to public beta during Agents Week. Any agent can now send email without integrating a third-party service like SendGrid or AWS SES.

Combined with the onEmail handler already in the SDK, agents can now handle both inbound and outbound email entirely within Cloudflare's stack. An agent that receives a customer email, processes it, creates a task, and sends a reply — with no external email service in the loop. For customer support agents, notification pipelines, or email-based task management, this is a meaningful simplification.

The Bigger Picture

Looking at Agents Week as a whole, it reads less like a feature release and more like a positioning statement. Twenty-plus announcements, all pointing the same direction: Cloudflare intends to be the infrastructure layer for the agent era the way AWS became the infrastructure layer for the web era.

The single thing I'd actually go build with first from this week: Sandboxes. Not the headline "agents create accounts" story — the persistent isolated Linux environment for agent code execution. That's immediately useful for any code-generation or code-testing agent, and it works today without novel legal or financial risk.

@cloudflare/agents@0.0.16 tells you what you need to know about production readiness. But if you're serious about evaluating agent infrastructure options, run through the local setup and form your own opinion. Twenty minutes, no account required.

Test environment: @cloudflare/agents@0.0.16, wrangler@4.91.0, Node.js v22.22.0, macOS 14

Note: The autonomous agent account creation feature requires a real Cloudflare account and Stripe integration — out of scope for local testing.

Source: Cloudflare Agents Week 2026

AWS MCP Server GA Practical Guide — Connecting CloudWatch & IAM to Your AI Coding Agent

Jangwook Kim — Thu, 14 May 2026 06:42:39 +0000

A CloudWatch alarm fired. Lambda error rate crossed the threshold, and I needed to dig through logs — flipping between the AWS console and my terminal, copying log group names by hand. At some point I had a clear thought: what if Claude Code could just look at my CloudWatch directly?

On May 6, 2026, AWS shipped an answer. AWS MCP Server hit general availability.

What the AWS MCP Server Actually Is

AWS MCP Server gives AI coding agents — Claude Code, Cursor, Codex — a standardized way to query AWS services directly. It wraps AWS APIs as MCP tools, using the Model Context Protocol that Anthropic defined. One uvx command wires 31 CloudWatch tools and 29 IAM tools into your coding agent.

Instead of copying log group names from the console and pasting them into CLI commands, you can ask your agent: "Find the Lambda function with the highest error rate in the past hour and summarize the relevant logs." The agent runs the Logs Insights query itself and brings back results.

If you've built an MCP server from scratch, you already understand the protocol. AWS MCP Server is the official, AWS-maintained collection of MCP servers for AWS services, published at awslabs/mcp on GitHub and installable from PyPI.

What Changed at GA

Three things matter compared to pre-GA versions:

IAM condition context keys. Every API call routed through AWS MCP Server now carries aws:ViaAWSMCPService and aws:CalledViaAWSMCP condition keys automatically. Your IAM policies can differentiate agent-initiated calls from human-initiated calls.

Full CloudTrail integration. Every API call goes to CloudTrail. There's a complete audit trail of what the agent did.

Separate CloudWatch namespace. Metrics published under AWS-MCP let you monitor how much of your API traffic comes from agents versus direct calls.

The practical upshot: you can now enforce different IAM permissions for agents and humans while using the same AWS credentials.

Installation: One Line with uvx

I installed and ran both servers. Here is what it takes.

# Install uv if you don't have it
curl -LsSf https://astral.sh/uv/install.sh | sh

# Run CloudWatch MCP server (creates isolated env automatically)
uvx awslabs.cloudwatch-mcp-server@latest

# Run IAM MCP server
uvx awslabs.iam-mcp-server@latest

uvx handles the virtual environment. First run pulls 53 packages for the CloudWatch server — botocore, pandas, scipy, statsmodels, and more. The reason for scipy and statsmodels is that the CloudWatch server includes built-in anomaly detection and statistical analysis on metrics, not just passthrough queries.

Installed versions:

awslabs.cloudwatch-mcp-server v0.1.1
awslabs.iam-mcp-server v1.0.20

The 0.x version on the CloudWatch server signals the API is still stabilizing. That is worth keeping in mind before putting it in production workflows.

Wiring It Into Claude Code (.mcp.json)

Put this in your project root:

{
  "mcpServers": {
    "cloudwatch": {
      "command": "uvx",
      "args": ["awslabs.cloudwatch-mcp-server@latest"],
      "env": {
        "AWS_REGION": "ap-northeast-1",
        "AWS_PROFILE": "default",
        "FASTMCP_LOG_LEVEL": "WARNING"
      }
    },
    "iam": {
      "command": "uvx",
      "args": ["awslabs.iam-mcp-server@latest"],
      "env": {
        "AWS_REGION": "ap-northeast-1",
        "FASTMCP_LOG_LEVEL": "WARNING"
      }
    }
  }
}

Set FASTMCP_LOG_LEVEL to WARNING. Without it, INFO logs bleed into the agent's responses. You can also install via the Claude Code CLI: claude mcp add aws-mcp-server.

CloudWatch MCP Server: 31 Tools

When the server starts, it registers exactly 31 tools. Here is the breakdown.

Log group tools (8):

describe_log_groups         List log groups
analyze_log_group           AI-powered log pattern analysis
execute_log_insights_query  Run a Logs Insights query
get_logs_insight_query_results  Poll query results
cancel_logs_insight_query   Cancel a running query
execute_cwl_insights_batch  Batch query execution
recommend_indexes_loggroup  Index recommendations for a log group
recommend_indexes_account   Account-wide index recommendations

Metrics tools (11):

get_metric_data             Fetch metric data points
get_metric_metadata         Metadata lookup (1,179 entries indexed at startup)
analyze_metric              Anomaly detection on a metric
get_recommended_metric_alarms  Suggest alarm thresholds
execute_promql_query        Run a PromQL query
execute_promql_range_query  PromQL range query
get_promql_label_values     PromQL label values
get_promql_series           PromQL series
get_promql_labels           PromQL labels
get_active_alarms           List active alarms
get_alarm_history           Alarm state history

The get_metric_metadata detail is worth noting. At startup, the server loads and indexes 1,179 metric metadata entries covering EC2, Lambda, RDS, DynamoDB, and most other AWS services. The server logs show it explicitly:

INFO | Loaded 1179 metric metadata entries
INFO | Successfully indexed 1179 metric metadata entries

This is what allows the agent to answer "which metric measures Lambda cold start duration?" without hitting the AWS docs.

What I Found on My Account

I ran this against a real ap-northeast-1 account. The output:

Available log groups (5):
  /aws/lambda/remotax-renewal-fe-CustomCDKBucketDeployment: 331,695 bytes
  /aws/lambda/remotax-renewal-fe-CustomS3AutoDeleteObjects:   2,038 bytes
  /aws/lambda/remotax-renewal-fe-LambdaServerFunctionHandler:     0 bytes
  /aws/lambda/remotax-renewal-fe-LogRetentionaae0aa3c5b4d4f:     0 bytes
  RDSOSMetrics: 55,192,669 bytes

Active CloudWatch Alarms:
  OK    EC2-HighCPU-Alarm
        CPUUtilization >= 80% | Currently: OK
  ?     EC2-HighDiskUsage-Alarm
        disk_used_percent >= 80% | INSUFFICIENT_DATA
  ?     EC2-HighMemoryUsage-Alarm
        mem_used_percent >= 80% | INSUFFICIENT_DATA
  ?     LaravelErrorAlarm
        LaravelErrorCount >= 1 | INSUFFICIENT_DATA

EC2 metrics available: 85

Three alarms sitting in INSUFFICIENT_DATA. Disk and memory alarms with no data means CloudWatch Agent is not running or misconfigured on those EC2 instances. That is the kind of silent infrastructure problem that usually only surfaces when an alert should fire and doesn't. The agent picked it up immediately.

IAM MCP Server: 29 Tools and the Security Architecture That Matters

The IAM server ships 29 tools:

list_users / get_user / create_user / delete_user
list_roles / create_role
list_policies / get_managed_policy_document
attach_user_policy / detach_user_policy
create_access_key / delete_access_key
simulate_principal_policy    ← the important one
list_groups / create_group / delete_group
add_user_to_group / remove_user_from_group
put_role_policy / get_role_policy / delete_role_policy

I find simulate_principal_policy the most useful. It checks whether an IAM principal can perform specific actions without actually making those API calls. After reading about MCP ecosystem security vulnerabilities and 30 CVEs, having agents pre-validate their permissions before executing is a meaningful safety step.

Test run against my account:

response = iam.simulate_principal_policy(
    PolicySourceArn='arn:aws:iam::370193714718:user/remotax-fe',
    ActionNames=[
        'cloudwatch:DescribeAlarms',
        'logs:DescribeLogGroups',
        'iam:ListUsers',
        's3:ListBuckets'
    ],
    ResourceArns=['*']
)

# Results:
# ✓ cloudwatch:DescribeAlarms: allowed
# ✓ logs:DescribeLogGroups: allowed
# ✓ iam:ListUsers: allowed
# ✓ s3:ListBuckets: allowed

The Condition Key Architecture

This is the part I think matters most about the GA release. Every API call through AWS MCP Server automatically carries:

aws:ViaAWSMCPService — marks this as a request via an MCP service
aws:CalledViaAWSMCP — marks this as originating from an MCP client

An IAM deny policy using these keys:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Deny",
      "Action": [
        "iam:CreateUser",
        "iam:DeleteUser",
        "iam:AttachUserPolicy"
      ],
      "Resource": "*",
      "Condition": {
        "Bool": {
          "aws:ViaAWSMCPService": "true"
        }
      }
    }
  ]
}

With this policy, a human using the AWS console can manage IAM users. Claude Code using the same credentials cannot. Same key pair, different effective permissions. When I was implementing Tool Use in the Claude Agent SDK, I had to build agent permission scoping into application logic. AWS is solving that at the infrastructure layer here.

Architecture

Three layers: coding agent → AWS MCP Server (stdio) → AWS API (SigV4 auth). Every AWS API call goes to CloudTrail. Metrics land in the AWS-MCP CloudWatch namespace separately from direct human calls.

Available AWS MCP Servers

Server	Purpose	Version
`awslabs.cloudwatch-mcp-server`	Logs, Metrics, Alarms	v0.1.1
`awslabs.iam-mcp-server`	IAM management	v1.0.20
`awslabs.aws-api-mcp-server`	Any AWS API	separate
CloudWatch Application Signals	APM/SLO monitoring	separate
AWS Network MCP Server	VPC/network diagnostics	separate
AWS Pricing MCP Server	Cost estimation	separate
EKS MCP Server	EKS cluster management	separate

The aws-api-mcp-server is interesting. It exposes every AWS API through a single tool. When building a FastMCP-based MCP server, each API endpoint needed its own tool definition. The aws-api-mcp-server flips that — one tool, all APIs. The trade-off is that the agent needs more context to figure out which API to call.

Honest Assessment — What Works, What Doesn't

What I find genuinely useful:

The IAM condition key separation is real. If you've been hesitant to give agents AWS access because you can't restrict them beyond the IAM user's permissions, this changes the calculation. You can attach aws:ViaAWSMCPService deny statements to enforce read-only agent access while keeping full human access with the same credentials.

PromQL support surprised me. CloudWatch supports PromQL for Container Insights metrics, and the MCP server exposes it. If you run Kubernetes on EKS and already write PromQL, you can use that syntax directly through the agent.

The 1,179-entry metric metadata index means the agent can reason about AWS services it has never seen before in your specific account. It knows what metrics EC2, Lambda, RDS, and most other services expose without needing to query AWS each time.

What gives me pause:

CloudWatch server at v0.1.1. The AI analysis tools like analyze_log_group and analyze_metric look promising but I have not stress-tested them. A 0.x version in production tooling warrants caution.

Logs Insights cost. CloudWatch charges for scanned log data in Insights queries. An agent with unconstrained query access could run up meaningful charges. There are no cost guardrails at the tool level — that has to be managed at the IAM level (restricting query scope) or through agent instructions.

create_access_key in the IAM server. An agent tool that creates new AWS access keys is, by default, accessible. The condition key approach can block it, but you have to set that up deliberately. I would not wire up the IAM server in a production environment without first adding explicit deny policies for the write operations.

My recommendation: start with cloudwatch-mcp-server in read-heavy workflows. Treat the IAM server as a development tool until you have the deny policies in place.

Getting Started

If AWS credentials are configured:

# Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh

# Test immediately
uvx awslabs.cloudwatch-mcp-server@latest

# Add to a project
cat > .mcp.json << 'EOF'
{
  "mcpServers": {
    "cloudwatch": {
      "command": "uvx",
      "args": ["awslabs.cloudwatch-mcp-server@latest"],
      "env": {
        "AWS_REGION": "us-east-1",
        "FASTMCP_LOG_LEVEL": "WARNING"
      }
    }
  }
}
EOF

Official docs: awslabs.github.io/mcp. Source: github.com/awslabs/mcp. Free to use — you pay only for the AWS resources the agent touches.

AI agents having console-level visibility into AWS infrastructure is coming regardless. AWS MCP Server GA is the first production-ready step in that direction.

MemRL: Self-Evolving Agents via Episodic Memory RL

Jangwook Kim — Thu, 14 May 2026 04:16:52 +0000

There is a gap in how most AI agents handle experience. They reason well from the start, but they don't get smarter from what they do. Fine-tuning closes that gap, but it's expensive, slow, and prone to catastrophic forgetting. RAG-based memory is cheaper, but it retrieves by similarity — not by whether a past strategy actually worked.

MemRL, published on arXiv in January 2026, proposes a different approach: apply reinforcement learning directly to episodic memory at runtime, without touching model weights. The result is an agent that improves through trial and error, storing structured experiences and learning which ones to prioritize based on real task outcomes.

This guide breaks down how MemRL works, what the benchmarks show, and how the core mechanism looks in practice — including a minimal reproduction Effloow Lab ran to verify the concept.

The Problem MemRL Solves

Current agent memory systems face a fundamental tradeoff. On one end, fine-tuning embeds knowledge directly into model weights — but requires expensive compute, labeled data, and still risks overwriting previously learned behavior (catastrophic forgetting). On the other end, RAG-style retrieval keeps knowledge external, making it cheap to update. But standard RAG retrieves by semantic similarity alone. It surfaces documents that look similar to the current query, not documents associated with strategies that previously worked.

This is the stability-plasticity dilemma: agents either freeze their knowledge (stable but rigid) or update it continuously (plastic but forgetful). MemRL's claim is that this tradeoff is a false choice — you can have a frozen LLM backbone (stable) with an external memory that evolves through RL feedback (plastic).

What MemRL Is

MemRL (arXiv:2601.03192, from MemTensor, updated February 2026) is a non-parametric framework that enables agents to self-evolve through runtime reinforcement learning on episodic memory. The LLM's weights never change. Instead, MemRL maintains a structured external memory, refines it based on task outcomes, and uses a two-phase retrieval mechanism to surface the most useful experiences — not just the most similar ones.

The open-source code is available at MemTensor/MemRL, with support for ALFWorld, BigCodeBench, HLE, and Lifelong Agent Bench benchmarks.

The Intent-Experience-Utility Triplet

The core data structure in MemRL is not a document. It's a triplet:

Intent: the task or query the agent was addressing
Experience: the specific action trajectory or solution strategy used
Utility (Q-value): a learned score representing how successful that experience was

Where RAG stores raw text and retrieves by embedding similarity, MemRL stores structured (intent, experience, Q-value) records. The Q-value is not fixed at write time — it evolves as the agent receives environmental feedback across episodes.

This distinction matters. Two experiences with similar intents might have very different Q-values if one led to a successful outcome and the other failed. RAG can't distinguish these. MemRL can.

How Two-Phase Retrieval Works

When an agent faces a new task, MemRL retrieves relevant past experiences in two stages:

Phase A — Semantic Filter: The agent computes similarity between the current intent and all stored intents using dense embeddings. The top-k candidates (by semantic relevance) are kept. This narrows the search to experiences that are topically related to the current task.

Phase B — Q-Value Ranking: Among those filtered candidates, MemRL re-ranks by Q-value. Experiences with higher utility — those associated with successful outcomes — rise to the top. The agent retrieves the highest-Q candidates and uses them as in-context guidance for the current task.

The paper describes Phase A as analogical transfer (retrieving similar past events) and Phase B as mental rehearsal (selecting strategies proven to work). Together, they avoid the main failure mode of pure RAG: retrieving semantically similar but strategically useless memories.

Q-Value Learning: The RL Mechanism

After the agent completes a task using retrieved memories, it receives a reward signal from the environment — success, partial success, or failure. MemRL applies a Monte Carlo-style update to the Q-value of the used memory:

Q_new = Q_old + α × (reward - Q_old)

Where α is the learning rate. Positive outcomes increase the Q-value; failures decrease it. Over many episodes, Q-values diverge: experiences associated with reliable strategies accumulate higher scores, while noise and failed attempts are downweighted.

The entire optimization loop runs outside the LLM. No gradient computation, no retraining. The LLM reasons over whatever context it's given — MemRL just gets better at deciding what to put in that context.

Effloow Lab PoC: Core Mechanism in Python

Effloow Lab ran a minimal reproduction of the IEU triplet and two-phase retrieval to verify the concept. Full repo installation requires ALFWorld and LLM credentials, so this PoC uses word-overlap similarity instead of dense embeddings — a known limitation documented in the lab run.

import math

class SimpleMemRL:
    def __init__(self, top_k_semantic=5, top_k_q=2):
        self.memory = []
        self.top_k_semantic = top_k_semantic
        self.top_k_q = top_k_q

    def _cosine_sim(self, a, b):
        # word-overlap proxy for embeddings (sandbox limitation)
        set_a = set(a.lower().split())
        set_b = set(b.lower().split())
        if not set_a or not set_b:
            return 0.0
        return len(set_a & set_b) / math.sqrt(len(set_a) * len(set_b))

    def write(self, intent, experience, initial_q=0.5):
        self.memory.append({'intent': intent, 'experience': experience, 'q': initial_q})

    def update_q(self, intent, reward):
        alpha = 0.3
        for m in self.memory:
            if m['intent'] == intent:
                m['q'] += alpha * (reward - m['q'])

    def retrieve(self, query_intent):
        if not self.memory:
            return []
        # Phase A: semantic filter
        scored = [(self._cosine_sim(query_intent, m['intent']), m) for m in self.memory]
        scored.sort(key=lambda x: x[0], reverse=True)
        candidates = [m for _, m in scored[:self.top_k_semantic]]
        # Phase B: Q-value ranking
        candidates.sort(key=lambda m: m['q'], reverse=True)
        return candidates[:self.top_k_q]

Running this with a small set of coding strategy memories, then applying positive feedback to sort-related experiences and negative feedback to a debugging strategy, produced the expected result: sort strategies rose to Q≈0.62, while the debugging entry dropped to Q≈0.24. Subsequent queries for sorting tasks surfaced the higher-Q memories first.

The key limitation observed: word-overlap similarity doesn't capture semantic equivalence well, which caused some retrieval mismatches. Real MemRL uses dense embeddings (e.g., OpenAI text-embedding models or similar), resolving this. Full lab-run details and output are in data/lab-runs/memrl-self-evolving-agents-episodic-memory-rl-guide-2026.md.

Benchmark Results

The paper benchmarks MemRL across these tasks:

Benchmark	MemRL (Last Acc.)	MemP Baseline	No-Memory Baseline	Key Gain
ALFWorld	0.507	0.324	0.278	+56% over MemP
HLE	0.573	0.528	—	+8.5% over MemP
BigCodeBench	0.508	0.494	—	+2.8% over MemP
Lifelong Agent Bench	0.697 CSR	—	—	Best overall

The gains are largest on ALFWorld and Lifelong Agent Bench — multi-step sequential tasks where memory utility accumulates across episodes. BigCodeBench shows smaller gains because it's primarily single-turn: there's less opportunity for multi-episode Q-value refinement when each task is independent.

This pattern is important. MemRL's value is proportional to how much your agent loops over time. If your agent handles isolated, one-shot queries, you won't see ALFWorld-level improvements.

MemRL vs Traditional RAG

MemRL Strengths
<ul>
  <li>Learns from success/failure — not just semantic match</li>
  <li>No model fine-tuning required — frozen LLM backbone</li>
  <li>Q-values suppress noise and bad strategies over time</li>
  <li>Improves within a session and across sessions (transfer)</li>
  <li>Open-source with multi-benchmark validation</li>
</ul>


Where It Lags
<ul>
  <li>Needs an environmental feedback signal — not always available</li>
  <li>Less useful for purely one-shot tasks without episode loops</li>
  <li>Q-value cold start: early episodes have unrefined utility scores</li>
  <li>More complex to set up than a standard RAG pipeline</li>
</ul>

The underlying difference is what retrieval optimizes for. RAG finds memories that are similar. MemRL finds memories that are similar and proved useful. For long-running agents where failure has a cost — home automation, coding assistants, planning agents — this distinction is meaningful.

The Tempera MCP Server

A community implementation called Tempera applies MemRL concepts to AI coding workflows via Model Context Protocol (MCP). Tempera captures coding sessions as episodes, indexes them for semantic search, and uses RL to surface the most valuable memories at query time. All projects share a common memory database stored under ~/.tempera/, enabling cross-project learning — a direct practical application of the MemRL architecture.

This matters for developers already using MCP-compatible tools: Tempera is one path to experimenting with MemRL ideas without implementing the full research framework.

How to Get Started with MemRL

For developers interested in running the actual MemRL benchmarks, the setup flow is:

# 1. Clone the repo
git clone https://github.com/MemTensor/MemRL
cd MemRL

# 2. Create environment (Python 3.10 required)
conda create -n memrl python=3.10
conda activate memrl

# 3. Install dependencies
pip install -r requirements.txt

# 4. Configure LLM + embedding settings in configs/
# (YAML files per benchmark)

# 5. Run a benchmark runner
python memrl/run/alfworld_rl_runner.py

Results write to logs/ and results/ directories. The configs/ directory controls which LLM and embedding model you use — the paper uses frontier models but the code supports swapping these.

Full environment setup for ALFWorld requires additional installation steps documented in the repo's README.

Practical Implications for Agent Developers

MemRL's ideas translate to a few concrete questions worth asking about any agent system:

Does your agent run repeatedly over similar tasks? If yes, runtime Q-value learning could improve retrieval quality. If your agent handles purely isolated requests, the benefit is limited.

What's your feedback signal? MemRL needs a reward — task success, user rating, test pass/fail, something. Agents that get no structured outcome signal can't update Q-values. Designing a feedback loop is a prerequisite, not an afterthought.

Are you fighting retrieval noise? If your RAG-based memory system frequently surfaces semantically similar but strategically useless memories, MemRL's Phase B filtering is directly relevant. The Q-value layer exists precisely to downweight experiences that match the query but don't help.

Do you need to avoid retraining? MemRL's strongest argument is that agents can improve without compute-intensive fine-tuning cycles. For teams running agents at scale where fine-tuning is prohibitively expensive, this is a meaningful alternative.

Q: How is MemRL different from Reflexion or Voyager?

Reflexion stores verbal self-reflection notes in memory. Voyager builds a skill library. MemRL is distinct in applying Q-value learning to determine which stored experiences to retrieve. Reflexion and Voyager still rely on recency or semantic matching; MemRL's retrieval is utility-driven.

Q: Can MemRL work with any LLM?

Yes — the LLM backbone is frozen. MemRL is agnostic to the underlying model. The paper runs experiments with frontier models, but the memory and retrieval mechanism is entirely external to the LLM's weights.

Q: What happens if the reward signal is noisy?

Noisy rewards are a known challenge in RL. The paper applies Monte Carlo-style updates (averaging over episodes) which provides some robustness, but highly noisy reward signals will produce unreliable Q-values. The quality of MemRL's learning is bounded by the quality of the feedback signal.

Q: Does MemRL require embeddings?

Yes, Phase A requires dense vector similarity. The sandbox PoC used word-overlap as a proxy, but real MemRL uses embedding models to compute semantic similarity between stored intents and current queries. Any embedding model compatible with your stack works.

Key Takeaways

MemRL addresses a genuine gap: the cost of fine-tuning versus the limitations of static retrieval. Its approach — structure memory as IEU triplets, filter by semantics, rank by learned Q-values, update Q-values from task outcomes — is conceptually clean and benchmarked across four tasks.

The gains are largest for multi-step, episodic tasks (ALFWorld: +56% over MemP) and more modest for single-turn workloads (BigCodeBench: +2.8%). The framework needs a feedback signal, and Q-values start uninformed — so there's a cold-start cost on early episodes.

For teams building agents that loop repeatedly over tasks, interact with real environments, and can capture task success as a signal, MemRL is a well-evidenced alternative to both fine-tuning and standard RAG. The code is open, the benchmarks are public, and the Tempera MCP server offers a path to experimenting without setting up the full research framework.

Bottom Line

MemRL is one of the more rigorous proposals for non-parametric agent learning published in early 2026. If you're running agents that repeat tasks and can capture feedback, the two-phase retrieval mechanism is worth understanding — and the open-source code makes it possible to test on your own benchmarks without writing the RL layer from scratch.

Sources:

OpenAI Realtime Audio API: Voice Agents Guide 2026

Jangwook Kim — Thu, 14 May 2026 00:12:11 +0000

On May 7, 2026, OpenAI quietly made voice agents production-viable. Three new realtime audio models landed in the API at the same time: GPT-Realtime-2 (voice with GPT-5-class reasoning), GPT-Realtime-Translate (live speech-to-speech translation across 70+ languages), and GPT-Realtime-Whisper (streaming speech-to-text billed by the minute). Each model has its own pricing, endpoint, and use-case fit.

If you have been waiting for a stable, production-ready voice API before building, the wait is over. This guide walks through what each model does, how to connect to the API, what it costs, and the production patterns that separate a working demo from a robust voice agent.

Effloow Lab inspected the Realtime API protocol and validated client-side event structures locally as part of this article's research. Full live testing requires an OpenAI API key; where relevant, we note what we verified and what we did not.

Why This Release Matters

Previous versions of the Realtime API required working around a 32K-token context ceiling, managing your own speech-to-text pipeline, and accepting that the model would sometimes lose the thread of a long conversation. GPT-Realtime-2 removes these constraints:

Context window expanded to 128K tokens — four times the previous limit, enough for multi-turn conversations spanning tens of minutes
GPT-5-class reasoning integrated directly — the model can call tools, reason through steps, and respond, all without leaving the audio stream
Three specialized models instead of one general voice model, each optimized for a specific cost-performance point

The split into three models is also a pricing move. If you only need transcription, GPT-Realtime-Whisper at $0.017/minute is dramatically cheaper than running voice inference at $32/1M tokens. Choose the right model and you can cut costs by 80–90% relative to using GPT-Realtime-2 for everything.

Model	Purpose	Pricing	Context
gpt-realtime-2	Voice reasoning agent	$32/1M input · $64/1M output tokens	128K tokens
gpt-realtime-translate	Live speech translation	$0.034/min	Translation-only
gpt-realtime-whisper	Streaming transcription	$0.017/min	STT-only

GPT-Realtime-2: Voice Reasoning for Production Agents

GPT-Realtime-2 is the flagship of the trio. It brings GPT-5-level intelligence into the audio stream: the model can reason through multi-step requests, call functions, handle tool results, and continue speaking — all without pausing the conversation for a round trip to a separate text model.

How audio tokens are billed

OpenAI encodes audio duration into tokens rather than sampling audio at a fixed rate. The billing math is:

User speech (input): 1 token per 100 ms of audio → 600 tokens per minute
Model response (output): 1 token per 50 ms of audio → 1,200 tokens per minute

For a typical bidirectional voice call where the user talks roughly as much as the model:

Input cost:  600 tokens × ($32 / 1,000,000) = $0.0192 / min
Output cost: 1,200 tokens × ($64 / 1,000,000) = $0.0768 / min
Total uncached: ~$0.096 / min (~$5.76 / hour)

With prompt caching applied to system instructions and persistent session context, real-world costs can drop to roughly $0.05–$0.10/min according to third-party production estimates published by OpenAI partners.

Connecting via WebSocket

The Realtime API uses a persistent WebSocket connection. Every interaction is modeled as an exchange of typed JSON events — the client sends events, the server sends events back. Effloow Lab validated that the client-side event structures serialize and round-trip correctly in Python:

import asyncio
import json
import websockets

OPENAI_API_KEY = "sk-..."  # your key

async def voice_agent_session():
    uri = "wss://api.openai.com/v1/realtime?model=gpt-realtime-2"
    headers = {
        "Authorization": f"Bearer {OPENAI_API_KEY}",
        "OpenAI-Beta": "realtime=v1"
    }

    async with websockets.connect(uri, additional_headers=headers) as ws:
        # 1. Configure the session
        await ws.send(json.dumps({
            "type": "session.update",
            "session": {
                "modalities": ["audio", "text"],
                "voice": "alloy",
                "turn_detection": {
                    "type": "server_vad",
                    "threshold": 0.5,
                    "silence_duration_ms": 500
                },
                "tools": [
                    {
                        "type": "function",
                        "name": "lookup_order",
                        "description": "Look up a customer order by ID",
                        "parameters": {
                            "type": "object",
                            "properties": {
                                "order_id": {"type": "string"}
                            },
                            "required": ["order_id"]
                        }
                    }
                ],
                "tool_choice": "auto"
            }
        }))

        # 2. Stream audio (PCM16, 24kHz, base64-encoded chunks)
        # await ws.send(json.dumps({
        #     "type": "input_audio_buffer.append",
        #     "audio": base64_chunk
        # }))

        # 3. Listen for server events
        async for raw_msg in ws:
            event = json.loads(raw_msg)
            event_type = event.get("type", "")

            if event_type == "response.audio.delta":
                # stream audio bytes to speaker
                pass
            elif event_type == "response.function_call_arguments.done":
                # handle tool call, then send result back
                await ws.send(json.dumps({
                    "type": "conversation.item.create",
                    "item": {
                        "type": "function_call_output",
                        "call_id": event["call_id"],
                        "output": json.dumps({"order_status": "shipped"})
                    }
                }))
                await ws.send(json.dumps({"type": "response.create"}))

asyncio.run(voice_agent_session())

The OpenAI Agents Python SDK (openai-agents) wraps this pattern into a higher-level RealtimeAgent class if you prefer avoiding raw WebSocket management. The underlying transport is the same.

Tool calls mid-conversation

GPT-Realtime-2 can call functions while speaking. The agent does not stop talking and wait — it continues the audio stream with a phrase like "Let me look that up" while dispatching the tool call in parallel. When the result arrives, it folds it into the ongoing response. This pattern is what makes GPT-Realtime-2 meaningfully different from a text model with TTS bolted on.

Interruption handling

Voice activity detection (VAD) is built in when you set turn_detection.type = "server_vad". When the user starts speaking mid-response, the API sends a response.cancelled event, truncates the current audio output, and starts a new inference cycle. The 128K context window means the model retains everything said before the interruption without a context reset.

Three things to get right in production:

VAD threshold (threshold: 0.5 in the example above) — lower values detect softer speech but increase false triggers in noisy environments. Tune per your deployment channel (phone line vs browser microphone vs call center headset).
Silence duration (silence_duration_ms) — how long a pause triggers end-of-turn. 500ms works for conversational speech; customer support scripts may need 700–1000ms.
Barge-in state management on your server — when response.cancelled fires, flush any queued tool results from the cancelled turn or you'll deliver stale data to the next response cycle.

GPT-Realtime-Translate: Live Speech-to-Speech Translation

GPT-Realtime-Translate is a single-purpose model trained on thousands of hours of professional interpreter audio. It takes live speech in any of 70+ input languages, detects the source language automatically, and returns translated speech plus text transcripts in one of 13 output languages.

Target output languages as of May 2026: Spanish, Portuguese, French, Japanese, Russian, Chinese, German, Korean, Hindi, Indonesian, Vietnamese, Italian, and English.

The dedicated endpoint is /v1/realtime/translations:

uri = "wss://api.openai.com/v1/realtime/translations"

session_config = {
    "type": "session.update",
    "session": {
        "output_language": "ja",   # target language code
        # source language is auto-detected
        "voice": "alloy"
    }
}

You stream 24 kHz PCM16 audio into input_audio_buffer.append exactly as you would with GPT-Realtime-2. The model processes input audio while simultaneously streaming translated audio back, which keeps perceived latency low over continuous speech.

Unlike a general-purpose voice model, GPT-Realtime-Translate will not answer questions or carry on conversation. It is translation-only by design. If a user asks "what time is it?" in French and your output language is English, the model translates the question into English — it does not answer it. Build a routing layer in front if your product needs both translation and reasoning.

At $0.034/minute, a one-hour multilingual support call costs $2.04 in translation credits. A 30-person conference session with real-time translation for 60 minutes costs around $60 — cheaper than a human interpreter for a short session, and it runs at scale.

GPT-Realtime-Whisper: Streaming Speech-to-Text

GPT-Realtime-Whisper is the transcription-only model in the trio. It starts producing text output as the speaker talks rather than waiting for an utterance to finish. This keeps the UI feeling responsive — a transcription bar can update word-by-word instead of appearing in blocks.

Pricing at $0.017/minute makes it among the cheapest options for streaming STT in the OpenAI ecosystem. An eight-hour workday of continuous transcription costs about $8.16.

# Whisper Realtime session uses the standard /v1/realtime endpoint
# with model=gpt-realtime-whisper
uri = "wss://api.openai.com/v1/realtime?model=gpt-realtime-whisper"

# Server returns transcript deltas as speech is detected:
# { "type": "conversation.item.input_audio_transcription.delta", "delta": "Hello, " }
# { "type": "conversation.item.input_audio_transcription.delta", "delta": "can you hear me?" }
# { "type": "conversation.item.input_audio_transcription.completed", "transcript": "Hello, can you hear me?" }

GPT-Realtime-Whisper is the right choice when you need transcription but not inference — meeting recorders, live captioning systems, accessibility tools, voice-search preprocessing, and call analytics pipelines where a separate LLM processes the transcript downstream.

Practical Application: Choosing the Right Model

The three models are not interchangeable. Use this decision tree:

Does your user need a spoken response from the AI?

Yes, and it involves reasoning, tool calls, or multi-turn logic → gpt-realtime-2
Yes, but it is a direct translation of what another person said → gpt-realtime-translate
No, you only need the text of what the user said → gpt-realtime-whisper

A customer support agent that looks up orders and reads statuses aloud: gpt-realtime-2.
A multilingual conference call platform where each attendee hears their own language: gpt-realtime-translate.
A meeting transcription SaaS that feeds into a separate summarizer: gpt-realtime-whisper.

For hybrid products, you can run models side-by-side. A global customer support pipeline might use gpt-realtime-translate for non-English callers to produce an English transcript, then pass that transcript to a text-only GPT-5 for classification and routing, and only invoke gpt-realtime-2 when the agent needs to speak back. This layering can reduce per-call cost significantly compared to routing all audio through gpt-realtime-2.

Common Mistakes in Production Voice Agents

Ignoring prompt caching on system instructions. The session configuration message is sent at the start of every WebSocket connection. For long system prompts, this is the largest per-session input cost. OpenAI caches inputs at $0.40/1M tokens vs $32/1M for uncached. Keep your system prompt stable and reuse session configurations where possible.

Treating response.cancelled as an error. Interruptions are a normal part of conversation. Your application should handle the cancel event cleanly — flush pending state, log the cancelled turn, and let the model proceed with the new input. Applications that surface interruption events as errors create broken UX and noisy logs.

Forgetting that context grows. The 128K context window means gpt-realtime-2 can hold a very long conversation without a reset. But it also means costs accumulate. A one-hour conversation with balanced speaking time can push well past $10 in audio tokens alone. For high-volume deployments, consider session time limits or periodic context compaction using a text-model summarization step.

Using gpt-realtime-2 for transcription-only use cases. If you only need the text of what the user said, run gpt-realtime-whisper at $0.017/min instead of gpt-realtime-2 at $0.096+/min. The cost difference is roughly 5–6x.

Hard-coding the VAD threshold. Different audio channels have different noise floors. A browser tab with a decent microphone is not the same as a phone call over PSTN. Ship a configuration option, even if only for internal deployment channels.

FAQ

Q: Does gpt-realtime-2 use GPT-5 under the hood?

OpenAI describes gpt-realtime-2 as bringing "GPT-5-class reasoning" to live voice, and their Big Bench Audio benchmark shows +15.2% audio intelligence over GPT-Realtime-1.5. OpenAI has not confirmed whether the underlying weights are shared with GPT-5 or whether this is a separate model trained to the same capability level.

Q: Can I use the Realtime API from a browser (client-side)?

Yes. OpenAI supports ephemeral session tokens for client-side WebSocket connections. Generate a short-lived token from your backend (POST /v1/realtime/sessions), pass it to the browser, and open the WebSocket from JavaScript. Do not embed your main API key in client-side code.

Q: How does server VAD compare to manual turn detection?

Server VAD (turn_detection.type = "server_vad") lets OpenAI's infrastructure handle speech segmentation — it detects when the user stops speaking and triggers inference automatically. Manual turn detection (turn_detection: null) gives your application full control: you decide when to commit an audio buffer and request a response. Manual mode is more predictable in noisy environments but requires more engineering. Start with server VAD and switch to manual if you hit false-trigger issues.

Q: Is gpt-realtime-translate available on Azure OpenAI?

Microsoft's Azure AI Foundry announced support for the new realtime audio models including gpt-realtime-whisper and gpt-realtime-translate shortly after the OpenAI release. Check the Azure OpenAI pricing page for regional availability and pricing, which may differ from direct OpenAI API pricing.

Q: What audio format does the Realtime API accept?

The API accepts PCM16 audio at 24 kHz, base64-encoded and sent as input_audio_buffer.append events. Most browser MediaRecorder APIs require a format conversion step. The OpenAI cookbook includes a realtime_translation_guide example with a JavaScript AudioWorklet for in-browser PCM16 capture.

Q: What happens if the WebSocket connection drops mid-conversation?

The session state is held server-side for the duration of the connection. If the connection drops, the session is lost — there is no resume or reconnect mechanism as of May 2026. Build reconnect logic in your client and design conversations to be resumable from the last committed turn. Store transcript deltas locally and replay context if a reconnect is needed.

Key Takeaways

The May 2026 Realtime Audio API update is the first time all three voice agent primitives — reasoning, translation, and transcription — are available in a single unified API with clear per-minute or per-token pricing.

For most developers building voice agents, the practical starting point is gpt-realtime-2 for prototyping and gpt-realtime-whisper for any transcription path that feeds a separate model. GPT-Realtime-Translate is genuinely useful and underpriced compared to traditional translation infrastructure — a multilingual product that previously required third-party translation services can now route entirely through one API.

The 128K context window and built-in VAD make gpt-realtime-2 a legitimate foundation for production voice agents rather than a demo novelty. The remaining work is on your side: audio channel handling, graceful interruption management, prompt caching discipline, and cost modeling before you scale.

Bottom Line

OpenAI's three-model voice API split is the right architecture: specialized models at specialized prices, all behind one WebSocket protocol. GPT-Realtime-2 is finally production-ready with 128K context and native tool calling. GPT-Realtime-Whisper at $0.017/min is the new default for any transcription-only pipeline. Build the routing layer between them and you can cover most voice AI use cases without leaving the OpenAI ecosystem.

AWS Kiro: Spec-Driven IDE for Agentic Development

Jangwook Kim — Wed, 13 May 2026 08:20:17 +0000

There is a quiet argument happening inside every engineering team that uses AI coding tools: should the AI write code directly from a chat prompt, or should it first commit to a plan you can actually verify?

Cursor and Windsurf answer "write from the prompt." AWS Kiro answers "write the spec first."

That is not a small difference. It changes what you version, what you review in a pull request, and who on the team can understand what the agent actually built. This guide covers what Kiro does, how the spec workflow is structured, how agent hooks automate the repetitive parts, and where it fits relative to the other agentic IDEs competing for your workflow in 2026.

What Kiro Is

Kiro is a desktop IDE built on Code OSS — the open-source base that VS Code also runs on — developed by Amazon Web Services and released to the public in late 2025. It reached general availability in November 2025 after hitting capacity limits within days of its July preview launch.

The product is positioned as AWS's successor to Amazon Q Developer. AWS ended new Q Developer signups effective May 15, 2026, explicitly directing new users to Kiro. That matters for team context: if your organization is already on AWS and was evaluating Q Developer, Kiro is now the answer.

The core design principle: specs are the source of truth, and code is a build artifact derived from them. Rather than asking an agent to "add a rate limiter," you write a spec that describes what the rate limiter should do, under what conditions, and what the acceptance criteria are. The agent then generates code to satisfy the spec, not just a prompt.

The Spec Workflow

When you start a feature, Kiro creates three structured markdown files under .kiro/specs/{feature-name}/:

requirements.md captures user stories and acceptance criteria using EARS notation (Easy Approach to Requirements Syntax). EARS structures each requirement as a conditional assertion:

WHEN the user submits the registration form
THEN the system SHALL validate the email format before saving
AND the system SHALL return a 422 with field-level errors when validation fails

That format is not just documentation. It maps directly to testable assertions, which is why Kiro can generate test stubs from requirements with reasonable accuracy.

design.md documents the technical architecture for the feature — data models, sequence diagrams in text form, interface contracts, and any relevant infrastructure considerations. This file lives in the repo alongside the feature code, so anyone reviewing a pull request can see the design intent without reconstructing it from the implementation.

tasks.md contains a discrete task list that Kiro generates from the requirements and design. Tasks are tracked as in-progress or completed as the agent works through them. You can pause, redirect, or reassign tasks manually; Kiro treats them as a checkpoint-able queue, not a linear script.

The three-document structure is also the surface where human review happens. Before the agent touches code, you can edit requirements to narrow scope, add edge cases to the design, or reprioritize tasks. That is the mechanism Kiro offers for keeping the human in the loop on complex features without turning every step into a manual approval.

Strengths
<ul>
  <li>Specs survive code refactors — the "why" stays versioned in the repo</li>
  <li>EARS format produces testable acceptance criteria, not vague prose</li>
  <li>Spec review is a natural code review gate that any team member can participate in</li>
  <li>Free tier (50 requests/month) requires no AWS account or credit card</li>
  <li>Powers bundle MCP servers + hooks into reusable, context-aware packages</li>
</ul>


Limitations
<ul>
  <li>Spec-first workflow adds planning time — not suited for fast prototyping</li>
  <li>Credits deplete quickly on multi-file specs (community-reported)</li>
  <li>Deep AWS integrations require an AWS account and Bedrock access</li>
  <li>Smaller extension/plugin ecosystem compared to VS Code or Cursor</li>
</ul>

Agent Hooks: Automating the Repetitive Parts

One feature that distinguishes Kiro from competitors is its hook system. Hooks are event-driven automations configured in .kiro/hooks/ as JSON files. When a trigger event fires, the hook either runs a natural-language agent prompt or executes a shell command.

The available triggers as of Kiro's 0.10 changelog:

file:save — fires whenever you save a file
file:create — fires when a new file is created
task:pre — fires before a spec task begins executing
task:post — fires after a spec task completes

Common hook patterns from the official Kiro blog:

{
  "trigger": "file:save",
  "match": "src/components/**/*.tsx",
  "action": {
    "type": "agent",
    "prompt": "Update the test file for the component that was just saved. Keep existing test cases; add new ones only for changed behavior."
  }
}

That hook means you never manually sync your test file after touching a component. The agent does it on save, automatically, every time.

The task:post hook is useful for quality gates. You can configure it to run linting, type checking, or test execution after each agent task completes — so that a multi-step spec run doesn't silently accumulate broken intermediate states.

Hooks are committed to the repository, not stored locally in user preferences. That means the automation behavior is consistent across the whole team and survives machine changes.

Kiro Powers and MCP Integration

Kiro supports both local and remote MCP servers. Its differentiated feature here is "Powers" — a packaging concept introduced in changelog 0.10.

A Power bundles three things into a single installable unit:

An MCP server providing tools
A steering file that defines when and how to activate those tools
Optional hooks that automate related tasks

Powers activate on-demand based on conversation context rather than loading all MCP tools upfront. This keeps the token budget clean: if you are working on a CloudFormation stack, the CloudFormation Power becomes active; the pricing tools stay dormant until they are relevant.

AWS ships first-party Powers for several of its own platforms: CDK, CloudFormation, Pricing, and HealthOmics workflows. Third-party Powers follow the same packaging spec. If you are building your own MCP server and want it to integrate cleanly with Kiro, the Powers format gives you a structured way to bundle it.

This is worth comparing to how Cursor handles MCP: Cursor supports MCP servers directly but without the packaging abstraction. All configured servers load simultaneously, and there is no built-in concept of context-aware activation. For teams with many MCP tools, the Powers approach reduces noise at the cost of an additional configuration layer.

Pricing and Getting Started

Kiro runs on a credit system. One agentic request equals one credit. Plans as of May 2026:

Plan	Monthly Credits	Price	AWS Account Required
Free	50	$0	No
Pro	Unlimited*	$20/mo	No
Pro+	Unlimited*	$40/mo	No
Power	Unlimited*	$200/mo	Optional

*Overage credits beyond the plan's included usage cost $0.04 each, billed at month-end.

To install: download from kiro.dev/downloads. The installer is available for macOS, Windows, and Linux. Sign in with GitHub, Google, AWS Builder ID, or IAM Identity Center. No credit card for the free tier.

Your first project follows this path:

Open a folder in Kiro
Open the Kiro panel and type a feature description in natural language
Kiro generates .kiro/specs/your-feature/requirements.md — review and edit it
Approve the requirements → Kiro generates design.md and tasks.md
Approve the design → Kiro begins working through tasks.md sequentially
Hooks run automatically on file saves during implementation

The full quickstart is at kiro.dev/docs/getting-started/first-project/.

How Kiro Compares to Cursor and Windsurf

The agentic IDE space has three dominant positions heading into mid-2026:

Cursor (1M+ daily active users, $20/mo Pro) is the market leader. Its strength is codebase indexing: semantic embeddings of the entire repo, @-file references, and a polished multi-file editing experience. Agent mode handles large refactors well. The weakness is that "prompt → code" means the agent's intent is implicit in the output, not in a verifiable artifact.

Windsurf ($15/mo) targets enterprise teams. Its Cascade feature auto-discovers context without manual file tagging, which works well on large codebases. First-pass success on complex tasks is reported as higher than Cursor's agent mode.

Kiro is the most opinionated of the three. It trades speed for verifiability. The spec workflow adds 15–30 minutes of upfront planning to any non-trivial feature. In return, you get requirements that you can reference in code review, design decisions that survive refactors, and hooks that keep tests and documentation in sync automatically.

A useful heuristic: if your team already writes design documents before implementing, Kiro formalizes that workflow and connects it to the code generation loop. If your team goes from Jira ticket straight to code, Kiro will feel like it is adding ceremony without clear return.

For further context on the broader agentic IDE landscape, see the cursor vs windsurf vs zed comparison and the best AI coding agents roundup for 2026.

Who Should Actually Use Kiro

Good fit:

Teams already on AWS who want AI coding integrated with their existing Bedrock and IAM setup
Projects where requirements traceability matters: regulated industries, complex APIs, multi-team codebases
Engineers who write design documents by habit and want to close the gap between the doc and the code
Anyone evaluating Amazon Q Developer alternatives (Kiro is now the official successor)

Less useful:

Solo developers doing rapid prototyping where the cost of planning exceeds the cost of mistakes
Projects where the team does not review design artifacts — specs without readers add overhead with no return
Teams wanting the largest VS Code extension ecosystem (Kiro's is smaller, though growing)

The Real Question: Is Spec-Driven Development a Better Default?

The honest answer is that spec-driven development is better for some teams and worse for others — and Kiro does not resolve that ambiguity for you.

What Kiro does resolve is the artifact gap that exists in every other agentic IDE: the mismatch between what you asked for and what the code actually does, documented nowhere. The spec files live in the repository. When something breaks three months later, you can read what the system was supposed to do instead of reverse-engineering it from the output.

Whether that is worth the additional workflow overhead depends on how much of your team's time currently goes into maintaining context versus generating new code. For teams where "why does this work this way" is a common question in standups, the spec overhead pays back quickly. For solo builders iterating fast, the overhead stays overhead.

Kiro's MCP Powers concept is worth watching independently of the spec workflow. Bundling MCP servers with activation context and hooks is a packaging idea that other IDEs will likely adopt — it solves a real problem with how multiple MCP tools currently have to be configured and managed.

FAQ

Q: Does Kiro work without an AWS account?

Yes. The free tier (50 agentic requests/month) and the paid Pro plans ($20/mo) work with GitHub or Google sign-in. An AWS account only becomes relevant if you want to use Bedrock directly or connect to AWS-specific Powers like CloudFormation or CDK.

Q: Are Kiro specs committed to the repository?

Yes. The .kiro/specs/ and .kiro/hooks/ directories are intended to be committed. Specs and hooks are team artifacts, not personal IDE settings. This is deliberate: Kiro's design assumes the spec files are part of the code review surface.

Q: How are Kiro credits consumed?

Each agentic request consumes one credit. Generating a spec from a prompt, executing a task from tasks.md, or running an agent hook each count as one request. Autocomplete and inline suggestions do not consume credits. On the free tier (50 credits/month), a medium-complexity feature with 8–10 spec tasks plus several hooks will use most of the monthly allowance.

Q: What is the difference between Kiro Powers and regular MCP servers?

A Power is an MCP server plus a steering file plus optional hooks, packaged together. The steering file tells Kiro when to activate the Power's tools based on conversation context. Regular MCP servers load all their tools upfront; Powers load on-demand. The practical difference is a shorter tool list in the agent's context window, which reduces token usage and improves relevance on complex tasks.

Q: Is Kiro open source?

The Kiro codebase repository is at github.com/kirodotdev/Kiro. The IDE is built on Code OSS (VS Code open-source base). The agent runtime and Bedrock integrations are proprietary AWS services.

Bottom Line

Kiro is the first agentic IDE that makes the design document part of the build process rather than a separate artifact that decays. The spec workflow adds overhead that pays back on team codebases where requirements traceability matters. For solo prototyping or teams that run Cursor smoothly, there is no compelling reason to switch today — but the Powers and hooks concepts are worth watching as patterns the rest of the IDE market will absorb.

Claude Agent SDK Practical Guide — Building Tool-Using AI Agents from Scratch

Jangwook Kim — Wed, 13 May 2026 06:41:24 +0000

I ran into the Tool Use moment while building a FastAPI streaming backend with the Claude API. The trigger was simple: a user asked "how many days are left in this year?" and Claude answered wrong. Not just wrong — confidently wrong. I remember thinking, "OK, a chatbot can't handle this."

Tool Use fixes that structurally. Instead of the model calculating directly, it calls a calculation function and uses the result to answer. That difference is what separates a chatbot from an agent.

This guide covers the Tool Use patterns I validated by directly installing and running anthropic SDK 0.101.0. From basic tool definitions to the agentic loop, error handling, and cost — practical code you can actually use.

Why Tool Use Is Different from a Chatbot — The Structural Gap

An LLM samples tokens from a probability distribution. Tasks like date arithmetic, precise numerical calculations, or live API lookups are structurally unreliable — the model recreates patterns from training data, not ground truth.

Tool Use addresses this at a different layer. The model decides what to do, and actual execution is delegated to external code. Instead of computing directly, the model emits something like calculate("365 - today.day_of_year"), and Python runs it and returns the result.

# Chatbot: model answers directly
# "Doesn't know today's date, has to compute directly -> can be wrong"
response = client.messages.create(
    model="claude-opus-4-7",
    messages=[{"role": "user", "content": "How many days left in this year?"}]
)

# Agent: delegates to a tool
# "Model picks the tool, Python computes accurately"
response = client.messages.create(
    model="claude-opus-4-7",
    tools=tools,  # includes date calculation tool
    messages=[{"role": "user", "content": "How many days left in this year?"}]
)

The decisive difference is reliability. Python's datetime module doesn't get dates wrong.

Setup — Sandbox Verification Results

python3 -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate
pip install anthropic

Results from running this directly in a temp directory:

anthropic version: 0.101.0
Client instantiated: ✓
Client type: Anthropic

0.101.0 is the latest as of 2026-05-13. This is the official Anthropic SDK — completely different from packages like pyautogen that were common before 2025.

import anthropic
import json
from typing import Any

client = anthropic.Anthropic(api_key="your-api-key")  # or set ANTHROPIC_API_KEY env var

The SDK auto-loads the API key from ANTHROPIC_API_KEY. Don't hard-code it.

Defining Your First Tool — JSON Schema Is All You Need

Tool Use uses a structure similar to OpenAI Function Calling. Each tool has three parts:

name: Tool identifier (like a function name)
description: The basis for the model's decision on when to use this tool
input_schema: JSON Schema for input parameters

tools = [
    {
        "name": "get_current_date_info",
        "description": "Returns current date and time information. Use for questions about 'today', 'now', or anything requiring current date knowledge.",
        "input_schema": {
            "type": "object",
            "properties": {
                "timezone": {
                    "type": "string",
                    "description": "IANA timezone (e.g. America/New_York, Asia/Seoul). Default: UTC"
                }
            },
            "required": []
        }
    },
    {
        "name": "calculate",
        "description": "Performs mathematical operations. Handles addition, subtraction, multiplication, division, exponentiation, and modulo.",
        "input_schema": {
            "type": "object",
            "properties": {
                "operation": {
                    "type": "string",
                    "enum": ["add", "subtract", "multiply", "divide", "power", "modulo"],
                    "description": "The operation to perform"
                },
                "a": {"type": "number", "description": "First operand"},
                "b": {"type": "number", "description": "Second operand"}
            },
            "required": ["operation", "a", "b"]
        }
    }
]

The description field matters more than it looks. The model reads only the description to decide whether to use this tool. When I tested with vague descriptions, the model picked the wrong tool or skipped it entirely.

Validated tool definition structure from my sandbox:

Tool: get_current_date_info
  Description: Returns current date info
  Required params: []

Tool: calculate
  Description: Performs math operations
  Required params: ['operation', 'a', 'b']

Implementing the Agentic Loop — The Core of Tool Use

This is the core. Tool Use doesn't finish in a single API call. When the model calls a tool → we execute it → we feed the result back. This cycle repeats until the model returns end_turn.

def run_agent(user_message: str, tools: list, max_iterations: int = 10) -> str:
    messages = [{"role": "user", "content": user_message}]

    for i in range(max_iterations):
        response = client.messages.create(
            model="claude-opus-4-7",
            max_tokens=4096,
            tools=tools,
            messages=messages,
        )

        # No tool call — return the final answer
        if response.stop_reason == "end_turn":
            for block in response.content:
                if hasattr(block, "text"):
                    return block.text

        # Handle tool calls
        if response.stop_reason == "tool_use":
            # Add the full assistant response to messages (including tool calls)
            messages.append({"role": "assistant", "content": response.content})

            # Collect all tool results and add together
            tool_results = []
            for block in response.content:
                if block.type == "tool_use":
                    result = process_tool_call(block.name, block.input)
                    tool_results.append({
                        "type": "tool_result",
                        "tool_use_id": block.id,
                        "content": result,
                    })

            # Tool results go under the "user" role (API requirement)
            messages.append({"role": "user", "content": tool_results})

    return "Max iterations exceeded"

Two things are easy to miss here.

First, add the entire response.content to messages — not just the text block. The model needs to know which tool it called in order to generate its next response correctly.

Second, tool results go under the user role. Counterintuitive, but the API treats tool execution results as coming from the environment (the user side), not the assistant.

Building Real Tools — Calculator, Date, File Reader

The tool execution function is straightforward. It takes a name and input, returns a string:

from datetime import datetime
import pytz
import json
import operator
from typing import Any

# Safe math — uses operator mapping instead of string expression execution
SAFE_OPERATIONS = {
    "add": operator.add,
    "subtract": operator.sub,
    "multiply": operator.mul,
    "divide": operator.truediv,
    "power": operator.pow,
    "modulo": operator.mod,
}

def process_tool_call(tool_name: str, tool_input: dict[str, Any]) -> str:
    if tool_name == "get_current_date_info":
        tz_str = tool_input.get("timezone", "UTC")
        try:
            tz = pytz.timezone(tz_str)
            now = datetime.now(tz)
            day_of_year = now.timetuple().tm_yday
            days_remaining = 365 - day_of_year
            return json.dumps({
                "date": now.strftime("%Y-%m-%d"),
                "time": now.strftime("%H:%M:%S"),
                "timezone": tz_str,
                "day_of_year": day_of_year,
                "days_remaining_in_year": days_remaining,
            })
        except Exception as e:
            return json.dumps({"error": str(e)})

    elif tool_name == "calculate":
        op_name = tool_input.get("operation")
        a = tool_input.get("a", 0)
        b = tool_input.get("b", 0)
        op_func = SAFE_OPERATIONS.get(op_name)
        if op_func is None:
            return f"Error: Unknown operation: {op_name}"
        try:
            if op_name == "divide" and b == 0:
                return "Error: Cannot divide by zero"
            result = op_func(a, b)
            return str(result)
        except Exception as e:
            return f"Error: {e}"

    elif tool_name == "read_file":
        import os
        filepath = tool_input.get("path", "")
        # Path traversal prevention: only allow within designated base directory
        allowed_base = "/app/data"
        abs_path = os.path.realpath(filepath)
        if not abs_path.startswith(allowed_base):
            return "Error: Path not allowed"
        try:
            with open(abs_path, "r") as f:
                content = f.read(2000)  # 2KB limit
            return content
        except FileNotFoundError:
            return f"Error: File not found: {filepath}"

    return f"Error: Unknown tool: {tool_name}"

Actual sandbox results:

calculate(multiply, 15, 7) = 105
calculate(add, 105, 3) = 108
calculate(divide, 100, 4) = 25.0
Input validation (required field present): True
Input validation (missing required field): False — Missing required field: location

The error classification strategy from the FastAPI + Claude API streaming guide applies here too — categorize tool errors as retryable vs. non-retryable for better production stability.

Handling Multiple Tool Calls — Can We Run in Parallel?

Claude can call multiple tools simultaneously in a single turn. Ask "compare the weather in Seoul and Tokyo" and it returns two get_weather calls at once.

# When Claude calls multiple tools in one turn
tool_use_blocks = [b for b in response.content if b.type == "tool_use"]

# Technically possible to run in parallel
from concurrent.futures import ThreadPoolExecutor, as_completed

with ThreadPoolExecutor(max_workers=4) as executor:
    futures = {
        executor.submit(process_tool_call, block.name, block.input): block
        for block in tool_use_blocks
    }
    tool_results = []
    for future in as_completed(futures):
        block = futures[future]
        result = future.result()
        tool_results.append({
            "type": "tool_result",
            "tool_use_id": block.id,
            "content": result,
        })

Sandbox-verified multi-tool results:

{"type": "tool_result", "tool_use_id": "tool_1", "content": "25.0"}
{"type": "tool_result", "tool_use_id": "tool_2", "content": "{\"temp\": 18, \"condition\": \"Sunny\"}"}

I'd only apply parallel execution to idempotent read tools. External API calls with side effects need careful rate-limit and ordering consideration.

Error Handling — Failing Gracefully

When a tool fails, return is_error: true. The model reads this, recognizes the error, and either tries something else or gives the user contextual guidance.

def safe_process_tool_call(tool_name: str, tool_input: dict) -> tuple[str, bool]:
    """Tool execution with error handling. Returns (content, is_error)."""
    try:
        result = process_tool_call(tool_name, tool_input)
        return result, False
    except Exception as e:
        error_msg = f"Tool execution failed: {type(e).__name__}: {str(e)}"
        return error_msg, True

for block in response.content:
    if block.type == "tool_use":
        content, is_error = safe_process_tool_call(block.name, block.input)
        tool_result = {
            "type": "tool_result",
            "tool_use_id": block.id,
            "content": content,
        }
        if is_error:
            tool_result["is_error"] = True
        tool_results.append(tool_result)

When is_error: true is set, the model doesn't just skip past it. From my testing, it reads the error content and responds with something like "The file couldn't be found — please double-check the path." Returning empty strings or ignoring errors tends to produce confused or hallucinated responses.

The Real Cost of Tool Use — How Many Tokens Does It Add?

Honestly, Tool Use costs more. According to Anthropic's documentation, each tool definition adds roughly 200–300 tokens of overhead.

5 tool definitions → ~1,250 tokens fixed overhead (every request)
1 tool call → additional input + output tokens
3-turn agentic loop → accumulating context

The agentic loop accumulates context. After 5 turns, everything from the first message to the fifth tool result is in context. Costs can compound quickly in long-running agents.

Two ways to manage this:

1. Combine with Prompt Caching: Tool definitions are the same on every request. As covered in the Claude API Prompt Caching guide, caching the system prompt with cache_control: {"type": "ephemeral"} applies here too, and tool definitions benefit similarly from repeated identical structures.

2. Pass only the tools you need: Always including 10 tool definitions is worse than passing the 2–3 that matter for the current task. More tools consume more tokens and occasionally lead the model to pick the wrong one.

Streaming Tool Use

Tool Use works with streaming responses. In anthropic 0.101.0, use client.messages.stream:

with client.messages.stream(
    model="claude-opus-4-7",
    max_tokens=4096,
    tools=tools,
    messages=messages,
) as stream:
    # Stream text chunks in real time
    for text_chunk in stream.text_stream:
        print(text_chunk, end="", flush=True)

    # Get the final message after streaming completes
    final_message = stream.get_final_message()

if final_message.stop_reason == "tool_use":
    # ... same handling as above

When streaming with tool use: if you're showing text chunks to the user in real time and also need to process tool calls, design the UX flow before you start. The Vercel AI SDK approach is worth looking at to see how this gets abstracted on the frontend side.

Production Pattern: GitHub Issue Monitor Agent

A complete example tying everything together — a simple agent that fetches and summarizes GitHub issues:

import anthropic
import json
from typing import Any

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY

tools = [
    {
        "name": "list_github_issues",
        "description": "Fetches the issue list for a GitHub repository.",
        "input_schema": {
            "type": "object",
            "properties": {
                "repo": {"type": "string", "description": "owner/repo format"},
                "state": {"type": "string", "enum": ["open", "closed", "all"]},
                "limit": {"type": "integer", "description": "Max issues to return (default: 10)"}
            },
            "required": ["repo"]
        }
    },
    {
        "name": "get_issue_detail",
        "description": "Fetches the details of a specific GitHub issue.",
        "input_schema": {
            "type": "object",
            "properties": {
                "repo": {"type": "string", "description": "owner/repo format"},
                "issue_number": {"type": "integer", "description": "Issue number"}
            },
            "required": ["repo", "issue_number"]
        }
    }
]

def process_tool_call(tool_name: str, tool_input: dict[str, Any]) -> str:
    if tool_name == "list_github_issues":
        # Real impl: requests.get(f"https://api.github.com/repos/{repo}/issues", ...)
        return json.dumps([
            {"number": 42, "title": "TypeError in data processor", "state": "open"},
            {"number": 41, "title": "Add streaming support", "state": "open"},
        ])
    elif tool_name == "get_issue_detail":
        return json.dumps({
            "number": tool_input["issue_number"],
            "body": "Reproduce: pass an empty list as input. Stack trace attached.",
            "comments": 3
        })
    return "Unknown tool"

def run_issue_agent(query: str) -> str:
    messages = [{"role": "user", "content": query}]

    for _ in range(10):
        response = client.messages.create(
            model="claude-opus-4-7",
            max_tokens=4096,
            tools=tools,
            messages=messages,
        )

        if response.stop_reason == "end_turn":
            return next(
                (block.text for block in response.content if hasattr(block, "text")),
                "No response"
            )

        if response.stop_reason == "tool_use":
            messages.append({"role": "assistant", "content": response.content})
            tool_results = []
            for block in response.content:
                if block.type == "tool_use":
                    result = process_tool_call(block.name, block.input)
                    tool_results.append({
                        "type": "tool_result",
                        "tool_use_id": block.id,
                        "content": result,
                    })
            messages.append({"role": "user", "content": tool_results})

    return "Loop limit exceeded"

What's Still Unresolved — Honest Limitations

Here's what I find genuinely frustrating about Tool Use in practice.

Context accumulation: The agentic loop keeps growing the context. After 10 turns, everything from the first message to the tenth tool result is in there. Long-running agents need a context management strategy — summarize intermediate results, prune stale messages — and there's no standard pattern for this yet.

Non-deterministic tool selection: Same question, different tool selection on different runs. Even with temperature=0, you can't guarantee identical behavior across invocations. This makes testing harder than it should be.

Description quality is everything: Vague description → wrong tool selection or no tool use at all. Writing good tool descriptions is its own prompt engineering discipline. No framework solves this for you.

I think Tool Use is underappreciated. Agent frameworks offer impressive abstractions, but this pattern is what's running underneath all of them. PydanticAI's type-safe tool definitions are a convenient layer that auto-generates the JSON schema, but understanding the underlying mechanism is what gets you unstuck when things break.

Summary

Validated findings from anthropic 0.101.0:

Tool definitions: name + description + input_schema. Description quality determines whether the tool gets used correctly.
Agentic loop: Detect stop_reason == "tool_use" → execute tool → append tool_result → repeat. Simple pattern, but the message structure has to be exactly right.
Error handling: Use is_error: true so the model recognizes failures and responds appropriately. Never return empty strings.
Cost: ~250 tokens overhead per tool definition. Combine with Prompt Caching. Watch context accumulation in multi-turn agents.
Parallel tool calls: ThreadPoolExecutor works for idempotent read tools. Apply selectively.

Tool Use is the most direct path from chatbot to agent. You don't need a complex framework — this pattern alone is enough to build practical agents.

A-Mem: Agentic Memory for LLM Agents Explained

Jangwook Kim — Wed, 13 May 2026 04:18:18 +0000

Your agent forgets everything between sessions. You bolt on a vector database, retrieve the top-5 similar chunks at query time, and call it memory. It works — until the agent needs to reason across multiple related memories it cannot connect on the fly, or until a new fact should change how it interprets older ones.

That is the problem A-Mem (Agentic Memory for LLM Agents, arXiv:2502.12110) was built to solve. Accepted at NeurIPS 2025, A-Mem introduces a memory system where the agent actively organizes, links, and evolves its memories on write — not just at retrieval time. The result is a system that handles multi-hop reasoning tasks at roughly six times the accuracy of standard vector retrieval baselines on the LoCoMo benchmark.

Effloow Lab inspected the paper, codebase (MIT license, GitHub: agiresearch/A-mem), and documented the architecture. This guide explains what A-Mem does differently and when it is worth reaching for.

Why Static Memory Systems Fall Short

Most agent memory setups follow the same pattern: embed a document or conversation turn, store it in a vector database, retrieve by cosine similarity at query time. The pattern is fast and simple, but it has three structural weaknesses.

Weak multi-hop reasoning. If memory A is about "Redis sorted sets" and memory B is about "leaderboard query optimization," a query about "how to build a fast leaderboard" may retrieve either memory but not both in the right relationship. The agent has to reconstruct the connection itself — often unreliably.

No retroactive updating. When you add a new memory that changes the interpretation of an older one, the old memory stays unchanged. The agent may retrieve the old, stale context and draw the wrong conclusion.

Fixed retrieval patterns. Standard RAG requires you to predefine how memories are accessed: top-k by similarity, keyword filter, or graph traversal. Each new task type may need a new access pattern that you have not engineered.

Graph-enhanced RAG systems (like MemGPT) address the third problem partially by adding explicit entity-relationship graphs, but they still rely on a predefined schema. A-Mem addresses all three by making memory organization an active, agentic process rather than a fixed retrieval mechanism. (For a practical foundation on building RAG pipelines before layering on agentic memory, see Build a RAG App with LlamaIndex.)

What A-Mem Is

A-Mem treats memory the way a thoughtful knowledge worker treats a Zettelkasten — a note-taking methodology where every note is a structured unit linked to related notes. Rather than storing raw text and embedding it once, A-Mem constructs a rich note for each memory, analyzes its relationship to existing memories, creates explicit links, and can update existing notes when new knowledge changes the picture.

The project is open-source under the MIT license and was accepted at NeurIPS 2025. The primary repositories are:

Official: github.com/agiresearch/A-mem
Paper author mirror: github.com/WujiangXu/A-mem
Community MCP server extension: github.com/tobs-code/a-mem-mcp-server

Core Architecture: Three Operations

A-Mem's architecture centers on three operations that run every time a new memory is added.

1. Note Construction

When a new piece of information enters the system — a conversation turn, a tool result, an observation — A-Mem does not just embed and store it. It generates a structured note containing:

Contextual description: a short LLM-generated summary that captures the meaning, not just the surface text
Keywords and tags: structured labels for categorical retrieval
Embedding vector: stored in ChromaDB for similarity search

This enrichment step is the first departure from vanilla RAG. The embedding is of a richer, LLM-synthesized representation rather than raw text.

2. Link Generation

After note construction, A-Mem scans the existing memory store for related notes. When meaningful semantic overlap exists — shared keywords, similar contextual descriptions, or high embedding similarity — it creates an explicit directed link between the notes. These links are stored in a NetworkX graph alongside the ChromaDB vector store.

The combination of ChromaDB (vector similarity) and NetworkX (graph traversal) means the system can answer both "what is similar to this?" (ChromaDB) and "what is connected to this?" (graph walk) without choosing one or the other.

3. Memory Evolution

This is A-Mem's most distinctive operation. When a new memory is integrated, the system checks whether any existing linked memories should be updated. If the new information changes or deepens the context of an older note, the older note's contextual description is rewritten to reflect the new understanding.

Consider an agent that first learns "the team uses Redis for session storage" and later learns "the team is migrating from Redis to Valkey for cost reasons." With vanilla RAG, both facts sit independently. With A-Mem, the second memory triggers an evolution of the first: its contextual description is updated to reflect that this is an in-progress migration, not a stable architecture decision.

This makes A-Mem's memory graph a living structure — not an append-only log.

Storage Backend

The implementation combines two storage layers:

Layer	Technology	Purpose
Vector store	ChromaDB	Fast approximate similarity search on enriched embeddings
Graph store	NetworkX	Explicit inter-memory links for multi-hop traversal
LLM backend	OpenAI / other	Note enrichment, link scoring, evolution reasoning

ChromaDB handles retrieval when you query by concept similarity. NetworkX handles traversal when the agent needs to follow a chain of related memories. The LLM backend drives the intelligent parts: note enrichment, deciding which links to create, and whether evolution should happen.

Benchmark Results on LoCoMo

A-Mem's paper evaluates on the LoCoMo (Long Conversational Memory) benchmark, a dataset of long-form conversations designed to test multi-session memory recall. The multi-hop category is most revealing — these are questions that require reasoning across two or more distinct stored memories.

System	Multi-Hop ROUGE-L	Temporal Reasoning F1
LoCoMo baseline	4.68	—
ReadAgent	2.81	—
MemGPT (GPT-4o-mini)	—	25.52
A-Mem (Qwen2.5-15b)	27.23	—
A-Mem (GPT-4o-mini)	—	45.85

The multi-hop ROUGE-L improvement with Qwen2.5-15b is roughly 5.8x over the LoCoMo baseline (27.23 vs 4.68). On temporal reasoning tasks with GPT-4o-mini, A-Mem reaches 45.85 F1 against MemGPT's 25.52 — nearly double. These gains are structural, not prompt tricks: they come from having precomputed the links between related memories at write time, so the agent does not need to reconstruct connections at query time under token pressure.

A-Mem's multi-hop advantage is more pronounced than its gains on simpler single-fact retrieval. Open Domain tasks — where the question maps to a single stored fact — show improvements too, but smaller. This tells you something important about when to use A-Mem: it earns its complexity for tasks that require chaining related facts, not for simple key-value lookups.

How to Use A-Mem

The project is installed from source. The core API is straightforward once the dependencies are in place.

Installation:

git clone https://github.com/agiresearch/A-mem
cd A-mem
python -m venv .venv && source .venv/bin/activate
pip install .

Dependencies include chromadb, networkx, and an LLM backend (OpenAI by default, but the backend is configurable).

Initializing the memory system:

from memory import AgenticMemorySystem

memory = AgenticMemorySystem(
    model_name='all-MiniLM-L6-v2',   # Embedding model (SentenceTransformers)
    llm_backend="openai",
    llm_model="gpt-4o-mini"           # Used for note enrichment + evolution
)

The model_name controls the embedding model. all-MiniLM-L6-v2 is a compact, fast option. For higher quality embeddings, substitute a larger SentenceTransformers model.

Adding a memory:

# Simple content
memory_id = memory.add_note("Learned that batch size of 16 reduces GPU OOM errors on A100s.")

# With metadata
memory_id = memory.add_note(
    content="Redis sorted sets are efficient for leaderboard queries.",
    tags=["redis", "database"],
    category="Engineering",
    timestamp="202503021500"
)

Every add_note call triggers the full Note Construction → Link Generation → Memory Evolution pipeline. The call blocks while the LLM enriches the note and evaluates links, so latency is higher than a plain vector insert. This is the write cost you pay for smarter retrieval.

Retrieving memories:

results = memory.search("database performance optimization")

The search returns notes ordered by relevance, now including notes that are linked to the top matches — so a query about "database performance" can surface both the Redis sorted sets note and a linked note about index strategy, even if the latter does not match the query embedding closely on its own.

A-Mem vs. Other Memory Systems

Feature	Vanilla RAG	MemGPT	Mem0	A-Mem
Storage type	Vector only	Vector + graph (schema)	Fact extraction	Vector + graph (dynamic)
Write-time enrichment	No	Partial	Yes (facts)	Yes (full note + links)
Memory evolution	No	No	No	Yes
Multi-hop reasoning	Weak	Moderate	Weak	Strong
Write latency	Low	Medium	Medium	High (LLM call per write)
Schema flexibility	None needed	Predefined	Fact-based	Fully flexible
Best for	Static corpora	Structured entities	Fact-heavy chat	Multi-session reasoning

Mem0 (which uses a fact extraction pattern and scores 66.9% on LOCOMO) is a reasonable middle ground for production: lower write latency than A-Mem, better multi-hop than vanilla RAG. A-Mem wins on the hardest multi-hop tasks but at a real cost: every write requires an LLM call for enrichment and link evaluation.

Common Mistakes

Using A-Mem for simple key-value lookups. If your agent stores "user prefers dark mode" and retrieves it verbatim, a plain vector store is faster and sufficient. A-Mem's overhead is only justified when you need cross-memory reasoning.

Ignoring write latency in production. The note enrichment LLM call is synchronous in the base implementation. For high-throughput applications, this needs to be moved to an async queue. The community MCP server (tobs-code/a-mem-mcp-server) is one starting point for integration patterns.

Choosing the wrong embedding model. all-MiniLM-L6-v2 is fast but loses nuance for specialized domains (code, legal text, medical). For domain-specific agents, use a domain-adapted embedding model.

Not monitoring memory graph growth. As the note graph grows, link evaluation cost scales. For agents running thousands of sessions, you need a graph pruning strategy. The paper does not fully address this; it is an open implementation concern.

Expecting zero-shot plugin behavior. A-Mem requires a different design philosophy than RAG. You need to think in terms of notes and links, not documents and embeddings. Teams that treat it as a drop-in RAG replacement will not see the multi-hop gains.

Frequently Asked Questions

Q: How does A-Mem compare to MemMachine?

MemMachine (see Effloow's MemMachine guide) focuses on ground-truth-preserving memory: it ensures memories are never silently corrupted or overwritten without provenance. A-Mem focuses on dynamic organization and cross-memory evolution. They address different failure modes — A-Mem solves the multi-hop reasoning gap, MemMachine solves the reliability gap. The two approaches are complementary rather than competing.

Q: Is A-Mem ready for production use?

A-Mem is an MIT-licensed research implementation, not a managed service. The GitHub codebase is functional and documented, but it has not been stress-tested at enterprise scale. For production use, you would need to wrap it in an async worker queue, add monitoring, and handle ChromaDB persistence and backup. Teams who want the architecture without the ops overhead should watch for managed implementations.

Q: How does A-Mem compare to Mem0 for agent memory?

Mem0 uses a fact-extraction approach: it identifies discrete facts from conversations and stores them as atomic units. This is efficient and production-friendly, scoring 66.9% on LOCOMO. A-Mem builds richer structured notes and evolves them — winning on multi-hop tasks but with higher write cost. If your agent needs to chain across multiple related memories, A-Mem has a structural advantage. For simpler recall, Mem0's lower latency is more practical.

Q: Does A-Mem work with local LLMs?

The llm_backend parameter is configurable. The codebase supports OpenAI out of the box and can be adapted to other backends. For local LLMs (Ollama, vLLM, LM Studio), you would configure an OpenAI-compatible endpoint. Note enrichment quality depends on the LLM: a stronger model produces better contextual descriptions and more accurate link decisions.

Q: What is the LoCoMo benchmark?

LoCoMo (Long Conversational Memory) is a dataset of long-form multi-session conversations designed to test whether memory systems can recall facts and relationships across extended interactions. The multi-hop subset specifically tests questions that require connecting two or more stored facts. It is the primary benchmark used in the A-Mem paper.

Q: What is memory evolution and when does it trigger?

Memory evolution is the process by which A-Mem updates the contextual description of an existing note when a new, related note is added. It triggers when the system determines — via LLM evaluation — that the new memory meaningfully changes the interpretation of an existing linked memory. In practice, this is most useful in long-running agents where knowledge compounds over time.

Key Takeaways

A-Mem (NeurIPS 2025, arXiv:2502.12110) builds structured, evolving memory graphs for LLM agents using Zettelkasten-inspired note construction.
The three core operations — Note Construction, Link Generation, Memory Evolution — happen at write time, not retrieval time.
On the LoCoMo benchmark multi-hop tasks, A-Mem achieves roughly 5.8x better ROUGE-L than the standard vector baseline with GPT-4o-mini.
Storage uses ChromaDB for vector similarity and NetworkX for graph traversal, giving both similarity search and relationship-aware retrieval.
The write latency cost (LLM call per memory) is real: A-Mem is not a drop-in replacement for RAG. It is a deliberate upgrade for agents where multi-session, multi-hop reasoning quality matters.
The codebase is MIT-licensed on GitHub and installable from source.

Bottom Line

A-Mem solves the multi-hop memory problem that vanilla RAG cannot — by making memory organization agentic at write time rather than patchwork at query time. If your agent needs to reason across sessions and chain related facts reliably, the architecture is worth the added write latency. For simpler recall tasks, stick with Mem0 or a plain vector store.

Cloudflare Project Think: Durable Agent Runtime Guide

Jangwook Kim — Wed, 13 May 2026 01:13:13 +0000

Most AI agents on serverless platforms share the same fatal flaw: they can't survive a restart. If the underlying worker crashes or cold-starts mid-task, the agent's progress disappears. The typical workaround is to keep tasks short and stateless — which means you cannot run a 10-minute research loop, a multi-file refactor, or an autonomous investigation that makes 50 external calls.

Cloudflare's Project Think, announced during Agents Week 2026 (April 2026), is a direct answer to that constraint. It ships a set of primitives — fiber checkpointing, sub-agents, a persistent Session API, and a 5-tier execution ladder — all wired into an opinionated base class (@cloudflare/think) that runs on Durable Objects.

Effloow Lab inspected the SDK packages, confirmed installability, and traced the API surface from official docs and the open-source cloudflare/agents repository. The following is a source-based guide to how Project Think works and when to use it. See data/lab-runs/cloudflare-project-think-durable-agent-runtime-2026.md for the full evidence note.

Why Serverless Agents Break — and Why Project Think Fixes It

A standard Cloudflare Worker is a request handler: it starts, does work, returns a response, and dies. Cloudflare Workflows added durable multi-step execution, but the state machine is managed outside your code and requires a separate infrastructure primitive.

Project Think takes a different approach. Each agent runs inside a Durable Object — a stateful micro-server with its own SQLite database, WebSocket connections, and scheduling. That alone gives agents persistence. But Project Think goes further by introducing fibers: durable invocations that can checkpoint their own instruction pointer directly into the co-located SQLite database.

The practical result: an agent can run a 30-step task, checkpoint after each step, survive a server restart, and resume exactly where it left off — without any external workflow orchestrator.

This is the critical architectural distinction from Cloudflare Dynamic Workers (covered in an earlier Effloow article on Dynamic Workers), which handle sandboxed code execution but are stateless by design. Project Think layers durable execution on top of the full Cloudflare platform stack.

The Five Primitives of Project Think

1. Fibers — Checkpointed Execution

The fiber is the foundational primitive. Unlike a regular async function, a fiber can call ctx.stash() to serialize the current state of its local variables into SQLite. If the Durable Object restarts, runFiber rehydrates from the last stash point.

import { runFiber } from "@cloudflare/think";

export class ResearchAgent extends Think<Env, unknown> {
  async onTask(query: string) {
    return runFiber(this.ctx, async (ctx) => {
      const sources = await searchWeb(query);
      await ctx.stash({ sources });         // checkpoint 1

      const summaries = await summarize(sources);
      await ctx.stash({ sources, summaries }); // checkpoint 2

      return synthesize(summaries);
    });
  }
}

Each ctx.stash() call writes to the Durable Object's SQLite database. On resume, the fiber fast-forwards to the last stash point. For long-horizon tasks — multi-file code reviews, iterative search loops, automated report generation — this removes the "start over" failure mode entirely.

Fibers also include automatic keepalive for long-running operations and handle non-deterministic workloads that would time out in a standard Worker context.

2. Sub-Agents (Facets) — Isolated Child Agents with Typed RPC

Project Think supports spawning child agents as facets — child Durable Objects colocated with the parent on the same machine. Each facet has:

Its own isolated SQLite database (no shared state)
Its own execution context and fiber support
A typed RPC stub returned to the parent for method calls

// Parent agent spawning a specialist sub-agent
const extractor = await this.spawnFacet("data-extractor", DataExtractorAgent);
const structured = await extractor.parseDocument(rawText);

const validator = await this.spawnFacet("validator", ValidationAgent);
const result = await validator.check(structured);

This pattern is more predictable than passing messages through a shared queue. Because the facet RPC is typed, TypeScript catches mismatches at compile time. And because facets are colocated, the latency for inter-agent calls is dramatically lower than network-based agent-to-agent communication.

Facets are useful when you need to decompose a task into specialist roles — a researcher, a writer, a fact-checker — without those roles sharing any mutable state.

3. The Session API — Relational Conversation Trees

Standard chat agent implementations append messages to a flat array. That works for simple Q&A but breaks when you need to explore alternatives without polluting the main reasoning path.

Project Think's Session API stores messages as a relational tree, with each message carrying a parent_id. This enables three capabilities that flat-list approaches cannot support:

Forking: The agent can branch off a conversation node to explore an alternative without modifying the main path. If the alternative fails, the original path is untouched.

Non-destructive compaction: Rather than truncating context when the window fills, the Session API creates a compaction overlay — a summary that sits beside the original messages without replacing them. The full history is still queryable.

Full-text search: FTS5 indexing over all stored messages lets the agent retrieve relevant earlier context without re-reading the entire history into the LLM context window.

export class LongHorizonAgent extends Think<Env, unknown> {
  configureSession() {
    return {
      systemPrompt: "You are a thorough technical researcher.",
      contextBlocks: [
        { type: "text", content: this.env.DOMAIN_KNOWLEDGE }
      ]
    };
  }
}

All session storage runs on the Durable Object's local SQLite — no external vector database required for the conversation layer.

4. The Execution Ladder — Graduated Code Trust

One of Project Think's most distinctive ideas is the execution ladder: a tiered system of code execution environments that agents escalate through based on the trust level required by a task.

Tier	Name	Package / API	Capability	Trust Level
0	Workspace	`@cloudflare/shell`	Durable filesystem (SQLite + R2)	Fully trusted
1	Dynamic Worker	`@cloudflare/codemode`	Sandboxed V8 isolate, no network	LLM-generated code
2	npm	`@cloudflare/worker-bundler`	Fetch npm pkgs, esbuild, load into DW	Third-party packages
3	Browser	Cloudflare Browser Run	Navigate, click, extract	Web content
4	Sandbox	`cloudflare/sandbox-sdk`	Full Linux env, git, cargo, npm test	Untrusted workloads

Agents do not jump directly to Tier 4 for every task. A simple data transformation can run in Tier 1 (a sandboxed V8 isolate that starts in milliseconds). A task requiring npm packages escalates to Tier 2. A task that needs to test a full Rust codebase goes to Tier 4.

The ladder enforces the principle of least privilege: agents operate at the lowest tier that can handle the task, escalating only when needed. This keeps the security surface small and execution fast for common cases.

5. Self-Authored Extensions — Agents Writing Their Own Tools

The final primitive is the most experimental: agents can write their own tools at runtime. An agent inspects a task, decides it needs a capability it doesn't have, generates a tool implementation, and loads it into a Dynamic Worker for execution — all within the same session.

This is not the same as calling an external tool-use API. The agent generates actual TypeScript code, bundles it with @cloudflare/worker-bundler, and executes it in a Tier 1 or Tier 2 environment. The generated tool becomes part of the agent's toolkit for the duration of the session.

In practice, this is useful for tasks where the required transformation or extraction logic cannot be fully specified in advance — for example, parsing a novel API response format or implementing a domain-specific calculation that varies per client.

The `@cloudflare/think` Base Class

All five primitives are exposed through the Think base class, which handles the full chat lifecycle: agentic loop, message persistence, streaming, tool execution, stream resumption, and extensions.

Installation:

npm install @cloudflare/think agents ai @cloudflare/shell zod workers-ai-provider

Minimal example:

import { Think } from "@cloudflare/think";
import { createWorkersAI } from "workers-ai-provider";

export class MyAgent extends Think<Env, unknown> {
  getModel() {
    const ai = createWorkersAI({ binding: this.env.AI });
    // Workers AI free tier includes @cf/meta/llama-3.3-70b-instruct
    return ai("@cf/meta/llama-3.3-70b-instruct");
  }

  configureSession() {
    return {
      systemPrompt: "You are a helpful assistant.",
    };
  }
}

The wrangler.toml binding wires the Durable Object:

[[durable_objects.bindings]]
name = "MY_AGENT"
class_name = "MyAgent"

[[migrations]]
tag = "v1"
new_sqlite_classes = ["MyAgent"]

The cloudflare/agents GitHub repository contains 30+ self-contained example agents demonstrating fibers, facets, sessions, and execution ladder integration. The docs/think/index.md file in that repository is the most complete reference beyond the official documentation.

Project Think vs. Dynamic Workers vs. Cloudflare Workflows

Developers familiar with Cloudflare's existing primitives will have one question: how does this fit alongside Dynamic Workers and Workflows?

Dynamic Workers (covered in Effloow's Dynamic Workers guide) are stateless sandboxed V8 isolates for executing LLM-generated code. They correspond to Tier 1 of Project Think's execution ladder. They are not durable.

Cloudflare Workflows provide durable multi-step execution, but the state machine lives outside your Worker. Steps are defined declaratively, and Cloudflare's infrastructure manages replay. This is powerful for ETL pipelines and scheduled jobs, but the agent has no access to its own state between steps.

Project Think puts the state machine inside the agent itself via fibers and the co-located SQLite database. The agent is both the executor and the state store. This gives more flexibility for agentic patterns where the next step depends on reasoning about the previous step's output — not just a declared execution graph.

The right choice depends on your workload:

Stateless code execution only → Dynamic Workers
Declarative multi-step pipeline with retry guarantees → Cloudflare Workflows
Autonomous agents with reasoning-driven state transitions → Project Think

Common Mistakes When Building Durable Agents

Checkpoint too infrequently. If you only call ctx.stash() at the end of a multi-minute operation, a crash at minute 8 means re-running 8 minutes of work. Checkpoint after each meaningful unit — after a web request, after a parsing step, after a tool call returns.

Share state through the parent's SQLite instead of facet isolation. Facets exist precisely so specialist sub-agents do not see each other's state. Routing everything through the parent's database re-introduces the coupling you were trying to avoid.

Escalate to Tier 4 for every code execution task. Cloudflare Sandbox (Tier 4) has more overhead than Dynamic Workers (Tier 1). Use Tier 4 only when the task genuinely needs a Linux environment — git operations, compiled languages, or full test runners.

Ignore compaction until the context window overflows. Plan compaction as a regular scheduled step, not an emergency measure. The Session API's non-destructive overlay lets you compact early and often without losing history.

Treat @cloudflare/think as production-stable. As of May 2026, Project Think is in experimental preview. The package version is 0.0.1-experimental.x. The API surface is intended to be stable, but Cloudflare explicitly says it will continue to evolve. Treat it as early-adopter infrastructure.

Practical Application: When to Choose Project Think

Project Think is well-suited to agent workloads that:

Exceed Cloudflare Worker's standard CPU time limits
Require specialist sub-tasks that should not share state
Need to explore multiple reasoning paths without forking the entire agent
Generate and execute code as part of their reasoning loop
Must maintain conversation history across days or weeks for personalization

It is less well-suited to:

Simple request/response pipelines (standard Worker is simpler)
Batch jobs without agent reasoning (Cloudflare Workflows is more appropriate)
Workloads requiring GPUs or dedicated compute (no GPU support on Workers)

FAQ

Q: Does Project Think work with any LLM or only Workers AI?

Project Think's Think base class is model-agnostic — getModel() can return any model compatible with the Vercel AI SDK's provider interface. Workers AI (workers-ai-provider) is the zero-egress option for Cloudflare-hosted models, but you can wire in OpenAI, Anthropic, or any other provider via the AI SDK.

Q: What's the cost of fiber checkpointing?

Each ctx.stash() writes to the Durable Object's SQLite database — a local write, not a network call. The overhead is the same as any SQLite write on the same machine. Cloudflare does not charge extra for SQLite writes beyond the standard Durable Object storage pricing. For most agents, checkpointing 10–50 times per session adds negligible cost.

Q: Can sub-agents (facets) span multiple geographic regions?

Facets are colocated with the parent Durable Object on the same machine by design — this is what makes their typed RPC low-latency. They do not span regions. If you need geographically distributed agent coordination, that requires a different architecture (message queues or service bindings across Workers).

Q: Is Project Think production-ready in May 2026?

No. It is in experimental preview. Cloudflare describes the API surface as stable but explicitly notes it will evolve. For production workloads, monitor the cloudflare/agents GitHub repository and the Cloudflare changelog for GA announcements.

Q: How does the Session API relate to a vector database?

The Session API is not a semantic search layer — it is a relational message store with FTS5 full-text search. It handles conversation history, forking, and compaction well. For semantic retrieval over large external knowledge bases, you still need a vector database (Cloudflare Vectorize, Pinecone, etc.). They are complementary, not alternatives.

Key Takeaways

Project Think solves the fundamental durability problem in serverless AI agents: agents can now checkpoint progress and survive restarts without re-running from the beginning.
The five core primitives — fibers, sub-agents (facets), the Session API, the execution ladder, and self-authored extensions — address distinct failure modes in long-horizon agentic workloads.
@cloudflare/think is the opinionated base class that wires all primitives together; it is model-agnostic and works with any Vercel AI SDK provider.
The 5-tier execution ladder enforces least-privilege code execution, keeping fast tasks in lightweight V8 isolates and escalating to full Linux environments only when necessary.
As of May 2026, Project Think is in experimental preview. The API is intended to be stable but will continue to evolve — suitable for early adoption and evaluation, not yet for production-critical deployments.

Bottom Line

Project Think is the most complete answer Cloudflare has given to "how do I run an AI agent that lasts longer than a serverless function?" The fiber + facet + session combination solves real architectural problems, not theoretical ones. Get familiar with it now — when it reaches GA, it will become the default pattern for serious agent infrastructure on the Workers platform.

Building a Python MCP Server in 30 Minutes with FastMCP 3.x — One @tool Decorator Is All You Need

Jangwook Kim — Tue, 12 May 2026 09:42:26 +0000

Building an MCP (Model Context Protocol) server from scratch is more work than it looks. stdio transport handling, JSON-RPC 2.0 serialization, handler registration — if you've gone through implementing an MCP server with Streamable HTTP, you know the moment where you think: "I just want to add one AI tool, why does this need so much boilerplate?"

FastMCP exists to fix that. Today, I installed it in a sandbox via pip and had a working MCP server running in under 30 minutes. Here's what I found.

What FastMCP Actually Is

FastMCP is a high-level layer on top of the MCP Python SDK — similar to how Express.js wraps Node's http module. The official tagline: "The fast, Pythonic way to build MCP servers and clients." After hands-on testing, I'd say that's accurate.

Version check first:

$ fastmcp version

FastMCP version:   3.2.4
MCP version:       1.27.0
Python version:    3.12.8
Platform:          macOS-15.6-arm64

My backlog had this noted as "v2.0," but it's already at 3.x. The MCP protocol itself is at 1.27.0. This version gap means one thing: the API has changed, and docs don't always reflect that. I had to verify things directly by running code rather than trusting older articles.

Install and First Server — This Really Is All of It

pip install fastmcp

Installation takes about ten seconds. Here's the first server I built in the sandbox — two weather-related tools:

from fastmcp import FastMCP
from datetime import datetime

mcp = FastMCP("weather-tools", version="1.0.0")

@mcp.tool()
def get_current_time(timezone: str = "UTC") -> str:
    """Returns the current time."""
    return f"Current time ({timezone}): {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}"

@mcp.tool()
def calculate_temp(celsius: float) -> dict:
    """Converts Celsius to Fahrenheit and Kelvin."""
    return {
        "celsius": celsius,
        "fahrenheit": round(celsius * 9/5 + 32, 2),
        "kelvin": round(celsius + 273.15, 2)
    }

@mcp.resource("data://server-info")
def server_info() -> str:
    """Returns server info."""
    return "FastMCP 3.x weather server"

@mcp.prompt()
def weather_analysis(location: str) -> str:
    """Weather analysis prompt template."""
    return f"Analyze the weather in {location} and recommend appropriate clothing."

if __name__ == "__main__":
    mcp.run()  # runs in stdio mode

That's it. Add a decorator to a Python function and it becomes an MCP tool. Type hints are automatically converted to JSON Schema and passed to Claude.

Inspect the server with the CLI:

$ fastmcp inspect server.py

Server
  Name:         weather-tools
  Version:      1.0.0
  Generation:   2

Components
  Tools:        2
  Prompts:      1
  Resources:    1
  Templates:    0

Three Building Blocks: Tool, Resource, Prompt

FastMCP has three core abstractions. Getting these right is what makes a well-designed server.

@mcp.tool() — A function Claude can directly invoke. It takes parameters, does work, and returns results. Search, compute, file operations, API calls — anything with execution behavior goes here. If I want Claude to interact with my filesystem or an external API, @mcp.tool() is the answer.

@mcp.resource() — A read-only data source. Register it with a URI like data://, file://, or https://, and Claude reads it as context. Unlike tools, this is "read" not "execute." Database schemas, config files, documentation — put these here and they flow into Claude's context window.

@mcp.prompt() — A reusable prompt template. Takes parameters, returns a structured prompt message. Works like a slash command in Claude Desktop or claude.ai.

The Tool vs Resource distinction trips people up. My rule: if it has side effects, it's a Tool; if it's read-only, it's a Resource.

Sending Progress Updates with Context

When a tool runs a long operation, you can stream progress back to the client in real time. Add a Context parameter and FastMCP injects it automatically.

from fastmcp import FastMCP, Context

mcp = FastMCP("dev-tools")

@mcp.tool()
async def list_files(directory: str, ctx: Context) -> list[str]:
    """Returns a list of files in the specified directory."""
    import os
    await ctx.info(f"Reading directory: {directory}")  # streams log to client

    try:
        files = os.listdir(directory)
        await ctx.report_progress(100, 100, "complete")
        return sorted(files)
    except FileNotFoundError:
        raise ValueError(f"Directory not found: {directory}")

I ran this in the sandbox and confirmed that ctx.info() actually streams to the client side:

INFO  Received INFO from server: {'msg': 'Reading directory: /tmp', 'extra': None}

When this works inside Claude Desktop, users see real-time feedback about what the tool is doing. It's a meaningful UX improvement for long-running operations.

Testing with FastMCP Client

You don't need an actual Claude Desktop to test. FastMCP provides an in-process client. This is also handy when implementing agentic workflow patterns — tests stay self-contained.

import asyncio
from fastmcp import FastMCP
from fastmcp.client import Client

mcp = FastMCP("dev-tools")

@mcp.tool()
def search_text(text: str, pattern: str) -> dict:
    """Searches for a pattern in text."""
    import re
    matches = re.findall(pattern, text)
    return {"pattern": pattern, "matches": matches, "count": len(matches)}

@mcp.tool()
def word_count(text: str) -> dict:
    """Returns word count, character count, and line count."""
    words = text.split()
    return {
        "words": len(words),
        "characters": len(text),
        "lines": len(text.splitlines())
    }

async def test():
    async with Client(mcp) as client:
        tools = await client.list_tools()
        print(f"Registered tools ({len(tools)}):")
        for t in tools:
            print(f"  [{t.name}] {t.description}")

        result = await client.call_tool("search_text", {
            "text": "FastMCP is fast. FastMCP is easy.",
            "pattern": "FastMCP"
        })
        print(f"\nsearch_text result: {result.data}")
        # → {'pattern': 'FastMCP', 'matches': ['FastMCP', 'FastMCP'], 'count': 2}

        result2 = await client.call_tool("word_count", {
            "text": "Hello World from FastMCP 3.x"
        })
        print(f"word_count result: {result2.data}")
        # → {'words': 5, 'characters': 27, 'lines': 1}

asyncio.run(test())

Access the structured return value directly through result.data. Ran this in the sandbox — zero errors.

HTTP Deployment for Remote Access

Beyond local stdio mode, you can run the server over HTTP. Useful when sharing an MCP server across Cursor instances or deploying remotely.

# HTTP mode (default port 8000)
if __name__ == "__main__":
    mcp.run(transport="http", host="0.0.0.0", port=8000)

# Or run directly with uvicorn
uvicorn server:mcp.http_app() --host 0.0.0.0 --port 8000

The FastMCP HTTP app is Starlette-based (StarletteWithLifespan under the hood). That means you can mount it inside a FastAPI app:

from fastapi import FastAPI
from fastmcp import FastMCP

app = FastAPI()
mcp = FastMCP("my-tools")

@mcp.tool()
def my_tool() -> str:
    return "result"

app.mount("/mcp", mcp.http_app())

Connecting Claude Desktop to the HTTP server:

{
  "mcpServers": {
    "my-tools": {
      "url": "http://localhost:8000/mcp/"
    }
  }
}

The fastmcp CLI

FastMCP ships with a CLI that I didn't notice at first. Running fastmcp --help reveals quite a bit:

Commands:
  inspect      — Print server component summary
  list         — List registered tools
  call         — Directly call a tool (useful for debugging)
  install      — Auto-register to Claude Desktop / Cursor
  dev          — Run dev server with hot reload
  discover     — Find MCP servers configured in editors
  run          — Start the server

fastmcp install server.py --client claude is supposed to automatically patch your Claude Desktop config. No more hand-editing JSON. I couldn't verify this directly since I don't have Claude Desktop installed in my sandbox environment — check the official docs for exactly which config path it touches.

The fastmcp dev command seems more immediately useful: hot reload during development means no manual server restarts as you iterate.

Type Hints Are Your API Schema

The feature I found most impressive: type hints become JSON Schema automatically. With the raw SDK, you write an inputSchema dict for every tool by hand. FastMCP delegates that to Python's type system.

from typing import Literal
from pydantic import BaseModel

class FileFilter(BaseModel):
    extension: str
    min_size_kb: int = 0
    exclude_hidden: bool = True

@mcp.tool()
def list_files_advanced(
    directory: str,
    filter: FileFilter | None = None,
    sort_by: Literal["name", "size", "modified"] = "name",
    limit: int = 50
) -> list[dict]:
    """Returns filtered and sorted file listing."""
    import os
    files = []
    for f in os.scandir(directory):
        if filter and filter.exclude_hidden and f.name.startswith("."):
            continue
        if filter and not f.name.endswith(f".{filter.extension}"):
            continue
        info = f.stat()
        size_kb = info.st_size / 1024
        if filter and size_kb < filter.min_size_kb:
            continue
        files.append({"name": f.name, "size_kb": round(size_kb, 2), "modified": info.st_mtime})
    key_map = {"name": "name", "size": "size_kb", "modified": "modified"}
    files.sort(key=lambda x: x[key_map[sort_by]])
    return files[:limit]

Register this with @mcp.tool() and Claude automatically knows the structure of FileFilter, the valid values for sort_by (name/size/modified), and limit's default. Pydantic models work too, so complex nested inputs don't need any extra wiring.

The docstring becomes the tool description Claude sees. A well-written docstring is the usage manual you send to the model.

A Real-World Example: Code Analysis MCP Server

Here's something I'd actually ship — a Python code analysis tool server:

from fastmcp import FastMCP, Context
import ast
import os

mcp = FastMCP("code-analyzer", version="1.0.0")

@mcp.tool()
async def analyze_python_file(filepath: str, ctx: Context) -> dict:
    """Analyzes a Python file with AST and returns functions and classes."""
    await ctx.info(f"Analyzing: {filepath}")
    if not os.path.exists(filepath):
        raise ValueError(f"File not found: {filepath}")
    with open(filepath, "r", encoding="utf-8") as f:
        source = f.read()
    tree = ast.parse(source)
    functions, classes = [], []
    for node in ast.walk(tree):
        if isinstance(node, ast.FunctionDef):
            functions.append({
                "name": node.name, "line": node.lineno,
                "args": [a.arg for a in node.args.args],
                "docstring": ast.get_docstring(node)
            })
        elif isinstance(node, ast.ClassDef):
            classes.append({"name": node.name, "line": node.lineno})
    await ctx.report_progress(100, 100, "complete")
    return {"total_lines": source.count("\n") + 1, "functions": functions, "classes": classes}

@mcp.tool()
def count_todo_comments(filepath: str) -> dict:
    """Finds TODO/FIXME/HACK comments in a file."""
    markers = ["TODO", "FIXME", "HACK", "XXX"]
    results = {m: [] for m in markers}
    with open(filepath, "r", encoding="utf-8") as f:
        for i, line in enumerate(f, 1):
            for marker in markers:
                if f"# {marker}" in line:
                    results[marker].append({"line": i, "text": line.strip()})
    return {k: v for k, v in results.items() if v}

@mcp.resource("data://project-structure")
def project_structure() -> str:
    """Returns Python file list in the current directory."""
    py_files = []
    for root, dirs, files in os.walk("."):
        dirs[:] = [d for d in dirs if not d.startswith(".")]
        for f in files:
            if f.endswith(".py"):
                py_files.append(os.path.join(root, f))
    return "\n".join(py_files[:50])

if __name__ == "__main__":
    mcp.run()

Connect this to Claude Desktop and you can ask in plain English: "Show me all classes in this file" or "How many TODO comments are there?" No Python required from the user's side. That's the point of an MCP tool server.

FastMCP vs Raw MCP SDK

Compare with building a Streamable HTTP MCP server directly:

Raw SDK approach:

Create a Server instance
Register @server.list_tools() and @server.call_tool() separately
Manually parse input parameters
Combine anyio.run() + stdio_server() to run

FastMCP approach:

One FastMCP instance
@mcp.tool() registers functions directly as tools
JSON Schema auto-generated from type hints
mcp.run() — one line

Fewer lines is secondary. The real point: you focus on business logic, not transport mechanics.

That said, FastMCP trades off control for convenience. If you need to customize low-level MCP messages, use non-standard transports, or access MCP features FastMCP hasn't exposed, you'll end up digging under the abstraction. In those cases, reach for the MCP Python SDK directly — like in MCP code execution scenarios that need finer control.

When to Use FastMCP

My practical take:

Use FastMCP when: You're building a server for standard MCP clients (Claude, Cursor, VS Code). Especially for rapid AI tool prototyping, or exposing existing Python functions as MCP tools for your team.

Use the raw SDK when: You need custom transport, non-standard message formats, or MCP features FastMCP hasn't wrapped. Performance-critical paths where every layer matters.

One honest complaint about FastMCP: 3.x moved faster than the docs. I found get_tools() referenced in older content but it doesn't exist — list_tools() is the actual method. Trust dir(mcp) and the source code over older blog posts. Including mine.

Before going to production, also look at MCP Gateway for controlling which tools agents can call. Once you've exposed a server, you'll want some control over what actually gets invoked and when.

Summary

FastMCP 3.x is the fastest path for a Python developer to ship an MCP server. One pip install fastmcp, one @mcp.tool() decorator, one mcp.run(). Under 30 minutes to a working AI tool server that Claude Desktop can call.

MCP's ecosystem is maturing fast. My MCP server toolkit covers what's already available before you build your own. Check there first — but if you need something custom, FastMCP makes building it genuinely quick.

Verified versions today: FastMCP 3.2.4, MCP 1.27.0. This space moves fast — check the FastMCP official docs for the latest API before you ship anything.

Vercel AI SDK 6: First-Class Agents for TypeScript

Jangwook Kim — Tue, 12 May 2026 09:33:54 +0000

When Vercel released AI SDK 6 on December 22, 2025, the headline feature was not a new model integration or a faster streaming API. It was a different kind of addition: agents became a first-class primitive in the SDK, not an afterthought patched on top of generateText.

Effloow Lab installed ai@latest (currently 6.0.177) and inspected the package exports, constructor signatures, and available methods directly. This guide covers what actually changed, what the new ToolLoopAgent API looks like in practice, and how to move existing code from SDK 5.x to 6.

Why This Matters: The Shift from Loops to Primitives

In AI SDK 5.x, building an agent meant writing your own loop. You called generateText, checked for tool calls in the response, executed those tools, passed results back, and repeated until done — all in application code that you maintained yourself.

The result was that every team ended up writing a slightly different, slightly buggy version of the same control loop. Edge cases around retries, step limits, streaming, and type safety accumulated per project.

SDK 6 formalizes this pattern. ToolLoopAgent handles the loop. Your code defines what the agent can do and when it should stop. The runtime handles how it executes.

This is not just a convenience wrapper. The Agent interface and ToolLoopAgent class are designed so that:

The same agent definition works in API routes, background jobs, and UI streaming contexts without modification
TypeScript type inference flows end-to-end from tool schemas to UI message types
Human-in-the-loop approval, stop conditions, and structured output are all opt-in per-agent rather than bolt-ons

Installation and Current Version

The package name is still ai. Installing the latest v6 release:

npm install ai@latest
# or
npm install ai@^6.0.0

Effloow Lab confirmed that npm install ai@latest pulls 10 packages with zero vulnerabilities. The install is lean.

Peer dependency: Zod 3.x or Zod 4.x are both supported.

"zod": "^3.25.76 || ^4.1.8"

If you are on Zod 3, nothing changes. If you want Zod 4, SDK 6 now accepts it natively.

The Core Change: ToolLoopAgent

The central addition is ToolLoopAgent. In SDK 5.x, an Experimental_Agent class existed (it is still exported in 6.0.177 as a compatibility shim, but it is deprecated). In SDK 6, the production API is:

import { ToolLoopAgent } from 'ai';
import { anthropic } from '@ai-sdk/anthropic';
import { z } from 'zod';
import { tool } from 'ai';

const searchTool = tool({
  description: 'Search documentation for a given query',
  inputSchema: z.object({ query: z.string() }),
  execute: async ({ query }) => {
    // your search implementation
    return { results: [] };
  },
});

const agent = new ToolLoopAgent({
  model: anthropic('claude-sonnet-4-5'),
  instructions: 'You are a helpful developer assistant.',
  tools: { search: searchTool },
});

The ToolLoopAgent constructor accepts:

Parameter	Type	Default	Purpose
`model`	`LanguageModel`	required	The model to use
`instructions`	`string`	optional	System prompt
`tools`	`Record<string, Tool>`	optional	Available tools
`stopWhen`	`StopCondition	StopCondition[]`	`stepCountIs(20)`
`output`	`Output`	optional	Structured output format
`toolChoice`	`ToolChoice`	optional	Force/auto/none
`temperature`, `topP`, etc.	number	optional	Model parameters
`onStepFinish`	callback	optional	Hook for each step
`onFinish`	callback	optional	Hook on completion

The instance exposes two methods: generate() (returns Promise<GenerateTextResult>) and stream() (returns a streaming result). Both accept the same parameters:

const result = await agent.generate({
  prompt: 'What does the useEffect hook do?',
});

console.log(result.text);

Agent Is an Interface

ToolLoopAgent implements the Agent interface. This matters because you can create custom agent implementations that slot into the same API surface. If you need a routing agent, a retrieval-augmented agent, or an agent that checks a policy before every step, you implement Agent directly rather than subclassing ToolLoopAgent.

This design rewards dependency injection: pass an Agent type through your application, and the concrete implementation can swap out in tests or staging environments without changing call sites.

Human-in-the-Loop: needsApproval

Any tool can be marked as requiring human approval before execution:

const deployTool = tool({
  description: 'Deploy a new version to production',
  inputSchema: z.object({
    version: z.string(),
    environment: z.enum(['staging', 'production']),
  }),
  needsApproval: true,
  execute: async ({ version, environment }) => {
    // deployment logic
    return { deployed: true, version };
  },
});

When needsApproval: true, the tool call pauses and surfaces a pending approval in the UI stream. Your application handles the approval UI. The agent resumes execution only after the user confirms.

You can also pass an async function to needsApproval for conditional approval logic:

needsApproval: async ({ toolInput }) =>
  toolInput.environment === 'production',

This pattern makes it straightforward to build agents that handle low-risk actions automatically (staging deploys, read-only lookups) while requiring human sign-off on high-risk ones (production deploys, account deletions).

Structured Output from Agents

In SDK 5.x, getting structured data back from a multi-step agent required parsing the final text output yourself. SDK 6 introduces the Output helper that works at the agent level:

import { Output } from 'ai';
import { z } from 'zod';

const reportAgent = new ToolLoopAgent({
  model: anthropic('claude-sonnet-4-5'),
  tools: { fetchData: dataFetchTool },
  output: Output.object({
    schema: z.object({
      summary: z.string(),
      keyFindings: z.array(z.string()),
      confidenceScore: z.number().min(0).max(1),
    }),
  }),
});

const result = await reportAgent.generate({
  prompt: 'Analyze Q1 sales data and provide a structured report.',
});

// result.object is typed as { summary: string; keyFindings: string[]; confidenceScore: number }
console.log(result.object.keyFindings);

Supported output types are Output.object(), Output.array(), Output.choice(), Output.json(), and Output.text(). The model calls tools as many times as needed, then produces the final structured output in one pass at the end.

Multi-Agent Patterns: Agents as Tools

The most powerful pattern in SDK 6 is composing agents into multi-agent systems by wrapping them as tools:

const summarizeAgent = new ToolLoopAgent({
  model: anthropic('claude-haiku-4-5'),
  instructions: 'Summarize the given text in 3 bullet points.',
});

const factCheckAgent = new ToolLoopAgent({
  model: anthropic('claude-sonnet-4-5'),
  instructions: 'Fact-check the claims in the given text against the web.',
  tools: { search: webSearchTool },
});

// Wrap subagents as tools
const summarizeTool = tool({
  description: 'Summarize a long block of text',
  inputSchema: z.object({ text: z.string() }),
  execute: async ({ text }) => {
    const result = await summarizeAgent.generate({ prompt: text });
    return result.text;
  },
});

const factCheckTool = tool({
  description: 'Fact-check claims in a passage',
  inputSchema: z.object({ passage: z.string() }),
  execute: async ({ passage }) => {
    const result = await factCheckAgent.generate({ prompt: passage });
    return result.text;
  },
});

// Orchestrator agent
const orchestrator = new ToolLoopAgent({
  model: anthropic('claude-sonnet-4-5'),
  instructions: 'Research topics thoroughly: summarize sources, then fact-check key claims.',
  tools: { summarize: summarizeTool, factCheck: factCheckTool },
});

Subagents call .generate() (which returns the final result) rather than .stream(). The orchestrator calls tools, gets back text results from subagents, and continues reasoning.

This decomposition keeps each agent focused on a narrow task, makes individual agents testable in isolation, and avoids the context bloat that comes from cramming all capabilities into one massive system prompt.

Stop Conditions

By default, ToolLoopAgent stops after 20 steps. You can configure this explicitly or combine multiple conditions:

import { stepCountIs } from 'ai';

const agent = new ToolLoopAgent({
  model: anthropic('claude-sonnet-4-5'),
  tools: { search: searchTool },
  stopWhen: stepCountIs(10),
});

The prepareStep callback gives you per-step control if you need dynamic stop conditions based on intermediate results:

const agent = new ToolLoopAgent({
  model: anthropic('claude-sonnet-4-5'),
  tools: { search: searchTool },
  prepareStep: async ({ steps }) => {
    if (steps.some(s => s.text.includes('DONE'))) {
      return { stopCondition: true };
    }
    return {};
  },
});

MCP Support

Model Context Protocol support ships in a separate package to keep the core ai bundle lean:

npm install @ai-sdk/mcp

SDK 6 adds OAuth authentication handling for HTTP-based MCP servers, plus resources and prompts discovery and elicitation support (server-initiated user input). The createMCPClient function now handles PKCE, token refresh, and session management transparently:

import { createMCPClient } from '@ai-sdk/mcp';

const client = await createMCPClient({
  transport: {
    type: 'http',
    url: 'https://your-mcp-server.com/mcp',
    authProvider, // handles OAuth flow automatically
  },
});

const tools = await client.tools();

This makes it practical to connect agents to external MCP servers (databases, APIs, enterprise systems) without writing OAuth boilerplate.

Migrating from AI SDK 5.x

Most of the migration is mechanical. Vercel provides a codemod that handles the majority of changes:

npx @ai-sdk/codemod upgrade v6

The key manual changes:

Rename Experimental_Agent to ToolLoopAgent

// Before
import { Experimental_Agent } from 'ai';
const agent = new Experimental_Agent({
  model: ...,
  system: 'You are a helpful assistant.',
});

// After
import { ToolLoopAgent } from 'ai';
const agent = new ToolLoopAgent({
  model: ...,
  instructions: 'You are a helpful assistant.',
});

Note the parameter rename: system becomes instructions. The default stopWhen also changed from stepCountIs(1) to stepCountIs(20) — if your existing agent relied on single-step behavior, set this explicitly.

CoreMessage to ModelMessage

// Before
import type { CoreMessage } from 'ai';

// After
import type { ModelMessage } from 'ai';
// Use convertToModelMessages() (now async) for conversion

generateObject / streamObject Deprecation

These functions still work in 6.0.177 but are on the deprecation path. Migrate to generateText/streamText with an output setting:

// Before
const { object } = await generateObject({
  model: anthropic('claude-sonnet-4-5'),
  schema: z.object({ title: z.string() }),
  prompt: 'Generate a blog post title',
});

// After
const { object } = await generateText({
  model: anthropic('claude-sonnet-4-5'),
  output: Output.object({ schema: z.object({ title: z.string() }) }),
  prompt: 'Generate a blog post title',
});

Token Usage Fields

// Before
usage.cachedInputTokens;
usage.reasoningTokens;

// After
usage.inputTokenDetails.cacheReadTokens;
usage.outputTokenDetails.reasoningTokens;

Common Mistakes

Ignoring the default step count change. If you migrated from Experimental_Agent and assumed single-step behavior, your agent now runs up to 20 steps. This affects cost and latency. Set stopWhen: stepCountIs(1) explicitly if you need the old behavior.

Calling .stream() on subagents. In multi-agent patterns, subagents called via tool execution should use .generate(). Streaming only makes sense at the top level where you pipe output to a UI.

Missing the async on convertToModelMessages. The function became async in v6 to support async Tool.toModelOutput(). Forgetting await produces a Promise instead of a message array — a TypeScript type that might not catch this at the call site.

Assuming MCP is in the core package. createMCPClient requires @ai-sdk/mcp separately. The ai core package does not include MCP to keep bundle size down.

Not using the codemod first. The automated codemod handles renaming, import changes, and common method signature updates. Running it before manual review saves time and reduces manual errors.

FAQ

Q: Is AI SDK 6 compatible with Next.js 14 and 15?

Yes. The SDK works with any React framework including Next.js App Router and Pages Router. The ai/react sub-package provides hooks (useChat, useCompletion) that remain unchanged in v6.

Q: Can I use Anthropic Claude models with ToolLoopAgent?

Yes. Any provider supported by the SDK works with ToolLoopAgent. Install the provider package (@ai-sdk/anthropic, @ai-sdk/openai, @ai-sdk/google) and pass the model to the constructor. Provider-specific tools (Anthropic memory tool, OpenAI file patching, Google Maps grounding) are also accessible through the standard tools API.

Q: Does ToolLoopAgent support streaming to the UI?

Yes. Call .stream() instead of .generate(). Use createAgentUIStream and pipeAgentUIStreamToResponse to pipe the stream to a Next.js API route or other HTTP handler. The InferAgentUIMessage<typeof myAgent> TypeScript type infers the correct message shape for type-safe UI components.

Q: What is the difference between stopWhen and prepareStep?

stopWhen is a declarative stop condition — "stop after N steps" or "stop when no tool calls remain." prepareStep is a per-step callback that lets you inspect intermediate results and dynamically modify the agent's behavior (change tools, update instructions, or signal a stop) based on what happened so far.

Q: Are there breaking changes in the OpenAI provider?

Yes. strictJsonSchema now defaults to true for OpenAI. This improves JSON reliability but requires stricter Zod schema compliance (no .optional() on required fields, no .default()). If you see validation errors after upgrading, check your OpenAI-specific schemas first.

Key Takeaways

Bottom Line

AI SDK 6 is the version where Vercel's SDK stops being a collection of LLM utility functions and starts being an agent framework. The lean install (10 packages), stable MCP support, and type-safe multi-agent composition make it a credible foundation for production TypeScript agent systems in 2026. Run the codemod first, check your stop conditions, and migrate generateObject calls when you have time — the migration is not urgent but the new APIs are cleaner.

ToolLoopAgent is the production API. Experimental_Agent still exports in 6.0.177 but is deprecated.
The Agent interface design lets you swap implementations without changing call sites — valuable for testing.
needsApproval on individual tools gives you granular human-in-the-loop control without restructuring your agent.
Agents as tools is the idiomatic multi-agent pattern: wrap agent.generate() inside a tool() definition and compose freely.
MCP lives in @ai-sdk/mcp, separate from core. OAuth handling is now built in.
The codemod (npx @ai-sdk/codemod upgrade v6) handles most migration automatically.
Zod 4 is now supported alongside Zod 3, so you are not blocked from upgrading Zod independently.

MemMachine: Ground-Truth Memory for AI Agents

Jangwook Kim — Tue, 12 May 2026 04:15:52 +0000

Every time an agent summarizes a conversation to save memory, it loses information. That trade-off has been accepted as unavoidable — LLMs produce long outputs, context windows are finite, and token costs are real. MemMachine, presented in arXiv paper 2604.04853 (April 2026), rejects that premise. Instead of extracting facts at write time, it stores entire conversational episodes verbatim and does the heavy lifting at retrieval time. The result: 93.0% on LongMemEvalS (ICLR 2025) and approximately 80% fewer input tokens compared to Mem0 under matched conditions.

This article walks through the architecture, explains why ground-truth preservation changes the memory equation for agent developers, and shows how to integrate MemMachine into a Python-based agent using the open-source SDK.

Why Agent Memory Is Still Broken

Most production agents handle long-term memory with one of two approaches: stuff everything into the context window (expensive and bounded) or summarize with an LLM before storing (lossy and irreversible).

The summarization approach — used by Mem0 and many RAG-based systems — runs an extraction pass at write time. The LLM reads a conversation and outputs a set of facts or a condensed summary. Those facts go into a vector store. When the user comes back, retrieved facts are injected into the prompt.

The problem is structural: LLM extraction at write time introduces irreversible information loss. What looks like a minor paraphrase today becomes a missed fact next month when the user returns with a follow-up. Multi-hop reasoning across multiple sessions is especially fragile because each hop must rely on the lossy summaries produced at previous write points.

The LoCoMo benchmark makes this concrete — it tests whether an agent can recall facts from extended conversations, and Mem0's token-heavy pipeline still trails open-source alternatives on accuracy. MemMachine reaches 0.9169 on LoCoMo (with gpt-4.1-mini), above published scores for Mem0, Zep, Memobase, and LangMem.

The Core Idea: Ground-Truth Preservation

MemMachine's defining design choice is deferred extraction. Raw conversational episodes are stored verbatim in a graph database. No LLM extraction pass runs at write time. When the agent needs to recall something, a Retrieval Agent queries the episode store and surfaces the original conversation context.

This flips the cost curve. Write operations become cheap — store an episode, index it, done. Read operations become slightly richer — contextualized retrieval expands nucleus matches with neighboring episode context, so a query about "my dietary restrictions" pulls not just the turn where that phrase appeared, but the surrounding dialogue that gives it meaning.

The architecture has four distinct memory tiers:

Short-term workspace — the current conversation buffer. Limited capacity, cleared between sessions (standard context window behavior).

Long-term episodic memory — the ground-truth store. Full conversational episodes in a graph database. This is the structural difference from Mem0-style systems.

Semantic/profile memory — high-level user facts (preferences, identity, stated goals) stored in a SQL database. This tier does run LLM extraction, but only for stable profile data, not for ephemeral conversation content.

Procedural memory — learned patterns, action sequences, and strategies the agent has acquired over interactions.

The episodic tier does the heavy lifting. Because episodes are stored verbatim, the system can always return to the source of truth rather than a derivative. That is what "ground-truth-preserving" means in practice.

The Retrieval Agent: Three Routing Strategies

Raw storage would be useless without smart retrieval. MemMachine introduces a companion Retrieval Agent with a ToolSelectAgent classifier that routes each incoming query to one of three strategies:

Direct retrieval — semantic similarity search over the episode store. Used for simple, single-fact queries ("What is the user's preferred language?"). Fast and low-latency.

Parallel decomposition — splits multi-part queries into sub-queries and executes them concurrently, then merges results. Used when a question has several independent dimensions ("What does the user prefer about both their IDE setup and their deployment workflow?").

Chain-of-query — iterative retrieval where each step informs the next. Used for multi-hop reasoning ("What framework was the user migrating to, and what deployment platform did they choose for it?"). Each query builds on what the previous step retrieved.

This adaptive routing is what enables the multi-hop benchmark numbers. On HotpotQA-hard the Retrieval Agent reaches 93.2%. On WikiMultiHop (2WikiMultiHopQA with randomized noise) it reaches 92.6%, while reducing input tokens by 59% (from 103k to 42k) compared to context-window-stuffing approaches.

The crucial insight: the routing strategies are layered on top of the storage model without modifying it. Improving retrieval does not require changing how episodes are stored. That separation makes the system extensible — you can swap in a new routing strategy without touching the episode graph.

LongMemEvalS: What the 93% Actually Means

LongMemEval (ICLR 2025) benchmarks five long-term memory abilities:

Information extraction — can the agent find a stated fact?
Multi-session reasoning — can it connect facts across sessions?
Temporal reasoning — can it handle time-sensitive updates ("I changed jobs last month")?
Knowledge updates — does it override stale information correctly?
Abstention — does it correctly say "I don't know" rather than hallucinate?

LongMemEvalS is the subset used in the MemMachine paper's ablation study. The 93.0% overall accuracy comes from stacking six optimization dimensions, with retrieval-stage improvements contributing more than ingestion-stage changes:

Retrieval depth tuning: +4.2%
Context formatting: +2.0%
Search prompt design: +1.8%
Query bias correction: +1.4%
Sentence chunking (ingestion): smaller contribution

The ablation is honest — it shows that the biggest gains come from how you retrieve, not from a magic storage algorithm. The ground-truth-preserving model matters because it gives retrieval something accurate to work with. But retrieval engineering is where the performance headroom lives.

Benchmarks at a Glance

Benchmark	MemMachine	Mem0	Zep / Graphiti	Notes
LongMemEvalS (ICLR 2025)	93.0%	[DATA NOT AVAILABLE]	[DATA NOT AVAILABLE]	6-dimension ablation, gpt-4.1-mini
LoCoMo (F1/accuracy)	0.9169	Lower (paper claim)	Lower (paper claim)	gpt-4.1-mini; best published open-framework result
HotpotQA-hard	93.2%	[DATA NOT AVAILABLE]	[DATA NOT AVAILABLE]	Retrieval Agent multi-hop
WikiMultiHop (noisy)	92.6%	[DATA NOT AVAILABLE]	[DATA NOT AVAILABLE]	103k → 42k tokens (59% reduction)
Input tokens vs Mem0 (LoCoMo)	~80% fewer	Baseline	—	Write-time savings; no LLM extraction pass

Independent benchmark comparisons beyond what the MemMachine paper reports are not available at the time of writing. The figures above come from arXiv:2604.04853 and the MemMachine official blog; treat the relative comparisons as claims to verify when production-scale testing is feasible.

Getting Started: Installation and Basic Usage

MemMachine is open-source at github.com/MemMachine/MemMachine and published as memmachine on PyPI. The quickstart requires Docker and Docker Compose because the system uses both a graph database and a SQL database for its memory tiers.

Prerequisites

Docker 24+ and Docker Compose
Python 3.10+
An OpenAI API key (or compatible LLM endpoint for the Retrieval Agent)

Installation

# Download the latest release tarball from the GitHub releases page,
# extract it, then run the setup script
./setup.sh  # walks through Docker config and API key setup

# Alternatively, install the Python SDK standalone:
pip install memmachine

Core Workflow

The SDK follows a four-step pattern: retrieve → enrich → generate → store.

from memmachine import MemMachineClient

client = MemMachineClient(base_url="http://localhost:8000", api_key="YOUR_API_KEY")

user_id = "user_42"
message = "I prefer TypeScript over Python for backend work."

# 1. Retrieve relevant memories before generating a response
memories = client.memory.search(query=message, producer=user_id, limit=5)

# 2. Enrich context with retrieved episodes
context = "\n".join([m.content for m in memories.results])
enriched_prompt = f"User context:\n{context}\n\nUser message:\n{message}"

# 3. Generate response using your LLM (not shown — use your preferred client)
response = your_llm.complete(enriched_prompt)

# 4. Store the full episode verbatim
client.memory.add(
    messages=[
        {"role": "user", "content": message},
        {"role": "assistant", "content": response}
    ],
    producer=user_id
)

The producer field ties episodes to a specific user. When that user returns, every search() call scoped to their producer ID retrieves from their episode history — across all past sessions.

Accessing the Retrieval Agent

For multi-hop queries, use the Retrieval Agent directly:

# The Retrieval Agent automatically selects the routing strategy
result = client.retrieval_agent.query(
    question="What database was the user migrating to last month, and what hosting provider did they pick?",
    producer=user_id
)
print(result.answer)
print(result.sources)  # episodes that contributed to the answer

The agent classifies the query, selects direct, parallel, or chain-of-query routing, and returns both the answer and the source episodes. This traceability — knowing which stored episodes contributed to the answer — is a practical advantage over LLM-extraction-based systems where the provenance chain is broken at write time.

MemMachine vs Mem0: When to Choose Which

The ground-truth-preserving approach is not free. Storing raw episodes grows the database faster than storing extracted summaries. For applications where storage cost is a hard constraint and conversation length is short, Mem0's extraction approach may be a reasonable trade-off. For applications where accuracy and multi-session reasoning matter — personalized coding assistants, customer support agents with long histories, companion AI — the accuracy gains from MemMachine's architecture are likely to outweigh the storage overhead.

Key questions to guide the choice:

Are your sessions long and multi-turn? Ground-truth preservation gains more from longer episodes.
Do you need multi-hop reasoning across sessions? The Retrieval Agent's chain-of-query routing is designed for this.
Is write-time token cost a concern? MemMachine's no-extraction-at-write-time model cuts ingestion costs substantially.
Do you need audit trails? Storing raw episodes lets you trace every memory back to its source conversation.

Common Mistakes When Building Agent Memory

Over-indexing on single-session performance. Most agent memory benchmarks run in a single session. LongMemEval is one of the few that tests across sessions. Evaluate your memory system on multi-session workloads before deploying.

Assuming RAG extraction is lossless. Every LLM extraction pass introduces paraphrase drift. Test by storing a conversation, extracting from it, then asking questions the original conversation answers but the summary might not.

Ignoring retrieval depth. The MemMachine ablation shows retrieval depth tuning (+4.2%) has more impact than chunking strategy. Most teams optimize chunking obsessively while leaving retrieval depth at defaults.

Skipping abstention testing. A memory system that hallucinations when it does not know something is worse than one with lower recall. LongMemEval's abstention dimension is worth including in your eval suite.

Not scoping memories to users. Mixing memories across users is a privacy and accuracy risk. Always use a user-scoped key (like MemMachine's producer field) from day one.

FAQ

Q: Does MemMachine work with models other than OpenAI?

The Retrieval Agent is LLM-agnostic — it uses the LLM for query classification and chain-of-query reasoning, so any model with tool-use capability should work. The documentation references OpenAI by default in the quickstart, but the SDK supports custom LLM endpoints.

Q: How does MemMachine handle knowledge updates?

Newer episodes naturally take precedence in retrieval ranking. For explicit corrections ("actually, I switched from TypeScript to Go last week"), the Retrieval Agent's temporal reasoning handles the update — it surfaces the more recent episode and the semantic/profile memory layer can be updated with the corrected fact.

Q: Is there a hosted / cloud version?

The GitHub repository and PyPI package are open-source. A managed cloud offering (memmachine.ai) appears to be available based on the official site, but pricing and tier details were not independently verified at time of writing.

Q: How does the graph database fit into the architecture?

Episodic memory is stored in a graph database rather than a flat vector store. This allows the system to represent relationships between episodes (same user, same topic, same session) and traverse those relationships during contextualized retrieval — expanding a nucleus match with connected episodes without running a second embedding search.

Q: Can I use MemMachine with LangChain or LlamaIndex?

Integration guides for major agent frameworks were not available in the documentation at the time of writing. The Python SDK and RESTful API are framework-agnostic, so wrapping them for LangChain or LlamaIndex is straightforward. An MCP server interface is also listed as available.

Key Takeaways

Ground-truth-preserving memory is a principled response to a real problem: LLM extraction at write time is lossy, and that loss compounds across sessions. MemMachine's approach — store raw episodes, retrieve intelligently — trades slightly more storage for substantially better accuracy and traceability.

The benchmark results (93.0% LongMemEvalS, 0.9169 LoCoMo, 93.2% HotpotQA-hard) place it at the top of published open-framework results. The 80% token reduction on write operations is a meaningful cost argument for high-volume applications.

For developers building agents that need to remember users across sessions — and get the details right — MemMachine is the most technically grounded option in the open-source memory layer space as of April 2026.

Bottom Line

MemMachine's ground-truth-preserving architecture solves the write-time extraction problem that makes most agent memory systems degrade over long conversations. If you're building personalized agents that need accurate multi-session recall, it's worth evaluating against your current Mem0 or RAG-based setup — the token savings alone may offset the storage overhead.

Sources: arXiv:2604.04853 · MemMachine GitHub · PyPI: memmachine · LongMemEval benchmark · MemMachine blog: LoCoMo results · MemMachine blog: WikiMultiHop

Cursor SDK: Build AI Coding Agents in TypeScript

Jangwook Kim — Tue, 12 May 2026 00:13:01 +0000

When Cursor released its TypeScript SDK on April 29, 2026, it changed the framing of what a coding assistant is. Before the SDK, Cursor agents lived inside the IDE. After it, they're programmable infrastructure — agents you can invoke from a CI pipeline, a cron job, or a backend service with the same capabilities as the desktop app.

This guide covers what the Cursor SDK actually is, how its deployment model works, and how to wire it into real workflows. Effloow Lab verified that @cursor/sdk v1.0.12 is installable and inspected its public API surface as part of this write-up (see data/lab-runs/cursor-sdk-typescript-agent-deployment.md for the raw commands and outputs).

What the Cursor SDK Is (and Isn't)

The Cursor SDK (@cursor/sdk) is a TypeScript package that gives you programmatic access to the same agent runtime that powers the Cursor desktop app, CLI, and web interface. You get:

Codebase indexing — the agent understands your repo structure out of the box
Full tool access — read, write, glob, grep, shell, semantic search
Model choice — Composer 2 (Cursor's in-house model), Claude Opus 4.7, GPT-5.5, Gemini 3.1 Pro, and others
MCP server support — connect external tools over stdio or HTTP
Hooks — lifecycle callbacks to observe, modify, or block agent actions
Subagents — delegate subtasks to named sub-agents with different prompts and models

What it isn't: a general-purpose LLM client or a thin wrapper over the OpenAI API. The SDK is specifically built around the Cursor agent loop — the same harness that handles multi-step coding tasks with tool use, checkpointing, and repo context.

As of v1.0.12 (published 2026-05-01), the package has been in public beta for all users since launch.

Installation and Environment Setup

Installing the SDK is a single command:

npm install @cursor/sdk

The Effloow Lab sandbox confirmed this installs successfully on Node.js v25.9.0 with npm 11.12.1, pulling 131 packages total. The main dependencies are @bufbuild/protobuf and @connectrpc/connect (for the gRPC-based communication layer), sqlite3 (used to store local run events), and zod for schema validation.

One note from the sandbox run: npm audit reports 10 vulnerabilities (2 low, 1 moderate, 7 high) in the current dependency tree, mostly stemming from sqlite3's native bindings. This is common for public beta SDKs. Run npm audit before any production deployment and monitor for updates.

You'll also need a Cursor API key. Set it as an environment variable:

export CURSOR_API_KEY="your_key_here"

API keys are available from your Cursor account settings. The SDK reads CURSOR_API_KEY from the environment automatically, or you can pass it explicitly in the Agent.create() call.

The Agent API: Three Deployment Modes

The SDK exposes an Agent class with a static Agent.create() factory. The Effloow Lab inspection confirmed these static methods on the Agent class: create, resume, prompt, list, listRuns, getRun, get, archive, unarchive, delete, and messages.

The create call accepts three mutually exclusive deployment targets via the options object:

Mode 1: Local Agent

The agent runs on your machine, using your local filesystem. Good for one-off scripts, local automation, and development.

import { Agent } from "@cursor/sdk";

const agent = await Agent.create({
  apiKey: process.env.CURSOR_API_KEY,
  model: { id: "composer-2" },
  local: {
    cwd: process.cwd(),
  },
});

const run = await agent.send("Summarize what this repository does and list its main entry points");

for await (const event of run.stream()) {
  if (event.type === "text-delta") {
    process.stdout.write(event.text);
  }
}

The run.stream() method yields typed InteractionUpdate events — text deltas, tool call starts and completions, thinking deltas, and shell output. This lets you build real-time UIs or pipe the output to logs without buffering the whole response.

Mode 2: Cloud Agent

Each cloud run gets its own sandboxed VM managed by Cursor. The VM clones the target repository, sets up the development environment, and continues running even if the invoking script exits. You can reconnect and stream the conversation later using Agent.resume().

import { Agent } from "@cursor/sdk";

const agent = await Agent.create({
  apiKey: process.env.CURSOR_API_KEY,
  model: { id: "claude-opus-4-7" },
  cloud: {
    repo: "github.com/your-org/your-repo",
    branch: "main",
  },
});

const run = await agent.send(
  "Review the changes in the last PR and open a follow-up issue for any technical debt introduced"
);

// Optionally stream, or fire-and-forget
const result = await run.wait();
console.log("Agent status:", result.status);

Cloud agents are the right choice for long-running tasks, CI-triggered workflows, or anything where you don't want the agent tied to a process lifecycle. The agent can push branches and open pull requests directly from the VM.

Mode 3: Self-Hosted Agent

You can run the agent runtime on your own infrastructure — useful for air-gapped environments or when you need to control where code is executed. Self-hosted mode requires a separate runner binary and additional setup documented in the Cursor SDK docs.

Hooks: Observing and Controlling the Agent Loop

Hooks let you intercept the agent at specific points in its execution loop. They're defined in .cursor/hooks.json at the repo root and apply across all three deployment modes.

{
  "hooks": [
    {
      "event": "onFileEdit",
      "command": "npx prettier --write {file}"
    },
    {
      "event": "beforeShellCommand",
      "command": "scripts/guard-destructive.sh {command}"
    },
    {
      "event": "onRunComplete",
      "command": "scripts/notify-slack.sh {runId} {status}"
    }
  ]
}

Common hook events include:

onFileEdit — fires after each file write, useful for formatters and linters
beforeShellCommand — runs before shell execution, good for safety guardrails
onRunComplete — fires when the agent finishes, useful for notifications or post-processing
onToolCall — fires on any tool use, giving you a full audit log

Hooks receive context variables like {file}, {command}, and {runId} as string interpolations in the command. The hook's exit code determines whether the agent proceeds (exit 0) or aborts the action (any non-zero exit).

Subagents: Multi-Agent Orchestration Without Custom Glue

Subagents let the main agent delegate subtasks to specialized agents. You define them either in .cursor/agents/*.md files or inline in the Agent.create() call:

const agent = await Agent.create({
  apiKey: process.env.CURSOR_API_KEY,
  model: { id: "composer-2" },
  local: { cwd: process.cwd() },
  agents: {
    "code-reviewer": {
      prompt: "You are a strict code reviewer focused on security and performance. Review changes and output a list of issues.",
      model: { id: "claude-opus-4-7" },
    },
    "doc-writer": {
      prompt: "You write concise, accurate technical documentation in Markdown.",
      model: { id: "composer-2" },
    },
  },
});

When the main agent calls the Agent tool with a name matching one of these definitions, the subagent runs as a separate call with its own prompt context and model. The main agent receives the subagent's output and continues. This is how teams build multi-step pipelines — review agent → fix agent → doc agent — without custom orchestration code.

MCP Server Integration

MCP (Model Context Protocol) servers extend the agent's tool set with external data sources and actions. You can configure them in .cursor/mcp.json or pass them directly to Agent.create():

const agent = await Agent.create({
  apiKey: process.env.CURSOR_API_KEY,
  model: { id: "composer-2" },
  local: { cwd: process.cwd() },
  mcpServers: [
    {
      name: "linear",
      transport: "stdio",
      command: "npx",
      args: ["-y", "@linear/mcp-server"],
      env: { LINEAR_API_KEY: process.env.LINEAR_API_KEY },
    },
    {
      name: "postgres",
      transport: "http",
      url: "http://localhost:8080/mcp",
    },
  ],
});

With MCP servers configured, the agent can query Linear for ticket context, pull database schemas, or call any custom tool you expose over the MCP protocol — without any special prompting. The agent discovers available tools automatically.

CI/CD Patterns

The most common production use case teams are building with the Cursor SDK is CI/CD integration. Here are three patterns that work today:

Pattern 1: PR Summary on Push

// ci/summarize-pr.ts
import { Agent } from "@cursor/sdk";

const prNumber = process.env.PR_NUMBER;
const branch = process.env.GITHUB_HEAD_REF;

const agent = await Agent.create({
  apiKey: process.env.CURSOR_API_KEY,
  model: { id: "composer-2" },
  cloud: { repo: process.env.GITHUB_REPOSITORY, branch },
});

const run = await agent.send(`
  Review the changes in PR #${prNumber} compared to main.
  Write a concise summary covering: what changed, why it matters, and any risks.
  Output as a GitHub comment body in Markdown.
`);

const result = await run.wait();
// Post result.output to the PR via GitHub API

Pattern 2: Auto-Fix CI Failures

// ci/fix-failures.ts
import { Agent } from "@cursor/sdk";
import { readFileSync } from "fs";

const failureLog = readFileSync("/tmp/ci-failures.txt", "utf-8");

const agent = await Agent.create({
  apiKey: process.env.CURSOR_API_KEY,
  model: { id: "claude-opus-4-7" },
  cloud: {
    repo: process.env.GITHUB_REPOSITORY,
    branch: `fix/auto-${Date.now()}`,
    baseBranch: process.env.GITHUB_HEAD_REF,
  },
});

const run = await agent.send(`
  The following tests are failing:\n\n${failureLog}\n\n
  Identify the root cause and apply the minimal fix.
  Push the changes and open a pull request with a description of what you changed and why.
`);

await run.wait();

Pattern 3: Scheduled Documentation Sync

// scripts/sync-docs.ts
import { Agent } from "@cursor/sdk";

const agent = await Agent.create({
  apiKey: process.env.CURSOR_API_KEY,
  model: { id: "composer-2" },
  cloud: { repo: "github.com/your-org/docs", branch: "main" },
});

await agent.send(
  "Review all public API functions in src/api/ for missing or outdated JSDoc comments. Update them to match the current implementation. Commit directly to main."
);

These patterns work because cloud agents persist independently — the CI runner just fires the Agent.create() call and optionally monitors the result. The actual work happens in Cursor's infrastructure.

Pricing Considerations

The Cursor SDK uses the same token-based pricing as Cursor's Max Mode. A few reference points from Cursor's published pricing:

Auto mode (Cursor selects model): ~$0.25/M tokens (cache read), $1.25/M tokens (input), $6.00/M tokens (output)
Claude Opus 4.7 and GPT-5.5 are frontier-tier models priced higher
A complex task with 150 tool calls, 200K input tokens, and 20K output tokens costs roughly $3–8 depending on the model

Cloud agents also incur VM compute costs during execution. For typical CI tasks (PR summary, test fix), an agent run of 5–15 minutes will consume a manageable amount of tokens — but high-frequency automation on large repos can accumulate cost quickly. Start with composer-2 for volume use cases and reserve Opus 4.7 for tasks where accuracy matters most.

Strengths
<ul>
  <li>Same runtime as the desktop app — no capability gap between IDE and API agents</li>
  <li>Three deployment modes (local, cloud, self-hosted) in a single API surface</li>
  <li>Hooks and subagents enable complex pipelines without custom orchestration code</li>
  <li>MCP server support extends the agent's tool set to any external system</li>
  <li>Cloud agents persist independently of the invoking process</li>
</ul>


Limitations
<ul>
  <li>Public beta — API surface may change between minor versions</li>
  <li>10 npm audit vulnerabilities in v1.0.12 (monitor for fixes)</li>
  <li>Token-based pricing can accumulate at scale; no flat-rate option for SDK use</li>
  <li>Self-hosted mode requires a separate runner binary setup</li>
  <li>No offline or air-gapped cloud VM option; cloud mode phones home to Cursor infrastructure</li>
</ul>

What to Build First

If you're evaluating the Cursor SDK, the fastest path to value is a PR summary script. It's a contained task, produces immediately useful output, and doesn't require write access to your repo. From there, the escalation path is clear: CI failure diagnosis, documentation sync, and eventually multi-agent pipelines with subagents.

Rippling, Notion, Faire, and C3 AI are confirmed early adopters from the April 2026 launch announcement — all using cloud agents for CI/CD automation at various scales.

The SDK is in active development. Check the Cursor SDK changelog and the cursor/cookbook GitHub repo for recipes that the Cursor team adds as real-world patterns emerge.

Bottom Line

The Cursor SDK turns a coding assistant into programmable infrastructure. If your team already uses Cursor and runs any manual agent tasks on a schedule, this SDK makes those tasks repeatable, auditable, and composable. Start with cloud mode for CI/CD and move to subagents once the basic pattern works.

FAQ

Q: Do I need a Cursor Pro subscription to use the SDK?

You need a Cursor account with API access. As of the public beta, SDK usage is billed separately via token-based pricing, not against your plan's monthly request quota. Check your Cursor account settings for API key availability.

Q: Can I use the SDK without the Cursor desktop app installed?

Yes. The SDK is a standalone npm package. You don't need the desktop app on the machine running the agent. You do need a CURSOR_API_KEY and network access to Cursor's API endpoints.

Q: How does cloud agent persistence work if my script exits?

When you launch a cloud agent, Cursor's infrastructure takes ownership of the VM and the agent run. Your script can exit immediately after calling agent.send(). You can reconnect later with Agent.resume(agentId) and stream the conversation from wherever it left off.

Q: Are there rate limits on the SDK?

The SDK inherits Cursor's standard rate limits. High-frequency automation (many concurrent cloud agents) may hit limits. The Cursor.models() call returns the current model list, and responses include rate limit headers when limits are approached.

Q: What models are available?

Cursor.models() returns the full list of available models at runtime. At launch, this includes Composer 2, Claude Opus 4.7, GPT-5.5, Gemini 3.1 Pro, and additional frontier models as they become available inside Cursor.