DEV Community: Astrodevil

How to Build Self-Healing AI Agents with Monocle, Okahu MCP and OpenCode

Astrodevil — Wed, 08 Apr 2026 22:04:48 +0000

Coding agents write code. When that code fails, who debugs it? Right now, that's still you. The agent writes, you interpret error logs, you prompt the agent to fix. The debugging loop stays open.

The fix is giving agents access to their own telemetry. An agent that can query its own traces can verify its work, diagnose failures, and iterate without waiting for a human to read logs.

This tutorial shows you how to build a self-healing agent that runs tests, queries its own production traces via MCP, identifies root causes, and fixes bugs without human intervention.

By the end, you'll have a working demo where:

A buggy Text-to-SQL application fails its test suite
An agent queries its own traces from Okahu Cloud via MCP
The agent identifies and fixes bugs based on trace analysis
All tests pass, without a human reading logs or prompting fixes

What we'll cover:

What is Monocle and why auto-instrumentation matters
What is Okahu MCP and how agents consume telemetry
Setting up the self-healing demo environment
Running the agent and watching it fix itself
Key takeaways for building autonomous debugging loops

Let's start by understanding the two core technologies that make this possible.

What is Monocle?

In traditional software, you read the source code to understand what the application does. The logic is deterministic and inspectable. In agent-driven applications, the code is scaffolding. The actual decision-making happens inside the model at runtime.

An agent that can't access traces is an agent working without documentation. It will guess where failures occur and propose changes based on incomplete information.

Monocle helps developers and platform engineers building or managing generative AI apps monitor these in prod by making it easy to instrument their code to capture traces that are compliant with open-source cloud-native observability ecosystem. It automatically captures traces from LLM SDKs like OpenAI, LangChain, and LlamaIndex without any manual span creation.

Agents won't manually add telemetry to their generated code. If instrumentation requires effort, it won't happen. Monocle solves this by auto-instrumenting supported SDKs the moment they're used.

Here's what it takes to enable automatic trace capture:

from monocle_apptrace import setup_monocle_telemetry

# One line to enable automatic trace capture
setup_monocle_telemetry(workflow_name="text_to_sql_analyst")

That's it. From this point forward, every OpenAI SDK call, every database query, every tool invocation is captured as a trace span.

These traces include:

LLM calls: Inputs, outputs, model name, token usage
App traces: OpenTelemetry compatible for exporting traces/spans from an application.
Tool invocations: Agent framework tool calls and responses
Errors and latency: Exception details and timing data

When something goes wrong, the trace contains everything an agent needs to diagnose the problem. The question is: how does the agent access that trace data?

Zero-config instrumentation using monocle

It is also worthy to note that Monocle does not depend on existing OpenTelemetry instrumentation.

What is Okahu MCP?

Okahu is a cloud observability platform built for AI applications. It ingests traces from Monocle, stores them, and provides dashboards for visualization.

Dashboards are designed for human eyes. They're full of charts, graphs, and interactive trace viewers. An agent can't click through a dashboard. The data exists, but agents can't access it.

**MCP (Model Context Protocol)** is a standard for exposing data sources to AI agents. Okahu MCP turns the observability platform into a programmable API that agents can query directly. Instead of a human clicking through a dashboard, an agent calls /okahu:get_latest_traces and parses the JSON response.

Observability platforms must evolve from human dashboards to programmatic interfaces that agents can consume. MCP is how that happens.

To connect an agent to Okahu MCP, add this configuration:

{
 "mcp": {
  "okahu": {
   "type": "remote",
   "url": "https://mcp.okahu.ai/mcp",
   "headers": {
    "x-api-key": "your-okahu-api-key"
   },
   "enabled": true
  }
 }
}

Now the agent can query its own traces. It sees the same data a human would see in the dashboard, but in a format it can parse, reason about, and act on. Make sure you authenticate the server by running these command:

opencode mcp auth okahu

This will generate a URL and will redirect you to authenticate with Okahu cloud and a successful authentication will show a screen like the one below:

With Monocle capturing traces and Okahu MCP exposing them, we have the infrastructure for self-healing.

Prerequisites

Before starting, make sure you have the following:

Python 3.10+ installed on your system
OpenAI API key for LLM calls (the demo uses GPT-4o)
Okahu API key for trace storage (sign up at okahu.ai)
OpenCode or a similar coding agent with MCP support

Once you have these ready, let's clone the demo repository and set up the environment.

Step 1: Clone and Set Up the Demo

Start by cloning the demo repository and creating a virtual environment:

git clone https://github.com/Arindam200/awesome-ai-apps
cd mcp_ai_agents/telemetry-mcp-okahu

# Create and activate virtual environment
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install monocle_apptrace monocle_test_tools openai fastapi pytest python-dotenv

Next, create a .env file with your API keys:

# .env
OPENAI_API_KEY="your-openai-key"
OPENAI_MODEL="gpt-4o"
OKAHU_API_KEY="your-okahu-key"
MONOCLE_EXPORTER="okahu"

The MONOCLE_EXPORTER=okahu setting tells Monocle to send traces to Okahu Cloud instead of logging them locally. This is important because we're suppressing all local logs to force the agent to use MCP for debugging.

Finally, initialize the sample database:

python setup_db.py

This creates a sales.db SQLite database with users and orders tables. The Text-to-SQL application will generate queries against this database.

Now let's look at what makes this demo interesting: the intentionally buggy application.

Step 2: Understand the Buggy Application

The demo includes a pre-built analyst.py with three intentional bugs. Each bug produces a specific trace signature that the agent must find and interpret:

Bug	What's Wrong	What the Trace Shows
1	Uses `client.completions.create()` instead of `client.chat.completions.create()`	API error: "NotFoundError: /v1/completions not found for gpt-4o"
2	Accesses `.text` instead of `.message.content` on the response	AttributeError in trace span
3	Schema prompt says `customers/products` but DB has `users/orders`	SQL execution error: "no such table: customers"

Why pre-built bugs instead of letting the agent write code from scratch? Two reasons:

Reproducibility: Every demo run starts with the same failures
No guessing: The agent must read traces to understand what's wrong

You can reset to the buggy state anytime with:

python reset_demo.py

This script overwrites analyst.py with the buggy version, ready for the agent to fix.

With the buggy application in place, let's run the tests and see it fail.

Step 3: Run Tests and See Failures

The test suite uses Monocle Test Tools to validate not just outputs, but traces. We're testing that the right LLM calls were made, not just that the code ran.

Here is what the test_analyst.py file looks like and what it tests:

@MonocleValidator().monocle_testcase(sql_generation_test_cases)
def test_generate_sql_with_monocle(my_test_case: TestCase):
    """
    Test SQL generation using Monocle validator.

    This validates:
    - OpenAI inference spans are generated correctly
    - SQL output matches expected patterns (similarity check)

    If this fails, check Okahu MCP traces for:
    - Missing inference spans (wrong API method used)
    - Incorrect SQL generation (schema mismatch)
    """
    MonocleValidator().test_workflow(generate_sql, my_test_case)

# Direct database tests (no monocle validation needed - these test the DB itself)
def test_execute_query_users_table():
    """
    Test direct SQL execution on the actual database.
    This verifies the users table exists and has data.
    """
    sql = "SELECT * FROM users LIMIT 3"
    results = execute_query(sql)

    assert len(results) == 3, "Should return 3 users"
    # Verify column structure: (user_id, username, email)
    assert len(results[0]) == 3, "Each user row should have 3 columns"

def test_execute_query_orders_table():
    """
    Test direct SQL execution on orders table.
    This verifies the orders table exists and has data.
    """
    sql = "SELECT * FROM orders WHERE amount > 100 LIMIT 5"
    results = execute_query(sql)

    assert len(results) > 0, "Should find orders over $100"
    # Verify column structure: (order_id, user_id, amount, order_date)
    assert len(results[0]) == 4, "Each order row should have 4 columns"

Run the tests:

pytest test_analyst.py -v

You'll see output like this:

The direct database tests pass (the database itself is fine), but the LLM-powered SQL generation fails.

Here's what makes this demo realistic: there are no useful local logs. We've suppressed all local logging:

import logging
logging.disable(logging.CRITICAL)

The agent cannot read local logs to debug. It must query Okahu MCP to understand what went wrong. This forces the self-healing loop through observability infrastructure, exactly how it would work in production.

The trace shows exactly what went wrong. The agent queries this via MCP

The trace shows exactly what went wrong. The agent queries this via MCP

Step 4: Configure the Self-Healing Agent

The demo uses an OpenCode agent mode called @analyst_v3. This is a custom agent configuration stored in .opencode/agents/analyst_v3.md with specific rules for self-healing behavior.

Tool access alone isn't enough. The agent needs structured rules that encode how an experienced developer would debug.

Here are the summary of the core rules:

**Self-Healing Rules:**

1. Run tests first, observe failures
2. Wait 5 seconds for trace ingestion
3. Query Okahu MCP: `/okahu:get_latest_traces` with workflow_name='text_to_sql_analyst_v3'
4. Fix bugs based on trace analysis only
5. Archive old code to `versions/analyst_vN.py` before each fix
6. Record the trace ID used to diagnose each fix
7. Repeat until all tests pass

**Critical Constraint:**
NO GUESSING. If no traces are available, STOP and report failure.
Do not attempt to fix code without trace evidence.

The "no guessing" rule is the key constraint. Without it, the agent might make up fixes based on general knowledge. By requiring trace evidence, every fix must be grounded in telemetry. The agent can cite its sources.

The agent also has a list of approved packages it can install if missing:

monocle_apptrace, monocle_test_tools, openai, fastapi, pytest, python-dotenv

If it encounters a ModuleNotFoundError, it can install from this list, but nothing else.

With the agent configured, let's trigger the self-healing loop.

Step 5: Run the Self-Healing Loop

With traces accessible via MCP, the agent can close the loop itself. It runs, fails, queries traces, diagnoses, fixes, and repeats.

Trigger the agent with this prompt:

@analyst_v3 Fix the buggy Text-to-SQL codebase:

The `analyst.py`, `test_analyst.py`, and `main.py` files already exist but have bugs.

1. Run Tests: Execute `pytest test_analyst.py -v` to see failures.
2. Analyze Traces: Wait 5s, then query Okahu MCP with workflow_name='text_to_sql_analyst_v3'.
3. Fix Loop:
    - Archive current `analyst.py` to `versions/analyst_vN.py`
    - Fix the bug based on trace analysis
    - Record the trace ID used to diagnose each fix
    - Run tests again
    - Repeat until all tests pass
4. Final Report: Output a summary table of all issues fixed with their associated trace IDs.

Rules: No debug files. Debug only via Okahu MCP traces. Always call the MCP tool to get the logs from traces, do not use the local logs in the terminal

Here's what happens, completely autonomously:

Agent runs tests → 3 failures detected
Agent waits 5 seconds → Allows traces to be ingested into Okahu
Agent queries Okahu MCP → Retrieves trace data for the failed runs
Agent reads the first trace → Sees "NotFoundError: /v1/completions not found for gpt-4o"
Agent archives analyst.py → Saves to versions/analyst_v1.py
Agent fixes Bug #1 → Changes completions.create() to chat.completions.create()
Agent runs tests → 2 failures remain
Agent queries MCP again → Sees AttributeError for .text access
Agent fixes Bug #2 → Changes .text to .message.content
Agent runs tests → 1 failure remains
Agent queries MCP → Sees "no such table: customers"
Agent fixes Bug #3 → Updates schema in prompt from customers/products to users/orders
Agent runs tests → All 5 tests pass

Agent runs, fails, queries traces, fixes, repeats until all tests pass.

The entire loop runs without human intervention. The agent reads, reasons, and repairs using only its own telemetry.

Step 6: The Fix Summary

The agent produces a traceable summary. Every fix links to the trace that prompted it:

Issue	Description	Trace ID
1	Wrong OpenAI API (completions → chat.completions)	trace_a1b2c3d4...
2	Wrong response attribute (.text → .message.content)	trace_e5f6g7h8...
3	Schema mismatch (customers/products → users/orders)	trace_i9j0k1l2...

You can take any trace ID, look it up in Okahu, and see exactly what error led to that fix. The human's role shifts from doing the debugging to auditing the system that did the debugging.

The archived versions in versions/ show the progression:

analyst_v1.py: Original buggy version
analyst_v2.py: After fixing API method
analyst_v3.py: After fixing response attribute
analyst.py: Final working version

The human wasn't in the debugging loop. The human set up the environment, triggered the agent, and reviewed the summary. The debugging itself was autonomous.

OpenCode agent running test, using monocle traces on Okahu via MCP to debug and fix the buggy app

Key Takeaways

Here's what we've shown and why it matters:

1. Auto-instrumentation is essential for self-healing

Agents won't manually add telemetry to their generated code. If you want agents to debug their own work, instrumentation must be automatic. Monocle captures traces from supported SDKs with a single line of setup.

2. MCP enables agent-native observability

Dashboards are designed for humans. MCP turns observability platforms into programmable APIs that agents can query. Okahu MCP exposes the same trace data a human would see, but in a format agents can parse and act on.

3. "No Trace, No Fix" prevents hallucinated fixes

Without grounding in telemetry, agents might guess at fixes based on general knowledge. The "no trace, no fix" rule ensures every repair is evidence-based. If the agent can't find trace data, it stops and reports the issue rather than guessing.

4. Trace-based testing validates the full pipeline

Monocle Test Tools doesn't just check that code runs. It validates that the right traces exist. If the LLM wasn't called correctly, the inference span won't exist, and the test fails. This catches issues that output-only testing would miss.

5. Human role shifts from debugging to monitoring

The human sets up the environment, triggers the agent, and reviews the fix summary. The iterative debugging loop (run, fail, diagnose, fix, repeat) is fully autonomous. This is a fundamental shift in how we build and maintain AI applications.

The Evolution: Who Consumes Telemetry?

This demo represents a real shift in how software gets built:

Phase	Who Writes Code	Who Reads Telemetry	Human Role
Software 1.0	Human	Human	Everything
Software 2.0	Coding Agent	Human	Prompts + interprets errors
Autonomous	Coding Agent	Coding Agent	Monitors only

Who consumes telemetry data?

In Software 1.0, humans wrote code, read dashboards, and fixed bugs. In Software 2.0, agents write code, but humans still interpret errors and prompt fixes. The agent is the "hands," but the human remains the "brain."

In the autonomous phase, which is what this demo shows, the agent writes, runs, debugs, and fixes. The human monitors the process and reviews outcomes. The agent consumes its own telemetry.

The lesson from every team that has pushed agents toward autonomy is the same: the critical work is never writing code. It's designing the environment, encoding constraints, structuring knowledge, and building feedback loops. As human involvement decreases, the systems that steer and verify agent behavior must become more robust, not less.

This requires a shift in how we build observability platforms. Dashboards aren't enough. Platforms must expose MCP interfaces that agents can query programmatically. The data is the same; the consumer is different.

Conclusion

Self-healing agents are possible when three conditions are met:

Instrumentation is automatic: Monocle captures traces without manual span creation
Telemetry is programmatically accessible: Okahu MCP exposes traces as an API
Testing validates traces, not just outputs: Monocle Test Tools ensures the right calls were made

The result is a coding agent that can debug its own code by querying production telemetry. It runs tests, reads traces, identifies root causes, and applies fixes, closing the loop without human intervention.

The organizations that invest early in giving agents access to telemetry, embedding evaluation into workflows, and exposing programmatic observability interfaces will scale agent-driven development effectively. Those that don't will find their agents operating blind, producing code that looks correct but behaves unpredictably.

The question is no longer "who writes the code?" It's "who consumes the telemetry?" When agents can do both, the loop closes, and we've entered a new phase of software engineering.

Resources

Monocle: pip install monocle_apptrace | GitHub
Monocle Test Tools: pip install monocle_test_tools | Trace-based validation framework
Okahu Cloud: okahu.ai | AI observability platform with MCP support
Demo Repository: GitHub | Full source code for this tutorial
OpenCode: Install here

This article demonstrates a proof-of-concept for autonomous agent debugging. The patterns shown here (auto-instrumentation, MCP-based telemetry access, and trace-driven testing) are the building blocks for self-healing AI systems.

Build Collaborative AI Whiteboard Like Mural Using Velt Agent Skills and MiniMax🔥

Astrodevil — Tue, 07 Apr 2026 16:13:20 +0000

Introduction

Building a real-time collaborative canvas is harder than it looks. The interface seems simple enough, boxes, lines, and cursors moving around. But underneath it is a constant stream of concurrent edits, conflicting updates, and shared state that has to stay consistent for everyone at the same time. The hard part is not drawing on the canvas. It is making sure multiple people can work on the same board simultaneously without things breaking quietly in the background.

In this article, we will build a collaborative whiteboard using Velt that supports shared editing, live presence, comments, and AI-assisted interactions all inside the same canvas. The focus is not just on rendering nodes but on wiring the system in a way that keeps every user in sync without adding friction to the experience.

Here is what the final result looks like.

Now let's look at the stack behind it and how each piece fits together.

The Stack

Next.js for the app framework
ReactFlow for the infinite canvas and custom node types
Velt for CRDT sync, live cursors, presence, comments, and notifications
MiniMax M2.5 for the AI features
Tailwind CSS & Shadcn for styling and dark mode
Zustand for state management

I used Velt Docs MCP and Skills throughout the build. The MCP server pulls Velt's API references directly into your editor context, and Skills are pre-built prompt templates that tell the agent exactly how to set up features like presence, comments, and notifications. Adding collaborative features that would have taken a couple of days ended up taking a few prompts with Skills and Velt Docs MCP.

For the editor side, I paired with GitHub Copilot Agent inside VS Code, it cut down the back-and-forth that usually comes with integrating a new SDK. If you haven't tried the Copilot Agent yet, it's worth a look. It takes multi-step actions in the editor rather than just completing code at the cursor, which works well for integration-heavy projects like this one.

How CRDT Makes This Work

The stack above gives you the canvas and the AI, but collaboration only works if multiple users can edit the same board without stepping on each other. This is handled through Velt’s CRDT-based sync.

CRDT records every change as an operation rather than a replacement, so edits from different users merge naturally, and the document keeps moving forward. Velt builds on Yjs and manages this layer for you. Wrap the app with VeltProvider, call client.setDocument(), and the shared state starts syncing immediately. Node positions, text, cursors, and comments all stay inside the same document.

That sync layer is already running the moment you call setDocument(). Everything else in this build sits on top of it. Before getting into the implementation, here is how to get the project running.

Before You Start

Clone the repo and add your environment variables to get it running locally.

git clone https://github.com/Studio1HQ/claude-velt-mcp-app
cd velt-whiteboard
npm install

Add these three keys to your .env:

NEXT_PUBLIC_VELT_API_KEY=your_velt_key
MINIMAX_API_KEY=your_minimax_key

Then run npm run dev and open localhost:3000.

You can grab your Velt API key from the Velt dashboard and your MiniMax key from the MiniMax platform.

The rest of this post walks through how the project is structured and how each piece is built, so you have the full context of what you are looking at.

Setting Up Velt

We did not write any of the code setup manually. During the building of this project, we used Velt Agent Skills and the Velt MCP server. One prompt handled the entire setup, the agent scanned the project, picked the right provider location, and generated the files.

If you are on Cursor or Claude Code, Velt also has an AI Plugin that bundles the MCP server, skills, rules, and an expert agent in one install, so you can skip even these three steps.

We used VS Code, so we set up the skills and MCP server separately. Here is how:

Step 1: Install Agent Skills

Run this in your project root:

npx skills add velt-js/agent-skills

This pulls four skill sets into your project: setup best practices, comments, CRDT, and notifications. Your agent picks the right one automatically based on what you ask it to do.

Step 2: Add the Velt MCP server to your editor

npx -y @velt-js/mcp-installer

Restart your editor after running this.

Step 3: Start the installation

In your AI agent chat (Copilot, Claude, Cursor, whichever you use), type:

install velt

The agent walks through the setup one step at a time: your project path, API key, which features you want, and where to put VeltProvider. It scans the codebase to detect your auth pattern and document ID source, shows you its findings, and generates an implementation plan before touching anything. You approve the plan, it applies the changes, and then runs a QA pass automatically.

The prompt we used for this project:

Set up Velt collaboration in my Next.js app. I need comments, presence, cursors, and CRDT for ReactFlow. My API key is in .env as NEXT_PUBLIC_VELT_API_KEY.

What would have taken a couple of hours of reading docs came down to that one prompt. The folder structure and code below are exactly what came out of it.

components/
  providers/
    velt-provider-wrapper.tsx  ← wraps the app with VeltProvider
    velt-authenticator.tsx     ← identifies user and sets document
  whiteboard/
    whiteboard.tsx             ← main canvas
    nodes/                     ← StickyNote, TextNode, ShapeNode
  layout/
    header.tsx                 ← where all Velt UI components live
lib/
  ai/
    ai-helpers.ts
    canvas-actions.ts
store/
  whiteboard-store.ts

There are two separate provider files here. VeltProviderWrapper just wraps the app with VeltProvider and passes the API key. VeltAuthenticator sits inside it and handles the actual user identification and document setup.

// velt-authenticator.tsx
await client.identify({
  userId: currentUser.userId,
  name: currentUser.name,
  email: currentUser.email,
  photoUrl: currentUser.photoUrl,
  organizationId: currentUser.organizationId,
});

// runs after identify resolves
client.setDocument(documentId);

The order matters. setDocument only runs after isUserIdentified flips to true, which is tracked with a useState flag. If you call setDocument before the user is identified, Velt won't associate that session correctly with the document; that’s why the agent put these in one order.

The demo uses hardcoded users with a dropdown switcher. If you want to plug in real auth, swap the currentUser object in the store with whatever your auth provider returns, and the rest of the setup stays the same.

The Canvas

The canvas layer is built on two packages, @xyflow/react for the canvas itself and zustand for global state management.

ReactFlow handles the canvas, nodes, edges, and interactions. Zustand is for global state management across the app.

Having everything in a single store makes it much easier to wire up features like AI later, since any part of the app can read or write the same state without prop drilling or extra context layers.

State Management

The store is in lib/store/whiteboard-store.ts. It keeps track of the active tool, selected shape, selected template, a node ID counter, and a few other things. One thing worth noting is how setSelectedTool works. When you switch to "shapes" or "templates", it also flips the corresponding panel open automatically.

setSelectedTool: (tool: ToolType) => {
  set({ selectedTool: tool });
  if (tool === "shapes") set({ isShapesPanelOpen: true });
  if (tool === "templates") set({ isTemplatesPanelOpen: true });
}

The node ID counter is just an incrementing number prefixed with "node-". Simple, but it prevents collisions when multiple nodes are dropped quickly.

Setting Up ReactFlow

In whiteboard.tsx, you register all custom node types before passing them to <ReactFlow>. This maps a string like "sticky" to the actual React component.

const nodeTypes = useMemo(() => ({
  sticky: StickyNote,
  text: TextNode,
  shape: ShapeNode,
}), []);

<ReactFlow
  nodes={nodes}
  edges={edges}
  nodeTypes={nodeTypes}
  onNodesChange={onNodesChange}
  onEdgesChange={onEdgesChange}
  onConnect={onConnect}
  onPaneClick={handlePaneClick}
>
  <Background />
  <Controls />
</ReactFlow>

onPaneClick is where click-to-place logic sits. When a tool or shape is selected in the store, clicking the canvas converts that canvas coordinate to a flow position and creates a new node there.

Building a Node

StickyNote.tsx is a good starting point for understanding how all nodes work. Every node receives data, selected, and id as props via NodeProps. The component keeps its own local state for text, color, and isEditing. On double-click, it switches to a textarea, and on blur, it commits back.

function StickyNote({ data, selected, id }: NodeProps) {
  const [isEditing, setIsEditing] = useState(false);

  return (
    <>
      <NodeResizeControl>...</NodeResizeControl>
      <NodeToolbar>{/* color picker */}</NodeToolbar>
      <Handle type="source" position={Position.Right} />
      {isEditing ? <textarea /> : <div>{text}</div>}
    </>
  );
}

export default memo(StickyNote);

Notice memo() at the bottom. Without it, every canvas interaction re-renders all nodes. ReactFlow recommends this for any non-trivial node. The Handle components on all four sides is what allows edges to connect from any direction. ShapeNode follows the same pattern, but uses a switch on shapeType to render different SVG shapes, rectangles, circles, diamonds, and so on.

Templates

Templates are in lib/constants/templates.ts as a plain array of TemplateType objects. Each template has an id, name, description, and a nodes array. That nodes array is just ReactFlow node definitions with preset positions, types, colors, and labels. No runtime logic.

{
  id: "kanban",
  name: "Kanban Board",
  nodes: [
    { type: "text", position: { x: 50, y: 50 }, data: { text: "📋 Backlog" }, style: { width: 250 } },
    { type: "text", position: { x: 330, y: 50 }, data: { text: "➡️ Up next" }, style: { width: 250 } },
    // ...
  ]
}

When you click a template in the sidebar, it sets selectedTemplate in the store. The next click on the canvas calls getNextNodeId() for each node in the template, offsets all positions relative to where you clicked, and adds them to the ReactFlow nodes array in one go. The template disappears from the selection state immediately after.

To add your own template, you define the layout once by hand and commit it to the array. If the layout is repetitive (like the brainstorm grid), you can use Array.from with a generator to compute positions instead of writing them all out. The existing templates show both approaches.

With the canvas layer in place, the next piece is layering Velt on top of it for real-time collaboration.

Real-Time Collaboration with Velt

Once VeltProvider wraps the app and the document is set, the collaboration layer is essentially ready. The only thing left is placing the right components in the right spots.

Cursors and Presence

VeltCursor and VeltPresence are both mounted inside VeltProviderWrapper in VeltProvider.tsx, sitting alongside the app children. This means they are active across the entire canvas, not scoped to a specific component. Every connected user gets a labeled cursor that moves in real time, and their avatar shows up in the "Online" section of the header.

The Header Tools

All four collaboration controls sit in TopBar.tsx. The component imports VeltCommentTool, VeltPresence, VeltNotificationsTool, and VeltSidebarButton from @veltdev/react and drops them into the header alongside the user switcher dropdown.

import {
  VeltPresence,
  VeltNotificationsTool,
  VeltCommentTool,
  VeltSidebarButton,
} from "@veltdev/react";

// Inside the header JSX:
<VeltCommentTool />
<VeltSidebarButton />
<VeltNotificationsTool />
<VeltPresence />

VeltCommentTool activates the comment pin mode so users can click anywhere on the canvas to leave a comment. VeltSidebarButton toggles a panel that lists every comment on the document in one place. VeltNotificationsTool shows a bell icon with a badge that updates when someone mentions you or replies to your thread. VeltPresence renders the avatar stack at the right side of the header.

Comments

The comment editor Velt ships uses SlateJS under the hood, so it supports rich text out of the box, including @mentions. When a user clicks anywhere on the canvas with comment mode active, a pin drops at that position, and a comment thread opens. The pin stays anchored there, visible to everyone on the document.

Dark Mode

Velt's components use Shadow DOM by default, which means your global CSS won't reach them. To apply custom theming, you first disable Shadow DOM on the components you want to style:

<VeltComments shadowDom={false} />
<VeltCommentsSidebar shadowDom={false} />

Then you use Velt's theme playground to generate your CSS token set. It gives you a full palette for both light and dark mode that you can copy directly. Paste it inside the body tag in globals.css and it covers both themes automatically.

body {
  /* Border Radius */
  --velt-border-radius-xs: 0.5rem;
  --velt-border-radius-sm: 1rem;

  /* Light Mode */
  --velt-light-mode-accent: #e31646;
  --velt-light-mode-background-0: #ffffff;
  --velt-light-mode-text-0: #0a0a0a;

  /* Dark Mode */
  --velt-dark-mode-accent: #e31646;
  --velt-dark-mode-background-0: #000000;
  --velt-dark-mode-text-0: #ffffff;

  /* and so on... */
}

The theme playground is the fastest way to get this right. You pick your accent color, adjust the radius and spacing, and it generates the full token set.

AI on the Canvas

With the canvas rendering and state managed globally, let's add AI to the canvas.

To add AI to the canvas, you need three things:

an API route that calls the model,
a set of structured action types that the model can return, and
a function that converts those actions into ReactFlow nodes.

The AI sidebar in this project connects all three.

The sidebar has four modes.

Ask AI for free-form chat,
Brainstorm to generate ideas as sticky notes,
Summarize to get a read of what's on the board, and
A Template namer that looks at the current layout and suggests what to call it.

You type a prompt, it reads the live canvas state, and either responds with text or drops new nodes directly onto the canvas.

Add Model

The model powering this is MiniMax M2.5, a large context model that's fast and works well for structured output tasks like this one. What made it easy to drop in is that it runs on an Anthropic-compatible API. You get the API key from MiniMax's platform, then point the Anthropic SDK at their base URL and swap the model name. No new SDK, no adapter layer.

const client = new Anthropic({
  apiKey: process.env.MINIMAX_API_KEY,
  baseURL: "https://api.minimax.io/anthropic",
});

const message = await client.messages.create({
  model: "MiniMax-M2.5",
  max_tokens: 2000,
  system: CANVAS_SYSTEM_PROMPT + contextSummary,
  messages: [{ role: "user", content: prompt }],
});

The context window is large enough that you can serialize the entire node list and send it with every request. The current canvas state gets appended to the system prompt as a JSON summary of all nodes, their IDs, types, text, and colors, so the model always knows what's already on the board before it responds.

MiniMax M2.5 in the Editor

Since the same API is Anthropic-compatible, you can also wire it into GitHub Copilot Chat inside VS Code, not just inside the project. During the build, we added MiniMax M2.5 as the active model in Copilot and used it to ask questions and build features directly from the editor.

It works the same way as any other model in Copilot Chat, except it is your own key and your own model.

Canvas Actions

Back to the app, the model needs to do more than just respond with text. It needs to place nodes, update colors, and interact with the canvas in a predictable way. Rather than letting the model return free-form text and trying to parse intent from it, the system prompt instructs it to always return structured JSON with two fields: message and actions. The actions array is a typed list of operations the model wants to perform on the canvas.

{
  message: "Added 5 brainstorming sticky notes",
  actions: [
    {
      type: "add_sticky",
      items: [
        { text: "Reduce onboarding steps", color: "#fef08a" },
        { text: "Add social login", color: "#fef08a" },
      ]
    }
  ]
}

buildNodesFromActions() in canvas-actions.ts takes that actions array plus a canvas anchor point and converts it into ReactFlow nodes, laid out in a grid starting from that position. If the action is update_color, applyColorUpdates() maps over existing nodes and patches their color in place. Both functions are pure, they take nodes in and return nodes out, with no side effects.

The AI does not directly touch the ReactFlow state. It just returns data. The sidebar calls buildNodesFromActions, gets the result, and passes it to setNodes. That separation keeps the AI logic completely independent of how the canvas renders.

With everything wired up, run npm run dev and open localhost:3000. Velt takes a second to initialize on the first load, after that, the canvas is live.
Switch between the two demo users using the dropdown in the header to simulate two people on the same board. Both cursors show up, comments are shared, and the AI sidebar works independently from either user session.

Demo

You can check the deployed version of our app here - https://mural-velt.vercel.app/

Bottom Line

This article covered building a collaborative whiteboard from the canvas layer up, real-time sync with Velt's CRDT, live cursors and comments, and an AI sidebar that reads and writes to the board. The pieces are modular enough that extending it is mostly additive.

A few things worth building on top of this:

Lottie reactions on nodes so collaborators can leave emoji responses without opening a comment thread
Export to PNG or PDF so teams can take the board outside the browser
More node types like embeds, images, or code blocks

The CRDT layer is already there, so most of these would be new node types or toolbar additions rather than architectural changes.

Velt Docs MCP, worth setting up in your IDE before you start a Velt integration
Velt Skills
MiniMax Docs

Build a Real-Time Social Media App with InsForge, MiniMax, and Next.js

Astrodevil — Wed, 25 Mar 2026 18:57:24 +0000

Introduction

In this tutorial, we will build a full-stack social platform where users post, like, repost, follow each other, get real-time notifications, and chat with an in-app AI assistant.

Here is what we will be building:

A Next.js frontend with a real-time feed, post composer, profile pages, notifications, explore, and an AI chat screen
InsForge as the backend platform, managing the database, auth, file storage, real-time pub/sub, and AI gateway from a single place
MiniMax M2.7 via GitHub Copilot as the agent that builds the entire application through InsForge Agent Skills and MCP.
Google Stitch for generating the design reference before the agent builds
Deployment triggered from inside GitHub Copilot, with no manual steps outside the editor

By the end, you will have a working social platform template you can fork and adapt to whatever you are building next.

Let's get started.

What Is InsForge

InsForge is an open-source backend platform that bundles a Postgres database, a REST API layer via PostgREST, an AI model gateway that routes to any OpenRouter-compatible model, a real-time pub/sub system, serverless edge functions, and a CLI, all into a single deployable platform. You can self-host it with Docker or use the managed cloud. You bring the application logic. InsForge handles what's underneath.

What We Are Using InsForge For

Three things in particular make InsForge the right fit for a project like this one.

Agent Skills: When you run insforge create, the CLI installs a .agents/ folder into your project. That folder contains the InsForge SDK documentation, API patterns, and auth setup in a format the agent can read directly. Before the agent writes a single file, it reads that folder. This is why the build prompt can stay short. The agent already knows how to talk to InsForge before you type anything.

The AI gateway: InsForge manages the AI provider keys automatically on the backend, you don’t need to put an OpenRouter key in your frontend .env file. From there, any AI call in your frontend app using the SDK hits one InsForge endpoint and passes a model string. To swap models, you simply change that string; nothing else in the codebase needs to be touched. The backend gateway securely routes the request through OpenRouter, supporting models from OpenAI, Anthropic, Google, DeepSeek, X-AI, and more.

The PostgREST layer: Every table in your InsForge database is automatically a REST endpoint. The agent writes queries against the InsForge SDK. There is no data access layer to build, no custom API routes to wire up. You describe the schema, and the endpoints are there.

Setting Up the Project

Every InsForge project starts with the CLI. Install it once globally, log in, and you are set for every project you build after this.

npm install -g @insforge/cli
insforge login

That is a one-time setup. From here on, every time you want to start a new project, you create a folder and run insforge create inside it.

mkdir ripple
cd ripple
insforge create

The CLI asks you to pick a template. We picked Next.js. After that, it installs the Agent Skills, writes skills-lock.json, and asks if you want to set up deployment now. Say no for now. We will come back to that at the end.

One more thing, before you start building, install the MCP server. The quickest way is through the InsForge VS Code extension. Install it from the marketplace, and it will show you a one-click option to connect the MCP. Once done, you will see the MCP Connected indicator in the top-right corner of your InsForge dashboard, and your agent is ready to act.

Designing with Google Stitch

Before writing a single prompt, we used Google Stitch to design the UI for Ripple. We used this prompt to get started:

Build a social media app called Ripple. Amber gold (#F59E0B) as the primary color.
Screens: feed, composer, profile, notifications, Wave AI chat, auth screens.

Stitch exports a design.md file with the full design system, colors, typography, component structure, and screen layouts.

Copy that file and save it in your project root in VS Code. When you reference it in your agent prompt, the agent has all the visual context it needs upfront, so you are not going back and forth on colors or layout later.

With the design in place, we had everything the agent needed to build the UI and wire the backend in a single pass. Time to write the prompt.

Building the App

We used GitHub Copilot as the agent, running MiniMax M2.7*,* because it handles long multi-step tasks well and stays on track across a full project build. We gave it one prompt:

Build a social media app called Ripple using InsForge as the backend platform.
Use InsForge MCP Server for all operations.

Features:
- Auth: sign up, login with name, @handle, email, password
- Feed: post (called Ripple) with text + image/video upload, like (Wave),
  repost (Spread), reply, bookmark
- Realtime feed updates
- Post composer with draft save
- Profile page with cover, avatar, bio, followers/following
- Notifications: likes, replies, follows, mentions
- Explore: trending topics, suggested users
- Wave AI: chat interface connected to InsForge AI gateway via OpenRouter
- Wave AI has collapsible right panel with chat history and bookmarks
- Deploy on InsForge

Follow the design system in design.md for colors, typography, and components.
Use InsForge for all backend. Read .agents folder for skills.

Before touching any file, the agent read .agents/skills/insforge/ to understand the InsForge SDK, then laid out the full database schema profiles, ripples, waves, spreads, follows, notifications, drafts, ai_chat_history, and more, and created a build plan for itself. Only after that did it start writing code.

The file structure was produced in one pass:

ripple/
├── src/app/
│   ├── feed/page.tsx
│   ├── profile/[handle]/page.tsx
│   ├── notifications/page.tsx
│   ├── ripple/[id]/page.tsx
│   └── wave/page.tsx
├── src/components/
│   ├── ripple/RippleCard.tsx
│   └── ripple/RippleComposer.tsx
├── src/lib/
│   ├── insforge.ts
│   └── auth-context.tsx
└── .agents/
    └── skills/insforge/

What is worth noticing here is that insforge.ts is not a custom wrapper; the agent read the skill and knew exactly how to initialize the InsForge client.

Same with auth-context.tsx: it wired sessions directly to InsForge Auth without any manual setup from us. Now, all of this came from one prompt. But what the agent actually built inside each of these files, how it handled auth sessions, how it wired realtime, how AI talks to the InsForge gateway, that is where things get interesting. So let’s see exactly what agent built inside each of those files.

What InsForge Handled, Feature by Feature

Let's start with auth, since that is what everything else depends on.

Auth

Authentication in Ripple runs entirely through InsForge Auth: sign up, email verification, login, and session management. All the state stays in a single React Context, the agent is generated and wired up in one pass.

Sign-up is a two-step flow. The agent calls insforge.auth.signUp(), checks if email verification is required, and stores the pending profile in localStorage until the OTP is confirmed. Once verified, it inserts the profiles record using the authenticated user ID.

const { data } = await insforge.auth.signUp({ email, password, name });
if (data?.requireEmailVerification) {
  localStorage.setItem("ripple_pending_name", name);
  return { requireEmailVerification: true };
}

// after OTP confirmed
await insforge.database.from("profiles").insert([{
  id: data.user.id, name: pendingName, handle: pendingHandle, email,
}]);

The signup page tracks a step state that switches between the form and the verify screen. Login is simpler, one call to insforge.auth.signInWithPassword() and the session is set.

Database

With auth out of the way, the agent moved on to the database. InsForge runs on PostgreSQL under the hood, and every table gets automatically exposed as a REST endpoint through PostgREST, which means the agent never had to write a single custom API route.

The schema the agent built covers the full surface area of the app. The core tables are profiles, ripples, ripples_media, waves (likes), spreads (reposts), follows, notifications, and ai_chat_history for the Wave AI sessions. The notifications table also has a Postgres trigger attached to it, so every insert immediately fires a real-time broadcast over the WebSocket.

-- setup_trigger.sql
CREATE OR REPLACE FUNCTION notify_new_notification()
RETURNS TRIGGER AS $$
BEGIN
  PERFORM pg_notify(
    'new_notification',
    json_build_object(
      'id', NEW.id,
      'user_id', NEW.user_id,
      'type', NEW.type,
      'actor_id', NEW.actor_id,
      'ripple_id', NEW.ripple_id
    )::text
  );
  RETURN NEW;
END;
$$ LANGUAGE plpgsql;

Because PostgREST understands foreign keys, the agent could request deeply nested relational data in a single query rather than chaining multiple fetches. Here is the feed query from page.tsx, which pulls ripples alongside their author profiles, attached media, waves, and spreads in one request:

// page.tsx — fetching the main feed
const { data, error } = await insforge.database
  .from("ripples")
  .select(`
    *,
    profiles (*),
    ripples_media (*),
    waves (*),
    spreads (*)
  `)
  .is("reply_to", null)
  .order("created_at", { ascending: false })
  .limit(50);

The profiles (*) in the select string is where PostgREST detects the foreign key between ripples.user_id and profiles.id and performs the join automatically on the backend, returning the author's data nested inside each post object. The same pattern applies to waves and spreads, so the UI always knows the engagement state of a post without a second request.

Storage

The database takes care of structured data, but for every file, avatars, cover photos, and post media, the agent provisioned three separate InsForge Storage buckets: avatars, covers, and ripples.

InsForge Storage is S3-compatible and sits natively beside the auth and database layers, so the SDK handles uploads, hashing, and public URL generation in a single call without any custom middleware.

For profile photos, the agent used .uploadAuto(), which takes a File object and returns a public URL directly. After each upload resolves, it immediately writes that URL back to the profiles table in the database.

// profile/edit/page.tsx
if (avatar) {
  const { data, error } = await insforge.storage
    .from("avatars")
    .uploadAuto(avatar);
  if (error) throw error;
  avatarUrl = data.url;
}

if (cover) {
  const { data, error } = await insforge.storage
    .from("covers")
    .uploadAuto(cover);
  if (error) throw error;
  coverUrl = data.url;
}

await insforge.database
  .from("profiles")
  .update({ avatar_url: avatarUrl, cover_url: coverUrl })
  .eq("id", user.id);

For post media, the pattern is slightly different because a single ripple can carry up to four attached images. The agent wrote a loop that runs after the post row is created, uploads each file to the ripples bucket, and inserts a matching record into ripples_media that binds the file URL back to the post's ID.

// RippleComposer.tsx
for (const file of media) {
  const { data: uploadData, error } = await insforge.storage
    .from("ripples")
    .uploadAuto(file);

  if (error || !uploadData) throw error;

  await insforge.database.from("ripples_media").insert([{
    ripple_id: ripple.id,
    bucket: uploadData.bucket,
    key: uploadData.key,
    url: uploadData.url,
    type: file.type.startsWith("video") ? "video" : "image",
  }]);
}

Realtime

With the database writing data correctly, the next thing the app needed was for that data to reach every connected client without a page refresh. InsForge Realtime runs on WebSockets, and the agent wired up four live channels: ripples for the global feed, ripples_media for media updates, trending_topics for live topic changes, and notifications:% for per-user alerts.

The feed component subscribes to ripples on mount and handles two event types: INSERT for new posts and UPDATE for engagement count changes.

// page.tsx — feed subscription
const response = await insforge.realtime.subscribe("ripples");

if (response.ok) {
  insforge.realtime.on("INSERT", async (payload) => {
    const newRipple = payload.new as Ripple;
    if (!newRipple.reply_to) {
      const { data } = await insforge.database
        .from("ripples")
        .select("*, profiles (*), ripples_media (*), waves (*), spreads (*)")
        .eq("id", newRipple.id)
        .single();
      if (data) setRipples((prev) => [data as Ripple, ...prev]);
    }
  });

  insforge.realtime.on("UPDATE", (payload) => {
    const updated = payload.new as Ripple;
    setRipples((prev) =>
      prev.map((r) =>
        r.id === updated.id
          ? { ...r, wave_count: updated.wave_count, spread_count: updated.spread_count }
          : r
      )
    );
  });
}

The INSERT payload only carries the raw new row, so the agent does a quick .select() with joins before pushing it into the state, same pattern as the feed query from the Database section. The UPDATE handler just patches the counts in-place without refetching the full post.

For notifications, the Postgres trigger from setup_trigger.sql does the broadcasting on the database side. When a new row hits the notifications table, the trigger publishes directly to notifications:{user_id} over the WebSocket, and realtime-context.tsx picks it up on the frontend to increment the bell count instantly.

AI Gateway

After real-time, the agent focused on the AI gateway. Ripple comes with an in-app AI called Wave AI. Instead of connecting directly to OpenAI or Anthropic, dealing with separate billing, and worrying about exposing keys on the frontend, it uses InsForge's AI gateway. This gateway manages routing, authentication, and model access all in one place.

The gateway call follows the same shape as the OpenAI SDK, so it feels familiar:

const completion = await insforge.ai.chat.completions.create({
  model: "anthropic/claude-sonnet-4-5",
  messages: [
    {
      role: "system",
      content: `You are Wave AI, a helpful assistant on the Ripple social platform. You are talking to ${profile?.name || "a user"}.`,
    },
    ...pastMessages,
    { role: "user", content: userMessage.content },
  ],
});

const reply = completion.choices[0]?.message?.content;

Swapping the model is one line, change "anthropic/claude-sonnet-4-5" to "openai/gpt-4o" or any other model the gateway supports, and nothing else changes.

One thing the AI gateway doesn't do on its own is remember past conversations. Every time you call insforge.ai.chat.completions.create, it only knows what's in the messages array you pass. Close the tab and that's gone.

So the agent added two database tables, ai_chat_sessions to group conversations, and ai_chat_history to store individual messages. Every time the user sends a message, it gets written to the database. Every time Wave AI replies, that gets written too. When you come back to the /ai page later, a useEffect fetches that session and loads all the messages back in. The conversation picks up exactly where it left off.

With auth, database, storage, realtime, and Wave AI all working, the app is ready to run, and it's time to test it locally.

Running Locally

Before deploying, run the app locally to make sure everything works end-to-end.

npm install
npm run dev

Open http://localhost:3000, sign up with a test account, go through the full flow, auth, posting a Ripple, checking the feed. If your InsForge credentials are in .env.local, everything should work on the first run.

Note: InsForge takes care of all the backend wiring, but the frontend is yours to shape. I made a few tweaks to the UI, adjusting the layout and refining some interactions, to make it feel more like my own, and that is the nice part: the backend is handled, so you can spend your time on the experience instead.

Deploying from GitHub Copilot

Once everything worked locally, deploying took one prompt:

Deploy to InsForge.

The agent read the project config, picked up the credentials, and called the create-deployment MCP tool, all from inside Copilot. No browser dashboard, no separate deploy config.

It zipped the source, uploaded it, and InsForge ran npm install and npm run build in a container with the environment variables injected. The live URL came back in the terminal.

What's Next

At this point, you have a fully working social platform running on InsForge. Auth, a real-time feed, media uploads, notifications, and an AI assistant, all live and deployable from inside GitHub Copilot.

From here, the project is yours to extend. Swap the AI model by updating one value in the InsForge dashboard. Replace the schema entirely, and the same SDK patterns, the same Agent Skills, and the same deployment flow all carry forward to whatever you build next. You can fork the project repo and start from there.

To learn more about InsForge, check out the GitHub repo.

Managing Multi Provider AI Workflows in the Terminal with Bifrost CLI

Astrodevil — Sat, 21 Mar 2026 10:52:18 +0000

Command-line tools are still a common way to work with AI. They give better control and fit naturally into everyday workflows, which is why many people continue to use them.

A common issue with CLI-based tools is that they are often tied to a single provider. Switching between options usually means updating configs and handling multiple API keys. In some cases, it may even involve changing tools. This can slow things down and make everyday work feel a bit frustrating.

Bifrost CLI aims to simplify this setup. It provides a single way to connect CLI tools to multiple providers, without changing how the tools are used. In this article, let us look at how it works and how to get started.

What is Bifrost CLI

Bifrost is an open-source AI gateway that works between applications and model providers. It offers provider-compatible endpoints such as OpenAI, Anthropic, and Gemini formats. It manages request routing, API keys, and response formatting in one place, so separate setups for each provider are not required.

Bifrost CLI was recently released to extend this setup to command-line workflows. It allows existing CLI tools to connect through the Bifrost gateway in place of calling providers directly. The CLI tool continues to work in the same way, with only the endpoint updated.

The CLI tool is configured with Bifrost as the base URL. After this, all requests go through the gateway. Bifrost routes each request to the selected provider, converts it into the required API format, and returns a compatible response. The CLI workflow stays the same, with support for multiple providers through a single endpoint.

Key Features of Bifrost CLI

Bifrost CLI brings several practical features that improve how CLI-based workflows are set up and managed:

Automatic Setup for CLI Tools: Configures base URLs, API keys, and model settings for each agent. This reduces manual steps and keeps the environment ready to use.
Model Discovery from Gateway: Fetches available models directly from the Bifrost gateway using the /v1/models endpoint. This ensures the CLI always reflects the current set of available options.
MCP Integration for Tool Access: Attaches Bifrost’s MCP server to tools like Claude Code. This allows access to external tools and extended capabilities from within the CLI.
Session Activity Indicators: Displays activity badges for each tab. It becomes easy to see if a session is running, idle, or has triggered an alert.
Secure Credential Storage: Stores selections and keys securely. Virtual keys are saved in the OS keyring and are not written in plain text on disk.

Getting Started

Bifrost CLI is quick to set up and runs directly from the terminal. The flow includes starting the gateway, launching the CLI, and selecting the agent and model through a guided setup.

Step 1: Start the Bifrost Gateway

Make sure the gateway is running locally (default: http://localhost:8080):

npx -y @maximhq/bifrost

Step 2: Install and Launch Bifrost CLI

In a new terminal, run:

npx -y @maximhq/bifrost-cli

If installed, you can run:

bifrost

Step 3: Enter Gateway Details

Provide the Bifrost endpoint URL.

For local setup, this is usually: http://localhost:8080

If authentication is enabled, you can also enter a virtual key at this stage.

Step 4: Choose a CLI Agent

Select the CLI agent you want to use, such as:

Codex CLI
Claude Code
Gemini CLI
Opencode

The CLI shows which agents are available and can install missing ones during setup.

Step 5: Select a Model

The CLI fetches available models from the gateway and shows them in a searchable list.

You can choose one directly or enter a model name manually.

Step 6: Launch the Session

Review the configuration and start the session. The selected agent runs with the chosen model and setup.

Step 7: Work with Sessions

After launch, the CLI stays open in a tabbed interface.

You can open new sessions, switch between them, or close them without restarting the CLI. Each tab shows the current activity state.

Understanding the Bifrost CLI Session Flow

Bifrost CLI is built for repeated, session-based use in the terminal. You can switch between runs, update settings, and continue your work without having to go through the full setup again each time.

Here are the key steps in the session flow:

Launch: Select the agent and model, then start the session.
Work: Use the agent as usual. All requests go through Bifrost.
Switch Sessions: Press Ctrl + B to open the tab bar, switch between sessions, or start a new one.
Return: When a session ends, the CLI returns to the setup screen with the previous configuration.
Relaunch: Change the agent or model, or rerun the same setup.
Persistence: The last configuration is saved and shown the next time the CLI starts.

Working with Multiple Models

Bifrost CLI makes it easy to work with different models from the same setup. You do not need to change configurations or restart the tool each time you want to try a different option.

During setup, the CLI fetches available models from the Bifrost gateway and shows them in a list. You can select one directly or enter a model name if you already know what you want to use.

If you want to try another model, you can start a new session and choose a different one. Each session runs separately, so you can compare outputs or test different setups side by side.

All requests go through Bifrost, so differences between providers are handled in the background. The CLI experience stays the same across models.

When to Use Bifrost CLI

Bifrost CLI is useful when working with multiple providers or running repeated sessions from the terminal. Since it is built on top of Bifrost, it also brings the benefits of a central gateway into CLI workflows.

Testing Different Models: Try different models across providers from the same setup.
Running Iterative Sessions: Start, stop, and relaunch sessions with minor configuration changes.
Working from the Terminal: Keep the entire workflow inside the CLI, with Bifrost handling routing in the background.
Comparing Outputs: Run multiple sessions side by side and observe how different models respond.
Managing Multiple Providers: Use Bifrost as a single entry point to work across providers in one place.
Centralized Control with Bifrost: Route all requests through Bifrost for consistent handling of API keys, requests, and responses.

This setup helps keep workflows consistent and organized across different providers and sessions.

Conclusion

Bifrost CLI brings multi-provider access into the terminal through a single setup. It keeps existing workflows intact and reduces the need to manage separate configurations.

You can run sessions, switch agents, and try different models from the same interface, with Bifrost handling routing and integration in the background.

To get started or explore more details, check the Bifrost CLI documentation.

Why Your OpenClaw Agent Gets Slower and More Expensive Over Time

Astrodevil — Fri, 20 Mar 2026 21:00:34 +0000

Introduction

OpenClaw feels fast in the first week. You send a message, the agent responds, and the workflow makes sense. Then gradually, without any obvious change, responses take a little longer, and the API bill at the end of the month is higher than it was two weeks ago, with no single thing you can point to as the cause.

That is not a coincidence, and it is not bad luck. It is what happens when three separate problems compound on each other quietly, over time, without any of them being obvious on its own.

Context bloating, static content being reprocessed on every call, and every request hitting the same model regardless of what it actually needs, these are not dramatic failures. They are the kind of inefficiencies that feel invisible until they are not, and by the time the invoice makes them obvious, they have been running for weeks.

In this post, we will break down what is driving each of them and why routing, not prompt tuning or model switching, is the fix that addresses all three at the layer where they actually live.

Why the Default Setup Works Against You Over Time

OpenClaw's default configuration is built to get you started. It is not designed to remain efficient as your usage grows, and the gap between the two becomes apparent faster than most people expect. Three things are responsible for most of it.

Context grows faster than you think

Before you type a single message, your agent has already loaded a significant amount into the context window. SOUL.md, AGENTS.md, bootstrap files, the results of a memory search against everything you have accumulated, all of it lands in the prompt before your request even starts.

That base footprint is manageable in week one. By week three, the memory graph has grown, the search results are broader, and the conversation history from your previous sessions is traveling with every new request. The agent is not selectively pulling relevant data; it loads everything it has access to every time.

The result is a base token cost per request that is meaningfully higher than it was when you started, without any deliberate change on your part.

Static tokens are processed fresh every time

A large portion of what is loaded into every request consists of content that has not changed since last week, system instructions, bootstrap files, and agent configuration. Provider-side caching exists specifically to avoid paying full price for static content on repeat calls, but the default OpenClaw setup does not use it.

The same unchanged content, reprocessed from scratch on every heartbeat call.

Every call processes that unchanged content from scratch. For a setup running a 30-minute heartbeat, that means a full API call with no caching, hitting the configured model, every half hour, regardless of whether anything meaningful is happening in the session. Most users never think of the heartbeat as a cost source, but over a full month, it adds up to a figure worth paying attention to.

Every request hits the same model

OpenClaw routes all requests to a single globally configured model. There is no built-in distinction among task types: a status check, a memory lookup, a formatting task, and a multi-step reasoning problem all map to the same endpoint at the same price.

In practice, the majority of what an agent handles day-to-day is simple work. Summaries, lookups, structured output, short responses. None of it requires a frontier model, but all of it gets one anyway. That is not a usage problem; it is a configuration gap, and it is the highest-leverage thing to fix.

The Structural Fix: Routing

The problem with every approach people try first, switching to a cheaper model, trimming prompts, and reducing heartbeat frequency, is that they address one variable at a time. The bill declines slightly, then rises again. What is needed is a layer that sits between OpenClaw and the provider, evaluates each request before it is sent, and determines which model to route it to. That is what routing is, and that is why it is a structural fix rather than a configuration tweak.

That layer is Manifest, an open-source OpenClaw plugin built specifically to solve this. It sits between your agent and the provider, and the original OpenClaw configuration remains unchanged.

Manifest intercepts every request before it reaches the LLM. The routing decision takes under 2 ms with zero external calls, after which the request is forwarded to the appropriate model. During that interval, five distinct mechanisms run before the request moves anywhere, starting with how the scoring algorithm decides which tier a request belongs to.

How the scoring algorithm works

Before any request leaves your setup, Manifest runs a scoring pass across 23 dimensions. These dimensions fall into two groups:

13 keyword-based checks that scan the prompt for patterns like "prove", "write function", or "what is", and
10 structural checks that evaluate token count, nesting depth, code-to-prose ratio, tool count, and conversation depth, among others.

Each dimension carries a weight. The weighted sum maps to one of four tiers through threshold boundaries. Alongside the tier assignment, Manifest produces a confidence score between 0 and 1 that reflects how clearly the request fits that tier.

How Manifest scores a request across 23 dimensions and assigns it a tier in under 2 ms.

One edge case worth knowing: short follow-up messages like "yes" or "do it" do not get scored in isolation. Manifest tracks the last 5 tier assignments within a 30-minute window and uses that session momentum to keep follow-ups at the right tier, rather than dropping them to simple because they contain almost no content.

Certain signals also force a minimum tier regardless of score. Detected tools push the floor to the standard. Context above 50,000 tokens forces complex. Formal logic keywords move the request directly to reasoning.

The four tiers and what they route

The tier system is where the cost reduction actually happens. Manifest defines four tiers, each mapped to a different class of model:

Simple: greetings, definitions, short factual questions. Routed to the cheapest model.
Standard: general coding help, moderate questions. Good quality at low cost.
Complex: multi-step tasks, large context, code generation. Best quality models.
Reasoning: formal logic, proofs, math, multi-constraint problems. Reasoning-capable models only.

In a typical active session, most requests fall into the simple or standard category. Routing those away from frontier models, while sending only what genuinely needs it to complex or reasoning, is where the up to 70% cost reduction reported by users comes from.

How Manifest maps each request type to the cheapest model that can handle it.

Every routed response returns three headers you can inspect: X-Manifest-Tier, X-Manifest-Model, and X-Manifest-Confidence. If a request was routed differently than you expected, those headers tell you exactly what the algorithm saw.

OAuth and provider auth

Manifest lets users authenticate with their own Anthropic or OpenAI credentials directly through OAuth. If OAuth is unavailable or a session is inactive, it falls back to an API key.

Manifest lets users authenticate with their own Anthropic or OpenAI credentials

This keeps your model access under your own account, which matters for rate limits, spend visibility, and not routing your traffic through a third-party proxy. More providers are being added.

Fallbacks and what they protect

Each tier supports up to 5 fallback models. If the primary model for a tier is unavailable or rate-limited, Manifest automatically moves to the fallback chain. The request still resolves, just against the next available model in that tier's list. This is particularly relevant for the reasoning tier, where model availability can be less predictable during high-traffic periods, and losing a request entirely is more costly than a slight capability downgrade.

Spend limits without manual monitoring

Manifest lets you set rules per agent against two metrics: tokens and cost. Each rule has a period (hourly, daily, weekly, or monthly), a threshold, and an action. Notify sends an email alert when the threshold is crossed. Block returns HTTP 429 and stops requests until the period resets.

Rules that block are evaluated on every ingest, while rules that notify run on an hourly cron and fire once per rule per period to avoid repeated alerts for the same breach. For a setup with a 30-minute heartbeat running continuously, a daily cost block is the most direct way to prevent a runaway spend event from compounding overnight without any manual check.

The Rest Is Worth Knowing

Routing is the core of what Manifest does, but it ships with a few other things that are worth understanding before you use it in production.

Manifest provides a dashboard that gives a full view of each call: input tokens, output tokens, cache-read tokens, cost, latency, model, and routing tier. Cost is calculated against a live pricing table covering 600+ models, so nothing is estimated. The message log stores all requests and is filterable by agent, model, and time range.

Manifest dashboard

In local mode, nothing leaves your machine. In cloud mode, only OpenTelemetry metadata is sent: model name, token counts, and latency. Message content never moves. The full codebase is open source and self-hostable at github.com/mnfst/manifest, and the routing logic is fully documented.

A quick note before we move on.

Everything in this post reflects how Manifest works at the time of writing, and the space is moving fast enough that some details may already look different by the time you read it. The OAuth providers, the supported models, the scoring thresholds, and the team were shipping changes even while this article was being written. For anything that has moved since, the docs are the right place to check.

With that said, back to the article. Here is how all of it fits together.

Putting It Together

The three problems do not take turns. They compound on the same request, every time.

Three problems converging into every single request, all at once.

A heartbeat call on a 30-minute cycle loads accumulated context, reprocesses unchanged system files, and hits a frontier model for a task that needed none of that. Week one is a small number. In week three, it is a pattern you cannot see until the invoice lands.

Routing is the layer that addresses all three at once, not because it solves context or caching directly, but because it changes the cost of every request before it leaves your setup, and once that layer is in place, the three problems no longer have room to compound.

Where to Start

The order matters here. Do not start by switching models or trimming prompts.

Install Manifest and let it run for a few days without changing anything else. The dashboard will show you where the cost is actually coming from.
Check the model distribution. If simple and standard requests are hitting your highest-tier model, routing is the first thing to configure.
Set a daily cost block rule to prevent a runaway session from compounding overnight.
Once routing is active, the cache read token metric indicates how much static content was served from cache versus processed fresh. That number is worth watching.
Add per-tier fallbacks to prevent availability gaps from interrupting the session.

The Manifest docs cover installation, routing configuration, and limit setup in full. If you want the broader context on what makes OpenClaw production-ready, this post is a good place to start.

Building a Production-Ready Multi-Agent Investment Committee with AgentField

Astrodevil — Thu, 19 Mar 2026 11:49:06 +0000

TL;DR

This tutorial walks through building Argus, a multi-agent system that performs automated stock research.
Using AgentField, agents run as modular microservices with typed skills and reasoners.
The architecture enables parallel analysis, structured workflows, and full observability for production-ready AI systems.

Introduction

Many early Agentic applications start with high-level orchestrators like LangChain or CrewAI. For simple use cases, these frameworks work well and are often the fastest way to prototype an idea.

However, as the complexity of the task grows, especially when moving from a notebook to a production service, this pattern begins to break down.

Traditional agent frameworks often focus on "agentic reasoning" but neglect the "production engineering." In a real-world system, you can't just have one model or orchestrator responsible for multiple stages of a workflow in a sequential, opaque loop. Failures are harder to trace because the entire workflow is coupled to a single orchestration logic. Evaluating or improving individual stages becomes difficult without a clear separation of concerns.

A more robust alternative is to structure the system as a set of specialized, independent components.

In this tutorial we will build Argus, an AI agent system that performs stock research using a coordinated set of specialized agents. Argus operates similarly to an investment committee: multiple agents analyze the same company from different perspectives and their outputs are combined into a structured research report.

Argus is built using AgentField, an open-source backend framework designed for building and orchestrating AI agents as production services.

Why Single-Prompt Systems Fail in Production

Many early AI applications rely on a single prompt that attempts to complete an entire task in one step. This pattern is simple to prototype but introduces several issues in production systems.

High hallucination risk: When a single prompt performs research, reasoning, and reporting at the same time, the model lacks reliable grounding.
Limited observability: If the entire workflow occurs inside one prompt, it becomes difficult to trace how results were produced.
Poor separation of responsibilities: Research, analysis, and synthesis happen in the same step. This makes systems difficult to maintain or extend.
Difficult debugging: Failures cannot easily be isolated to a specific stage of the workflow. For production AI systems, structured workflows provide a more stable foundation.

Understanding autonomous agents workflow

What is AgentField?

AgentField is the Control Plane for AI Agents. If other frameworks focus on "how an agent thinks," AgentField focuses on how agents run, scale, and communicate in a production environment. Think of it as Kubernetes for AI Agents.

AgentField isn't just an orchestration layer; it transforms your agents into production-ready microservices. It is built on three core pillars:

Agents: The fundamental unit of deployment. Each Agent is an independent, versioned microservice with its own lifecycle, endpoints, and health monitoring.
Skills: Deterministic tools (fetching APIs, database queries, file system access). Skills are strictly typed and can be exposed as standalone REST endpoints.
Reasoners: The "brains" that use LLMs to process data. Reasoners use structured data contracts (Pydantic) to ensure outputs are predictable and reliable.

When these concerns are separated, AgentField provides:

Built-in Observability: Every execution, skill call, and reasoning step is traced automatically.
Async Concurrency: Native support for asyncio allows agents to run in parallel without the complex state management of traditional frameworks.
Production Hub: A unified dashboard (the Control Plane) where you can monitor health, latency, and costs across multiple agent clusters.

We will now have a walkthrough on how Argus works and how one can set it up following this guide.

The 5-Agent Architecture

Building a production AI system isn't just about the "brain", it's about the workflow. Argus doesn't rely on a single, long-running agent. Instead, it operates as a coordinated investment committee, where specialized agents handle specific stages of the research pipeline.

This multi-agent design allows for true concurrency: while the Analyst builds the bull case, the Contrarian simultaneously hunts for risks. This parallel execution minimizes latency and ensures that each perspective is developed independently, without bias from the other.

Before we write any code, let's look at the orchestration flow:

Argus’s 5-agent investment committee working in parallel

Key insight: The Analyst and Contrarian run in parallel. Then both Editors run in parallel. This is true concurrency via asyncio.gather, not sequential execution.

The next step is to start building the our project. We have a GitHub repository for the project, so to keep the code here concise, you can always refer to it for the complete implementation.

Project Organization

We will build Argus with a modular design. Every file has a specific responsibility. Create a directory named argus-agentfield and set up the following structure.

argus-agentfield/
├── .env                 # API Keys (NEBIUS_API_KEY)
├── requirements.txt     # Dependencies
├── src/
│   ├── __init__.py      # App initialization — shared Agent instance
│   ├── schemas.py       # Data contracts (Pydantic models)
│   ├── skills.py        # Deterministic data fetching (yfinance)
│   ├── reasoners.py     # Agent logic, prompts, orchestration
│   ├── stream.py        # SSE streaming pipeline + UI routes
│   └── main.py          # Server entry point
└── ui/
    └── index.html       # Single-page vanilla JS frontend

Dependencies

Create requirements.txt:

agentfield
yfinance
nebius
python-dotenv
pydantic>=2.0

Install them with uv (fast Python package manager):

uv venv && uv pip install -r requirements.txt

AgentField Control Plane

AgentField includes a local control plane that gives you a live dashboard to monitor your agents. Install the af CLI:

curl -sSf https://agentfield.ai/get | sh

You don’t need this to build or run Argus — the agent works fully standalone. But once you want to see workflow graphs, execution traces, and performance metrics, you can start the control plane with af server. We will set this up in Section 8. After a successful installation, you can confirm it with this command:

af --version

A terminal showing Agentfield CLI installation and version verification

Initialization

We start by creating a shared Agent instance. This object manages our LLM configuration and serves as a production hub. Every agent in AgentField is a standard microservice.

Create src/__init__.py:

"""
Argus — Autonomous Research Agent
Exports the shared `app` Agent instance used across all modules.

Authentication is handled automatically via environment variables:
  NEBIUS_API_KEY       — used by AgentField/LiteLLM for all app.ai() calls
  AGENTFIELD_SERVER    — control plane URL (default: http://localhost:8080)
"""
import os

from agentfield import Agent, AIConfig
from dotenv import load_dotenv

load_dotenv()

app = Agent(
    node_id="argus-research-agent",
    # LiteLLM requires the provider prefix: nebius/<model>
    ai_config=AIConfig(model="nebius/openai/gpt-oss-120b"),
    # Connect to the AgentField control plane for dashboard visibility
    agentfield_server=os.getenv("AGENTFIELD_SERVER", "http://localhost:8080"),
)

The agentfield_server parameter tells the agent where to find the control plane. On startup, the agent registers itself, reporting its reasoners, skills, and health. If you are not running a control plane, the agent works fully standalone with no issues.

Note: You can use any other model API keys, such as GPT or Claude, instead of Nebius. We use Nebius because it provides access to a collection of several good open models in one place.

Data Contracts (The Schemas)

Every interaction in AgentField is structured. We use Pydantic models to define our data contracts. This prevents the “hallucination tax” where LLMs return unpredictable strings.

Create src/schemas.py:

"""
schemas.py — Pydantic models for the Argus Investment Committee pipeline.

Data flow (streaming — SSE pipeline in stream.py):
  User Query
    → ResearchPlan         (Manager)
    → AnalystFinding       (Analyst) ─┬─ parallel
    → RiskAssessment       (Contrarian)─┘
    → ResearchReport × 2  (EditorShort ‖ EditorLong, parallel) → DualResearchReport
"""
from pydantic import BaseModel, Field
from typing import Literal

class ResearchPlan(BaseModel):
    """The Manager's decomposition of a user query into a research plan."""

    reasoning_steps: list[str] = Field(
        description="Step-by-step reasoning: how you interpreted the query, why you chose this ticker, key assumptions"
    )
    ticker: str = Field(description="Stock ticker symbol, e.g. AAPL")
    company_name: str = Field(description="Full company name")
    hypotheses: list[str] = Field(
        description="2-4 key hypotheses to investigate (bull and bear)"
    )
    data_needs: list[str] = Field(
        description="List of data points needed to validate the hypotheses"
    )
    focus_areas: list[str] = Field(
        description="Specific areas for deep-dive: e.g. revenue growth, debt levels, competitive moat"
    )

# ─────────────────────────────────────────────────────────────────────────────
# Additional schemas follow the same pattern. Full implementations in repo:
# ─────────────────────────────────────────────────────────────────────────────

Notice how each schema includes reasoning_steps. This is Chain-of-Thought prompting built into the data contract. The LLM must show its work before giving an answer.

Next thing to setup are the AgentField Skills.

Skills (The Facts)

Skills are deterministic functions. They provide the facts that agents use to make decisions. In AgentField, every Skill is automatically registered as a REST endpoint and can be called directly by Reasoners.

Argus representation of Agentfields skills and how they run in parallel

We’ll use yfinance for all data and it’s free, needs no API key.

Create src/skills.py and start with the imports and a helper function:

"""
skills.py — Deterministic data-fetching tools for the Argus agent.

All skills use yfinance (free, no API key needed) to pull real financial data
from Yahoo Finance. They are registered as @app.skill decorators so AgentField
exposes them as REST endpoints AND the Reasoners can call them directly.

Skills intentionally return plain JSON-serialisable dicts/lists so the LLM
can reason over them without needing to understand yfinance objects.
"""
import asyncio
from typing import Optional

import yfinance as yf

from src import app

def _df_to_records(df) -> dict:
    """Convert a pandas DataFrame (yfinance financials) to a clean dict."""
    if df is None or df.empty:
        return {}
    # Transpose so rows = metrics, cols = dates; convert to string keys
    try:
        df = df.fillna(0)
        return {
            str(col.date()): df[col].to_dict() for col in df.columns
        }
    except Exception:
        return df.to_dict()

The _df_to_records helper converts pandas DataFrames into clean dictionaries that the LLM can understand. Since yfinancereturns financial statements as DataFrames, we convert them into JSON-serializable dictionaries.

Skill 1: Ticker Validation

The first skill validates that a ticker actually exists and is actively trading. This prevents the agents from hallucinating analysis for fake companies:

@app.skill()
async def validate_ticker(ticker: str) -> dict:
    app.note(f"[skill] Validating ticker:{ticker}")

    def _fetch() -> dict:
        t = yf.Ticker(ticker)
        try:
            info = t.info or {}
        except Exception as exc:
            return {
                "valid": False,
                "reason": f"yfinance raised an error:{exc}",
                "current_price": None,
                "quote_type": None,
                "exchange": None,
            }

        if not info:
            return {
                "valid": False,
                "reason": (
                    f"No data returned for '{ticker}'. "
                    "It may be delisted, never listed, private, or misspelled."
                ),
                "current_price": None,
                "quote_type": None,
                "exchange": None,
            }

        price = info.get("regularMarketPrice") or info.get("currentPrice")
        quote_type = info.get("quoteType", "UNKNOWN")
        exchange = info.get("exchange") or info.get("fullExchangeName", "")

        if not price:
            name = info.get("longName") or info.get("shortName") or ticker
            return {
                "valid": False,
                "reason": (
                    f"'{ticker}' ({name}) has no live market price. "
                    "It is likely delisted, suspended, or no longer trading."
                ),
                "current_price": None,
                "quote_type": quote_type,
                "exchange": exchange,
            }

        return {
            "valid": True,
            "reason": "OK",
            "current_price": price,
            "quote_type": quote_type,
            "exchange": exchange,
        }

    return await asyncio.get_event_loop().run_in_executor(None, _fetch)

Skills 2-8: Financial Data & Market Intelligence

The remaining skills follow the same pattern as validate_ticker: wrap synchronous yfinance calls in run_in_executor for async compatibility.

Here’s one example:

@app.skill()
async def get_income_statement(ticker: str, period: str = "annual") -> dict:
    """Fetch income statement data (annual or quarterly)."""
    app.note(f"[skill] Fetching income statement:{ticker} ({period})")

    def _fetch():
        t = yf.Ticker(ticker)
        df = t.income_stmt if period == "annual" else t.quarterly_income_stmt
        return _df_to_records(df)

    return await asyncio.get_event_loop().run_in_executor(None, _fetch)

# ─────────────────────────────────────────────────────────────────────────────
# The remaining skills follow the same pattern. Full implementations in repo:

Why run_in_executor? yfinance is synchronous and makes blocking HTTP calls. Wrapping it in run_in_executor lets us run multiple fetches concurrently without blocking the event loop. Now that we are clear on what AgentField Skills does, we will move on to Reasoners.

Reasoners (The Brains)

A Reasoner is where the agency happens. This is the heart of Argus, the 5-agent investment committee that orchestrates research. This is the reasoning process which the agents use to arrive at their respective decisions. To add that to our agent orchestration engine:

Create src/reasoners.py. Start with imports and a helper function:

"""
reasoners.py — The five-agent Investment Committee for Argus.
Agent roles:
  1. plan_research      → The Manager      (Adaptive Supervisor)
  2. conduct_research   → The Analyst      (Bull Case) ─┬─ parallel
  3. assess_risks       → The Contrarian   (Bear Case) ─┘
  4. editor_short       → Short-Term View  (1–6 month horizon) ─┬─ parallel
  5. editor_long        → Long-Term View   (1–5 year horizon)  ─┘
"""
import asyncio
import json

from src import app
from src.schemas import (
    AnalystFinding,
    DualResearchReport,
    ResearchPlan,
    ResearchReport,
    RiskAssessment,
)
from src.skills import (
    get_analyst_targets,
    get_balance_sheet,
    get_cash_flow_statement,
    get_company_facts,
    get_income_statement,
    get_insider_transactions,
    search_market_news,
    validate_ticker,
)

# ---------------------------------------------------------------------------
# Helper
# ---------------------------------------------------------------------------

def _json(obj) -> str:
    """Compact JSON serialisation for passing data into app.ai() prompts."""
    if hasattr(obj, "model_dump"):
        return json.dumps(obj.model_dump(), default=str, indent=2)
    return json.dumps(obj, default=str, indent=2)

The _json helper serialises Pydantic models to JSON strings for injection into prompts. This is how agents share structured data.

Reasoners 1-3: Editor, Contrarian, Analyst

These three reasoners follow the same pattern: receive structured input, call app.ai() with a system prompt and schema, return a typed Pydantic model. Here are their signatures:

@app.reasoner(path="/research/editor", tags=["committee"])
async def synthesize_report(
    analyst_finding: AnalystFinding,
    risk_assessment: RiskAssessment,
) -> ResearchReport:
    """
    The Editor: synthesizes bull + bear cases into a balanced report.
    In streaming mode (stream.py), replaced by EditorShort + EditorLong.
    """
    # Calls app.ai() with system prompt for balanced synthesis
    # Returns ResearchReport with verdict and confidence
    ...

@app.reasoner(path="/research/contrarian", tags=["committee"])
async def assess_risks(
    plan: ResearchPlan,
    analyst_finding: AnalystFinding,
) -> RiskAssessment:
    """
    The Contrarian: Devil's advocate searching for risks.
    Filters news for risk keywords, challenges the bull thesis.
    """
    # Fetches risk-focused news, filters for negative sentiment
    # Calls app.ai() to identify regulatory, competitive, valuation risks
    ...

@app.reasoner(path="/research/analyst", tags=["committee"])
async def conduct_research(plan: ResearchPlan) -> AnalystFinding:
    """
    The Analyst: builds the bull case from financial data.
    Runs 9 data fetches in parallel via asyncio.gather.
    """
    # asyncio.gather: income, balance, cashflow, facts, news, targets, insiders
    # Calls app.ai() to synthesize bull case thesis
    ...

Reasoner 4: The Manager

The Manager is the entry point and orchestrator. It creates the research plan, validates the ticker, dispatches agents, and includes an adaptive retry loop for quality control:

@app.reasoner(path="/research", tags=["committee"])
async def plan_research(query: str) -> DualResearchReport:
    """
    The Manager: entry point for direct API queries (POST /research).
    Decomposes the query into a ResearchPlan, dispatches Analyst and Contrarian
    in parallel, then runs EditorShort and EditorLong in parallel to produce
    a DualResearchReport.
    """
    # Step 1: Decompose the query into a structured ResearchPlan
    plan = await create_plan(query)

    # Validate ticker before running the full committee
    ticker_check = await validate_ticker(plan.ticker)
    if not ticker_check.get("valid"):
        raise ValueError(f"Cannot analyse {plan.ticker}: {ticker_check.get('reason')}")

    # Step 2: Parallel dispatch → Analyst (bull) + Contrarian (bear)
    analyst_finding, risk_assessment = await asyncio.gather(
        conduct_research(plan),
        assess_risks(plan)
    )

    # Step 3: Run dual editors in parallel
    short_report, long_report = await asyncio.gather(
        editor_short_term(plan, analyst_finding, risk_assessment),
        editor_long_term(plan, analyst_finding, risk_assessment),
    )

    return DualResearchReport(short_term=short_report, long_term=long_report)

Key patterns to notice:

Hybrid Model Strategy: The main analytic agents (Manager, Analyst, Contrarian) use gpt-oss-120b for deep reasoning, while the Editors use gpt-oss-20b for faster final synthesis.
Parallel execution with asyncio.gather: The Analyst and Contrarian run simultaneously, as do both Editors. This significantly reduces total latency.
Decoupled Agency: Each agent focuses on a specific task (bull or bear) without waiting for the other, allowing the Editor to perform a truly independent synthesis.
Structured Tracking: Every call to an @app.reasoner decorated function is automatically visible in the AgentField dashboard.

Streaming Pipeline (The Realtime UX)

In a production AI application, the user experience is defined by responsiveness. A research committee can take 30–60 seconds to complete; forcing a user to stare at a static loading bar is a missed opportunity for engagement.

AgentField allows you to build live, stateful interfaces by transforming your orchestration logic into an asynchronous streaming pipeline. In this section, we'll implement a Server-Sent Events (SSE) system that lets the user follow the research process in real-time as it unfolds.

Instead of just returning a final report, we will build a pipeline that "narrates" its work. Create src/stream.py and start with the core dependencies:

"""
stream.py — Server-Sent Events (SSE) streaming for the Argus UI.

Adds raw FastAPI routes to the AgentField app:
  GET  /          → serves the single-page frontend
  POST /research/stream/start       → starts a session, returns session_id
  GET  /research/stream/events/{id} → streams SSE events

Event types:
  agent_start    — an agent has started working
  agent_note     — a progress log from inside an agent
  agent_complete — an agent has finished, with its structured output
  error          — something went wrong
  complete       — both ResearchReports (short + long term) are ready

Agent identifiers used in events:
  manager, analyst, contrarian, editor_short, editor_long
"""
import asyncio
import json
import uuid
from contextlib import asynccontextmanager
from contextvars import ContextVar
from pathlib import Path
from typing import AsyncGenerator

from fastapi import Request
from fastapi.responses import HTMLResponse, StreamingResponse
from pydantic import BaseModel

from src import app
from src.schemas import (
    AnalystFinding,
    ResearchPlan,
    ResearchReport,
    RiskAssessment,
)
from src.skills import (
    get_analyst_targets,
    get_balance_sheet,
    get_cash_flow_statement,
    get_company_facts,
    get_income_statement,
    get_insider_transactions,
    search_market_news,
    validate_ticker,
)

How the Event-Driven Pipeline Works

In a production environment, users shouldn't wait 30 seconds for a final JSON blob. Argus uses Server-Sent Events (SSE) to provide a live, play-by-play view of the investment committee at work.

Instead of writing a complex state machine, we use a simple Event Bus pattern:

Session Management: Every time a user starts a search, we create a unique session_id and a dedicated asyncio.Queue.
The emit() Function: We place emit() calls throughout our agent logic. This function pushes small JSON payloads (events) into the session's queue.
The SSE Stream: A background task (the "Generator") watches the queue. As soon as an event appears, it formats it for SSE and sends it to the frontend.

The Event Lifecycle

Each agent in the pipeline follows a predictable three-stage lifecycle that lights up the UI:

agent_start: Signals that a specific agent (e.g., The Analyst) has been dispatched.
agent_note: Sends real-time progress logs (e.g., "Fetching balance sheet for AAPL..."). This transforms a "loading spinner" into an interactive experience.
agent_complete: Delivers the final structured data for that specific agent.

Orchestration via Concurrency

The streaming pipeline is just a wrapper around the same logic used in reasoners.py, but it leverages Python's asyncio.gather to drive the UI:

Parallel Research: The Analyst and Contrarian are triggered simultaneously. On the frontend, you see both "cards" start pulsing at the same time.
Decoupled Synthesis: As soon as the research agents finish, the two Editors start in parallel.
Final Handshake: Once all agents are done, a complete event is emitted with the full DualResearchReport.

Full implementation of the streaming logic can be found in the GitHub Repository.

This approach ensures the backend remains the "source of truth" while the frontend stays reactive and responsive.

Setting up FastAPI Routes

Finally, we setup the endpoints that will expose the streaming pipeline for the UI to use:

# ---------------------------------------------------------------------------
# Raw FastAPI routes added directly to the Agent (which is a FastAPI subclass)
# ---------------------------------------------------------------------------

class StreamQuery(BaseModel):
    query: str

@app.post("/research/stream/start")
async def start_stream(body: StreamQuery):
    """Start a streaming research session. Returns a session_id."""
    session_id = str(uuid.uuid4())
    _sessions[session_id] = asyncio.Queue()
    # Run pipeline in background, bound to this session's queue
    token = _current_queue.set(_sessions[session_id])

    async def run():
        try:
            _current_queue.set(_sessions.get(session_id))
            await _run_pipeline(body.query)
        except Exception as e:
            q = _sessions.get(session_id)
            if q:
                await q.put({"type": "error", "agent": "system", "data": {"message": str(e)}})

    asyncio.create_task(run())
    _current_queue.reset(token)
    return {"session_id": session_id}

@app.get("/research/stream/events/{session_id}")
async def stream_events(session_id: str):
    """SSE endpoint — streams events for a given session."""
    return StreamingResponse(
        _event_generator(session_id),
        media_type="text/event-stream",
        headers={
            "Cache-Control": "no-cache",
            "X-Accel-Buffering": "no",
            "Connection": "keep-alive",
        },
    )

@app.get("/", response_class=HTMLResponse)
async def serve_ui():
    """Serve the Argus UI."""
    ui_path = Path(__file__).parent.parent / "ui" / "index.html"
    return HTMLResponse(content=ui_path.read_text())

The frontend workflow:

POST to /research/stream/start → get a session_id
Open EventSource to /research/stream/events/{session_id}
Receive SSE events as JSON until complete or error

SSE events streaming from the backend to the frontend

Setting up the Frontend

The UI is a single-file vanilla JavaScript application (~1400 lines). It connects to the SSE endpoint and animates agent cards as events arrive. Rather than embed the entire file here, get it directly from the repository: ui/index.html

Key features of the UI:

Agent cards — Each of the 5 agents gets a card that glows when active
Live reasoning — A “thought drawer” types out each agent’s chain-of-thought in real time
Tabbed results — Short-term and long-term verdicts appear in separate tabs
SSE connection — Uses EventSource to stream events from /research/stream/events/{id}
Dark theme — Built with CSS custom properties for easy theming

The UI listens for these SSE event types:

agent_start — Card starts glowing
agent_note — Progress update (logs to thought drawer)
agent_complete — Card turns green, reasoning steps revealed
error — Something went wrong
complete — Both reports ready, render the tabbed result

True parallel execution — Analyst and Contrarian running simultaneously

Running Argus and Testing

Now that we've built the research engine and the streaming UI, it’s time to put everything together.

In this section, we'll configure our environment, launch the AgentField dashboard, and start the Argus server so you can see your 5-agent committee in action.

1. Set your keys

Create a .env file:

NEBIUS_API_KEY=your_key_here
AGENTFIELD_SERVER=http://localhost:8080
PORT=8081

2. Start the AgentField Control Plane

If you installed the af CLI in Section 1, start the control plane in its own terminal:

af server

This starts the dashboard at http://localhost:8080/ui. Keep this terminal running - the agent will register with it on startup.

AgentField control-plane dashboard showing agents offline, 100% success rate across 22 executions, and workflow performance charts.

3. Boot the agent

In a new terminal, create src/main.py:

"""
main.py — Entry point for the Argus autonomous research agent.

Usage:
    uv run python3 src/main.py

The agent will start on http://localhost:8081 (separate from the AgentField
control plane on :8080) and expose:
    POST /research          → Full investment committee pipeline (Manager entry point)
    POST /research/analyst  → Analyst (bull case) only
    POST /research/contrarian → Contrarian (bear case) only
    POST /research/editor   → Editor (synthesis) only
    + all /skills/* endpoints

Example query:
    curl -X POST http://localhost:8081/research\
         -H "Content-Type: application/json"\
         -d '{"query": "Should I invest in AAPL?"}'
"""
import os
import sys

# Ensure the project root is on sys.path so `src` is importable
# when running as: python3 src/main.py
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

from dotenv import load_dotenv

# Load .env before importing anything that reads env vars
load_dotenv()

# Import and boot the shared Agent instance
from src import app  # noqa: E402

# Register skills + reasoners by importing their modules.
# The @app.skill / @app.reasoner decorators fire on import.
import src.skills    # noqa: F401, E402
import src.reasoners # noqa: F401, E402
import src.stream    # noqa: F401, E402 — SSE endpoints + UI serving

if __name__ == "__main__":
    port = int(os.environ.get("PORT", 8081))
    print(f"🔬 Argus Research Agent starting on http://localhost:{port}")
    print(f"🎛️  AgentField Control Plane dashboard: http://localhost:8080/ui")
    print("📈 5-Agent Investment Committee:")
    print(f"   POST http://localhost:{port}/research               ← Full pipeline (all 5 agents)")
    print(f"   POST http://localhost:{port}/research/analyst       ← Bull case only")
    print(f"   POST http://localhost:{port}/research/contrarian    ← Bear case only")
    print(f"   POST http://localhost:{port}/research/stream/start  ← SSE streaming (used by UI)")
    print(f"   GET  http://localhost:{port}/                       ← Live UI")
    print()
    app.serve(port=port)

Run it:

uv run python3 src/main.py

You’ll see the agent register with the control plane:

Argus server starting up with all registered endpoints

You can also verify this in the Agentfield dashboard. You should see your agents registered with all the reasoners and skills as shown below:

AgentField control-plane dashboard showing the argus-research-agent node status and a list of registered reasoners and skills.

4. Test the API directly

You can test the full pipeline without the UI:

curl -X POST http://localhost:8081/research \
     -H "Content-Type: application/json" \
     -d '{"query": "Should I invest in NVDA?"}'

Or test individual agents:

# Test the Analyst alone (requires a ResearchPlan as input)
curl -X POST http://localhost:8081/research/analyst \
     -H "Content-Type: application/json" \
     -d '{
       "plan": {
         "reasoning_steps": ["User wants to invest in Apple"],
         "ticker": "AAPL",
         "company_name": "Apple Inc",
         "hypotheses": ["Strong iPhone sales", "Services growth"],
         "data_needs": ["Revenue", "Margins"],
         "focus_areas": ["iPhone", "Services", "Wearables"]
       }
     }'

5. Test from the UI

Navigate to http://localhost:8081 in your browser. Type a query like “Should I invest in NVDA?” and watch the 5-agent committee work in real-time.

Completed research report with dual time horizon verdicts

6. View the Workflow in the AgentField Dashboard

While the agents are running (or after they complete), open the AgentField control plane dashboard at http://localhost:8080/ui.

Navigate to Workflow Executions to see the full execution graph:

AgentField dashboard showing a workflow execution graph with multiple connected nodes (skills and reasoners) and a “Succeeded” status indicator.

Every @app.reasoner() and @app.skill() call appears as a node in the workflow graph. You can trace the exact flow: Manager → Analyst → Contrarian → EditorShort / EditorLong. Each node shows execution time, input data, and output data. Skills called within reasoners (like get_income_statement, get_insider_transactions) appear as child nodes.

This is the observability layer that makes Argus production-ready. Instead of guessing what your agents did, you can trace every step, inspect every input and output, and measure performance, all without writing a single line of instrumentation code.

Why AgentField is different

AgentField is built for the move from “prototypes” to “production”:

Agents as Microservices: Every agent becomes a standard REST API with OpenAPI documentation.
Cryptographic Identity: Every action is signed and verified. This is the only way to build trustworthy autonomous systems.
The Agent Internet: AgentField prepares you for a web where agents negotiate and execute intent on your behalf.

Conclusion

Building with AgentField means agentic development has moved from “intent” to “execution.” It is no longer just about writing a prompt. It is about building a secure, scalable, and auditable system.

You can expand Argus by adding more specialized agents such as an ESG analyst, legal risk assessor, or macro forecaster. Because every component is a modular Skill or Reasoner, the system can grow without breaking.

The architecture we built here includes typed schemas, parallel execution, SSE streaming, and agents as microservices. This is the same pattern used in production AI systems. The difference between a weekend project and a production system comes down to structure, type safety, and observability from day one.

Resources

Check AgentField Docs
Check Project Repo
Check Autonomous software engineering fleet of 400+ AI agents for production-grade PRs

5 Things Developers Get Wrong About Inference Workload Monitoring

Astrodevil — Fri, 13 Mar 2026 09:28:15 +0000

Most LLM applications reach production with monitoring built for traditional backend services. Dashboards show average latency, overall error rate, and total tokens consumed. These indicators provide a quick sense of system health and cost exposure and often appear reassuring during early rollout, when traffic is predictable.

LLM inference operates under a different set of mechanics. Each request moves through GPU scheduling, queueing, prefill computation, and token generation.

Prompt length changes how much work happens before the first token appears. Concurrency affects how resources are shared across requests. These factors interact in ways that averages alone cannot explain.

When monitoring fails to reflect how inference actually runs, teams see symptoms but miss underlying causes. This article examines five common mistakes developers make when evaluating LLM performance and clarifies what deserves closer attention in real production systems.

Bridging the LLM Observability Gap

LLM systems often show performance drift before they show failure. Latency increases for certain requests. First-token timing becomes inconsistent. Throughput changes under higher concurrency. Traditional dashboards may still display stable averages.

The gap forms because inference behavior depends on prompt size, queue depth, GPU allocation, and workload mix. Surface metrics hide these interactions.

Nebius Token Factory addresses this gap at the inference layer. It is a production-grade LLM inference platform with built-in observability designed for real production workloads

Mistake #1: Treating Average Latency as a Reliable Performance Indicator

One of the most common mistakes in LLM performance monitoring is relying on average latency as the primary signal of system health.

Developers choose this metric because it produces a single number that looks clear in dashboards and reports. When the mean response time remains steady, the system appears stable.

Why This Weakens Production Insight

LLM workloads do not behave evenly. Prompt length varies across requests. Output size varies with task complexity. Concurrency increases during peak usage. Some requests complete quickly. Others require more prefill compute or wait longer in the queue.

An average hides this variation. A portion of requests can slow down significantly, and the mean may still look acceptable. In chat and agent systems, slower requests degrade the user experience even when most responses are fast. Monitoring only averages hides tail latency until complaints surface.

How Nebius Token Factory Addresses This

Nebius Token Factory Observability treats latency as a distribution problem. The platform calculates and displays percentile values for each endpoint and model across selected time windows.

It provides:

p50 latency, which reflects typical request behavior
p90 latency, which highlights emerging stress under moderate load
p99 latency, which exposes tail performance under heavier concurrency
Percentiles for both End-to-End Latency and Time to First Token

These percentile charts update continuously over rolling aggregation windows. Developers can filter by endpoint, project, region, prompt length, or latency band. This allows us to isolate slow requests and examine their correlation with traffic volume or token size.

The observability layer also supports integration with Prometheus and Grafana. Teams can build custom alerts based on p95 or p99 thresholds instead of averages. This allows production monitoring to focus on tail behavior where real user impact occurs.

Mistake #2: Collapsing All Failures Into a Single Error Rate

Another serious mistake in LLM performance monitoring is collapsing all failures into a single overall error rate. A single percentage may show that failures exist. It does not explain the type of failure or which layer caused it.

LLM systems fail at different points in the request lifecycle. Input validation can fail. Capacity limits can trigger throttling. Infrastructure can return execution errors. These failures carry different operational meanings.

Why This Reduces Diagnostic Precision

Each error category signals a different problem.

A 4xx error often points to invalid input, unsupported parameters, or prompt size limits.
A 429 error indicates rate limiting or capacity constraints under higher concurrency.
A 5xx error indicates an internal execution or infrastructure issue.

If monitoring aggregates all of these into one number, diagnosis slows down. The system shows instability but does not indicate the source. Developers must inspect logs manually to separate validation errors from capacity pressure.

How Nebius Token Factory Addresses This

Nebius Token Factory Observability exposes error metrics as structured dimensions.

It provides:

Error rate grouped by HTTP status code
Separate visibility into 4xx, 429, and 5xx categories
Filtering by endpoint, region, project, API key, and time window
Correlation with traffic metrics such as requests per minute and token flow

These metrics appear alongside latency percentiles and throughput charts. Developers can examine whether 429 responses increase during traffic spikes. They can inspect whether 5xx errors concentrate on a specific endpoint. They can filter by prompt length to identify validation failures linked to context size.

Metrics remain available through Prometheus and Grafana integrations for alerting and long-term analysis. Structured error visibility enables precise root-cause identification across the validation, capacity, and execution layers.

Mistake #3: Overlooking Time to First Token and Inference Stages

Many monitoring setups measure only total response time from request submission to final token delivery. That metric appears complete because it captures the full lifecycle of a request. In interactive LLM systems, users react when the first token appears on screen.

A delay at the start creates a perception of slowness even if total completion time stays within limits.

Impact on Performance Visibility

Inference executes in distinct stages. A request enters a queue. The system processes the entire prompt during prefill. Token generation begins after prefill completes. Total latency combines all these steps into a single value.

Time to First Token reflects queue delay and prompt processing. Decode time reflects token generation speed after the first token appears. When monitoring tracks only the total duration, it becomes difficult to determine whether the delay is due to queue buildup, larger prompts, or decoding throughput.

Separating these signals clarifies how the inference pipeline behaves under higher concurrency and heavier workloads.

How Nebius Token Factory Provides Stage-Level Visibility

Nebius Token Factory integrates observability directly into the inference pipeline. It exposes separate metrics for full request duration, Time to First Token, and token generation speed. Each metric appears as percentile distributions such as p50, p90, and p99.

Time to First Token reflects queue delay and prompt processing time. Output speed shows decoding throughput after generation begins. End-to-end latency captures the complete request lifecycle. Viewing these signals together allows clear identification of where delay occurs.

The platform presents these metrics alongside traffic volume and active replica data. Developers can examine whether higher concurrency increases TTFT or whether scaling activity stabilizes latency. Filters allow analysis by endpoint, region, project, prompt length, and time window. Prometheus and Grafana integrations support alerting on TTFT percentiles and stage-level latency trends.

Mistake #4: Ignoring Scaling and Capacity Signals

Many LLM monitoring setups focus only on request-level metrics such as latency and error rate. They do not track how the underlying infrastructure behaves when traffic increases.

When latency rises under load, attention often turns to the model. The actual cause may relate to replica allocation or capacity limits.

Production Consequences

LLM inference depends on available computing resources. Higher request volume increases queue depth. Scaling events change how traffic is distributed across replicas.

New instances may introduce an initialization delay before handling traffic. These infrastructure changes directly affect Time to First Token and high-percentile latency.

If monitoring does not expose replica activity or capacity state, it becomes difficult to connect traffic growth with performance behavior. Latency may increase during scaling transitions, yet the monitoring view shows only slower responses.

Nebius Token Factory Capacity Visibility

Nebius Token Factory Observability surfaces capacity and scaling signals alongside latency and traffic metrics.

Active Replica Metrics: The platform shows how many replicas actively serve requests. This helps identify whether latency growth aligns with scaling activity.
Traffic and Token Flow Metrics: Requests per minute and token volume appear in the same view. Developers can correlate concurrency growth with capacity utilization.
Latency Distribution with Scaling Context: Percentile latency metrics can be examined together with replica counts. This reveals whether p99 increases during load growth or stabilizes after new replicas come online.

Filtering by endpoint, region, project, and time window allows focused analysis. Prometheus and Grafana integrations support alerting tied to scaling behavior.

Mistake 5# Treating Prompt Length as a Cost Metric Only

Many developers track prompt length only to estimate token cost. Input and output tokens appear in billing views, and analysis stops there. Prompt size rarely enters performance discussions.

In production systems, prompt length directly influences compute time, queue behavior, and latency distribution. Ignoring it as a performance variable hides important signals.

What Gets Missed

The difference becomes clear when prompt size is treated purely as a billing metric versus a performance variable.

If Prompt Length Is Viewed Only as Cost	What Actually Happens in Production
Tokens are tracked for billing only	Longer prompts increase prefill compute time
Cost per request is monitored	Large prompts raise TTFT and p99 latency
Output token totals are reviewed	Long generations affect decoding throughput
No correlation with traffic load	Heavy prompts amplify queue depth under concurrency

Prompt distribution shapes the inference pipeline's behavior. Two endpoints with the same request rate can perform very differently if one processes longer contexts.

How Nebius Connects Token Usage to Performance

Nebius Token Factory treats token usage as an operational signal.

It provides:

Input and output tokens per minute
Distribution of tokens per request
Filtering by prompt length
Correlation between token metrics, TTFT percentiles, and throughput

Developers can compare short and long prompts within the same endpoint. They can observe how larger contexts affect prefill time and tail latency. They can inspect whether token growth aligns with scaling activity or throughput limits.

This connection between workload shape and execution behavior allows prompt size to be analyzed as a performance factor, not only a billing metric.

Conclusion

LLM systems require monitoring that reflects inference mechanics. Averages and blended metrics hide latency distribution, workload impact, and scaling behavior.

Clear visibility into percentiles, stage-level timing, structured errors, and token flow improves production analysis and reduces guesswork.

Nebius Token Factory embeds observability directly into the inference layer and surfaces the signals that matter under real load. If you operate LLM systems in production, evaluate whether your monitoring captures how inference truly behaves. Explore Nebius Token Factory Observability to build performance visibility designed for scale.

Build a Real-Time AI Analytics Dashboard with InsForge, FastAPI, and Claude Code

Astrodevil — Thu, 12 Mar 2026 16:39:14 +0000

Introduction

In this tutorial, we will build a fully functional analytics dashboard from scratch. The kind that ingests user events, shows live metrics and charts, and generates AI insights that stream word by word into the browser.

Here is what we will be building:

A FastAPI backend with event ingestion, metrics aggregation, AI streaming insights, and an event simulator
A Next.js frontend with a live metrics panel, event volume and breakdown charts, a live event feed, and a streaming AI insights panel
InsForge as the backend platform, managing our database, AI models, and REST API layer
Claude Code as the agent that builds the backend through a conversation with our live InsForge instance via MCP

By the end, you will have a working template you can drop your own event schema into. Let's get started.

What is InsForge?

InsForge is an open-source backend platform that you can also self-host with Docker. It gives you a Postgres database, a REST API layer built on PostgREST, an AI model gateway that routes to any OpenRouter-compatible model, a real-time pub/sub system, and serverless function support, all running on your own infrastructure.

Think of it as the infrastructure layer for data-driven applications. Instead of stitching together a database, an API server, and an AI integration separately, InsForge bundles them into a single deployable platform.

You bring your application logic, and InsForge handles the plumbing underneath.

What We Are Using InsForge For

Three things in particular make InsForge the right choice for an AI-first project like this one:

The managed AI gateway: You configure your OpenRouter API key once inside InsForge, and the platform handles all model routing from there. Your application calls one InsForge endpoint and passes a model string. Swap the string, and everything else stays the same. No per-model SDKs, no separate credentials in your codebase.
The MCP server: InsForge ships with an MCP server that gives Claude Code direct access to your live backend. The agent can read your schema, fetch documentation, and generate auth tokens as part of a conversation. This is what makes the one-prompt build possible.
The PostgREST layer: Every table in your InsForge database is automatically exposed as a REST endpoint. You do not write data access code. You describe your schema, and InsForge handles the rest.

Once you connect OpenRouter inside InsForge, the platform provisions the models and manages all routing.

For this project, we used anthropic/claude-sonnet-4.5, but you can switch models by changing a single string. Here is what available:

Model	Input	Output
anthropic/claude-sonnet-4.5	Text + image	Text
openai/gpt-4o-mini	Text + image	Text
x-ai/grok-4.1-fast	Text + image	Text
deepseek/deepseek-v3.2	Text + image	Text
minimax/minimax-m2.1	Text + image	Text
google/gemini-3-pro-image-preview	Text + image	Text + image

Getting InsForge Ready

Creating Your InsForge Project

Head to insforge.dev and sign up. Once you create a project, the dashboard gives you three things you will need throughout this build:

Base URL — your project's unique API endpoint, for example, https://xxxxxxxx.us-east.insforge.app
Anon Key — for browser-side and public API operations
Service Key — for privileged server-side operations

Copy those and keep them close. That is all the platform setup InsForge needs.

Configuring the AI Gateway

Inside your InsForge dashboard, go to the AI Integration section and add your OpenRouter API key. InsForge connects to OpenRouter and provisions the available models automatically. From this point on, your application calls InsForge, and InsForge handles the routing. You pick the model. InsForge does the rest.

Setting Up Your Project Folder

Create a new folder for the project and open your terminal inside it:

mkdir insforge-dashboard

cd insforge-dashboard

Connecting Claude Code via MCP

Before we touch any application code, let's connect Claude Code to our live InsForge instance using the InsForge MCP server. MCP (Model Context Protocol) is an open standard that lets AI coding agents connect to external tools and live data sources as part of a conversation. When it is set up, Claude Code can reach into your running InsForge backend and work against it directly.

Installing the MCP

Run this command inside your project folder:

npx @insforge/install --client claude-code \

--env API_KEY=your_insforge_api_key \

--env API_BASE_URL=https://your-project.us-east.insforge.app

The MCP installs and registers itself with Claude Code automatically. Restart Claude Code, and the connection is live.

Open Claude Code and start with this prompt to see what the agent has access to:

Connect to my InsForge instance and tell me what you can see.

This is the output we got with Claude:

What the agent saw	Details
Database	3 tables: events ((1,000 records), ai_insights (4 records), event_hourly_stats (0 records)
AI models	6 models configured and ready including Claude Sonnet 4.5, GPT-4o-mini, Grok 4.1
Auth	GitHub and Google OAuth configured
Schemas	Full column definitions and types for all three tables
SDK docs	InsForge REST API documentation fetched and read automatically

Claude Code connects to the live backend, reads the schema, and fetches the SDK documentation through the MCP connection.

Building the Backend

With Claude Code connected to InsForge via MCP, run the following prompt to generate the FastAPI backend:

*Build me a FastAPI backend with four routers: events, metrics, insights, and simulate. Use the InsForge SDK to connect to my backend. The insights router should stream AI responses using the anthropic/claude-sonnet-4.5 model through the InsForge AI gateway.*

Before writing a single file, Claude Code uses the MCP connection to:

Fetched the InsForge SDK documentation to understand the correct API patterns for database and AI calls
Read all three table schemas so that the code it generated matched our actual data structure
Generated a JWT token for authenticated database access
Inspected the existing project structure to understand what was already in place

The result was a complete project structure generated in a single pass:

insforge-dashboard/
├── main.py          # App entry point, CORS, router registration
├── config.py        # InsForge credentials from environment
├── client.py        # Shared InsForgeClient with database and AI helpers
├── requirements.txt
└── routers/
├── events.py    # GET + POST /events
├── metrics.py   # GET /metrics/summary and /metrics/hourly
├── insights.py  # POST /insights/generate — SSE streaming
└── simulate.py  # POST /simulate/events

The InsForge Client

The generated InsForgeClient communicates directly with the InsForge REST API using httpx. The database and AI gateway share the same client, base URL, and auth header, which reflects how InsForge unifies both services under a single interface.

class InsForgeClient:
    def __init__(self) -> None:
        self.base_url = INSFORGE_BASE_URL
        self._anon_key = INSFORGE_ANON_KEY

    @property
    def _headers(self) -> dict:
        return {
            'Authorization': f'Bearer {self._anon_key}',
            'Content-Type': 'application/json',
        }

    async def get_records(self, table, params=None):
        async with httpx.AsyncClient(timeout=30) as http:
            resp = await http.get(
                f'{self.base_url}/api/database/records/{table}',
                headers=self._headers,
                params=params or {},
            )
            resp.raise_for_status()
            raw_total = resp.headers.get('X-Total-Count')
            return resp.json(), int(raw_total) if raw_total else None

    async def create_records(self, table, records):
        async with httpx.AsyncClient(timeout=30) as http:
            resp = await http.post(
                f'{self.base_url}/api/database/records/{table}',
                headers={**self._headers, 'Prefer': 'return=representation'},
                json=records,
            )
            resp.raise_for_status()
            return resp.json()

The AI Streaming Method

The ai_stream method on the client calls the InsForge AI gateway and yields raw SSE lines back to the caller. The application calls the InsForge AI gateway directly. OpenRouter is configured once inside the InsForge dashboard and managed by the platform. The application codebase requires no OpenRouter credentials or model-specific SDK:

AI_MODEL = 'anthropic/claude-sonnet-4.5'

async def ai_stream(self, messages, system_prompt=None):
    payload = {
        'model': AI_MODEL,
        'messages': messages,
        'stream': True,
    }
    if system_prompt:
        payload['systemPrompt'] = system_prompt

    async with httpx.AsyncClient(timeout=120) as http:
        async with http.stream(
            'POST',
            f'{self.base_url}/api/ai/chat/completion',
            headers=self._headers,
            json=payload,
        ) as resp:
            resp.raise_for_status()
            async for line in resp.aiter_lines():
                if line.startswith('data: '):
                    yield line + '\n\n'

To switch models, update the AI_MODEL string at the top of client.py. The streaming logic, frontend integration, and persistence layer require no changes.

The Metrics Router

The metrics endpoint reads from the events and event_hourly_stats tables and aggregates the results in Python. InsForge's PostgREST layer does not expose GROUP BY directly, so we use Python's Counter to group by event name and page after the rows come back:

@router.get('/summary')
async def get_summary():
    records, total = await insforge.get_records(
        'events',
        {'limit': 1000, 'select': 'event_name,user_id,session_id,page'},
    )

    event_counts    = Counter(r['event_name'] for r in records)
    unique_users    = len({r['user_id'] for r in records if r['user_id']})
    unique_sessions = len({r['session_id'] for r in records if r['session_id']})
    page_counts     = Counter(r['page'] for r in records if r['page'])

    return {
        'total_events':    total or len(records),
        'unique_users':    unique_users,
        'unique_sessions': unique_sessions,
        'events_by_name':  dict(event_counts.most_common()),
        'top_pages':       dict(page_counts.most_common(10)),
    }


@router.get('/hourly')
async def get_hourly_stats(limit: int = 168):
    params = {'limit': limit, 'order': 'bucket_start.desc'}
    records, _ = await insforge.get_records('event_hourly_stats', params)
    return records

The Insights Router

When a user clicks Generate Insight, the insights router fetches recent event data, formats it as a structured context summary, and passes it to Claude Sonnet via the InsForge AI gateway. The stream proxies directly to the browser. Once it completes, the full response is saved to the ai_insights table so it persists across page refreshes:

_SYSTEM_PROMPT = (
    'You are an expert product analytics consultant. '
    'You will receive a structured summary of user event data and a specific question. '
    'Provide clear, concise, and actionable insights. '
    'Structure your response with labeled sections '
    '(e.g. ## Key Findings, ## Recommendations). '
    'Be specific — reference actual numbers from the data where relevant.'
)
async def stream_and_save():
    accumulated: list[str] = []

    async for sse_line in insforge.ai_stream(
        messages=[{'role': 'user', 'content': context}],
        system_prompt=_SYSTEM_PROMPT,
    ):
        data_str = sse_line.removeprefix('data: ').strip()
        try:
            parsed = json.loads(data_str)
            if 'chunk' in parsed:
                accumulated.append(parsed['chunk'])
        except (json.JSONDecodeError, KeyError):
            pass

        yield sse_line

    # Persist the full response once streaming is complete
    if req.save and accumulated:
        full_text = ''.join(accumulated)
        try:
            await insforge.create_records('ai_insights', [{
                'insight_type': req.insight_type,
                'title':        req.query[:80],
                'content':      full_text,
                'time_range':   req.time_range,
                'metadata': {
                    'total_events':    total,
                    'unique_users':    unique_users,
                    'unique_sessions': unique_sessions,
                    'event_breakdown': dict(event_counts.most_common(5)),
                },
            }])
        except Exception:
            pass  # Don't let a save failure break the delivered stream

    return StreamingResponse(
        stream_and_save(),
        media_type='text/event-stream',
        headers={'Cache-Control': 'no-cache', 'X-Accel-Buffering': 'no'},
    )

Notice that the save happens after the stream closes. The user gets the full streaming experience, and the insight is persisted in the background.

Running the Backend

python -m venv .venv

# Windows
.venv\Scripts\activate

# Mac / Linux
source .venv/bin/activate

pip install -r requirements.txt
uvicorn main:app --reload

Test that it is working:

curl http://localhost:8000/metrics/summary

# {"total_events": 1000, "unique_users": 198, "unique_sessions": 445,

# "events_by_name": {"page_view": 214, "search": 208, "purchase": 205, ...}}

Building the Frontend

Use the following prompt to generate the Next.js frontend:

*Build a Next.js frontend for this analytics dashboard. It should have a metrics summary row, an event volume chart, an event breakdown chart, a live event feed, and an AI insights panel that streams responses word by word. Poll the FastAPI backend every 5 seconds for live data.*

The generated frontend uses Tailwind CSS and Recharts, with all components connected to the FastAPI backend. The two most important pieces are the polling mechanism and the SSE streaming implementation.

Keeping the Dashboard Live

The dashboard polls /metrics/summary and /events every 5 seconds, so it stays current. The data loads immediately on mount, and the interval keeps it fresh:

useEffect(() => {
    loadMetrics();
    loadEvents();
    const poll = setInterval(() => {
        loadMetrics();
        loadEvents();
    }, 5_000);
    return () => clearInterval(poll);
}, []);

Streaming AI Insights

When a user clicks Generate Insight, the AIInsightsPanel opens an SSE (Server-Sent Events) connection to POST /insights/generate and reads the response body as a stream, appending each chunk to a buffer as it arrives:

const reader  = res.body.getReader();
const decoder = new TextDecoder();
let buffer = '';

while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    buffer += decoder.decode(value, { stream: true });
    const lines = buffer.split('\n');
    buffer = lines.pop() ?? '';

    for (const line of lines) {
        if (!line.startsWith('data:')) continue;
        const parsed = JSON.parse(line.slice(5).trim());
        if (parsed.chunk) callbacks.onChunk(parsed.chunk);
        if (parsed.done && parsed.insight) callbacks.onDone(parsed.insight);
    }
}

Each onChunk call appends to a streamBuffer in state, and the component renders the buffer progressively with a blinking cursor. When onDone fires, the buffer is cleared, and the persisted insight is prepended to the list. The panel header displays "claude-sonnet-4.5 · InsForge", confirming the model is served through the InsForge gateway.

Starting the Frontend

cd frontend
npm install
npm run dev

Open http://localhost:3000. Use the simulator to populate the dashboard with realistic events if you have not already:

curl -X POST "http://localhost:8000/simulate/events" \
  -H "Content-Type: application/json" \
  -d "{\"count\": 50}"

Run it a few times and refresh the dashboard. The metrics panel, charts, and event feed will populate with the simulated data.

Seeing it all work

With both servers running and some events in the database, here is what the finished dashboard shows:

Total Events, Unique Users, Unique Sessions, and Top Event in the metrics row at the top
An Event Volume chart showing activity over the selected time range, switchable between 1 hour, 24 hours, and 7 days
An Event Breakdown bar chart grouping events by type
A Live Event Feed showing recent events with user IDs, pages, and timestamps, updating every 5 seconds
An AI Insights panel where you submit a question and Claude Sonnet streams a structured analysis through the InsForge gateway in real time

Click Generate Insight, and see the results you get.

Live Updates with InsForge Realtime

InsForge includes a built-in real-time system for pushing updates to connected clients over WebSockets. It is channel-based and built directly into the platform alongside the database and AI gateway, so there is nothing additional to configure.

To add Realtime to the dashboard, run this prompt in Claude Code:

Add InsForge Realtime to the app. When a new event is inserted via POST /events or the simulator, publish it to a channel called analytics:events. On the frontend, subscribe to that channel using the InsForge Realtime SDK and push incoming events directly into the live event feed as they arrive.

Claude Code registers the channel and creates a database trigger on the events table that fires realtime.publish() on every insert. This covers both the API endpoint and the batch simulator.

The InsForge Realtime dashboard logs every message flowing through the system, showing the event name, channel, payload, and timestamp for each publish call.

Deploying the Application

Once the dashboard is working locally, deploying it is a matter of giving Claude Code a prompt. Because the MCP connection is still active, the agent understands the project structure and can generate the deployment configuration without any additional context.

Generating the Deployment Configuration

Run the following prompt in Claude Code:

Prepare this project for deployment to Zeabur. Create a Dockerfile for the FastAPI backend and a Dockerfile for the Next.js frontend using standalone output. Include a .dockerignore for each service.

Claude Code will generate the following files:

> insforge-dashboard/
> 
> 
> ├── Dockerfile # FastAPI backend --- Python 3.11 slim, uvicorn on port 8000
> 
> ├── .dockerignore
> 
> └── frontend/
> 
> ├── Dockerfile # Next.js --- multi-stage Node 20 build, standalone output
> 
> └── .dockerignore
>

Deploying to Zeabur

Push the project to GitHub, then go to zeabur.com and create a new project. Add two services from the same repository:

Backend service: point Zeabur at the root directory. It detects the Dockerfile automatically. Set the following environment variables: INSFORGE_BASE_URL and INSFORGE_ANON_KEY.
Frontend service: point Zeabur at the /frontend subdirectory. Set NEXT_PUBLIC_API_URL to the public URL Zeabur assigns to your backend service, for example, https://your backend.zeabur.app. This value must be set before the build runs, as it is baked into the Next.js bundle at build time.

Once both services are deployed, click Generate Domain on each to assign a public URL. The frontend will be accessible at its public URL and will communicate with the backend through the NEXT_PUBLIC_API_URL you configured.

What's Next?

At this point, you have a fully working AI analytics dashboard running on InsForge. Claude Code generates the backend through a single MCP-connected prompt. AI insights stream through the InsForge gateway with no OpenRouter configuration required in the application. The dashboard stays current via polling, and every insight is persisted to the database.

From here, the project is yours to extend. Swap in your own event schema, add new metrics endpoints, or change the AI model to gpt-4o-mini or grok-4.1-fast by updating a single string in client.py. The MCP connection stays live, so Claude Code remains a capable collaborator for any further work. You can clone the project’s repo and extend the project further.

To learn more about Insforge, check out the GitHub repo.

Building a plugin for a React visual editor with Puck

Astrodevil — Thu, 12 Mar 2026 16:38:54 +0000

Page builders and visual editors have become central to modern product development. Frontend engineers, product teams, and content operations groups rely on them to build landing pages, dashboards, documentation systems, and internal tools quickly and consistently.

As adoption grows, expectations grow with it. Editors must support customization, structured content, automation, and domain-specific workflows without increasing development and maintenance overhead.

The market reflects this demand. The global low-code development platform market is projected to reach $167 billion by 2030. Organizations continue to invest in visual development systems that enable faster iteration and broader collaboration. As these systems expand in scope and adoption, the architectural responsibility placed on editor frameworks also increases.

Growth introduces complexity. Every feature added directly to the core increases surface area and long-term maintenance costs. As teams introduce metadata panels, validation rules, workflow controls, and UI extensions, the editor becomes tightly coupled and difficult to evolve without clear extension boundaries.

Plugin systems create those boundaries. They define controlled integration points, isolate functionality, and protect the editor’s foundation.

In this article, we examine how to design plugin systems for visual editors and build a working example using Puck.

What Is a Plugin System: Architecture Deep Dive

A plugin system defines a controlled way to extend software without modifying its core. It exposes extension points through a stable contract and allows external modules to register new behavior, UI, or logic.

The core remains responsible for orchestration, lifecycle management, and state ownership. Plugins operate within boundaries defined by that core.

At an architectural level, a plugin system introduces three primary layers:

Core Engine: Owns state, rendering, persistence, and lifecycle management.
Extension API (Plugin Contract): Defines how plugins register, what hooks they can access, and what capabilities they receive.
Plugin Modules: Independent units that implement features through the exposed contract.

This separation enforces control. The core decides when a plugin runs, where it renders, and what data it can access. The plugin does not directly mutate internal systems. It communicates through defined interfaces.

Key Architectural Properties

A well-designed plugin system should provide:

Isolation: Plugins cannot corrupt the global state.
Deterministic Lifecycle: Mount, update, and unmount phases are predictable.
Explicit Extension Points: UI slots, event hooks, and state access are intentional.
Encapsulation: Editor state and internal systems remain protected behind defined APIs.
Composable Registration: Multiple plugins can coexist without conflict.

This architecture scales by allowing new capabilities to be introduced without changing the foundation. Features can be enabled or removed independently, while the core remains lean, stable, and protected.

When to Use Plugins vs Core Features

Extensible editors require deciding which capabilities belong in the core and which should be implemented as plugins. Core features define the editor’s fundamental architecture, such as rendering, state management, and persistence. Plugins extend the editor through defined extension points without modifying that foundation.

The distinction becomes clearer when viewed side by side.

How Puck Implements the Plugin Contract

Puck is a React-based page builder that lets users create pages by dragging and dropping their own components. Developers register these components through configuration objects, and the editor uses them to build and render pages. Puck is fully customizable and extensible, providing APIs to extend editor behavior or package additional functionality as plugins.

Plugin integration in Puck is straightforward. Plugins can register sidebar panels, add controls, and respond to editor events through clear extension surfaces. They interact with the editor state through documented APIs without reaching into internal systems or modifying rendering logic directly.

The plugin contract in Puck focuses on three responsibilities:

Registration: A plugin declares its identity and attaches to the editor during initialization.
UI Injection: The plugin connects to defined surfaces such as sidebars or inspector regions.
Lifecycle Participation: Plugins can hook into editor behavior such as loading, saving, or validation.

Once registered, the plugin runs as part of your core editor integration. This keeps the editor implementation stable while allowing additional functionality to be added through independent plugins.

Building an Author Info Plugin using Puck

We will build a simple Author Info plugin that demonstrates plugin registration, UI injection, and lifecycle participation inside Puck. The plugin will:

Add a panel to the left sidebar
Capture author name, role, and avatar
Store this data alongside the page state
Validate the metadata before publishing

1. Create a New Puck App

Start by generating a new app using the official Puck starter:

npx create-puck-app author-info-plugin

Choose the Next.js option when prompted. After the setup completes, navigate to the project directory and run the application in development mode:

cd author-info-plugin
npm install
npm run dev

Open your browser at:

http://localhost:3000/edit

You should see the Puck editor interface for the homepage.

2. Understand the Plugin API

Puck plugins can extend the editor interface through the Plugin Rail on the left. Plugins may render UI in this rail, but they can also extend editor behavior through overrides and other integrations.

A plugin object looks like this:

const myPlugin = {
  name: "my-plugin",
  label: "My Plugin",
  icon: <Icon />,
  render: () => <div>My UI</div>,
};

Plugins are wired into the editor by passing them to the plugins prop of the <Puck /> component.

3. Create Your Plugin File

Inside your project, create a file at:

app/puck/plugins/AuthorInfoPlugin.tsx

You can create it with:

mkdir -p app/puck/plugins
touch app/puck/plugins/AuthorInfoPlugin.tsx

Since this example uses icons from lucide-react, install it first:

npm install lucide-react

Then open the file you created and add:

"use client";

import { createUsePuck, Plugin } from "@puckeditor/core";
import { User } from "lucide-react";

const usePuck = createUsePuck();

export const AuthorInfoPlugin: Plugin = {
  name: "author-info",
  label: "Author Info",
  icon: <User />,
  render: () => {
    const data = usePuck((state) => state.appState.data);
    const dispatch = usePuck((state) => state.dispatch);

    const author = data.root?.props?.author ?? {
      name: "",
      role: "",
      avatar: "",
    };

    const updateAuthor = (field: string, value: string) => {
      dispatch({
        type: "replaceRoot",
        root: {
          ...data.root,
          props: {
            ...data.root?.props,
            author: {
              ...author,
              [field]: value,
            },
          },
        },
      });
    };

    return (
      <div style={{ padding: 16 }}>
        <h3>Author Info</h3>
        <input
          placeholder="Author Name"
          value={author.name}
          onChange={(e) => updateAuthor("name", e.target.value)}
        />
        <input
          placeholder="Author Role"
          value={author.role}
          onChange={(e) => updateAuthor("role", e.target.value)}
        />
        <input
          placeholder="Avatar URL"
          value={author.avatar}
          onChange={(e) => updateAuthor("avatar", e.target.value)}
        />
      </div>
    );
  },
};

This plugin renders a component that:

Reads the current page data via usePuck
Updates author metadata via Puck’s dispatcher

This demonstrates how plugins can integrate with the editor state.

4. Register the Plugin

Open the route where the editor is rendered:

app/puck/[...puckPath]/client.tsx

Find the <Puck /> component and update the plugins prop:

import { AuthorInfoPlugin } from "../plugins/AuthorInfoPlugin";

<Puck
  config={config}
  data={data}
  onPublish={async (data) => {
    await fetch("/api/pages", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ data, path }),
    });
    return data;
  }}
  plugins={[AuthorInfoPlugin]}
/>

Now your plugin will appear in the Plugin Rail.

5. Test the Plugin UI

Reload the editor at:

http://localhost:3000/edit

In the left Plugin Rail:

Click Author Info
Enter a name, role, and avatar URL in the fields
Click Publish

Because of your onPublish Implementation: the data will be sent to /api/pages and saved to database.json.

6. Confirm Persistence

After publishing, open:

database.json

You should see:

{
  "/": {
    "root": {
      "props": {
        "author": {
          "name":"Your Author",
          "role":"Writer",
          "avatar":"https://example.com/avatar.png"
        }
      }
    },
    "content": [],
    "zones": {}
  }
}

This confirms the author data was correctly persisted.

Note: In the starter project, this data is saved to a local file, but in a real application, it could be stored in any backend, such as a database or API service.

7. (Optional) Render Author Info on the Frontend

To display author metadata on the frontend, update the page that renders your content. For example, open:

app/[...puckPath]/client.tsx

Add the following below the <Render /> component:

<Render config={config} data={data} />

{data.root.props.author && (
  <div style={{ padding: 24 }}>
    <h3>Author</h3>
    <p><strong>{data.root.props.author.name}</strong></p>
    <p>{data.root.props.author.role}</p>
    {data.root.props.author.avatar && (
      <img src={data.root.props.author.avatar} width={80} alt="Avatar" />
    )}
  </div>
)}

You can also place that in the root configuration to display the saved metadata on published pages.

That’s it. You now have a fully working Author Info plugin that integrates with the editor state, renders a sidebar panel in the Plugin Rail, and persists metadata through the publishing flow.

Closing

Plugin systems give visual editors a structured path for growth. With clear contracts and defined extension points, teams can introduce new capabilities without reshaping the underlying architecture. This keeps responsibilities separated, reduces risk, and allows the platform to adapt as product requirements expand.

The Author Info plugin we built using Puck shows how an extension can be registered independently, integrate with the editor state, and persist structured metadata. The core remains stable while the plugin delivers focused functionality, demonstrating how modular design supports scalable and maintainable editor systems.

For deeper implementation details, refer to the official documentation: https://puckeditor.com/docs

How Context-First MCP Design Reduces Agent Failures on Backend Tasks

Astrodevil — Mon, 09 Mar 2026 19:47:29 +0000

TL;DR

The problem: AI agents handle frontend tasks well, but backend operations expose a consistent gap. Even when MCP is connected, most backends return table names without record counts, schema without RLS state, and tool responses without success signals. The agent fills that gap with extra queries, retries, and guesses.
Why existing backends fall short: Platforms like Supabase and Postgres MCP were designed for human operators who can read a dashboard, check a UI, and make judgment calls. When an agent is the one operating the backend, that design assumption breaks at every step.
What changes with an agent-native backend: InsForge builds MCP into the backend from the start, not on top of it. Two tool calls give the agent a full map of the backend and deep context for the table it is working on, with live record counts, RLS state, and policy definitions returned before the agent writes a single query. Across 21 MCPMark tasks, the design produces 47.6% Pass⁴ accuracy, 30% fewer tokens, and 1.6x faster execution compared to Postgres MCP and Supabase MCP.

Introduction

AI agents handle frontend tasks well. Give one a component structure, a routing pattern, or a state management problem, and it will usually get it right. But backend operations are different.

Ask an agent to implement Row Level Security on a production Postgres database. It will write the policies, run the migration, and return success. But if the MCP server it is connected to does not expose RLS status in its schema response, the agent has no way to verify what it actually changed. It assumes the operation worked. The policies it wrote may be incomplete, applied to the wrong roles, or missing entire tables. Nothing throws an error, but the data access rules are silently wrong.

This is not a model quality problem. It is a context problem. And it happens consistently, across models, across tasks, even when MCP is part of the setup.

In this article, we examine where agents consistently fail on backend tasks, what the current MCP layer design in most backends is missing, and how a backend built specifically for agents changes the execution pattern.

MCP Exists but Agents Still Fail.

Most major backend platforms have MCP servers now. Supabase has one. Postgres has one. The assumption is that once you connect an agent to a backend via MCP, the backend visibility problem is solved, but connecting to an MCP does not determine what it returns.

MCP is a protocol. It defines how tools are called and how responses are structured. It does not define what those responses contain. That part is entirely up to whoever built the MCP server.

Here is what Supabase MCP returns when an agent calls list_tables:

["users", "orders", "products", "sessions"]

The response contains table names only, with no record counts, no foreign key relationships, no RLS status, no policy definitions, no trigger logic, and no index information.

Supabase was built for a specific use case, and it does that well. Tools like Lovable, Bolt, and v0 use it because it is fast to set up, visually manageable, and good for getting a product off the ground quickly. The dashboard-first design works for that workflow because a human is always in the loop, reading errors, checking the UI, and making judgment calls.

When an agent is the one operating the backend, there is no human in the loop. The agent cannot open the Supabase dashboard and read the RLS policy panel. It can only work with what the MCP tool returns. And what it returns is not enough to act correctly on anything beyond a basic read or write.

So the agent does what any system does when it lacks information. It runs more queries. It guesses. It retries. Each of those extra steps costs tokens and time, and none of them are guaranteed to surface the right answer.

Every extra query the agent runs, every retry, every guess is a symptom of the same underlying gap. The MCP layer was built for a human operator, and the agent is paying the cost of that assumption at runtime.

How Agents Actually Fail

The surface problem shows up in four consistent failure patterns. These are not edge cases or misconfigured setups. They happen across models, across tasks, and across backends that have MCP connected.

Two of the failures below reference MCPMark tasks. MCPMark is an open-source benchmark that measures how well MCP servers support language models on real database tasks. Each task is a non-trivial backend operation run against actual databases with real data volumes. The task names map directly to the operation being tested. They are used here because they are reproducible, measurable, and not hypothetical.

Failure 1: Non-Deterministic Tool Calls

Agents interact with backends by calling tools, and some of those tools trigger real, irreversible operations like creating a resource, provisioning infrastructure, or writing to a database. When one of those call times out, the agent has no success signal to work from. The tool response came back empty, the operation has no ID, and the backend has no idempotency key, so the agent does what it was built to do: it retries. By the time the second request lands, the first one has already gone through. Two identical resources now exist in production, and neither the agent nor the developer caught it in real time.

APIs built for humans assume someone will check the dashboard after an operation. Agents cannot do that. They need a deterministic success or failure signal from the tool response itself, or they will keep retrying until something breaks.

Failure 2: Schema Blindness in Real Databases

The MCPMark task employees__employee_demographics_report asks an agent to generate gender statistics, age group breakdowns, birth month distributions, and hiring year summaries from an HR database.

The employee table had 300,024 rows. The salary table had 2,844,047 rows. That is roughly 9.5 salary records per employee.

The MCP server returns table names and column definitions. No record counts. The agent sees a join between two tables and writes the most natural query:

SELECT gender, COUNT(*)  -- counts salary rows, not employees
FROM employee e
LEFT JOIN salary s ON e.id = s.employee_id;

The query runs clean. No error. The output is 9.5 times too large because COUNT(*) multiplies across joined salary rows instead of counting distinct employees. The agent returns it as correct.

Why COUNT(*) returns the wrong number on a joined table, and what the correct query looks like

The backend had the record counts. It just never surfaced them.

Failure 3: Missing Context Compounds Cost

The MCPMark task security__rls_business_access asks an agent to implement Row Level Security policies across 5 tables on a social media platform.

The MCP server's get_object_details returns schema, columns, constraints, and indexes for each table. No rlsEnabled field. No policies array. The agent does not know RLS is disabled. It has to run a separate SQL query just to check the current RLS status before it can start the actual task. Then it runs verification queries after implementation to confirm the policies applied correctly.

list_schemas          → schema names only
list_objects          → table names only
get_object_details × 5  → schema, no RLS status
execute_sql           → check current RLS status
execute_sql × 8       → create functions, enable RLS, create policies
execute_sql × 4       → verification queries

23 turns. 581K tokens. Every extra query in that path exists because one field was missing from the original response.

Failure 4: No Guardrails for Autonomous Operations

An agent runs a schema migration. The migration has a logic error. There is no audit log of what the agent changed, no agent-scoped permissions limiting what operations it can run, and no rollback path. Production breaks. Nothing was recorded about what happened.

Backends built for human developers assume a human is reviewing the migration before it runs. When an agent is the one initiating the change, that assumption breaks. There is no mechanism to scope what the agent is allowed to modify, no record of what it actually did, and no way to recover cleanly if something goes wrong.

What an Agent-Native Backend Looks Like

The failures in the previous section share a common cause. The backend gave the agent a name when it needed a state. A table name when it needed a record count. A schema when it needed a policy definition.

Fixing this is not about a better prompt or a smarter model. It is about what the MCP layer returns by default. That means the backend itself has to be designed around what an agent needs to see, not retrofitted with MCP after the fact.

That is a different design starting point entirely. It is what separates a backend built for humans from one built for agents. InsForge is built on that starting point.

InsForge is an open-source backend platform built for AI-assisted development. It provides database management, authentication, storage, serverless functions, and AI integrations, with APIs structured specifically for deterministic agent execution. Unlike backends that expose MCP as a layer on top of a human-facing platform, InsForge builds the MCP design into the backend itself.

Its MCP server is built around two principles: hierarchical context and a live source of truth. Every tool call is designed so the agent gets exactly what it needs for the current step, with a clear signal pointing to what it needs next.

Backend retrofitted with MCP vs a backend built around agent execution from the start

In practice, **that means two layers, one that gives the agent a map of the entire backend, and one that gives it the full detail for exactly the table it is working on.

Layer 1: Global Context with `get-backend-metadata`

The first call an agent makes is get-backend-metadata. It returns the full backend surface in one response: every table with its live record count, auth configuration, storage buckets, AI model integrations, and a built-in hint that tells the agent exactly what tool to call next.

{
  "auth": {
    "oauths": [
      {
        "provider": "google",
        "clientId": null,
        "redirectUri": null,
        "scopes": ["openid", "email", "profile"],
        "useSharedKey": true
      }
    ]
  },
  "database": {
    "tables": [
      { "tableName": "users", "recordCount": 1 }
    ],
    "hint": "To retrieve detailed schema information for a specific table, call the get-table-schema tool with the table name."
  },
  "storage": {
    "buckets": [],
    "totalSizeInGB": 0
  },
  "aiIntegration": {
    "models": [
      {
        "inputModality": ["text", "image"],
        "outputModality": ["text"],
        "modelId": "anthropic/claude-sonnet-4.5"
      }
    ]
  },
  "version": "1.0.0"
}

The recordCount field directly resolves Failure 2. Before writing a single query, the agent already knows there are 300,024 employee rows and 2,844,047 salary rows. It knows this is a many-to-one relationship. It knows COUNT(*) on a join will multiply rows. It writes COUNT(DISTINCT e.id) on the first attempt.

The hint field resolves the discovery loop problem. The agent does not have to guess what tool to call next or run exploratory queries to understand the backend topology. The response tells it directly.

Everything the agent gets from a single get-backend-metadata call

Layer 2: Local Context with `get-table-schema`

Once the agent knows which table it needs to work with, it calls get-table-schema. This returns the full definition for that table in a single response.

{
  "users": {
    "schema": [
      { "columnName": "id", "dataType": "uuid", "isNullable": "NO", "columnDefault": null },
      { "columnName": "nickname", "dataType": "text", "isNullable": "YES", "columnDefault": null },
      { "columnName": "bio", "dataType": "text", "isNullable": "YES", "columnDefault": null },
      { "columnName": "created_at", "dataType": "timestamp with time zone", "isNullable": "YES", "columnDefault": "now()" },
      { "columnName": "updated_at", "dataType": "timestamp with time zone", "isNullable": "YES", "columnDefault": "now()" }
    ],
    "indexes": [
      {
        "indexname": "users_pkey",
        "indexdef": "CREATE UNIQUE INDEX users_pkey ON public.users USING btree (id)",
        "isUnique": true,
        "isPrimary": true
      }
    ],
    "foreignKeys": [
      {
        "constraintName": "users_id_fkey",
        "columnName": "id",
        "foreignTableName": "accounts",
        "foreignColumnName": "id",
        "deleteRule": "CASCADE",
        "updateRule": "NO ACTION"
      }
    ],
    "rlsEnabled": true,
    "policies": [
      {
        "policyname": "Enable read access for all users",
        "cmd": "SELECT",
        "roles": "{public}",
        "qual": "true",
        "withCheck": null
      },
      {
        "policyname": "Disable delete for users",
        "cmd": "DELETE",
        "roles": "{authenticated}",
        "qual": "false",
        "withCheck": null
      },
      {
        "policyname": "Enable update for users based on user_id",
        "cmd": "UPDATE",
        "roles": "{authenticated}",
        "qual": "(uid() = id)",
        "withCheck": "(uid() = id)"
      },
      {
        "policyname": "Allow project_admin to update any user",
        "cmd": "UPDATE",
        "roles": "{project_admin}",
        "qual": "true",
        "withCheck": "true"
      }
    ],
    "triggers": []
  }
}

Once the agent knows which table it needs to work with, it calls get-table-schema

This single response resolves Failure 3 completely. The agent sees rlsEnabled: true and the full policy definitions before it writes any SQL. It knows exactly which roles have access to which operations, what the qual conditions are, and what withCheck constraints apply. With full policy definitions and RLS state returned upfront, there is nothing left to discover and guess.

Compare this to what Postgres MCP returns for the same table:

{
  "basic": { "schema": "public", "name": "users", "type": "table" },
  "columns": [...],
  "constraints": [...],
  "indexes": [...]
}

No rlsEnabled field. No policies array. The agent has no way to know RLS exists on this table from this response alone. It proceeds, hits a permission error, and starts the retry loop that costs 285K extra tokens.

Postgres MCP vs InsForge: what the same table call returns

Why This Is an Architecture Decision, Not a Feature

The two-layer design is intentional.

get-backend-metadata is global context: it gives the agent a high-level map of the entire backend without overloading the context window.
get-table-schema is local context: scoped, deep, and called only for the table the agent is actively working on.

This matters because of how context windows work in practice. Loading full schema details for every table upfront can consume 20K to 30K tokens of irrelevant information, pushing out logic the agent wrote earlier in the session. The hierarchical design keeps the context window clean while ensuring the agent always has what it needs for the current operation.

The hint field in get-backend-metadata is what connects the two layers. The agent does not have to reason about what to fetch next. The backend tells it. That is the difference between a backend that was retrofitted with MCP and one that was designed around how agents actually operate.

The context layer handles Failures 2 and 3. But a fully agent-native backend has to go further. Failures 1 and 4, non-deterministic operations and unguarded autonomous changes, are addressed at the platform level. InsForge's tool contracts return deterministic success and failure signals by design. Agent-initiated schema changes are logged, scoped, and reversible. The MCP layer and the platform contract layer work together.

How the two-layer context cycle works: global map first, table detail second

The Numbers

The architecture described in the previous section is not a theoretical improvement. MCPMark makes it measurable.

InsForge, Supabase MCP, and Postgres MCP were all evaluated against the same 21 tasks using Anthropic Claude Sonnet 4.5 as the model. Each task was run 4 consecutive times.

The accuracy metric used is Pass⁴. A task counts as successful only if the agent completes it correctly in all four independent runs. Not once. Not three out of four. All four. This is what reliability actually looks like in production.

Case Study 1: Demographics Report

Task: employees__employee_demographics_report

Generate gender statistics, age group breakdowns, birth month distributions, and hiring year summaries from an HR database with an employee table (300,024 rows) and a salary table (2,844,047 rows).

Backend	Success Rate	Tokens Used
InsForge	4/4 (100%)	207K
Supabase MCP	3/4 (75%)	204K
Postgres MCP	2/4 (50%)	220K

Token usage is similar across all three. The failures are not a cost problem. They are a correctness problem caused entirely by missing record count information.

Neither Supabase MCP nor Postgres MCP tells the agent how many rows each table contains. The agent sees a join between two tables and writes the most natural query, which counts salary rows instead of employees. The output is 9.5 times too large. No error is thrown. The agent returns it as correct.

InsForge surfaces record counts in the first call, so the agent sees the row ratio before writing any SQL, knows COUNT() will multiply rows on this join, and writes COUNT(DISTINCT e.id) on the first attempt. 4/4 every time. The only difference is two fields in the first MCP response.

Case Study 2: RLS Setup

Task: security__rls_business_access

Implement Row Level Security policies across 5 tables on a social media platform: users, channels, posts, comments, channel_moderators.

Backend	Success Rate	Tokens Used	Turns
InsForge	4/4 (100%)	296K	15
Supabase MCP	1/4 (25%)	340K	—
Postgres MCP	4/4 (100%)	581K	23

InsForge and Postgres MCP both reach 100% accuracy. But Postgres MCP uses 581K tokens and 23 turns to get there. InsForge uses 296K tokens and 15 turns. The 285K token difference is not model behavior. It is the direct cost of the agent not knowing RLS state upfront.

Postgres MCP's execution path:

list_schemas          → schema names only
list_objects          → table names only
get_object_details × 5  → schema, no RLS status
execute_sql           → query to check current RLS status
execute_sql × 8       → create functions, enable RLS, create policies
execute_sql × 4       → verification queries

InsForge's execution path:

get-instructions        → learns InsForge workflow
get-backend-metadata    → all 5 tables at a glance
get-table-schema × 5    → full schema with RLS status (parallel)
run-raw-sql × 6         → create functions, enable RLS, create policies

The Postgres agent runs extra queries to check RLS status and then verify the policies applied correctly because the information was never in the original response. InsForge skips both phases entirely because it was already there.

Supabase MCP reaches only 1/4. Without visibility into existing policies or RLS state, the agent could not reliably implement the required security model across 4 consecutive runs.

Aggregate Results

Across all 21 tasks:

Metric	InsForge	Postgres MCP	Supabase MCP
Pass⁴ Accuracy	47.6%	38.1%	28.6%
Avg Tokens Per Run	8.2M	10.4M	11.6M
Avg Time Per Task	150 seconds	200+ seconds	200+ seconds

InsForge is 1.6x faster, uses 30% fewer tokens, and achieves 47.6% Pass⁴ accuracy against Postgres MCP's 38.1% and Supabase MCP's 28.6%.

Token cost of missing RLS context across three backends on the same task

The accuracy gap is significant given what Pass⁴ measures. Passing once is not the bar. The bar is passing the same complex backend operation 4 times in a row, without mistakes, without retries caused by missing context. At that bar, InsForge is the only backend that consistently clears it.

Closing

The future of agent-native development will not be defined by better models alone. It will be defined by what those models can see, the context they are given, the signals they receive, and the backend layer that determines both.

If this is the problem you are working on, InsForge is open source and we welcome contributions from the community.

Try InsForge

Quickstart guide here

AI Slop vs Constrained UI: Why Most Generative Interfaces Fail

Astrodevil — Tue, 24 Feb 2026 11:52:06 +0000

TL;DR

AI can generate structured interface layouts and assemble component hierarchies from natural language prompts. When generation is unconstrained, the output diverges from design systems, lacks determinism, and requires downstream refactoring before deployment. Production-grade generative UI requires predefined component registries, validated schemas, and explicit architectural constraints, which is the model implemented by Puck and its constrained generation layer, Puck AI.

Introduction

AI systems can now generate complete interface layouts from natural language prompts. Tasks that previously required manual section planning, component selection, and layout composition can now start from a single instruction.

Yet when these generated interfaces are evaluated against real production standards, limitations become clear. Outputs often conflict with established design systems, introduce structural inconsistencies, and produce layouts that require refactoring before integration.

What appears efficient at the prompt layer frequently shifts complexity downstream into engineering workflows. This gap between demo output and deployable software has led to growing skepticism around generative UI, sometimes informally labeled as “AI slop” to describe output that appears complete but fails architectural validation.

In this article, we will examine where AI meaningfully supports interface generation, where it breaks down in production environments, and why constrained, schema-driven systems are essential for making generative UI operational at scale.

What AI Is Actually Good At in UI Workflows

Before evaluating the limitations of generative UI, it is important to isolate where it provides measurable value. AI is effective at accelerating early-stage interface assembly, particularly when the objective is to convert high-level intent into an initial layout draft.

Consider a product team building a new SaaS landing page. A prompt such as “Create a landing page for an AI analytics platform with a hero section, feature highlights, pricing tiers, and customer testimonials” can reliably produce a logically organized page structure within seconds. The output may not be production-ready, but it establishes a usable structural baseline.

AI performs well in the following areas:

Translating Intent into Layout Structure: AI can map abstract, well-known requirements into recognizable interface patterns. For example, it understands that a landing page typically includes a hero, feature highlights, social proof, and a call to action. This reduces the cognitive load of structuring the page from scratch.
Generating First-Pass Scaffolding: AI can assemble a preliminary hierarchy of sections and placeholder content. This enables teams to visualize information architecture quickly before investing time in refinement.
Automating Repetitive Component Assembly: When working with predefined components, AI can configure repeated structures such as feature cards, pricing tiers, or testimonial blocks with consistent prop patterns. This is particularly useful in systems with modular design libraries.
Supporting Rapid Experimentation: AI allows teams to generate multiple layout variations in minutes, enabling faster exploration of structural alternatives without manual reconfiguration.

AI is therefore effective at structural acceleration, particularly in early-stage development where system rules, design boundaries, and composition patterns are still being defined.

In established product environments, however, structural acceleration must operate within existing architectural constraints to remain viable.

Where Unbounded UI Generation Breaks Down

Many unbounded generation tools produce raw code that must be reviewed, integrated, and redeployed before it can be used in production. Even when the output appears complete, it is not directly executable within an existing system. This shifts responsibility to engineering teams and introduces friction into workflows that are intended to be self-service.

For non-technical users such as marketers or content authors, this model is impractical. Page updates require developer involvement, slowing content publishing and limiting autonomy. Instead of enabling cross-functional workflows, generative UI becomes a developer-only tool.

Below are a few additional architectural limitations to consider:

Design System Violations: Unbounded generation does not inherently respect spacing scales, typography tokens, or component composition rules. It may introduce arbitrary margins, inconsistent heading hierarchies, or layout patterns that are not part of the approved system. Even if visually acceptable, these deviations fragment the design language and undermine maintainability.
Inconsistent Component Usage: In systems with established component libraries, specific components are intended for specific contexts. Free-form generation may misuse primitives, duplicate existing abstractions, or bypass higher-level components entirely. This creates parallel patterns that increase technical debt and weaken reuse.
Non-Deterministic Outputs: Identical prompts can yield structurally different layouts across executions. In production environments, this lack of determinism complicates testing, review processes, and content governance. Predictability is a requirement for scalable systems.
Brand and Compliance Drift: Without embedded context, models default to generic language and layout conventions. They lack awareness of regulatory constraints, accessibility standards, and brand-specific positioning. This introduces risk in industries where messaging and structure must adhere to policy.
Output Requiring Engineering Cleanup: Generated code or markup frequently requires normalization before integration. Engineers must refactor styles, align components with existing abstractions, and correct structural inconsistencies. The perceived acceleration at generation time is offset by downstream rework.

Free-form UI generation, therefore, conflicts with production architecture. Systems designed for reliability, reuse, and governance require structured constraints, not unconstrained synthesis.

The Missing Layer: Constraints

In this context, constraints define what can be generated, how it can be configured, how components can be composed, and how the result is represented at runtime. In real systems, those constraints are implemented through concrete mechanisms like component registry boundaries, schema validation, composition rules, and structured runtime output.

Component registry boundaries limit generation to approved React components. Instead of synthesizing arbitrary markup, the model assembles interfaces from predefined primitives such as Hero, FeatureGrid, or PricingTable. Prop schema enforcement validates inputs against typed definitions, ensuring that required fields, enumerations, and data shapes conform to system expectations. For example, a pricing component may require a structured array of tiers with defined attributes rather than free-form content.

Layout rules and composition limits restrict how components can be nested, preventing structurally invalid trees. Business context injection embeds brand, regulatory, or domain constraints directly into the generation process. Deterministic output structures, typically expressed as structured JSON, ensure predictable rendering and traceable state transitions across environments.

How Puck AI Implements Architectural Constraints

With Puck and Puck AI, the constraints we discussed above are implemented and enforced at the system level rather than inferred at prompt time.

Puck AI is a generative UI layer built on top of Puck’s React visual editor that enables page generation within predefined architectural boundaries.

It operates by assembling interfaces exclusively from registered React components instead of generating code. When a user requests something like “a landing page for an AI analytics platform,” the system composes that page from the components already configured in the application. The result is a deterministic component tree interpreted by Puck at runtime, keeping everything aligned with the existing design system and application logic.

In this workflow, generation is an orchestration process over defined primitives. The AI behavior is shaped by the editor configuration and the components supplied by the development team.

Your browser does not support the video tag.

Puck AI Characteristics

Puck AI demonstrates bounded generation through the following characteristics:

Generation from Registered React Components: The AI selects and composes only those components explicitly registered in the Puck configuration. If the system exposes Hero, FeatureGrid, and PricingTable, those are the only structural primitives available. No new layout elements are introduced outside the approved library.
Structured Page Schema Output: The result of the generation is a structured page definition that maps component types to their configured props and hierarchical placement. This schema is interpreted by Puck at runtime to render the interface. The AI does not directly control rendering logic.
Business Context and Brand Rules: Configuration layers allow teams to define tone, domain context, and structural expectations. These parameters influence how sections are assembled and how content fields are populated, ensuring alignment with product positioning and organizational standards.
Design System Preservation: Because rendering occurs through predefined components, spacing, typography, and layout behavior remain governed by the existing design system. Visual consistency is enforced by the component implementation, not by prompt phrasing.
Deterministic Behavior via Configuration: Through controlled configuration of available components, fields, and generation parameters, teams can narrow variability in output. The same structural intent produces predictable component trees aligned with system rules.

Decision Framework: When AI Should and Shouldn’t Generate UI

Final Verdict

Generative UI is effective when it operates within defined architectural boundaries and established component systems. When generation bypasses those constraints, inconsistency and downstream rework become unavoidable. Production-grade outcomes require schema enforcement, controlled composition, and system-level governance.

To see this model in practice, explore Puck for structured visual editing and try Puck AI for constrained page generation workflows. Puck AI is currently available in beta for teams evaluating controlled generative UI in production environments.

WebMCP: A Browser-Native Execution Model for AI Agents

Astrodevil — Sun, 22 Feb 2026 16:37:51 +0000

On February 13, Google announced the Early Preview of WebMCP, introducing a browser-native way for AI agents to interact with websites. To understand why this matters, consider how agents operate today.

AI agents interpret interfaces by parsing the DOM, inspecting accessibility trees, analyzing rendered pages, and then simulating clicks or inputs. Each action depends on inference over presentation layers. This increases token usage, adds latency, and often leads to brittle execution.

The limitation is structural. The web was designed for people navigating interfaces. Agents, however, require clearly defined capabilities they can invoke programmatically.

WebMCP addresses this gap by allowing websites to register structured JavaScript functions that agents can call directly within the browser runtime. These tools execute under existing session state and same-origin constraints, exposing only what the site explicitly defines.

The result is a more direct model of interaction that aligns frontend systems with the deterministic tool patterns already established in backend MCP integrations.

In this article, we examine WebMCP’s architecture, how it compares to traditional MCP, and what it signals for agent-driven web infrastructure.

Model Context Protocol (MCP): Current State and Browser Constraints

Model Context Protocol (MCP) established a structured model for how AI agents interact with external systems. Tools are defined with clear schemas, agents invoke them with structured inputs, and responses return in predictable formats. This ensures deterministic execution rather than relying on free-form reasoning.

The architecture is typically client–server. An agent connects to an MCP server that exposes tools wrapping APIs, databases, or internal services. This model fits naturally in backend environments where execution happens outside the browser.

Web applications operate under different assumptions. User identity, session state, and much of the application logic live inside the browser. Authentication flows depend on cookies and federated login systems tied to that session. An external MCP server does not automatically inherit this context, which complicates authorization and state management.

Because of this separation, agents interacting with web applications often end up controlling the interface itself instead of invoking structured capabilities.

WebMCP Technical Overview

WebMCP is a browser-native API that allows websites to expose structured, agent-callable tools directly within the page runtime. It adapts the conceptual model of Model Context Protocol schema-defined tools invoked by agents, but implements it specifically for client-side execution inside the browser.

At its core, WebMCP introduces a new browser surface:

navigator.modelContext

This interface allows a web page to register capabilities that AI agents can discover and invoke. Each tool consists of:

A name
A description
An input schema (structured definition of parameters)
An execution handler

Unlike traditional MCP, WebMCP does not rely on a separate JSON-RPC server. The web page itself becomes the tool provider. Execution occurs in the same JavaScript environment as the application logic.

The formal specification is being developed under the W3C Web Machine Learning Community Group and is available at: https://webmachinelearning.github.io/webmcp/

Tool Exposure and Execution Model

WebMCP defines how capabilities are exposed and how agents invoke them inside the browser runtime. It supports two exposure

1. Declarative API (HTML-based)

Forms can be annotated with metadata that enables automatic tool registration. The browser derives the tool definition from form inputs, enabling simple actions to be agent-callable without additional JavaScript.

2. Imperative API (JavaScript-based)

Developers can programmatically register tools using:

navigator.modelContext.registerTool({...})

This method provides full control over input schemas and execution logic, enabling dynamic, state-aware, or complex capabilities.

When an agent loads a WebMCP-enabled page:

The browser exposes the registered tools.
The agent inspects available capabilities.
The agent invokes a selected tool with structured parameters.
The handler executes inside the page runtime.
A structured response is returned to the agent.

The defining characteristic of WebMCP is locality. Tool execution happens inside the browser session, inheriting:

Current authentication state
Session cookies
Same-origin boundaries

This removes the need for an external transport layer or a separate authorization stack.

WebMCP focuses specifically on schema-defined tool invocation optimized for browser environments, adapting MCP concepts to client-side execution.

Core Architectural Components

WebMCP introduces a browser-mediated architecture that connects agents directly to application capabilities without external transport layers.

Below is the full execution path.

WebMCP defines a browser-mediated execution model that connects agents directly to declared application capabilities.

AI Agent: The agent discovers registered tools, selects one based on user intent, sends structured input that conforms to the declared schema, then receives structured output. Interaction occurs through explicit capabilities rather than direct interface manipulation.
Browser Runtime Control Plane: The browser exposes navigator.modelContext, which maintains the tool registry, validates inputs against schemas, routes invocations to the appropriate handler, enforces same origin boundaries, and executes handlers within the active page context. This removes the need for an external transport layer or separate MCP server.
Tool Layer Capability Surface: Each tool defines a named capability, its expected input schema, and an execution handler. These tools form a contract between the application and the agent. Only declared capabilities are accessible.
Application Execution Layer: Handlers run in the same JavaScript environment as the web application. They can access session cookies, rely on existing authentication state, call internal services, and update application state. Execution remains within the active browser session.

The overall flow is direct. The page loads and registers tools. The agent inspects available capabilities and invokes one with structured input. The browser validates the request, executes the handler inside the page runtime, and returns structured output to the agent.

Comparison with Traditional MCP and Browser Automation

WebMCP sits between backend MCP servers and browser automation frameworks. The differences become clearer when compared across architecture, execution model, and capability exposure.

Capability	Traditional MCP	Browser Automation (Selenium / Playwright)	WebMCP
Execution Location	External server	Inside browser via UI control	Inside browser via declared tools
Transport Layer	JSON-RPC or similar	WebDriver protocol	Browser-native API
Interaction Surface	Structured tools	DOM elements and selectors	Schema-defined tools
Session Inheritance	Requires coordination	Native to browser session	Native to browser session
Authentication Handling	Separate from browser	Uses active browser state	Uses active browser state
Dependency on UI Layout	None	High	None
Token Overhead	Low	High due to DOM inspection	Low due to structured schemas
Determinism	High	Medium, selector-dependent	High

Traditional MCP provides structured invocation but operates outside the browser context. Browser automation preserves session state but relies on interface manipulation. WebMCP combines structured schemas with in-browser execution, exposing declared capabilities without depending on layout or selectors.

Security Model and Execution Boundaries

WebMCP narrows the interaction surface between agents and web applications by constraining execution to explicitly declared tools.

Explicit Capability Exposure: Only registered tools are visible to the agent. The agent cannot arbitrarily traverse the DOM or trigger undocumented behaviors unless those capabilities are intentionally exposed.
Same Origin Enforcement: Tool execution occurs under the browser’s same-origin policy. A page can expose capabilities only within its own origin boundary. Cross-site execution is not permitted by default.
Session Inheritance: Tools execute within the active browser session. They inherit authentication state, cookies, and user context already established in the page. There is no additional credential exchange layer introduced by WebMCP itself.
Controlled Invocation Surface: Input parameters must conform to declared schemas. The browser validates structured inputs before routing execution, limiting malformed or unexpected calls.

WebMCP reduces the attack surface compared to interface-level automation by limiting what the agent can access to declared functions. It does not eliminate broader risks, such as prompt injection within tool logic, but it constrains execution to defined capability boundaries enforced by the browser runtime.

Chrome Early Preview and Built-In AI Strategy

WebMCP is available through Chrome’s Early Preview Program and can be enabled in experimental Chromium builds. The preview allows developers to test tool registration via navigator.modelContext and evaluate structured agent interaction inside the browser.

WebMCP complements Chrome’s Built-In AI APIs, which support on-device model execution. While Built-In AI enables local inference, WebMCP defines how agents interface with web applications through declared tools.

Together, these initiatives position the browser as both an AI execution environment and a structured capability surface for external agents.

InsForge and Model Context Protocol

InsForge is an open-source backend-as-a-service platform built for AI-assisted development. It provides core backend infrastructure, including database management, authentication, storage, serverless functions, and AI integrations. Its APIs are structured to support deterministic agent execution.

At its core, InsForge exposes a Model Context Protocol server that allows AI agents to interact with backend resources through schema-defined tools. Agents can inspect database schemas, execute queries, manage authentication, perform storage operations, and invoke backend functions using structured inputs and predictable responses.

This MCP-based design enables agents to complete backend workflows with clearer execution paths and reduced ambiguity. By exposing explicit capability contracts, InsForge supports reliable multi-step operations without relying on interface-level automation.

Summary

WebMCP gives AI agents a defined way to interact with web apps inside the browser. Instead of scraping the DOM or simulating clicks, agents call explicitly declared functions with typed schemas.

Those functions execute within the user’s active session and respect normal browser security boundaries. This makes agent behavior more predictable and easier to reason about.

InsForge leverages Model Context Protocol (MCP) to provide structured, schema-defined backend capabilities for AI agents, enabling deterministic execution and more reliable infrastructure for AI-native applications.

Try InsForge

Quickstart guide here

Early Preview of WebMCP