How to Build a LangGraph Research Agent that Embeds Dynamic Charts via MCP Apps (CopilotKit & Tako)

AI research assistants are getting smarter, but turning that reasoning into something usable is still hard.

Real research involves more than text: searching the web, exploring structured data, comparing sources, and gradually assembling a report. Doing that well requires state, tools, and UI -- not just a chat box.

Today, we will build a chat + canvas research assistant that searches the web, queries structured data via MCP, and embeds interactive charts directly into the report, while streaming progress to the frontend in real time.

We will use CopilotKit (Next.js) for the frontend, LangGraph to orchestrate the agent workflow, and MCP (Model Context Protocol) to connect external tools like Tako, which returns embeddable chart UI.

Check out CopilotKit's GitHub ⭐️

You will find architecture, key concepts, how state flows between the UI and the agent, how MCP Apps fit into the system, and a step-by-step guide to building this from scratch.

Let’s build it.

---

What is covered?

In summary, we will cover these topics in detail.

1. What are we building?
2. Tech Stack and Project Structure
3. Building the Frontend (CopilotKit)
4. Agent Orchestration (LangGraph)
5. Backend: MCP Integration & Tools
6. Running the Application
7. Complete Data Flow between UI ↔ agent

Here’s the GitHub Repo if you are interested in exploring it yourself.

---

1. What are we building?

We are building an AI research assistant with a live chat + canvas UI that searches, collects sources, and assembles a report while streaming progress to the frontend in real time.

It will:

• Turn a user question into a research plan (research question + data sub-questions)
• Run parallel retrieval: Tavily for web search + Tako via MCP apps for charts
• Deduplicate results into a clean resources list
• Generate a report with [CHART:title] placeholders and then embed the real charts inline
• Stream intermediate state (logs/resources/report/charts) so the UI updates as the agent works.

We will see all of these concepts in action as we build the agent.

Core Components

A few days ago, the Anthropic team shipped MCP Apps as an official MCP extension, meaning tools can return interactive UI that the host can render, not just JSON or text.

That’s the pattern we will use here: our agent will call Tako (over MCP) to fetch charts as embeddable UI, so interactive visualizations can drop straight into the report/canvas instead of us building chart components.

At a high level, this project is built by combining:

• CopilotKit for the agent chat + canvas UI
• AG‑UI for streaming agent ↔ UI events (protocol)
• LangGraph for stateful agent orchestration
• MCP Apps for external tools
• Tako for charts (search engine)
• Tavily for web search

For context: AG-UI (Agent–User Interaction Protocol) is an event-based protocol that standardizes this “agent ↔ frontend” real-time communication.

Because it’s transport-agnostic (SSE, WebSockets, etc.), the same frontend can work across different agent backends without custom wiring. CopilotKit uses AG-UI as its synchronization layer.

Here's a simplified call request → response flow of what will happen:

User question
   ↓
CopilotKit UI (chat + canvas)
   ↓
LangGraph agent workflow
   ├─ chat: interpret intent + plan
   ├─ search: parallel retrieval (Tavily + Tako via MCP Apps)
   └─ report: narrative + [CHART:*] markers
   ↓
Streaming state updates (AG‑UI events)
   ↓
Canvas renders report + embedded charts

---

2. Tech Stack and Architecture

At the core, we are going to use this stack for building the agent:

• Next.js 15 : frontend framework with TypeScript

• CopilotKit 1.50 : agent UI components + streaming runtime (@copilotkit/react-core, @copilotkit/react-ui, @copilotkit/runtime)

• LangGraph: stateful agent orchestration (StateGraph nodes, conditional routing)

• FastAPI & Uvicorn : HTTP server for the agent

• Tavily: web search API (tavily-python)

• Model Context Protocol (MCP): connects the agent to Tako (mcp, httpx)

• CopilotKit LangGraph SDK (Python): streams LangGraph state to CopilotKit (copilotkit)

• OpenAI: LLM provider (openai)

There are also other libraries used, like react-markdown & remark-gfm for report rendering, langgraph-checkpoint-sqlite for state persistence, and Shadcn UI for components.

See package.json and agents/python/pyproject.toml in the repo for the complete dependency list.

 

Project structure

This is how our directory will look.

The agents/python/ directory hosts the Python LangGraph agent, exposed via FastAPI, which orchestrates the workflow (chat, search, report), streams state updates, and calls external tools (Tavily, Tako via MCP).

The src/ directory hosts the Next.js frontend, including the UI components, shared types, and the CopilotKit API route (/api/copilotkit/route.ts) that bridges the frontend to the agent backend.

.
├── src/                           ← Next.js frontend (TypeScript)
│   ├── app/
│   │   ├── page.tsx               ← CopilotKit provider & model selector
│   │   ├── Main.tsx               ← Chat + ResearchCanvas split layout
│   │   └── api/
│   │       └── copilotkit/
│   │           └── route.ts      ← CopilotRuntime bridge to agent
│   ├── components/
│   │   ├── ResearchCanvas.tsx    ← Main canvas (orchestrates report + resources)
│   │   ├── Resources.tsx         ← Displays Tavily + Tako resources list
│   │   ├── MarkdownRenderer.tsx  ← Renders report + embeds charts
│   │   └── ui/                   ← Reusable Shadcn UI components
│   └── lib/
│       ├── types.ts                     ← AgentState type
│       ├── utils.ts                     ← Utility functions
│       └── model-selector-provider.tsx  ← Model selection context
│
├── agents/
│   ├── python/                        ← Python LangGraph agent (primary)
│   │   ├── src/
│   │   │   ├── agent.py               ← StateGraph definition & compile
│   │   │   └── lib/
│   │   │       ├── state.py           ← AgentState (Pydantic)
│   │   │       ├── model.py       ← LLM factory (OpenAI / Anthropic / Gemini)
│   │   │       ├── chat.py            ← Chat node & tool definitions
│   │   │       ├── search.py          ← Parallel Tavily + Tako search
│   │   │       ├── mcp_integration.py ← MCP client & iframe helpers
│   │   │       ├── download.py        ← Download node
│   │   │       └── delete.py          ← Delete node
│   │   ├── main.py                    ← FastAPI/Uvicorn entrypoint
│   │   ├── requirements.txt
│   │   └── pyproject.toml
│   │
│   └── src/                       ← TypeScript agent (optional, local dev)
│       └── server.ts              ← Express + CopilotRuntime
│
├── package.json                   ← Frontend deps & scripts
├── .env.example
├── .env.local
└── README.md

Here's the GitHub repository and deployed live at tako-copilotkit.vercel.app if you want to explore yourself. I will cover the implementation and all key concepts in the following sections.

The easiest way to follow along is to clone the repo.

git clone https://github.com/TakoData/tako-copilotkit.git
cd tako-copilotkit

Add necessary API Keys

You can copy the environment template (.env.example) by using this command to create a .env.local in the root directory.

cp .env.example .env.local

Add your OpenAI API Key, Tavily API Key and Tako API Key to the file. I have attached the docs link so it's easy to follow.

OPENAI_API_KEY=sk-proj-...
TAVILY_API_KEY=tvly-dev-...
TAKO_API_TOKEN=your_api_token_here
TAKO_MCP_URL=https://mcp.tako.com  # URL of Tako's MCP server
TAKO_URL=https://tako.com  # URL of Tako's main API

The Tako/MCP integration is an optional data source for the research agent. If you want to query charts or structured datasets via a Tako MCP server, provide TAKO_API_TOKEN and the related URLs.

Otherwise, you can leave these unset & the agent will continue to work normally. For this tutorial, we will be using it.

OpenAI API Key

 

Tavily API Key

 

Tako API Key

---

3. Frontend: wiring the agent to the UI

Let's first build the frontend part.

The src/ directory hosts the Next.js frontend, including the UI components, shared types, and the CopilotKit API route (/api/copilotkit/route.ts) that bridges the frontend to the Python agent backend.

At a high level, the frontend is responsible for:

• Sending user queries to the agent backend via CopilotChat
• Receiving and rendering real-time state updates from the agent
• Embedding Tako chart iframes in the rendered report
• Managing the CopilotKit runtime bridge between UI and Python agent

If you are building this from scratch, the easiest approach is to copy the existing package.json from the repo.

It already includes all the required dependencies for CopilotKit, LangGraph integration, UI components, and local dev tooling, so you don’t have to assemble them manually.

I'm only covering the core frontend dependencies you actually need to understand.

Step 1: CopilotKit Provider & Layout

Install the necessary packages:

npm install @copilotkit/react-core @copilotkit/react-ui @copilotkit/runtime

• @copilotkit/react-core: Core React hooks and context that connect your UI to the agent backend via AG-UI protocol

• @copilotkit/react-ui: Ready-made UI components like <CopilotChat /> for building AI chat interfaces

• @copilotkit/runtime: Server-side runtime that exposes an API endpoint and bridges the frontend with the LangGraph agent using HTTP and SSE

Everything else in the repo (radix, tailwind, react-split, etc.) is there to support layout, styling, and developer experience -- not the core agent wiring.

The <CopilotKit> component must wrap the agent-aware parts of your application and point to the runtime endpoint so it can communicate with the agent backend.

This is the main entry point (page.tsx):

// page.tsx
"use client";

import { CopilotKit } from "@copilotkit/react-core";
import Main from "./Main";
import {
  ModelSelectorProvider,
  useModelSelectorContext,
} from "@/lib/model-selector-provider";

export default function ModelSelectorWrapper() {
  return (
    <ModelSelectorProvider>
      <Home />
    </ModelSelectorProvider>
  );
}

function Home() {
  const { agent, lgcDeploymentUrl } = useModelSelectorContext();

  // This logic is implemented to demonstrate multi-agent frameworks in this demo project.
  // There are cleaner ways to handle this in a production environment.
  const runtimeUrl = lgcDeploymentUrl
    ? `/api/copilotkit?lgcDeploymentUrl=${lgcDeploymentUrl}`
    : `/api/copilotkit${
        agent.includes("crewai") ? "?coAgentsModel=crewai" : ""
      }`;

  return (
    <div style={{ height: "100vh", overflow: "hidden" }}>
      <CopilotKit runtimeUrl={runtimeUrl} showDevConsole={false} agent={agent}>
        <Main />
      </CopilotKit>
    </div>
  );
}

Here, ModelSelectorProvider is a convenience for switching agents/models in development.

 

Step 2: Chat + Canvas Layout

The Main.tsx component sets up the core UI. It sets up:

• the chat interface (CopilotChat)
• the research canvas (where results & charts appear)
• state binding to the agent

// app/Main.tsx
import { useCoAgent } from "@copilotkit/react-core";
import { CopilotChat } from "@copilotkit/react-ui";
import Split from "react-split";
import { ResearchCanvas } from "@/components/ResearchCanvas";
import { ChatInputWithModelSelector } from "@/components/ChatInputWithModelSelector";
import { AgentState } from "@/lib/types";
import { useModelSelectorContext } from "@/lib/model-selector-provider";

export default function Main() {
  const { model, agent } = useModelSelectorContext();

  const { state, setState } = useCoAgent<AgentState>({
    name: agent,
    initialState: {
      model,
      research_question: "",
      resources: [],
      report: "",
      logs: [],
    },
  });

  return (
    <div style={{ height: "100%", display: "flex", flexDirection: "column" }}>
      <h1>Research Helper</h1>

      <Split sizes={[30, 70]} style={{ flex: 1, display: "flex" }}>
        {/* Chat Panel */}
        <CopilotChat
          Input={ChatInputWithModelSelector}
          onSubmitMessage={async () => {
            setState({ ...state, logs: [] });
            await new Promise((r) => setTimeout(r, 30));
          }}
        />

        {/* Canvas Panel */}
        <ResearchCanvas />
      </Split>
    </div>
  );
}

Here's what's happening:

• useCoAgent<AgentState>: bi-directional state synchronization between UI and agent. When the agent emits state updates via copilotkit_emit_state(), this hook automatically picks them up.

• CopilotChat: drop-in chat UI that sends messages to the agent and renders tool calls inline

• ResearchCanvas: custom component that renders the streaming report, resources list, and embedded charts

• Split: provides a resizable split-pane layout

 

Step 3: Building Key Components

I'm only covering the core logic behind the main components since the overall code is huge. You can find all the components in the repository at src\components.

These components use CopilotKit hooks (like useCoAgentStateRender) to tie everything together.

✅ Research Canvas Component

ResearchCanvas.tsx is where the agent’s current state is rendered:

• the accumulating report text
• linked resources
• embedded charts (iframes from MCP Apps like Tako)

This component listens to the agent state and displays elements as they arrive. It translates [CHART:title] markers into real embedded charts. This pattern is part of CopilotKit's support for generative UI.

// core logic
import { useCoAgent, useCoAgentStateRender, useCopilotAction } from "@copilotkit/react-core";
import { MarkdownRenderer } from "./MarkdownRenderer";
import { AgentState } from "@/lib/types";
import { useRef, useEffect } from "react";

export function ResearchCanvas() {
  const { state, setState } = useCoAgent<AgentState>({
    name: "research_agent",
  });

  // Use refs to prevent flicker during streaming updates
  const lastReportRef = useRef<string>("");
  const lastResourcesRef = useRef<Resource[]>([]);

  if (state.report) lastReportRef.current = state.report;
  if (state.resources?.length) lastResourcesRef.current = state.resources;

  const report = state.report || lastReportRef.current;
  const resources = state.resources || lastResourcesRef.current;

  // Render progress logs during execution
  useCoAgentStateRender({
    name: "research_agent",
    render: ({ state, status }) => {
      if (state.logs?.length) return <Progress logs={state.logs} />;
      return null;
    },
  });

  // Generative UI: Agent requests deletion confirmation from user
  useCopilotAction({
    name: "DeleteResources",
    description: "Prompt user for resource delete confirmation",
    available: "remote",
    parameters: [{ name: "urls", type: "string[]" }],
    renderAndWait: ({ args, handler }) => (
      <div>
        <h3>Delete these resources?</h3>
        <Resources resources={resources.filter(r => args.urls.includes(r.url))} />
        <button onClick={() => handler("YES")}>Delete</button>
        <button onClick={() => handler("NO")}>Cancel</button>
      </div>
    ),
  });

  return (
    <div>
      <h2>Research Question</h2>
      <div>{state.research_question || "Agent will identify your question..."}</div>

      {/* Resources Panel */}
      <Resources resources={resources} />

      {/* Report Panel with Embedded Charts */}
      <MarkdownRenderer content={report} />
    </div>
  );
}

The full component also includes collapsible sections for Tako charts vs web resources, and resource management dialogs. Check src/components/ResearchCanvas.tsx.

 

✅ Chart Embedding Pattern

The MCP Apps-style embedding happens in MarkdownRenderer.tsx. It embeds Tako charts without flickering during streaming updates. Instead of simple regex replacement, it uses a stable iframe pattern.

// core logic
import React, { useEffect, useRef, useMemo } from "react";
import ReactMarkdown from "react-markdown";

// Global registry: iframe src never changes for a given ID
const iframeRegistry = new Map<string, string>();

export function MarkdownRenderer({ content }: { content: string }) {
  // Split content into text and iframe segments
  const segments = useMemo(() => {
    const embedPattern = /<!doctype html>[\s\S]*?<\/html>/gi;
    const parts = [];
    let lastIndex = 0;

    content.replace(embedPattern, (match, offset) => {
      // Add text before iframe
      if (offset > lastIndex) {
        parts.push({ type: 'text', content: content.slice(lastIndex, offset) });
      }

      // Extract iframe src and create stable ID
      const src = match.match(/src=["']([^"']+)["']/)?.;
      if (src) {
        const id = `embed-${src.replace(/[^a-zA-Z0-9]/g, '-').slice(0, 50)}`;
        iframeRegistry.set(id, src);
        parts.push({ type: 'iframe', id, src });
      }

      lastIndex = offset + match.length;
      return match;
    });

    // Add remaining text
    if (lastIndex < content.length) {
      parts.push({ type: 'text', content: content.slice(lastIndex) });
    }

    return parts;
  }, [content]);

  return (
    <div>
      {segments.map((segment, i) => 
        segment.type === 'text' 
          ? <ReactMarkdown key={i}>{segment.content}</ReactMarkdown>
          : <StableIframe key={segment.id} src={segment.src} />
      )}
    </div>
  );
}

// Memoized iframe - never re-renders once mounted
const StableIframe = React.memo(({ src }: { src: string }) => {
  const iframeRef = useRef<HTMLIFrameElement>(null);

  // Handle Tako resize messages
  useEffect(() => {
    const handleResize = (event: MessageEvent) => {
      if (event.data?.type === "tako::resize" && 
          iframeRef.current?.contentWindow === event.source) {
        iframeRef.current.style.height = `${event.data.height}px`;
      }
    };
    window.addEventListener("message", handleResize);
    return () => window.removeEventListener("message", handleResize);
  }, []);

  return <iframe ref={iframeRef} src={src} style={{ height: "400px" }} />;
});

Tako returns fully-formed chart UI via MCP Apps so the frontend only needs to embed the iframe. Here's what's happening:

• Iframes are rendered as stable siblings (outside ReactMarkdown), preventing remounts during streaming updates

• Each embed uses a deterministic ID derived from its src, ensuring consistent DOM identity

• Resize events from Tako (tako::resize) are handled via postMessage to keep charts responsive

• rehypeRaw: Allows ReactMarkdown to render raw HTML (the iframe elements)

Check out the complete code at src/components/MarkdownRenderer.tsx.

 

✅ Model Selector Provider

The ModelSelectorProvider (src\lib\model-selector-provider.tsx) is a convenience wrapper for switching between different agent configurations during development. It reads URL query parameters to determine which agent/model to use.

"use client";

import React from "react";
import { createContext, useContext, useState, ReactNode } from "react";

type ModelSelectorContextType = {
  model: string;
  setModel: (model: string) => void;
  agent: string;
  lgcDeploymentUrl?: string | null;
  hidden: boolean;
  setHidden: (hidden: boolean) => void;
};

const ModelSelectorContext = createContext<ModelSelectorContextType | undefined>(undefined);

export const ModelSelectorProvider = ({ children }: { children: ReactNode }) => {
  // Read model from URL query params (e.g., ?coAgentsModel=google_genai)
  const model =
    globalThis.window === undefined
      ? "openai"
      : new URL(window.location.href).searchParams.get("coAgentsModel") ?? "openai";

  const [hidden, setHidden] = useState<boolean>(false);

  const setModel = (model: string) => {
    const url = new URL(window.location.href);
    url.searchParams.set("coAgentsModel", model);
    window.location.href = url.toString();
  };

  // Optional: LangGraph Cloud deployment URL for production
  const lgcDeploymentUrl =
    globalThis.window === undefined
      ? null
      : new URL(window.location.href).searchParams.get("lgcDeploymentUrl");

  // Map model to agent name
  let agent = "research_agent";
  if (model === "google_genai") {
    agent = "research_agent_google_genai";
  } else if (model === "crewai") {
    agent = "research_agent_crewai";
  }

  return (
    <ModelSelectorContext.Provider
      value={{ model, agent, lgcDeploymentUrl, hidden, setModel, setHidden }}
    >
      {children}
    </ModelSelectorContext.Provider>
  );
};

export const useModelSelectorContext = () => {
  const context = useContext(ModelSelectorContext);
  if (context === undefined) {
    throw new Error("useModelSelectorContext must be used within a ModelSelectorProvider");
  }
  return context;
};

The repo also includes UI components like Resources.tsx (resource cards), Progress.tsx (logs display), EditResourceDialog.tsx and AddResourceDialog.tsx for resource management, plus ChatInputWithModelSelector for model switching during development.

 

Step 4: Runtime Bridge (API Route)

This Next.js API route acts as a thin proxy between the browser and the Python LangGraph agent. It:

• Accepts CopilotKit requests from the UI
• Forwards them to the LangGraph agent via HTTP
• Streams agent state and events back to the frontend via SSE

Instead of letting the frontend talk to the FastAPI agent directly, all requests go through a single endpoint /api/copilotkit.

import {
  CopilotRuntime,
  copilotRuntimeNextJSAppRouterEndpoint,
  EmptyAdapter,
} from "@copilotkit/runtime";
import {
  LangGraphHttpAgent,
  LangGraphAgent,
} from "@copilotkit/runtime/langgraph";
import { NextRequest } from "next/server";

// const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
// const llmAdapter = new OpenAIAdapter({ openai } as any);
const llmAdapter = new EmptyAdapter();
const langsmithApiKey = process.env.LANGSMITH_API_KEY as string;

export const POST = async (req: NextRequest) => {
  const searchParams = req.nextUrl.searchParams;
  const deploymentUrl =
    searchParams.get("lgcDeploymentUrl") || process.env.LGC_DEPLOYMENT_URL;

  const baseUrl =
    process.env.REMOTE_ACTION_URL || "http://localhost:2024/copilotkit";

  let runtime = new CopilotRuntime({
    agents: {
      research_agent: new LangGraphHttpAgent({
        url: `${baseUrl}/agents/research_agent`,
      }),
      research_agent_google_genai: new LangGraphHttpAgent({
        url: `${baseUrl}/agents/research_agent_google_genai`,
      }),
    },
  });

  if (deploymentUrl) {
    runtime = new CopilotRuntime({
      agents: {
        research_agent: new LangGraphAgent({
          deploymentUrl,
          langsmithApiKey,
          graphId: "research_agent",
        }),
        research_agent_google_genai: new LangGraphAgent({
          deploymentUrl,
          langsmithApiKey,
          graphId: "research_agent_google_genai",
        }),
      },
    });
  }

  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter: llmAdapter,
    endpoint: "/api/copilotkit",
  });

  return handleRequest(req);
};

The frontend is fully reactive: as the agent works, the UI updates in real time.

---

4. Agent Orchestration (LangGraph)

Now that the frontend can communicate with an agent, we need to define how the agent thinks and operates. The backend agent lives in agents/python/src/.

It uses LangGraph to orchestrate a multi-step research workflow (chat → search → download → report...) through well-defined steps (nodes), each responsible for a specific task.

Each step:

• receives the current AgentState

• performs one responsibility

• returns partial state updates

• decides what node runs next

Step 1: Define the StateGraph (agent.py)

The workflow is defined using LangGraph's StateGraph in agent.py. It models the research assistant as a stateful workflow, where each step (node) operates on shared state and decides which step to run next.

At a high level:

• AgentState is the shared memory across the entire agent (messages, report, resources, logs, ...)

• Each node is an async function that:

  • reads from the current state
  • updates part of that state
  • returns control-flow instructions

• Edges control execution flow between nodes (download → chat → search → download loop)

One important detail is the interrupt_after hook, which allows the agent to pause execution for user confirmation before deleting a resource (delete_node).

# agents/python/src/agent.py
import os

from langgraph.graph import StateGraph

from src.lib.chat import chat_node
from src.lib.delete import delete_node, perform_delete_node
from src.lib.download import download_node
from src.lib.search import search_node
from src.lib.state import AgentState

workflow = StateGraph(AgentState)
workflow.add_node("download", download_node)
workflow.add_node("chat_node", chat_node)
workflow.add_node("search_node", search_node)
workflow.add_node("delete_node", delete_node)
workflow.add_node("perform_delete_node", perform_delete_node)


workflow.set_entry_point("download")
workflow.add_edge("download", "chat_node")
workflow.add_edge("delete_node", "perform_delete_node")
workflow.add_edge("perform_delete_node", "chat_node")
workflow.add_edge("search_node", "download")

# Compile with interrupt support for deletion confirmation
compile_kwargs = {"interrupt_after": ["delete_node"]}

# Check if we're running in LangGraph API mode
if os.environ.get("LANGGRAPH_FASTAPI", "false").lower() == "false":
    # When running in LangGraph API, don't use a custom checkpointer
    graph = workflow.compile(**compile_kwargs)
else:
    # For CopilotKit and other contexts, use MemorySaver
    from langgraph.checkpoint.memory import MemorySaver

    memory = MemorySaver()
    compile_kwargs["checkpointer"] = memory
    graph = workflow.compile(**compile_kwargs)

Why this structure works well:

• The download → chat → search → download loop naturally supports iterative research

• Optional checkpointing (MemorySaver) allows the same graph to run locally or on LangGraph Cloud (state persistence)

 

Step 2: Chat Node & Tools (chat.py)

The chat node (chat_node) is the brain of the agent. It orchestrates the LLM, defines tools, manages state streaming, and handles chart injection. It:

• Define LLM-callable tools (Search, WriteReport, WriteResearchQuestion, GenerateDataQuestions)

• Configure intermediate state emission (stream research question and data questions as they are generated)

• Build prompts with available resources and Tako chart context

• Inject Tako chart iframes into the report after generation

• Route to next node based on tool calls

Here are the core tools (agents/python/src/lib/chat.py).

@tool
def Search(queries: List[str]):
    """A list of one or more search queries to find good resources."""

@tool
def WriteReport(report: str):
    """Write the research report."""

@tool
def WriteResearchQuestion(research_question: str):
    """Write the research question."""

@tool
def GenerateDataQuestions(questions: List[DataQuestion]):
    """
    Generate 3-6 ATOMIC data-focused questions for Tako search.

    CRITICAL: Each query should ask for ONE metric/dimension only.
    BAD: "Compare US GDP and inflation" → compound query
    GOOD: "US GDP 2020-2024", "US inflation rate" → separate atomic queries
    """

Here is the chat node logic.

async def chat_node(state: AgentState, config: RunnableConfig):
    # Configure intermediate state emission
    config = copilotkit_customize_config(
        config,
        emit_intermediate_state=[
            {"state_key": "research_question", "tool": "WriteResearchQuestion", "tool_argument": "research_question"},
            {"state_key": "data_questions", "tool": "GenerateDataQuestions", "tool_argument": "questions"},
        ],
    )

    # Build Tako charts context for LLM
    tako_charts_map = {}
    available_tako_charts = []
    for resource in state["resources"]:
        if resource.get("resource_type") == "tako_chart":
            title = resource.get("title", "")
            card_id = resource.get("card_id")
            description = resource.get("description", "")
            tako_charts_map[title] = {"card_id": card_id, "embed_url": resource.get("embed_url")}
            available_tako_charts.append(f"  - **{title}**\n    Description: {description}")

    # Build prompt with workflow instructions
    response = await model.bind_tools([Search, WriteReport, WriteResearchQuestion, GenerateDataQuestions]).ainvoke([
        SystemMessage(content=f"""
        You are a research assistant. 

        WORKFLOW:
        1. Use WriteResearchQuestion to extract the core research question
        2. Use GenerateDataQuestions to create 3-6 ATOMIC data questions (one metric per query)
        3. Use Search for web resources
        4. Write a comprehensive report using WriteReport

        AVAILABLE CHARTS ({len(tako_charts_map)}):
        {available_tako_charts_str}

        WRITING GUIDELINES:
        - Write a detailed analysis with substantial narrative text
        - For EACH chart, write 1-2 paragraphs discussing insights and trends
        - DO NOT include chart markers or image syntax - charts will be inserted automatically
        - Reference specific data points from chart descriptions
        """),
        *state["messages"],
    ], config)

    # Handle tool calls
    if response.tool_calls:
        if response.tool_calls[0]["name"] == "WriteReport":
            report = response.tool_calls[0]["args"]["report"]

            # Clean up any markdown images LLM incorrectly added
            report = re.sub(r'!\[[^\]]*\]\([^)]+\)', '', report)

            # Inject charts using second LLM pass
            if tako_charts_map:
                chart_list = "\n".join([f"- {title}" for title in tako_charts_map.keys()])

                inject_response = await model.ainvoke([
                    SystemMessage(content=f"""Insert [CHART:exact_title] markers at appropriate positions.

                    AVAILABLE CHARTS:
                    {chart_list}

                    RULES:
                    - Place markers AFTER relevant paragraphs
                    - NEVER place more than two charts consecutively
                    - Each chart is used exactly once
                    """),
                    HumanMessage(content=f"Insert chart markers:\n\n{report}")
                ])

                report_with_markers = inject_response.content

                # Replace markers with actual iframe HTML
                async def replace_marker(match):
                    chart_title = match.group(1).strip()
                    chart_info = tako_charts_map.get(chart_title)
                    if chart_info:
                        iframe_html = await get_visualization_iframe(
                            item_id=chart_info.get("card_id"),
                            embed_url=chart_info.get("embed_url")
                        )
                        return "\n" + iframe_html + "\n" if iframe_html else ""
                    return ""

                # Apply replacements
                markers = list(re.finditer(r'\[CHART:([^\]]+)\]', report_with_markers))
                processed_report = report_with_markers
                for match in reversed(markers):
                    replacement = await replace_marker(match)
                    processed_report = processed_report[:match.start()] + replacement + processed_report[match.end():]

            return Command(goto="chat_node", update={"report": processed_report})

        elif response.tool_calls[0]["name"] == "GenerateDataQuestions":
            data_questions = response.tool_calls[0]["args"]["questions"]
            return Command(goto="search_node", update={"data_questions": data_questions})

    # Route based on tool call
    goto = "__end__"
    if response.tool_calls:
        tool_name = response.tool_calls[0]["name"]
        if tool_name == "Search":
            goto = "search_node"
        elif tool_name == "DeleteResources":
            goto = "delete_node"

    return Command(goto=goto, update={"messages": response})

How this node works in practice:

• The LLM never writes free-form text directly. Every meaningful action goes through an explicit tool (WriteResearchQuestion, GenerateDataQuestions, Search, WriteReport). This keeps state changes deterministic and easy to reason about.

• Intermediate state (research question and data questions) is streamed immediately using copilotkit_customize_config. The report itself is intentionally not streamed to avoid UI flicker while charts are being injected later.

• Chart handling is deliberately two-phase: the model first writes a clean narrative report, then a second pass inserts [CHART:title] markers, which are finally replaced with real Tako iframes. To keep the generation stable and layout-safe.

• Control flow is explicit. Each tool call results in a Command that routes execution to the next node (search_node, delete_node, or back to chat_node), making the agent’s behavior predictable and easier to debug.

 

Step 3: Search Node (search.py)

The search node (search_node) runs web search (Tavily) and structured chart data search (Tako via MCP) in parallel, streams progress to the UI, deduplicates results, and extracts the most relevant resources.

• Tavily and Tako searches run concurrently using asyncio.gather, so web pages and structured datasets are fetched at the same time

• TavilyClient is synchronous (uses requests library), so we wrap it with loop.run_in_executor() to avoid blocking the async event loop

• As each search completes, copilotkit_emit_state is called to update state.logs, allowing the frontend to show real-time progress

• Tako chart results are deduplicated using the title to avoid embedding the same visualization multiple times

• Web results and chart results are passed through an LLM tool (ExtractResources) to select only the most relevant resources

• Each resource is tagged as web or tako_chart, which the frontend uses to decide whether to render a link or an embedded chart

Here is the code for Tavily async wrapper (bridging sync client with async workflow):

# agents/python/src/lib/search.py

from tavily import TavilyClient
import asyncio

_tavily_client = None

def get_tavily_client():
    """Lazy initialization to avoid repeated client creation"""
    global _tavily_client
    if _tavily_client is None:
        _tavily_client = TavilyClient(api_key=os.getenv("TAVILY_API_KEY"))
    return _tavily_client

async def async_tavily_search(query: str) -> Dict[str, Any]:
    """
    Async wrapper for synchronous Tavily client.
    TavilyClient.search() is blocking, so we run it in a thread pool.
    """
    loop = asyncio.get_event_loop()
    return await loop.run_in_executor(
        None,
        lambda: get_tavily_client().search(
            query=query,
            search_depth="advanced",
            include_answer=True,
            max_results=5,
        ),
    )

Search node implementation (parallel execution and resource extraction):

# agents/python/src/lib/search.py (core search pattern)
async def search_node(state: AgentState, config: RunnableConfig):
    """
    Perform Tavily web search and Tako MCP chart search in parallel.
    Stream progress, dedupe results, extract top resources.
    """
    # Extract queries from tool_calls or fall back to research_question
    # (Actual code handles dual routing: Search tool OR GenerateDataQuestions)
    queries = state.get("search_queries", [])
    data_questions = state.get("data_questions", [])

    # PHASE 1: Build parallel tasks
    tavily_tasks = [async_tavily_search(query) for query in queries]
    tako_tasks = [
        search_knowledge_base(q["question"], search_effort="fast")
        for q in data_questions
    ]
    all_tasks = tavily_tasks + tako_tasks

    # Initialize logs for progress tracking
    for query in queries:
        state["logs"].append({"message": f"Web search: {query}", "done": False})
    for q in data_questions:
        state["logs"].append({"message": f"Tako search: {q['question']}", "done": False})

    if all_tasks:
        await copilotkit_emit_state(config, state)

    # PHASE 2: Run all searches concurrently
    all_results = await asyncio.gather(*all_tasks, return_exceptions=True)

    # Split results back into Tavily and Tako
    num_tavily = len(tavily_tasks)
    tavily_results = all_results[:num_tavily]
    tako_results_raw = all_results[num_tavily:]

    # PHASE 3: Stream completion and process results
    search_results = []
    tako_results = []

    # Process Tavily results
    for i, result in enumerate(tavily_results):
        if isinstance(result, Exception):
            search_results.append({"error": str(result)})
        else:
            search_results.append(result)
        state["logs"][i]["done"] = True
        await copilotkit_emit_state(config, state)

    # Process Tako results
    tako_log_offset = num_tavily
    for i, result in enumerate(tako_results_raw):
        if isinstance(result, Exception):
            tako_results.append({"error": str(result)})
        elif result:
            tako_results.extend(result)
        state["logs"][tako_log_offset + i]["done"] = True
        await copilotkit_emit_state(config, state)

    # PHASE 4: Deduplicate Tako charts by title
    # (Same chart may appear in multiple searches)
    seen_titles = set()
    unique_tako = []
    for chart in tako_results:
        title = chart.get("title", "")
        if title and title not in seen_titles:
            seen_titles.add(title)
            unique_tako.append(chart)
        elif not title:
            unique_tako.append(chart)

    # PHASE 5: Use LLM to extract the most relevant resources
    model = get_model(state)

    # Prepare search message with both web and Tako results
    search_message = f"Web search results: {search_results}"
    if unique_tako:
        search_message += f"\n\nTako chart results: {unique_tako}"

    # Build extraction prompt
    extract_messages = [
        SystemMessage(content="""
        Extract the 3-5 most relevant resources from the search results.
        Prioritize Tako charts when they match the research topic.
        """),
        *state["messages"],
        SystemMessage(content=f"Search results:\n{search_message}")
    ]

    # Call LLM with ExtractResources tool
    response = await model.bind_tools(
        [ExtractResources], 
        tool_choice="ExtractResources"
    ).ainvoke(extract_messages, config)

    # PHASE 6: Tag resources with type and attach content
    resources = response.tool_calls[0]["args"]["resources"]

    for resource in resources:
        # Check if this is a Tako chart by matching URL or title
        is_tako = False
        for tako_result in unique_tako:
            if (resource.get("url") == tako_result.get("url") or 
                resource.get("title") == tako_result.get("title")):
                is_tako = True
                resource["resource_type"] = "tako_chart"
                resource["source"] = "Tako"
                resource["card_id"] = tako_result.get("id")
                resource["embed_url"] = tako_result.get("embed_url")
                resource["content"] = tako_result.get("description", "")
                break

        if not is_tako:
            resource["resource_type"] = "web"
            resource["source"] = "Tavily Web Search"
            # Find matching Tavily result for content summary
            for search_result in search_results:
                if isinstance(search_result, dict) and "results" in search_result:
                    for tavily_item in search_result["results"]:
                        if tavily_item.get("url") == resource.get("url"):
                            resource["content"] = tavily_item.get("content", "")
                            break

    # Deduplicate and add to state resources
    # (Production code enforces MAX_TOTAL_RESOURCES limit and generates iframe HTML)
    state["resources"].extend(resources)

    return state

This simplified version shows the core search pattern.

The production code adds fallback searches when Tako returns no results, generates iframe HTML for chart embedding via get_visualization_iframe(), enforces resource limits, and handles dual routing for different tool paths.

Check out the complete implementation at agents/python/src/lib/search.py.

---

5. Backend: MCP Integration & Tools

At this point, the UI is wired, the agent graph is defined, and the execution flow is clear. What’s left is the backend work that actually fetches data, talks to external systems, and turns results into something the agent can reason about.

The most interesting part here is the MCP integration for Tako charts. The rest of the backend nodes are intentionally simple.

MCP Apps Pattern for Tako Charts

Tako is a data visualization platform with searchable charts on economics, markets, demographics, and more.

What Tako provides:

• Searchable chart catalog via knowledge_search tool

• Chart search with configurable effort: "fast" (quick lookups), "medium", or "deep" (thorough searches)

• Ready-to-embed iframe HTML via open_chart_ui tool

• Auto-resizing charts via postMessage protocol

This is the core idea behind MCP Apps: tools return UI components, not just data.

This module mcp_integration.py is where the agent connects to Tako via the Model Context Protocol (MCP). I have broken the core parts of this integration, so it's easier to follow along.

The MCP Client: Session Management

The SimpleMCPClient class handles the Model Context Protocol lifecycle:

• SSE connection establishes and maintains the session ID
• HTTP messages send JSON-RPC tool calls to the server
• Automatic reconnection on session expiry (404/410 errors)

# core client logic -- agents/python/src/lib/mcp_integration.py

class SimpleMCPClient:
    """
    Minimal MCP client following the Model Context Protocol spec.
    Handles connection lifecycle, session management, and message passing.
    """

    async def connect(self):
        """Connect to MCP server via SSE to get session ID."""
        self._sse_task = asyncio.create_task(self._sse_reader())

        # Wait for session_id establishment
        for _ in range(50):
            if self.session_id:
                return True
            await asyncio.sleep(0.1)
        return False

    async def reconnect(self):
        """Reconnect with new session on expiry."""
        if self._sse_task:
            self._sse_task.cancel()

        # Clear state and reconnect
        self.session_id = None
        self._responses.clear()
        await self.connect()
        await self.initialize()

    async def _send(self, method: str, params: dict = None, _retry: bool = True):
        """
        Send JSON-RPC message to server via HTTP.
        Automatically reconnects and retries once on session expiration (410/404).
        """
        msg = {"jsonrpc": "2.0", "id": self.message_id, "method": method}
        if params:
            msg["params"] = params

        future = asyncio.get_event_loop().create_future()
        self._responses[self.message_id] = future

        try:
            resp = await self._client.post(
                f"{self.base_url}/messages/?session_id={self.session_id}",
                json=msg,
            )

            # Handle session expiration
            if resp.status_code in (404, 410):
                if _retry:
                    await self.reconnect()
                    return await self._send(method, params, _retry=False)
                raise SessionExpiredException("Session expired")

        except httpx.HTTPStatusError as e:
            if e.response.status_code in (404, 410) and _retry:
                await self.reconnect()
                return await self._send(method, params, _retry=False)
            raise

        return await asyncio.wait_for(future, timeout=120.0)

# Global client instance (reused across all calls)
_mcp_client: Optional[SimpleMCPClient] = None

A single global client instance maintains the session across all tool calls. If a session expires mid-conversation, the client automatically reconnects and retries once.

 

Searching Tako's Knowledge Base

The search_knowledge_base function wraps the MCP knowledge_search tool to find relevant charts.

Each result becomes a Resource with resource_type = "tako_chart". The search_node runs these queries in parallel with Tavily web searches using asyncio.gather() for speed (as we discussed in the previous section).

async def search_knowledge_base(
    query: str,
    count: int = 5,
    search_effort: str = "fast"
) -> List[Dict[str, Any]]:
    """
    Search Tako knowledge base via MCP 'knowledge_search' tool.

    Returns a list of charts with metadata (id, title, description, embed_url).
    """
    result = await _call_mcp_tool("knowledge_search", {
        "query": query,
        "api_token": TAKO_API_TOKEN,
        "count": count,
        "search_effort": search_effort
    })

    if result and "results" in result:
        formatted_results = []
        for item in result["results"]:
            item_id = item.get("card_id") or item.get("id")
            title = item.get("title", "")
            description = item.get("description", "")
            embed_url = f"{DATA_SOURCE_URL}/embed/{item_id}/?theme=dark" if item_id else None

            formatted_results.append({
                "type": "data_visualization",
                "id": item_id,
                "title": title,
                "description": description,
                "embed_url": embed_url,
                "url": f"{DATA_SOURCE_URL}/card/{item_id}"
            })

        return formatted_results

    return []

 

Fetching Embeddable Chart UI

The get_visualization_iframe function is where MCP Apps shines. Instead of returning chart data, it returns complete iframe HTML.

There are two paths involved here:

• MCP UI (open_chart_ui): Returns complete iframe HTML with interactive controls and resize logic

• Fallback embed URL: Generates basic iframe HTML manually if the MCP call fails

The returned HTML is injected directly into the markdown report. The frontend's MarkdownRenderer extracts these iframe blocks and renders them as siblings to the markdown content (preventing flickering during streaming updates).

async def get_visualization_iframe(item_id: str = None, embed_url: str = None) -> Optional[str]:
    """
    Get embeddable iframe HTML for a Tako chart.
    Uses the MCP 'open_chart_ui' tool with automatic session reconnection.

    Returns ready-to-embed iframe HTML with resize script.
    """
    if item_id:
        result = await _call_mcp_tool("open_chart_ui", {
            "pub_id": item_id,
            "dark_mode": True,
            "width": 900,
            "height": 600
        })

        # Extract HTML from MCP response
        if result and result.get("type") == "resource":
            resource = result.get("resource", {})
            html_content = resource.get("htmlString") or resource.get("text")
            if html_content:
                return html_content

    # Fallback: Generate iframe HTML with embed_url
    if embed_url:
        return f'''<iframe
  width="100%"
  height="600"
  src="{embed_url}"
  scrolling="no"
  frameborder="0"
></iframe>

<script type="text/javascript">
window.addEventListener("message", function(e) {{
  if (e.data.type !== "tako::resize") return;

  for (let iframe of document.querySelectorAll("iframe")) {{
    if (iframe.contentWindow === e.source) {{
      iframe.style.height = e.data.height + "px";
    }}
  }}
}});
</script>'''

    return None

This is how MCP Apps actually works. The agent writes [CHART:title] markers in the report, and the backend replaces them with full iframe HTML from Tako.

The frontend never needs to know how to render Tako charts; it just embeds the iframe. Check out the complete code at agents/python/src/lib/mcp_integration.py.

 

Other backend nodes (briefly)

The repository includes a few more helper nodes, but they follow standard patterns and don’t introduce new concepts, so I have left them out to avoid unnecessary length.

• download_node: Downloads web resource content and caches it (truncated to 3000 chars to reduce context bloat)

• delete_node: handles resource deletion with user confirmation (interrupt pattern for generative UI)

• state.py: defines TypedDict schemas for AgentState, Resource, DataQuestion, and Log

• model.py: function to get LLM instance (OpenAI/Anthropic/Google) based on env config

---

6. Running the Application

After completing all the parts of the code, it's time to run it locally. Please make sure you have added the credentials to the .env in the root directory.

Install all dependencies (frontend + Python agent) by running this command from the root. This installs the Next.js frontend packages and also runs uv sync in agents/python to install Python dependencies.

npm install

This repository runs the frontend and backend together in development mode. Just run the following command from the root:

npm run dev

Behind the scenes, this is wired up using concurrently in package.json:

"dev": "concurrently -k -n ui,agent -c blue,red \"PORT=3000 npm run dev:ui\" \"PORT=2024 npm run dev:agent\"",
"dev:ui": "next dev",
"dev:agent": "cd agents/python && uv run main.py"

• dev:ui → runs Next.js frontend on port 3000
• dev:agent → runs Python FastAPI agent backend on port 2024 (using uv run)
• concurrently runs both in parallel with colored output (blue for UI, red for agent)

You can navigate to http://localhost:3000 in your browser to view it locally.

Here is the demo!

---

6. Data flow

Now that we've built the frontend (Next.js + CopilotKit), the agent orchestration (LangGraph) and the backend integrations (MCP + Tavily), this is how data actually flows between them.

This should be easy to follow if you have been building along so far.

[User enters research question]
      ↓
Next.js UI (CopilotKit Provider + CopilotChat + ResearchCanvas)
        ↓
POST /api/copilotkit (CopilotKit runtime endpoint)
        ↓
CopilotRuntime → Python Agent (LangGraph)
        ↓
LangGraph StateGraph orchestration
   ├─ chat_node: interpret intent, decide tools 
   │   └─ (generates report with [CHART:title] markers, then replaces with iframe HTML)
   ├─ search_node: parallel gather + dedupe/normalize resources
   │      ├─ Tavily search (web results)
   │      └─ Tako MCP search (returns chart metadata)
   └─ download_node: cache web resource content
        ↓
copilotkit_emit_state(): stream intermediate AgentState with report containing iframe HTML (SSE)
        ↓
Frontend receives state updates (resources/report with embedded iframes)
        ↓
MarkdownRenderer extracts <!doctype html>...</html> iframe blocks and renders as siblings to markdown
        ↓
User sees: streaming report + resources list + interactive embedded charts

---

That’s it! 🎉

You now have a research assistant that streams agent state into a chat + canvas UI, blends web search with MCP-powered charts, and produces an interactive report.

What makes this work is the structure: clear agent state, explicit tool boundaries, and a UI that renders progress instead of hiding it. Once those pieces are in place, adding new MCP tools or constraints becomes an implementation detail.

I hope you learned something valuable. Have a great day!

Comments

[Nathan Tarbert CopilotKit]

Whoa, amazing detailed tutorial Anmol, thanks for sharing!