Arindam Majumder

Posted on Jul 22

🤯 I Built an Agentic Deep Researcher & It's MCP Server 🕵🏻‍♂️

#ai #mcp #programming #python

Finding and analyzing quality information on the internet takes hours.

So I built something better.

After exploring Agno’s multi-agent capabilities and tools like Scrapegraph and Nebius AI, I thought, why not automate the boring research process?

So I built a multi-agent Deep Researcher that:

Searches the internet
Analyzes what it finds
And writes a clean report with insights + references

All with a single input: your research question.

In this Article, I'll show how to build this Workflow, also create an MCP server for it!

Let's Start!

What is the Deep Researcher Agent?

This agentic workflow is like a research intern that never sleeps.

Here’s the pipeline:

Searcher Agent: Scans the web for up-to-date, high-quality info using Scrapegraph
Analyst Agent: Uses Nebius AI to summarize and organize the content
Writer Agent: Generates a full research report, with citations and insights

You just type a question like:

“What’s the current state of AGI research in 2025?”

…and the agents do everything else.

If You like the video version, you can check this out:

Tools I Used

Here are the main tools I used to build this project.

I was looking for simple and reliable options, and these turned out to be perfect for the job.

1. Agno for Agentic Workflows

Agno is a lightweight Python library for building multi-agent workflows.

It’s deterministic, production-ready, and super flexible.

Agno lets me define workflows where each agent handles a different stage of research—without losing state.

If you’re building agentic apps, you should check this out.

Try Agno

2. Scrapegraph for Web Search

LLMs can’t browse the web. Scrapegraph can.

It connects to real-time websites, handles scraping, and extracts structured content.

With just a few lines, I had my Searcher Agent pulling fresh data from the internet.

Perfect for deep research where freshness = value.

Check out Scrapegraph

3. Nebius AI Studio for Reasoning

Open-source models are great, but you need good infrastructure to run them repeatedly in a long workflow.

Nebius AI Studio made this super easy.

I ran multiple steps of inference across agents, without worrying about latency or cost.

Bonus: it’s pay-as-you-go and supports OSS models out of the box.

Try Nebius Studio

4. Streamlit for UI

I wanted a dead-simple way to interact with the workflow.

Streamlit made it possible to go from CLI → clean web UI in 10 minutes.

Input your research topic, hit run, get the report.

No frontend code, just Python.

Try Streamlit

How to Use the Agent

We can use the Agent via 3 methods:

The CLI
The Streamlit UI
MCP Server

Building the Job Searching Agent

Enough talking, let's start building the Agent!🔥

Pre-requisites

Before running this project, make sure you have:

Python 3.10 or higher
A ScrapeGraph account and API credentials
Nebius AI Studio account and API key

Project Structure

To keep things clean and modular, we’ve organized the project like this:

deep_researcher_agent/
├── app.py              # Streamlit web UI
├── agents.py           # Multi-agent logic
├── server.py           # MCP server
├── assets/             # Images and demos
├── pyproject.toml      # Dependency config
└── README.md

Building the Agent

First, we'll create the Agents needed for our project. For that, let's go to the agents.py file.

Here, we'll import the required Modules:

import os
from agno.agent import Agent
from agno.models.nebius import Nebius
from dotenv import load_dotenv
from typing import Iterator
from agno.utils.log import logger
from agno.utils.pprint import pprint_run_response
from agno.tools.scrapegraph import ScrapeGraphTools
from agno.workflow import RunEvent, RunResponse, Workflow
from pydantic import BaseModel, Field

load_dotenv()

We will now define a class DeepResearcherAgent that starts the entire agent workflow.

class DeepResearcherAgent(Workflow):
    """
    A multi-stage research workflow that:
    1. Gathers information from the web using advanced scraping tools.
    2. Analyzes and synthesizes the findings.
    3. Produces a clear, well-structured report.
    """

Now We'll create our first agent, ie the Searcher Agent. This agent scrapes the website and finds relevant details via Scrapegraph Tool.

searcher: Agent = Agent(
        tools=[ScrapeGraphTools(api_key=os.getenv("SGAI_API_KEY"))],
        model=Nebius(
            id="deepseek-ai/DeepSeek-V3-0324", api_key=os.getenv("NEBIUS_API_KEY")
        ),
        show_tool_calls=True,
        markdown=True,
        description=("..."),
        instructions=(
            "...",
        ),
    )

Next, we'll create the Analyst agent. This agent summarizes and organizes the content extracted by the Searcher.

 # Analyst: Synthesizes and interprets the research findings
    analyst: Agent = Agent(
        model=Nebius(
            id="deepseek-ai/DeepSeek-V3-0324", api_key=os.getenv("NEBIUS_API_KEY")
        ),
        markdown=True,
        description=(
            "..."
        ),
        instructions=(
            "..."
        ),
    )

You can treat this like your personal note-taker who’s great at summarizing dense material.

Now, we'll create our third agent, the Writer Agent. This one writes the actual report, formatting everything neatly.

  # Writer: Produces a final, polished report
    writer: Agent = Agent(
        model=Nebius(
            id="deepseek-ai/DeepSeek-V3-0324", api_key=os.getenv("NEBIUS_API_KEY")
        ),
        markdown=True,
        description=(
            "..."
        ),
        instructions=(
            "..."
        ),
    )

Now we'll define workflow logic in the run() method. This is the main function that will be called when you run the workflow (the workflow entrypoint).

    def run(self, topic: str) -> Iterator[RunResponse]:
        """
        Orchestrates the research, analysis, and report writing process for a given topic.
        """
        logger.info(f"Running deep researcher agent for topic: {topic}")

        # Step 1: Research
        research_content = self.searcher.run(topic)
        # logger.info(f"Searcher content: {research_content.content}")

        logger.info("Analysis started")
        # Step 2: Analysis
        analysis = self.analyst.run(research_content.content)
        # logger.info(f"Analyst analysis: {analysis.content}")

        logger.info("Report Writing Started")
        # Step 3: Report Writing
        report = self.writer.run(analysis.content, stream=True)
        yield from report

Finally, we'll create a utility function run_research which takes the query and streams the response. We'll be using it in MCP, CLI, and Streamlit app.

def run_research(query: str) -> str:
    agent = DeepResearcherAgent()
    final_report_iterator = agent.run(
        topic=query,
    )
    logger.info("Report Generated")

    # Collect all streaming content into a single string
    full_report = ""
    for chunk in final_report_iterator:
        if chunk.content:
            full_report += chunk.content

    return full_report

Now, we'll create a function to test it in CLI.

if __name__ == "__main__":
    topic = "Extract information about Nebius AI Studio, including its features, capabilities, and applications from available sources."
    response = run_research(topic)
    print(response)

This will print the full research report in your terminal.

Creating the UI

The Agent Part is Done. Now we'll create the Streamlit UI for our DeepResearcher Agent.

First, we'll import the required models

import streamlit as st
from agents import DeepResearcherAgent
import time
import base64

We’ll set the app config and add a custom title with logos for some polish.

st.set_page_config(
    page_title="Deep Research Agent",
    page_icon="🔎",
)

Then we read and encode the ScrapeGraph logo so we can embed it directly in the title.

with open("./assets/scrapegraph.png", "rb") as scrapegraph_file:
    scrapegraph_base64 = base64.b64encode(scrapegraph_file.read()).decode()

    # Create title with embedded images
    title_html = f"""
    <div style="display: flex; justify-content: center; align-items: center; width: 100%; padding: 32px 0 24px 0;">
        <h1 style="margin: 0; padding: 0; font-size: 2.5rem; font-weight: bold;">
            <span style="font-size:2.5rem;">🔎</span> Agentic Deep Searcher with 
            <span style="color: #fb542c;">Agno</span> & 
            <span style="color: #8564ff;">Scrapegraph</span>
            <img src="data:image/png;base64,{scrapegraph_base64}" style="height: 60px; margin-left: 12px; vertical-align: middle;"/>
        </h1>
    </div>
    """
    st.markdown(title_html, unsafe_allow_html=True)

Now, we'll create the sidebar where we read and encode the ScrapeGraph logo so we can embed it directly in the title.


with st.sidebar:

    st.image("./assets/nebius.png", width=150)
    nebius_api_key = st.text_input("Enter your Nebius API key", type="password")
    st.divider()

    st.subheader("Enter Scrapegraph API key")
    scrapegraph_api_key = st.text_input(
        "Enter your Scrapegraph API key", type="password"
    )
    st.divider()

    st.header("About")
    st.markdown(
        """
    This application is powered by a `DeepResearcherAgent` which leverages multiple AI agents for a comprehensive research process:
    - **Searcher**: Finds and extracts information from the web.
    - **Analyst**: Synthesizes and interprets the research findings.
    - **Writer**: Produces a final, polished report.
    """
    )
    st.markdown("---")
    st.markdown(
        "Developed with ❤️ by [Arindam Majumder](https://www.youtube.com/c/Arindam_1729)"
    )

Now we create the chat-style prompt and hook it into our agent chain.

# Chat input at the bottom
user_input = st.chat_input("Ask a question about your documents...")

When the user types something, we kick off the agent pipeline:

if user_input:
    try:
        agent = DeepResearcherAgent()
        with st.status("Executing research plan...", expanded=True) as status:
            # PHASE 1: Researching
            phase1_msg = "🧠 **Phase 1: Researching** - Finding and extracting relevant information from the web..."
            status.write(phase1_msg)
            research_content = agent.searcher.run(user_input)

            # PHASE 2: Analyzing
            phase2_msg = "🔬 **Phase 2: Analyzing** - Synthesizing and interpreting the research findings..."
            status.write(phase2_msg)
            analysis = agent.analyst.run(research_content.content)

            # PHASE 3: Writing Report
            phase3_msg = (
                "✍️ **Phase 3: Writing Report** - Producing a final, polished report..."
            )
            status.write(phase3_msg)
            report_iterator = agent.writer.run(analysis.content, stream=True)

We stream the output to the screen in chunks as it's being generated:


        # Move report display outside of status block
        full_report = ""
        report_container = st.empty()
        for chunk in report_iterator:
            if chunk.content:
                full_report += chunk.content
                report_container.markdown(full_report)

    except Exception as e:
        st.error(f"An error occurred: {e}")

That’s the full Streamlit UI. It runs the agent, shows progress by phase, and renders the final result, all live.

Building the MCP Server:

Now, we'll create our MCP Server so that we can use it in MCP clients.

First, We'll import the required modules.

import asyncio
from mcp.server.fastmcp import FastMCP
from agents import run_research

Now, We'll define our MCP Server and tool using FastMCP :

mcp = FastMCP("deep_researcher_agent")


@mcp.tool()
def deep_researcher_agent(query: str) -> str:
    """Run Deep Researcher Agent for given user query. Can do both standard and deep web search.

    Args:
        query (str): The research query or question.

    Returns:
        str: The research response from the Deep Researcher Agent.
    """

    return run_research(query)

Finally, we run the server with stdio transport:

# Run the server
if __name__ == "__main__":
    mcp.run(transport="stdio")

Now, In order to try it, you can use this config json:

# add this inside ./.cursor/mcp.json
{
   "mcpServers": {
     "deep_researcher_agent": {
       "command": "python",
       "args": [
         "--directory",
         "/Users/arindammajumder/Developer/Python/awesome-llm-apps/advance_ai_agents/deep_researcher_agent",
         "run",
         "server.py"
       ],
       "env": {
         "NEBIUS_API_KEY": "your_nebius_api_key_here",
         "SGAI_API_KEY": "your_scrapegraph_api_key_here"
       }
     }
   }
 }

Running it:

Awesome! Our Setup is done. Time to run this!

Running the UI

Start the front-end interface with:

uv run streamlit run app.py

You’ll see something like this:

Run from CLI (Agent Script)

We can run the research agent directly from the command line. Run the following command:

uv run python agents.py

This is what shows up:

Run as MCP Server (Claude/Cursor)

To integrate with Claude Desktop or Cursor, drop this config into .cursor/mcp.json or claude_desktop_config.json :

{
  "mcpServers": {
    "deep_researcher_agent": {
      "command": "python",
      "args": [
        "--directory",
        "/Your/Path/to/directory/awesome-ai-apps/advance_ai_agents/deep_researcher_agent",
        "run",
        "server.py"
      ],
      "env": {
        "NEBIUS_API_KEY": "your_nebius_api_key_here",
        "SGAI_API_KEY": "your_scrapegraph_api_key_here"
      }
    }
  }
}

This lets Claude Desktop auto-launch your agent via MCP: