Pydantic AI + MCP + Advanced Web Scraping = The Key To Powerful Agentic AI

In this story, I have a super quick tutorial showing you how to create a multi-agent chatbot using Pydantic AI, MCP, and advanced Web Scraping to build a powerful agent chatbot for your business or personal use.

MCP is gradually being accepted because it is an open standard. I have made quite cool videos that you will definitely like. In the development of AI projects, the integration of AI models is complex, and there are problems with existing frameworks, such as LangGraph and Pydantic AI Tools. LangGraph and Pydantic AI have high code abstraction and are too commercialized.

The advantages of MCP are that open standards facilitate service providers' development of APIs, prevent developers from reinventing the wheel, and allow them to use existing MCP services to enhance Agents.

Recently, DeepSeek released a new open-source DeepSeek-V3, a large mixture of experts (MoE) model with 671B parameters, of which 37B are activated per token. Training took only two months and cost less than $6 million. It is enough to compete with ChatGPT and shake up the global AI community.

The full training of DeepSeek-V3 took only 2.788 million GPU hours. Assuming the rental price of an H800 GPU is $2 per GPU hour, the total training cost of DeepSeek-V3 is only $5.576 million.

Pydantic AI is a framework that I have described in Many Videos, so I will not describe it in detail here. However, the application here is easy to understand.

So, let me give you a quick demo of a live chatbot to show you what I mean.

Check the YouTube Video

I will ask two questions: “Search for the latest news about artificial intelligence " and “Create a bar chart showing the population of the five largest cities in the world.” You can also ask any questions you like.

The chatbot uses the Model Control Protocol (MCP), where different tools work together to complete tasks. The Python tool, powered by Fastmcp, runs Python code safely and creates visualisations using the PythonREPL class. It has two tools: python_repl(code: str) for running code and data_visualization(code: str) for making Matplotlib charts.

The web search tool uses the Exa API to get search results, setting up FastMCP, an API client, and a default five-result search setting, returning answers in Markdown format.

The AI agent connects to both MCP servers, retries if needed, and performs multi-step tasks like finding climate change data using exa_search.py or creating a bar chart with python_tools.py.

Disclaimer: This article is only for educational purposes. We do not encourage anyone to scrape websites, especially those web properties that may have terms and conditions against such actions.

How is The DeepSeek V3 Trained?

DeepSeek was able to develop a high-performance AI model with only $5.57 million in two years, These achievements are due to the following innovative technologies:

Accurately activate some “brain cells”

DeepSeek-V3 adopts a design called “hybrid expert architecture”. Simply put, it will only activate some “brain cells” instead of all when needed, greatly reducing the consumption of computing resources. Only 2048 NVIDIA H800 GPUS are used to train the model.

Data processing and energy-saving innovation

DeepSeek develops internal tools to generate high-quality training data and uses “distillation technology” to further compress computing resources. FP8 technology is used during training. This low-precision data format can significantly reduce video memory requirements while improving efficiency. The use of FP8 reduces memory requirements to half of traditional FP16 technology, while maintaining the same computing performance.

Let’s Start Coding

We initiate the code by importing classes from

MCPServerStdio: which runs the server as a subprocess and connects to it using the stdio transport

FastMCP makes building MCP servers simple and intuitive. Create tools, expose resources, and define prompts with clean, Pythonic code.

exa_py was formerly called Metaphor. Metaphor is a search engine 🔍 based on a large language model 🔗 that allows users to search using complete sentences and natural language, and can also simulate the way people share and talk about links on the Internet to query content.

#app.py
import os
import asyncio
from pydantic_ai import Agent
from pydantic_ai.mcp import MCPServerStdio

#exa_search.py
from mcp.server.fastmcp import FastMCP
from dotenv import load_dotenv
import os
from exa_py import Exa

#python_tools
from mcp.server.fastmcp import FastMCP
import io
import base64
import matplotlib.pyplot as plt
import sys
from io import StringIO
import traceback

Let’s define a FastMCP-powered Python tool that executes arbitrary Python code and generates data visualizations using Matplotlib. It includes a PythonREPL class that runs Python code in a restricted environment, redirects standard output sys.stdoutto capture execution results, and returns either the output or an error traceback.

It provides two MCP tools: python_repl(code: str), which executes Python code and returns its output, and data_visualization(code: str), which runs Python code, saves Matplotlib plots, and returns the image as a base64-encoded string, handling errors if they occur.

It uses FastMCP("python_tools") to define a multi-agent tool for executing Python tasks and running as an MCP server when executed directly

from mcp.server.fastmcp import FastMCP
import io
import base64
import matplotlib.pyplot as plt
import sys
from io import StringIO
import traceback

mcp = FastMCP("python_tools")

class PythonREPL:
    def run(self, code):
        old_stdout = sys.stdout
        redirected_output = sys.stdout = StringIO()

        try:
            exec(code, globals())
            sys.stdout = old_stdout
            return redirected_output.getvalue()
        except Exception as e:
            sys.stdout = old_stdout
            return f"Error: {str(e)}\n{traceback.format_exc()}"

repl = PythonREPL()

@mcp.tool()
async def python_repl(code: str) -> str:
    """Execute Python code."""
    return repl.run(code)

@mcp.tool()
async def data_visualization(code: str) -> str:
    """Execute Python code. Use matplotlib for visualization."""
    try:
        repl.run(code)
        buf = io.BytesIO()
        plt.savefig(buf, format='png')
        buf.seek(0)
        img_str = base64.b64encode(buf.getvalue()).decode()
        plt.close()  # Close the figure to free memory
        return f"data:image/png;base64,{img_str}"
    except Exception as e:
        return f"Error creating chart: {str(e)}"

if __name__ == "__main__":
    mcp.run()

Then we set up a web search tool using the Exa API within an MCP (Model Control Protocol) framework. It first loads environment variables and initializes FastMCP with metadata.

Then, it configures an Exa API client using an API key from the environment (or a default key). A default search configuration is defined, allowing five results per search unless overridden. The main function is registered as an MCP tool and asynchronously queries the Exa API, retrieving results formatted in Markdown.

The format_search_results() function structures the output, including titles, URLs, publication dates, and summaries (if available). If executed directly, the script runs the MCP instance.

from mcp.server.fastmcp import FastMCP
from dotenv import load_dotenv
import os
from exa_py import Exa


load_dotenv(override=True)

# Initialize FastMCP
mcp = FastMCP(
    name="websearch", 
    version="1.0.0",
    description="Web search capability using Exa API"
)

# Initialize the Exa client
exa_api_key = os.getenv("EXA_API_KEY", "")
exa = Exa(api_key=exa_api_key)

# Default search configuration
websearch_config = {
    "parameters": {
        "default_num_results": 5,
        "include_domains": []
    }
}

@mcp.tool()
async def search_web(query: str, num_results: int = None) -> str:
    """Search the web using Exa API and return results as markdown formatted text."""
    try:
        search_args = {
            "num_results": num_results or websearch_config["parameters"]["default_num_results"]
        }

        search_results = exa.search_and_contents(
            query, 
            summary={"query": "Main points and key takeaways"},
            **search_args
        )

        return format_search_results(search_results)
    except Exception as e:
        return f"An error occurred while searching with Exa: {e}"

def format_search_results(search_results):
    if not search_results.results:
        return "No results found."

    markdown_results = "### Search Results:\n\n"
    for idx, result in enumerate(search_results.results, 1):
        title = result.title if hasattr(result, 'title') and result.title else "No title"
        url = result.url
        published_date = f" (Published: {result.published_date})" if hasattr(result, 'published_date') and result.published_date else ""

        markdown_results += f"**{idx}.** [{title}]({url}){published_date}\n"

        if hasattr(result, 'summary') and result.summary:
            markdown_results += f"> **Summary:** {result.summary}\n\n"
        else:
            markdown_results += "\n"

    return markdown_results

if __name__ == "__main__":
    mcp.run()

And I developed an AI agent that leverages the DeepSeek Chat model for processing requests while integrating two MCP (Model Control Protocol) servers:

exa_search.py – Handles web searches using the Exa API.
python_tools.py – Provides Python-based utilities, including data analysis and visualization.

The agent is defined using agent.run_mcp_servers, which connects to these servers and retries up to three times if necessary. The main function runs the MCP servers and executes a multi-step task:

Step 1: Search for recent climate change statistics.
Step 2: Generate a bar chart visualising the global temperature rise over the last decade using Python.

import os
import asyncio
from pydantic_ai import Agent
from pydantic_ai.mcp import MCPServerStdio
from pydantic_ai.models.openai import OpenAIMode

deepseek_chat_model = OpenAIModel( #define the base as open AI
    'deepseek-chat',
    base_url='https://api.deepseek.com',
    api_key=os.environ["DEEPSEEK_API_KEY"],
)

# Define the MCP Servers
exa_server = MCPServerStdio(
    'python',
    ['exa_search.py']
)

python_tools_server = MCPServerStdio(
    'python',
    ['python_tools.py']
)

# Define the Agent with both MCP servers
agent = Agent(
    deepseek_chat_model, 
    mcp_servers=[exa_server, python_tools_server],
    retries=3
)

# Main async function
async def main():
    async with agent.run_mcp_servers():
        result = await agent.run("""
        I need to analyze some climate data. First, search for recent climate change statistics.
        Then, create a bar chart showing the increase in global temperature over the last decade.
        Use Python for the data visualization.
        """)
        print(result)

# Run the async function
if __name__ == "__main__":
    asyncio.run(main())

Conclusion :

MCP is more than just a protocol or architectural pattern; it represents a fundamental shift in designing AI-driven applications and distributed systems. By effectively separating models, contexts, and protocols, organizations can create adaptive, scalable, and maintainable solutions.

DeepSeek-V3 exemplifies this mindset by demonstrating that algorithm optimization and engineering innovation can produce top-tier AI models even with limited resources.

This breakthrough not only challenges the traditional AI business model but also opens the door for entrepreneurs and developers to leverage high-performance AI as a tool for inclusive innovation

Learning is a process, and as long as u learn, there will be challenges. God rewards hard work, and the harder you work, the better you will become