If LLMs Only Predict the Next Word, How Can They Search the Web Or Read Pdfs?

If you strip away the hype, Large Language Models (LLMs) are just glorified autocomplete text generator engines.

Mathematically, an LLM evaluates a sequence of tokens and predicts the probability distribution of the next token

It does not have hands. It cannot execute code, click a browser button, or open a local PDF on your hard drive.

So how does Claude execute a web search, or how does ChatGPT summarize a 100-page document?

Let's break down how text predictors turn into autonomous systems—and why the Model Context Protocol (MCP) is becoming the standard way to wire them up.

1. The Layers: AI vs. LLM vs. AI Agent
To understand tool use, we first need to distinguish three terms that get tossed around interchangeably:

Artificial Intelligence (AI): The broad field of creating systems that perform tasks requiring human-like intelligence.

Large Language Model (LLM): The core reasoning engine. It takes context as input and returns predicted text as output. It is stateless and completely isolated from the outside world.

AI Agent: An architectural pattern that wraps an LLM inside an environment equipped with Memory, Planning loops (e.g., ReAct / Thought-Action-Observation), and Tools.

When you ask an AI Agent to "Search the web for today's weather":

1. Prompting: The agent system passes your question along with a list of available tools (formatted as text definitions) to the LLM.

2. Intent Generation: The LLM does not search Google. Instead, it outputs special tokens or formatted JSON:
   

{
  "tool": "web_search",
  "parameters": { "query": "current weather in Lagos" }
}

1. Execution: The application runtime (Python/Node harness) intercepts this text, sees the tool request, and calls a real Web Search API.

2. Context Injection: The runtime receives the API's raw text response and appends it to the LLM's context window as an Observation.

3. Final Prediction: The LLM reads the fresh context and predicts the human-readable answer.

To read a PDF, the runtime parses the document into raw text (or vectors via a RAG pipeline) and injects that content directly into the model's context window. The model isn't "reading" a document; it's predicting tokens on text placed in front of it.

At the end of the day LLMS are still text generators that use clever ways to interact with tools and give it the illusion that it has hands

1. APIs vs. MCP: What's the Difference? Before MCP, if you wanted an LLM to read a PDF, run a SQL query, or search GitHub, you had to write custom "glue code" for every single tool and model combination.

What is an API?
An Application Programming Interface (API) is a set of rules that allows software applications to communicate with a specific service (e.g., GitHub API, Stripe API). Every API has its own authentication, endpoints, payloads, and response structures.

What is MCP?
The Model Context Protocol (MCP)—introduced by Anthropic—is an open standard that acts like a USB-C port for AI applications.

Instead of writing bespoke integration logic for every API:

Developers build an MCP Server that exposes tools, prompt templates, or resources.

Any MCP Client (Claude Desktop, Cursor, Custom Agent Runtime) can instantly discover and call those capabilities without custom integration code.

3. Why AI Needs MCP
The High-Level Perspective
AI applications need to interact with local databases, internal SaaS tools, file systems, and web APIs.
Without a universal standard, every AI host (ChatGPT, Claude, Cursor, LangChain) requires developers to rewrite tool definitions repeatedly. MCP solves the M * N integration complexity by reducing it to 1 * N

Key advantages:
Plug-and-Play Security: Tools run in isolated MCP servers with clear permissions.

Context Consistency: The standard defines how resources (files, logs) and tools (functions) are presented to models.
Separation of Concerns: Application developers build the UI/agent loop; tool authors build the MCP server once.

The Low-Level Mechanics:
How MCP Works Under the Hood
Under the hood, MCP uses JSON-RPC 2.0 over transports like stdio (for local tools) or Server-Sent Events (SSE) (for remote network tools).

Step 1: Capability & Discovery Phase
When an MCP Client connects to an MCP Server, it queries what capabilities are available via tools/list:

// Client -> Server
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/list"
}

The Server responds with standardized JSON Schema definitions:

// Server -> Client
{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "tools": [
      {
        "name": "search_pdf",
        "description": "Extracts text from a local PDF file by page range.",
        "inputSchema": {
          "type": "object",
          "properties": {
            "filePath": { "type": "string" },
            "pageStart": { "type": "integer" }
          },
          "required": ["filePath"]
        }
      }
    ]
  }
}

Step 2: Prompt Translation
The MCP Client converts these JSON schemas into system prompts that the LLM understands (e.g., standard Anthropic tool format or OpenAI function specs).

Step 3: Execution Phase
When the LLM decides to use the tool, the Client translates the model's text decision into a standardized tools/call request:

// Client -> Server
{
  "jsonrpc": "2.0",
  "id": 2,
  "method": "tools/call",
  "params": {
    "name": "search_pdf",
    "arguments": {
      "filePath": "./docs/architecture.pdf",
      "pageStart": 12
    }
  }
}

The MCP Server performs the actual OS/file/API operation and returns the result, which is appended back to the LLM's prompt window.

In summary
LLMs don't act on the world; they describe intentions.

An LLM outputs structured text saying "I want to execute search(X)", an agent harness executes the command via standard protocols like MCP, and the output is fed back as context for the next token prediction.

The leap from simple autocomplete to autonomous AI isn't a change in how neural networks work, it's a leap in how effectively we connect their text outputs to real-world software abstractions.