DEV Community: Gustavo M.

How to Use MCP Servers With Ollama and Local LLMs

Gustavo M. — Mon, 04 May 2026 22:16:00 +0000

Ollama makes it easy to run open-weight models locally, but it does not ship an MCP client. The MCP protocol is handled at the client layer, not inside the LLM itself. To use MCP servers with a local Ollama model, you need a bridge that speaks MCP on one side and the Ollama API on the other. MCPFind indexes 832 servers in the ai-ml category, averaging 114.27 stars per server, the highest average across all categories. Most of those servers are designed for cloud-hosted clients, but a meaningful subset runs well offline. This guide covers the bridge setup, model selection, and which server categories give the best results on local hardware.

Does Ollama Support MCP Natively?

Ollama does not implement the MCP protocol. It exposes an OpenAI-compatible REST API with a /api/chat endpoint and a tools parameter that mirrors OpenAI's function-calling format. When you pass tools to Ollama, it generates JSON tool calls in the response that your client code must parse and dispatch. That dispatch step is what an MCP client handles. The MCP protocol adds session management, capability negotiation, and a richer tool schema on top of the basic function-calling concept. Because Ollama's tool-calling format is compatible with the OpenAI spec, any MCP bridge that already supports OpenAI backends can usually target Ollama by pointing the base URL at http://localhost:11434/v1. The gap is session lifecycle and streaming, not the tool schema format itself.

How to Bridge Ollama With MCP Using MCPHost

MCPHost is a Go CLI that acts as an MCP client with a configurable LLM backend. Install it with:

go install github.com/mark3labs/mcphost@latest

Configure it with a JSON file that lists your MCP servers and sets Ollama as the backend:

{
  "mcpServers": {
    "filesystem": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-filesystem", "/Users/you/projects"]
    }
  }
}

Then run it pointing at your local model:

mcphost --config mcp-config.json \
  --model ollama:qwen2.5:14b \
  --ollama-url http://localhost:11434

MCPHost starts each server listed in the config, negotiates capabilities, and passes available tools to the model on each turn. The model's tool call responses are routed back to the correct MCP server. Use a model that supports tool calling: Qwen2.5, Llama 3.3, Mistral Nemo, and Gemma 3 are reliable options. Smaller quantizations (Q4_K_M and below) sometimes produce malformed JSON tool calls, so test before deploying.

Which MCP Servers Work Best With Local LLMs?

Not all servers are equally useful offline. We grouped the devtools category and the ai-ml category by how much benefit they provide in a local context. Filesystem and code tools perform well because they do not need an external API; the model reads files and the server reports results. The official filesystem, Git, and SQLite reference servers from Anthropic all fit this pattern. Search and web browsing servers still make external HTTP requests, but the LLM component runs locally, so they work as long as you have internet access. The highest-friction category is cloud services: servers for AWS, GitHub, and similar platforms require valid API credentials regardless of whether the LLM is local or remote. For pure offline use, stick to filesystem, database, and local code tools. The ai-ml category on MCPFind includes several model management servers worth exploring if you are running a multi-model setup.

Limitations and When to Use a Cloud Client Instead

Three constraints matter when running MCP with Ollama. First, tool-calling reliability: open-weight models produce malformed tool calls more often than frontier models. Complex tool schemas with nested objects or many optional fields are particularly prone to errors. Simplify your tool definitions when building for local use. Second, context length: long MCP tool responses eat into the model's context window. A server that returns a 10,000-token file listing may cause later turns to lose earlier context. Truncate large responses at the server level before returning them. Third, latency: a tool call adds one round trip, so multi-step agentic tasks accumulate latency. On consumer hardware, expect 2-10 seconds per tool call depending on model size and quantization. If your workflow requires frequent multi-tool calls or long responses, a Claude-backed client with a subset of the same MCP servers will be faster. Start with what MCP is if you want the protocol background before optimizing your setup.

Frequently Asked Questions

Does Ollama support MCP natively?

No. Ollama exposes an OpenAI-compatible API with tool-calling support, but it does not implement the MCP protocol. You need a bridge like MCPHost to connect MCP servers to Ollama.

Which Ollama models support tool calling?

Qwen2.5, Llama 3.1/3.2/3.3, Mistral Nemo, Command-R, and Gemma 3 all support tool calling through Ollama's API. Not all quantizations behave identically, so test with the exact model tag you plan to deploy.

Can I run MCP servers that require internet access with Ollama?

Yes, the local model and the MCP server are separate processes. A server that fetches web data or calls an API will still make those external requests; only the LLM component runs locally.

Is MCPHost the only way to use MCP with Ollama?

No. Any MCP client that supports configurable backends works, including Continue.dev and custom wrappers built with the official MCP SDK. MCPHost is the simplest standalone option for CLI workflows.

For background on how MCP compares to other tool-calling approaches, see MCP vs function calling. If you want to build your own MCP server to pair with a local model, the TypeScript MCP server guide covers the full setup.

Set Up Your First MCP Server in Claude Desktop (Step-by-Step 2026 Guide)

Gustavo M. — Mon, 04 May 2026 16:17:16 +0000

If you have Claude Desktop installed and want to make it dramatically more useful, adding MCP servers is the fastest path. You do not need to write code or understand how the protocol works under the hood.

This guide covers picking your first server, adding it to Claude Desktop, and confirming it works. The whole process takes about 15 minutes. If you want to understand what MCP servers are before diving in, read MCP Servers Explained first. For background on the protocol itself, see What Is the Model Context Protocol. If you use Cursor instead of Claude Desktop, how to configure MCP in Cursor covers the same process with Cursor-specific config details.

What You Need Before You Start

You need Claude Desktop installed and a terminal available on your machine. That is it.

Claude Desktop is the free desktop app from Anthropic available for macOS and Windows. If you do not have it yet, download it from claude.ai/download. The terminal is built into every Mac (search "Terminal" in Spotlight) and available on Windows via PowerShell or Command Prompt. You will use the terminal to install server packages, but the commands are simple copy-paste steps - nothing to memorize.

Step 1: Pick Your First MCP Server

Pick one server to start. Trying to add five at once makes it harder to debug if something goes wrong. The goal is to learn how to use MCP with Claude on a single working example, then expand from there.

A good first choice for non-developers is a search server, which lets Claude pull live web results into your conversation. MCPFind indexes 416 servers in the search category. Another strong beginner pick is a filesystem server, which gives Claude read access to a folder on your computer - useful for asking questions about documents you have saved locally. For something more powerful, the Supabase MCP server has 2,556 GitHub stars and solid documentation. Browse /categories/devtools if you want to see the full range of developer-focused options, or /categories/databases for data-focused servers.

Step 2 - Find the Install Config on MCPFind

Every server page on MCPFind shows a ready-to-copy configuration block for Claude Desktop.

Navigate to the server you chose and look for the "Claude Desktop" tab in the install section. You will see a JSON snippet that looks something like this:

{
  "mcpServers": {
    "server-name": {
      "command": "npx",
      "args": ["-y", "@scope/server-package"]
    }
  }
}

Copy that entire block. You will paste it into your config file in the next step.

Step 3 - Edit the Claude Desktop Config File

Open your Claude Desktop config file in any text editor. On macOS, the path is ~/Library/Application Support/Claude/claude_desktop_config.json. On Windows, it is %APPDATA%\Claude\claude_desktop_config.json.

If the file does not exist yet, create it. Paste the JSON snippet you copied from MCPFind. If you already have other servers in your config, add the new server as another entry inside the existing mcpServers object - do not replace the whole file. Save the file when you are done.

Step 4 - Restart Claude Desktop and Test

Close Claude Desktop completely and reopen it. The app reads the config file on startup, so a restart is required.

Once it reopens, start a new conversation and ask Claude to use the server you just added. For a search server, try: "Search the web for the latest MCP server releases." For a filesystem server, try: "List the files in my Documents folder." If Claude uses the tool and returns a result, the connection is working. If you see an error, double-check that your JSON is valid - a missing comma or bracket is the most common issue.

Frequently Asked Questions

Does Claude Desktop support MCP out of the box?

Yes. Claude Desktop has built-in MCP support. You just need to edit its configuration file to add the servers you want. No extensions or plugins are required.

Can I run multiple MCP servers at the same time in Claude Desktop?

Yes. Claude Desktop supports multiple simultaneous MCP server connections. You add each server as a separate entry in the mcpServers section of your config file, and Claude can use all of them in the same conversation.

What happens if an MCP server stops working?

Claude Desktop will show an error if it cannot connect to a configured server. Your other servers and Claude itself continue working normally. You can remove or fix the broken entry in your config file without affecting anything else.

Do MCP servers have access to my entire computer?

Only what the server is designed to access. A filesystem MCP server, for example, typically limits access to a specific directory you configure. Always read the server documentation to understand exactly what it can and cannot reach.

Where is the Claude Desktop config file located?

On macOS it is at ~/Library/Application Support/Claude/claude_desktop_config.json. On Windows it is at %APPDATA%\Claude\claude_desktop_config.json. Create the file if it does not exist.