DEV Community: Kees Heuperman

How to call tools with Mistral AI using MCP servers

Kees Heuperman — Mon, 31 Mar 2025 19:38:56 +0000

Giving Europe's leading AI models access to a wide range of tools by connecting Mistral's AI API to MCP servers.

In a previous post we've seen how to build a host that connects an AI model to Model Context Protocol (MCP) tools. In that post we used Anthropic's Claude model to do the tool calling. As MCP was originally created by Anthropic, it is not surprising their AI API is the most directly compatible with the protocol. However, due to recent events, we might want to use a different, European, model for our agents and tool calling workflows.

In this tutorial we will create helper functions to both convert MCP server output into a format Mistral's API can handle and convert the API's responses to an MCP compliant format. We'll then use those helpers to connect a couple of MCP servers to one of Mistral's free models and use it to fetch and summarise data from the web.

This tutorial assumes you are familiar with TypeScript and MCP.

Setting up the project

Before we write any code, we need to set up our environment and dependencies. First, create a folder for the project and change the current working directory to this folder: mkdir mistral-mcp && cd mistral-mcp.

Install pnpm if you haven't already. You can install it using Corepack which is included with Node.js: corepack enable pnpm

Then initialise pnpm with pnpm init, and install the required dependencies: pnpm add @mistralai/mistralai @modelcontextprotocol/sdk dotenv && pnpm add -D @types/node typescript.

We'll also need to install uv (a python package manager) to run MCP servers written in Python: brew install uv

Configuring TypeScript and Node.js

First we add some basic configuration for TypeScript. Create the config file: touch tsconfig.json, then add the following rules.

{
  "compilerOptions": {
    "module": "esnext",
    "moduleResolution": "node",
    "esModuleInterop": true,
    "target": "esnext",
    "types": ["node"]
  },
  "include": ["**/*.ts"]
}

Then we need to update our package.json to treat our files as ES modules by default, so we can use import syntax and top-level await.

{
  "name": "mistral-mcp",
  "type": "module",
  "scripts": {
    "dev": "npx tsx main.ts"
  }
  // Rest of the file can stay as is
}

Connecting to the Mistral API

To use AI models via their API, you will have to create and account on the Mistral AI console. For this project we can use their free models so you won't have to set up any payment method for your account.

Once you have registered, go to the API keys section and create a new key. Then create a file to safely store env variables in your project directory and add your API key.

echo "MISTRAL_API_KEY=[YOUR API KEY HERE]" > .env

Let's give the Mistral API a try without using any tools first. Create a TypeScript file using touch main.ts, then add the following code to call the API and log the result. Mistral's AI models love cheese, so let's have it write a poem about its favourite subject.

// main.ts

import "dotenv/config";
import { Mistral } from "@mistralai/mistralai";

const mistral = new Mistral({ apiKey: process.env.MISTRAL_API_KEY });

const response = await mistral.chat.complete({
  model: "mistral-small-latest",
  messages: [
    {
      role: "user",
      content: "Please write a poem about cheese",
    },
  ],
});

if (response.choices?.length) {
  console.dir(response.choices[0], { depth: null });
}

Run the code using pnpm dev and enjoy Mistral's poetic prowess.

Adding MCP servers

To access the tools we'll need, we can add two MCP servers; fetch, which can get the HTML for a site at a given URL, and filesystem, which will allow the AI model to read and write files. The third item in the args of filesystem is the path to the directory it is allowed to access. Change this to the path of the folder you created for this project.

We will need a connect function to initialise clients for each server and connect them. We will also need a disconnect function, which we will have to run after all tasks are done to close the connection between each client and server, and end the process.

Let's create an agent class that contains all the required functions and can keep track of the shared state, like the list of clients. We'll add logging to the connect function so we can see which tools the agent has connected to. Then we call the connect function with config for the servers we want and the call function. We won't use the tools yet, for now we just check if we can successfully connect to them.

// main.ts

import "dotenv/config";
import { Mistral } from "@mistralai/mistralai";
import { Client } from "@modelcontextprotocol/sdk/client/index";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio";

class Agent {
  ai: Mistral;
  clients: Client[] = [];

  constructor(ai: Mistral) {
    this.ai = ai;
  }

  async connect(servers: MCPServer[]) {
    for (const server of servers) {
      const transport = new StdioClientTransport(server);
      const client = new Client({
        name: server.name + "-client",
        version: "1.0.0",
      });
      await client.connect(transport);
      this.clients.push(client);
      console.log(`Connected to MCP server ${server.name}`);
    }
  }

  async disconnect() {
    for (const client of this.clients) {
      await client.close();
    }
  }

  async call(message: string) {
    return ai.chat.complete({
      model: "mistral-small-latest",
      messages: [{ role: "user", content: message }],
    });
  }
}

type MCPServer = {
  name: string;
  command: string;
  args: string[];
};

const servers: MCPServer[] = [
  {
    name: "fetch",
    command: "uvx",
    args: ["mcp-server-fetch"],
  },
  {
    name: "filesystem",
    command: "npx",
    args: [
      "-y",
      "@modelcontextprotocol/server-filesystem",
      "/Users/yourname/mistral-mcp", // Replace with path to folder for this project
    ],
  },
];

const ai = new Mistral({ apiKey: process.env.MISTRAL_API_KEY });
const agent = new Agent(ai);

await agent.connect(servers);
const response = await agent.call("Please write a poem about cheese");

if (response.choices?.length) {
  console.dir(response.choices[0], { depth: null });
}

await agent.disconnect();

Listing tools

Now that we can connect to the servers, we want to list the tools available from each server so we can tell the AI model which tools it can call. However, MCP servers list their servers in a format that can't be passed directly to the Mistral API, so we'll need to convert the data to the expected format.

Let's specify some types for the result of the list tools function of the MCP server and for the tool format Mistral is expecting. This will help us determine the data transformation we need to do. We can also move the MCPServer type here for consistency.

// types.ts

export type MCPServer = {
  name: string;
  command: string;
  args: string[];
};

type MCPTool = {
  name: string;
  description?: string;
  inputSchema: {
    type: "object";
    properties?: Record<string, unknown>;
  };
};

export type MCPListToolsResult = {
  tools: MCPTool[];
  nextCursor?: string;
};

export type MistralTool = {
  type?: "function";
  function: {
    name: string;
    description?: string;
    strict?: boolean;
    parameters: Record<string, unknown>;
  };
};

Looking at the types, we can see here that the structure of the MCP tool and the Mistral tool are not that different. We only need to specify that the tool is of type function and map the tool parameters to the expected function parameters. Let's write a simple converter to do so.

// converters.ts

import type { MCPListToolsResult, MistralTool } from "./types";

export function toMistralTools(
  listToolResult: MCPListToolsResult
): MistralTool[] {
  return listToolResult.tools.map((tool) => ({
    type: "function",
    function: {
      name: tool.name,
      description: tool.description,
      parameters: tool.inputSchema,
    },
  }));
}

Calling tools

Now that we have a way to pass the tools from the MCP server to Mistral we'll also need a way to convert the response data that the API returns when it wants to call a tool. Let's first look at the types again and then build another converter function. Add the following types to the types file.

// types.ts

// Previously added types stay the same

export type MistralToolCall = {
  id?: string;
  type?: "function" | string;
  function: {
    name: string;
    arguments: Record<string, unknown> | string;
  };
  index?: number;
};

export type MCPCallToolRequest = {
  name: string;
  arguments?: Record<string, unknown>;
};

Again we see the structures are fairly similar, with the MCP format being a little simpler. One thing to be aware of is that the arguments for the function call that Mistral return will be in stringified JSON format, which means we'll have to parse them. Also note the Mistral tool call includes a tool call ID. We do not need to pass that ID to the MCP server, but we do want to save it so we can include it when we send back the result of the tool call to Mistral.

// converters.ts

import type {
  MCPCallToolRequest,
  MCPListToolsResult,
  MistralTool,
  MistralToolCall,
} from "./types";

// toMistralTools function stays the same as before

export function toMcpToolCall(toolCall: MistralToolCall): MCPCallToolRequest {
  const call = toolCall.function;
  const toolCallArguments =
    typeof call.arguments === "string"
      ? (JSON.parse(call.arguments) as Record<string, unknown>)
      : call.arguments;

  return {
    name: call.name,
    arguments: toolCallArguments,
  };
}

Let's use our new converters to have Mistral make a tool call. We will gave the agent a more challenging command, which involves fetching data from a website. The model will return a request for a tool call, which we will convert into the right format. We will then use that to call the tool using the server and log the result.

// main.ts

import "dotenv/config";
import { Mistral } from "@mistralai/mistralai";
import { Client } from "@modelcontextprotocol/sdk/client/index";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio";
import {
  ChatCompletionResponse,
  Tool,
} from "@mistralai/mistralai/models/components";
import { toMcpToolCall, toMistralTools } from "./converters";
import { MCPServer } from "./types";

class Agent {
  ai: Mistral;
  clients: Client[] = [];
  tools: Tool[] = [];
  toolToClientMap: Record<string, Client> = {};

  constructor(ai: Mistral) {
    this.ai = ai;
  }

  async connect(servers: MCPServer[]) {
    for (const server of servers) {
      const transport = new StdioClientTransport(server);
      const client = new Client({
        name: server.name + "-client",
        version: "1.0.0",
      });
      await client.connect(transport);
      this.clients.push(client);
      console.log(`Connected to MCP server ${server.name}`);

      // Store the tools and their corresponding clients in a map,
      // so we can call the right client when a tool is called later.
      const toolsList = await client.listTools();
      const toolsForMistral = toMistralTools(toolsList);
      this.tools.push(...toolsForMistral);
      for (const tool of toolsForMistral) {
        this.toolToClientMap[tool.function.name] = client;
      }
    }
  }

  async disconnect() {
    for (const client of this.clients) {
      await client.close();
    }
  }

  async call(message: string) {
    const response = await this.ai.chat.complete({
      model: "mistral-small-latest",
      messages: [{ role: "user", content: message }],
      tools: this.tools, // Pass the tools to the API
    });

    if (response.choices?.length) {
      await this.handleResponse(response);
    }
  }

  async handleResponse(response: ChatCompletionResponse) {
    if (!response.choices?.length) {
      throw new Error("No response choices found");
    }

    const { message: assistantMessage, finishReason } = response.choices[0];

    if (typeof assistantMessage.content === "string") {
      console.log(`[Mistral]: ${assistantMessage.content}`);
    }

    if (finishReason === "tool_calls") {
      if (!assistantMessage.toolCalls) {
        throw new Error("No tool calls found in response");
      }

      for (const toolCall of assistantMessage.toolCalls) {
        if (!toolCall.id) {
          throw new Error("Tool call ID not found");
        }

        const mcpToolCall = toMcpToolCall(toolCall);
        const client = this.toolToClientMap[toolCall.function.name];
        console.log(`Using tool: ${toolCall.function.name} ...`);

        const toolResult = await client.callTool(mcpToolCall);
        console.log("Tool call result:");
        console.dir(toolResult, { depth: null });
      }
    }
  }
}

const servers: MCPServer[] = [
  {
    name: "fetch",
    command: "uvx",
    args: ["mcp-server-fetch"],
  },
  {
    name: "filesystem",
    command: "npx",
    args: [
      "-y",
      "@modelcontextprotocol/server-filesystem",
      "/Users/yourname/mistral-mcp", // Replace with path to folder for this project
    ],
  },
];

const ai = new Mistral({ apiKey: process.env.MISTRAL_API_KEY });
const agent = new Agent(ai);

await agent.connect(servers);
await agent.call(
  "Please fetch the content at https://mistral.ai/news/mistral-small-3, summarise it in your own words and write the summary to a markdown file."
);
await agent.disconnect();

Returning tool results to the API

Now that we the result of the tool call, we want to return it to the AI model so it can determine the next action to take. This means we'll have to convert the MCP tool call output to a message format Mistral can handle. Once again let's first specify the types so we can see what transformations are needed.

// types.ts

// Previously added types stay the same

type MCPTextContent = {
  type: "text";
  text: string;
};

type MCPImageContent = {
  type: "image";
  data: string;
  mimeType: string;
};

type MCPResourceContent = {
  type: "resource";
  resource:
    | {
        text: string;
        uri: string;
        mimeType?: string;
      }
    | { blob: string; uri: string; mimeType?: string };
};

type MCPContent = MCPTextContent | MCPImageContent | MCPResourceContent;

export type MCPCallToolResult =
  | {
      content: MCPContent[];
      isError?: boolean;
    }
  | { toolResult?: unknown };

type MistralImageURLChunk = {
  type: "image_url";
  imageUrl:
    | string
    | {
        url: string;
        detail?: string | null;
      };
};

type MistralTextChunk = {
  type: "text";
  text: string;
};

type MistralReferenceChunk = {
  type: "reference";
  referenceIds: number[];
};

type MistralContentChunk =
  | MistralTextChunk
  | MistralImageURLChunk
  | MistralReferenceChunk;

export type MistralToolMessage = {
  role: "tool";
  content: string | MistralContentChunk[] | null;
  toolCallId?: string | null;
  name?: string | null;
};

We can see here that tools can return not only text, but also images and different types of resources. For now we will focus on handling text results only. Converting other result types to the appropriate Mistral content type will be left as an exercise for the reader.

// converters.ts

import type {
  MCPCallToolRequest,
  MCPListToolsResult,
  MCPCallToolResult,
  MistralTool,
  MistralToolCall,
  MistralToolMessage,
} from "./types";

// toMistralTools and toMCPToolCall functions we added previously stay the same

export function toMistralMessage(
  toolCallId: string,
  callToolResult: MCPCallToolResult
): MistralToolMessage {
  let content: MistralToolMessage["content"];

  if ("content" in callToolResult && callToolResult.content) {
    content = callToolResult.content.map((item) => {
      if (item.type === "text") {
        return {
          type: "text",
          text: item.text,
        } as const;
      } else {
        throw new Error(`Unsupported content type: ${item.type}`);
      }
    });
  } else if ("toolResult" in callToolResult && callToolResult.toolResult) {
    // Handle case where content is not provided and toolResult is used instead
    content = JSON.stringify(callToolResult.toolResult);
  } else {
    throw new Error("No tool result found");
  }

  return {
    content: content,
    toolCallId,
    role: "tool",
  };
}

This will convert the result of the MCP tool call to the Mistral message format as well as handle some unexpected result data. Let's put it all together now. When receiving a response from Mistral that includes a tool call we call the tool, convert the result to a Mistral message and use the call function to send that message back to the API.

// main.ts

import "dotenv/config";
import { Mistral } from "@mistralai/mistralai";
import { Client } from "@modelcontextprotocol/sdk/client/index";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio";
import {
  AssistantMessage,
  ChatCompletionResponse,
  Messages,
  Tool,
} from "@mistralai/mistralai/models/components";
import { toMistralMessage, toMistralTools, toMCPToolCall } from "./converters";
import { MCPServer } from "./types";

class Agent {
  ai: Mistral;
  clients: Client[] = [];
  tools: Tool[] = [];
  toolToClientMap: Record<string, Client> = {};
  messages: Messages[] = [];

  constructor(ai: Mistral) {
    this.ai = ai;
  }

  // Connect and disconnect functions stay the same as before

  async call(message: string | Messages) {
    if (typeof message === "string") {
      this.messages.push({
        role: "user",
        content: message,
      });
    } else {
      this.messages.push(message);
    }

    const response = await ai.chat.complete({
      model: "mistral-small-latest",
      messages: this.messages,
      tools: this.tools,
    });

    if (response.choices?.length) {
      await this.handleResponse(response);
    }
  }

  async handleResponse(response: ChatCompletionResponse) {
    if (!response.choices?.length) {
      throw new Error("No response choices found");
    }

    const { message: assistantMessage, finishReason } = response.choices[0];

    if (
      assistantMessage.content &&
      typeof assistantMessage.content === "string"
    ) {
      console.log(`[Mistral]: ${assistantMessage.content}`);
    }

    if (assistantMessage.role === "assistant") {
      this.messages.push(
        assistantMessage as AssistantMessage & { role: "assistant" }
      );
    }

    if (finishReason === "tool_calls") {
      if (!assistantMessage.toolCalls) {
        throw new Error("No tool calls found in response");
      }

      for (const toolCall of assistantMessage.toolCalls) {
        if (!toolCall.id) {
          throw new Error("Tool call ID not found");
        }

        const mcpToolCall = toMCPToolCall(toolCall);
        const client = this.toolToClientMap[toolCall.function.name];
        console.log(`Using tool: ${toolCall.function.name} ...`);

        const toolResult = await client.callTool(mcpToolCall);
        const message = toMistralMessage(toolCall.id, toolResult);

        await this.call(message);
      }
    }
  }
}

// Instantiate the agent and call it, same as before

Now we call the AI with instructions, triggering a loop of tool calls and results being returned that will end when there are no more tool calls to make. The expectation is that the model will call the fetch tool to get the content of the website, receive the response, call the file system tool to write a summary of the content to a file, then send a response without a tool call to report the results, ending the loop.

Conclusion

Although Mistral's tool calling does not follow the MCP standard (yet), the format it uses for tools and messages is similar enough to make conversion possible. With a few simple converters we can build extendable AI agents that use Mistral instead of relying on American models.

We can provide more tools to our agents by simply adding additional MCP servers to the config and asking the agent to use them in the prompt when we call it. Just keep in mind the Mistral Small model is limited in capabilities (and the free use is rate limited), so switch to the larger or more specialised models for more complex workflows.

Powering up AI models by handing them tools

Kees Heuperman — Sat, 01 Mar 2025 11:27:48 +0000

A brief introduction to enabling LLMs to use tools via the Model Context Protocol by building your own host application in TypeScript.

Most LLMs are great for simple conversations, but they lack the ability to interact with their environment. Using Model Context Protocol (MCP) we can give our LLM access to a wide range of open-source tools that will allow it to take agentic action.

In this tutorial we will first go over the basic features of MCP, then build a simple AI agent that can use tools to access the web and write files. We do this by creating a host that uses multiple clients to connect to existing MCP servers, makes the tools from these servers available to an LLM and handles tool calls made by the LLM.

How Model Context Protocol works

MCP is an open protocol that standardises how applications provide context to LLMs. It can be used to create servers that provide tools and data to an MCP client in a standardised format, and hosts that connect one or more of these clients to an LLM. The LLM can then call on the tools and data made available via the host.

             ┌──>client_1<──>server_1
LLM<──>host<─┤
             └──>client_2<──>server_2

Diagram showing the connection between the host, LLM, clients and servers

There are many MCP servers available already, with more being added every day. Their functionality ranges from git use, to Spotify control, to web search and more. These servers can transform an LLM into a useful assistant that can perform complex tasks.

Tools

Tools are callable functions made available by the server. The LLM will be able to ask for a function to be called–with specific arguments if required–and for the outcome of the function to be returned to the LLM. For example a calculation function that take two values and returns the sum of them.

Resources

Resources are data made available to the LLM. Unlike calling a tool, calling a resource will not have any side-effects. For example fetching rows of error logs from a database, but not writing new data to the database.

Prompts

Prompts are basically templates made available by the server. They will take an argument, insert it in the template and return a message with instructions, ready to pass to the LLM. For example, a prompt called research might take a subject as an argument and return a prompt with detailed instructions on how the LLM is expected to research the subject.

Advanced features

There are other, more advanced features included in the protocol, such as sampling, roots, and Server-Sent Events transport. We won't need those for this tutorial, but they will give you an idea of the flexibility of MCP.

Setting up the project

Before we write any code, we need to set up our environment and dependencies.

1. Create an account with Anthropic

To make calls to the Anthropic API, you will need to create an account on the Anthropic API console and add some credit. $5 will be more than enough for this tutorial.

2. Create a project folder

Create a folder for the project and change the current working directory to this folder.

mkdir mcp-host && cd mcp-host

3. Install dependencies

Install pnpm if you don't have it already. You can install it using Corepack which is included with Node.js.

corepack enable pnpm

Then initialise pnpm and install the required dependencies.

pnpm init && pnpm add @anthropic-ai/sdk @modelcontextprotocol/sdk dotenv && pnpm add -D @types/node

We will also need uv to run MCP servers written in Python

brew install uv

4. Set up env variables and TypeScript config

Create a file to safely store env variables and add your Anthropic API key, which you can find in the Anthropic console.

echo "ANTHROPIC_API_KEY=[YOUR API KEY HERE]" > .env

We'll also add some basic configuration for TypeScript. First create the config file.

touch tsconfig.json

Then add the following rules to tsconfig.json so we can use top-level await.

{
  "compilerOptions": {
    "module": "esnext",
    "moduleResolution": "node",
    "esModuleInterop": true,
    "target": "esnext",
    "types": ["node"]
  },
  "include": ["**/*.ts"]
}

Creating a host that connects to tools

Before we start making calls to the LLM API, let's build and test the connection with the MCP servers. Create a file called host.ts. Add a class with a function that takes an array of servers to connect to, creates an MCP client for each one and then connects the client to the server. For each client we get all the tools available from the connected server and add them to a map that will allow us to quickly find the right client when the time comes to call the tool. For now we just log all the tool names so we can see if the connections worked and which tools we'll be making available to the LLM.

// host.ts

import { StdioServerParameters } from "@modelcontextprotocol/sdk/client/stdio";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio";
import { Client } from "@modelcontextprotocol/sdk/client/index";
import { Tool } from "@anthropic-ai/sdk/resources";

type Server = StdioServerParameters & { name: string };

export class Host {
  private clients: Client[] = [];
  private tools: Tool[] = [];
  private toolToClientMap: Record<string, Client> = {};

  async connect(servers: Server[]) {
    for (const server of servers) {
      const transport = new StdioClientTransport(server);
      const client = new Client({
        name: server.name + "-client",
        version: "1.0.0",
      });
      await client.connect(transport);
      this.clients.push(client);
      console.log(`Connected to ${server.name}`);

      const tools = (await client.listTools()).tools;
      for (const tool of tools) {
        if (tool.inputSchema) {
          this.tools.push({
            name: tool.name,
            description: tool.description,
            input_schema: tool.inputSchema as Tool.InputSchema,
          });
          this.toolToClientMap[tool.name] = client;
        }
      }
    }

    console.log(
      "Available tools: ",
      this.tools.map((tool) => tool.name)
    );
  }

  async disconnect() {
    for (const client of this.clients) {
      await client.close();
    }
  }
}

Passing tools to the host

Create a file called main.ts where we will initialise the host and pass it some tools to use. For this tutorial we will use the 'fetch' and 'filesystem' servers created by the MCP team. The 'fetch' server can be used to read a website at a given URL, and 'filesystem' can read, write and edit files in folders it is given access to. The third item in the args of 'filesystem' is the path to the directory it is allowed to access. Change this to the path of the folder you created for this project.

After we've connected to to the servers and listed all the tools we will need to disconnect to end the process.

// main.ts

import { Host } from "./host";

const servers = [
  {
    name: "fetch",
    command: "uvx",
    args: ["mcp-server-fetch"],
  },
  {
    name: "filesystem",
    command: "npx",
    args: [
      "-y",
      "@modelcontextprotocol/server-filesystem",
      "/Users/yourname/mcp-host", // Replace with path to folder for this project
    ],
  },
];

const host = new Host();
await host.connect(servers);
await host.disconnect();

Now run the app and check the logs to see the host connecting to servers and listing the available tools.

npx tsx main.ts

Connecting the host to an LLM

Once we've confirmed we can connect to the MCP servers and list their tools, we can connect the host to an LLM and pass that LLM a list of the available tools. Anthropic's Claude models conveniently accept a tools argument and will include a "tool use" block in its response when the LLM want to call the tool.

When we receive a response, we first log the text. Then, if a code use block is present, we use the map we created earlier to find the right client. We tell that client to run the tool and pass it the arguments provided by the LLM. We then pass that message back to Claude and wait for a response. We continue to loop over this whole process.

The final response will also have a stop_reason value of end_turn, indicating the model completed the request, which is safer to check, but unnecessary for this simple project.

// host.ts

import Anthropic from "@anthropic-ai/sdk";
import { StdioServerParameters } from "@modelcontextprotocol/sdk/client/stdio";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio";
import { Client } from "@modelcontextprotocol/sdk/client/index";
import {
  ContentBlockParam,
  ImageBlockParam,
  MessageParam,
  TextBlockParam,
  Tool,
} from "@anthropic-ai/sdk/resources";

type Server = StdioServerParameters & { name: string };
type ToolResult = {
  content: string | Array<TextBlockParam | ImageBlockParam>;
  isError: boolean;
};

export class Host {
  private llmClient: Anthropic;
  private clients: Client[] = [];
  private tools: Tool[] = [];
  private messages: MessageParam[] = [];
  private toolToClientMap: Record<string, Client> = {};

  constructor(llmClient: Anthropic) {
    this.llmClient = llmClient;
  }

  async connect(servers: Server[]) {
    for (const server of servers) {
      const transport = new StdioClientTransport(server);
      const client = new Client({
        name: server.name + "-client",
        version: "1.0.0",
      });
      await client.connect(transport);
      this.clients.push(client);
      console.log(`Connected to MCP server ${server.name}`);

      const tools = (await client.listTools()).tools;
      for (const tool of tools) {
        if (tool.inputSchema) {
          this.tools.push({
            name: tool.name,
            description: tool.description,
            input_schema: tool.inputSchema as Tool.InputSchema,
          });
          this.toolToClientMap[tool.name] = client;
        }
      }
    }
  }

  async disconnect() {
    for (const client of this.clients) {
      await client.close();
    }
  }

  private async handleResponse(response: Anthropic.Messages.Message) {
    if (response.content[0].type === "text") {
      console.log(`[Claude]: ${response.content[0].text}`);
    }

    this.messages.push({
      role: "assistant",
      content: response.content,
    });

    if (response.content[1]?.type === "tool_use") {
      const toolUse = response.content[1];
      const client = this.toolToClientMap[toolUse.name];
      console.log(`Using tool: ${toolUse.name} ...`);
      const toolResult = (await client.callTool({
        name: toolUse.name,
        arguments: toolUse.input as Record<string, unknown>,
      })) as ToolResult;

      const message = [
        {
          tool_use_id: toolUse.id,
          type: "tool_result",
          content: toolResult.content,
          is_error: toolResult.isError,
        } as const,
      ];

      await this.call(message);
    }
  }

  async call(message: string | Array<ContentBlockParam>) {
    this.messages.push({
      role: "user",
      content: message,
    });

    const response = await this.llmClient.messages.create({
      max_tokens: 2048,
      messages: this.messages,
      tools: this.tools,
      model: "claude-3-5-haiku-latest",
    });

    return await this.handleResponse(response);
  }
}

Calling the LLM

Finally we update the main script to instantiate the Anthropic client and pass it to the host. Then we pass the server config and tell the host to connect to the servers as before. Finally, we pass a message to the host, instructing it to fetch the introduction page of the MCP site, summarise the content and write the summary down in a file for us.

As the LLM is only allowed access to a single folder via the 'filesystem' client, it will write the file in the directory you have provided for it.

// main.ts

import "dotenv/config";
import Anthropic from "@anthropic-ai/sdk";
import { Host } from "./host";

const anthropicClient = new Anthropic({
  apiKey: process.env["ANTHROPIC_API_KEY"],
});

const servers = [
  {
    name: "fetch",
    command: "uvx",
    args: ["mcp-server-fetch"],
  },
  {
    name: "filesystem",
    command: "npx",
    args: [
      "-y",
      "@modelcontextprotocol/server-filesystem",
      "/Users/yourname/mcp-host", // Replace with path to folder for this project
    ],
  },
];

const host = new Host(anthropicClient);
await host.connect(servers);
await host.call(
  "Please fetch https://www.anthropic.com/news/model-context-protocol, summarise the content of that page, then write it to a markdown file."
);
await host.disconnect();

Run the app again. Once the process is complete, have a look at the new file that was just created. If all went well, you will find a nice summary of MCP inside.

npx tsx main.ts

Conclusion

We now have a simple way to greatly enhance the capabilities of Claude by giving it access to any number of tools. We can give it persistent memory, the ability to surf the web or even to send emails on our behalf. All thanks to the power of MCP.

Key takeaways

Giving an LLM access to tools allows it to take action to affect its environment. This greatly increases the usefulness of the LLM.
The Model Context Protocol provides an excellent standard for tool interfaces, allowing us to make use of the many available open-source servers.
By creating a relatively simple host ourselves, we can connect an LLM to tools, while remaining in full control.

How to get consistent structured output from Claude

Kees Heuperman — Sat, 01 Feb 2025 09:29:14 +0000

Using Anthropic's excellent Claude models in automated workflows can be tricky. Although they excel at conversational interaction, getting consistently formatted output that can be used programmatically requires some work. Unlike OpenAI's models, Claude does not have an option to request structured data, but it is possible with a simple trick.

In this tutorial we'll build a reusable component that calls Claude Haiku and returns structured data following a schema. We'll then use this component to create a workflow for checking user comments for inappropriate content.

This tutorial assumes you are familiar with Node.js and TypeScript.

LLM output is hard to parse

Large Language Models (LLMs) bias towards responding in human readable text and tend to be verbose, often giving additional context even when asked for specific output. This is great for casual chats, but becomes a problem when we try to use LLM responses in workflows where the output feeds into subsequent tasks and needs to be parsed programmatically.

For example, we might to use an LLM to sort customer support messages by category and urgency. To label the messages correctly we would need the LLM to return the category title and urgency score on a predefined scale. This could be achieved by custom prompts requesting a structured response, but there is no guarantee the model will follow the instructions for the output exactly.

Utilising Tool Use to get structured data

Fortunately there is a simple trick we can use to get consistent structured output from Anthropic's Claude models: create a tool spec for Claude with a defined input schema and force the model to use this tool. This will ensure the response follows the specified format. We can generate this input schema using Zod, a TypeScript-first schema library, which we can also use to validate the LLM output and infer TypeScript types.

Setting up the project

Before we write any code, we need to do some set up.

Creating an account with Anthropic

To make calls to the Anthropic API, you will need to create an account on the Anthropic API console and add some credit. Five USD will be more than enough for this tutorial.

Creating project folder

Create a folder for the project and change the current working directory to this folder.

mkdir claude-structured-output && cd claude-structured-output

Installing dependencies

Install pnpm if you don't have it already. You can install it using Corepack which is included with Node.js.

corepack enable pnpm

Then initialise pnpm and install the required dependencies.

pnpm init && pnpm add @anthropic-ai/sdk dotenv && pnpm add -D @types/node

Setting up env variables and TypeScript config

Create an file to safely store env variables and add your Anthropic API key, which you can find in the Anthropic console.

echo "ANTHROPIC_API_KEY=[YOUR API KEY HERE]" > .env

We'll also add some basic configuration for TypeScript. First create the config file.

touch tsconfig.json

Then add the following rules to tsconfig.json.

{
  "compilerOptions": {
    "module": "esnext",
    "moduleResolution": "node",
    "esModuleInterop": true,
    "target": "esnext",
    "types": ["node"]
  },
  "include": ["**/*.ts"]
}

Making a request

Before we look into using a schema for structured output, let's create a basic component and a workflow script to make an LLM call. We can inspect the result and see why specifying a consistently structured output would be useful.

Creating a basic component for LLM calls

Now that the project has been set up, we can create a class to call the Anthropic API. The class constructor creates a new client, which we can then use to send messages to the API via the call function.

touch model.ts

// model.ts

import Anthropic from "@anthropic-ai/sdk";

export default class Model {
  private client: Anthropic;

  constructor(apiKey: string) {
    this.client = new Anthropic({ apiKey });
  }

  async call(message: string) {
    console.log("[LLM] Getting LLM API response");
    const response = await this.client.messages.create({
      max_tokens: 1024,
      model: "claude-3-5-haiku-latest",
      messages: [{ role: "user", content: message }],
      temperature: 0, // minimal randomness injected into the response
    });

    const result = response.content[0];

    return result.type === "text" ? result.text : result.input;
  }
}

Using the component to analyse a message

We can instantiate the class, send a message to the API and log the output to see what it looks like. Note that because we're using an LLM to analyse the messages, our criteria for what is and isn't appropriate can easily be customised to fit our needs.

touch main.ts

// main.ts

import "dotenv/config";
import Model from "./model";

const instructions =
  "Check the following comment for inappropriate content, meaning foul language, harassment, or anti-union sentiment. If such content is detected, specify the type(s) of inappropriate content, explain the reasoning behind the assessment, and provide a confidence score.";

const comment =
  "These idiots deserve it. Should have spent more time working and less time on TikTok";

const model = new Model(process.env.ANTHROPIC_API_KEY || "");

const response = await model.call(`${instructions}\n\n${comment}`);

console.log("[Main] Comment analysed: ", {
  message: comment,
  response,
});

Inspecting the output

Run the project and have a look at the output.

npx tsx main.ts

You will see that although we get a detailed and likely correct analysis in response, the format of the response is unpredictable and would require a human (or an LLM) to parse. This is not great if we want to use the response to take some automatic actions such a removing or labelling inappropriate posts. To get a response in a more useful format, let's give Claude a tool to use.

Requesting a structured response

Managing output structure using a schema

First we'll add the Zod library, and a library to generate JSON schemas from Zod schemas.

pnpm add zod zod-to-json-schema

Now update the model to accept a Zod schema. The tool input specification requires a JSON schema, which we can generate from the Zod schema. Then we pass the tool spec as an argument in the LLM API call, along with the tool_choice parameter, telling Claude to use our tool. When we receive a response we can use the Zod schema to check if it conforms to the expected format and throw an error if it doesn't. Finally, we can infer the type from the Zod schema and assert the return type of the call.

// model.ts

import Anthropic from "@anthropic-ai/sdk";
import type { Tool } from "@anthropic-ai/sdk/resources/index";
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";

export default class Model {
  private client: Anthropic;

  constructor(apiKey: string) {
    this.client = new Anthropic({ apiKey });
  }

  async call<T extends z.ZodTypeAny>({
    message,
    schema,
  }: {
    message: string;
    schema: T;
  }) {
    const jsonSchema = zodToJsonSchema(schema, "schema");
    const schemaDefinition = jsonSchema.definitions?.schema;

    if (!schemaDefinition) {
      console.error(jsonSchema);
      throw new Error("Failed to generate JSON schema for provided schema.");
    }

    const tools = [
      {
        name: "json",
        description: "Respond with a JSON object.",
        input_schema: schemaDefinition as Tool.InputSchema,
      },
    ];

    console.log("[LLM] Getting LLM API response");
    const response = await this.client.messages.create({
      max_tokens: 1024,
      model: "claude-3-5-haiku-latest",
      messages: [{ role: "user", content: message }],
      tools,
      tool_choice: { name: "json", type: "tool" },
      temperature: 0,
    });

    if (response.content[0]?.type !== "tool_use") {
      console.error(response);
      throw new Error("Unexpected response from LLM API.");
    }

    const result = schema.safeParse(response.content[0].input);

    if (result.success === false) {
      console.error(result.error);
      throw new Error("Response did not conform to provided schema.");
    }

    return result.data as z.infer<T>;
  }
}

Defining and passing the schema

We'll need to add the schema and update the model call in the main function. We'll ask the model to flag if the content is inappropriate, give the reason for the assessment so it can be checked by a human if needed and tell us how confident it is in the assessment. We'll also ask it to return the type of violations committed. Let's also add a few more comments to analyse and loop over them, making a call for each one.

// main.ts

import "dotenv/config";
import { z } from "zod";
import Model from "./model";

const comments = [
  "This company is a f*cking joke. They just announced record profits and now they’re firing people??",
  "These layoffs are long overdue. The unions have made people lazy and complacent.",
  "I went through layoffs last year and it was a horrible time for me and my family.",
  "These idiots deserve it. Should have spent more time working and less time on TikTok",
];

const schema = z.object({
  isInappropriateContent: z.boolean(),
  reason: z.string(),
  confidence: z.number().min(0).max(1),
  type: z.object({
    foulLanguage: z.boolean(),
    harassment: z.boolean(),
    antiUnionSentiment: z.boolean(),
  }),
});

const instructions =
  "Check the following comment for inappropriate content, meaning foul language, harassment, or anti-union sentiment. If such content is detected, specify the type(s) of inappropriate content, explain the reasoning behind the assessment, and provide a confidence score.";

const model = new Model(process.env.ANTHROPIC_API_KEY || "");

for (const comment of comments) {
  const response = await model.call({
    message: `${instructions}\n\n${comment}`,
    schema,
  });

  console.log("[Main] Comment analysed: ", {
    message: comment,
    response,
  });
}

Inspecting structured output

Run the project again. You should be getting consistently formatted JSON output for each analysed comment.

npx tsx main.ts

Conclusion

Now we have a simple and reliable way to get consistent JSON output from Claude, which enables us to integrate LLM calls into automated workflows.

Key takeaways:

Default LLM responses are great for conversational interaction, but hard to parse programmatically, making them unsuitable for automated workflows.
By specifying a JSON schema as the input for a tool and instructing Claude to use that tool, we can consistently get output conforming to our required structure.
Using Zod we can specify schemas, validate response structure and get correctly typed return values, improving confidence and developer experience.

DEV Community: Kees Heuperman

How to call tools with Mistral AI using MCP servers

Setting up the project

Configuring TypeScript and Node.js

Connecting to the Mistral API

Adding MCP servers

Listing tools

Calling tools

Returning tool results to the API

Conclusion

Powering up AI models by handing them tools

How Model Context Protocol works

Tools

Resources

Prompts

Advanced features

Setting up the project

1. Create an account with Anthropic

2. Create a project folder

3. Install dependencies

4. Set up env variables and TypeScript config

Creating a host that connects to tools

Passing tools to the host

Connecting the host to an LLM

Calling the LLM

Conclusion

Key takeaways

How to get consistent structured output from Claude

LLM output is hard to parse

Utilising Tool Use to get structured data

Setting up the project

Creating an account with Anthropic

Creating project folder

Installing dependencies

Setting up env variables and TypeScript config

Making a request

Creating a basic component for LLM calls

Using the component to analyse a message

Inspecting the output

Requesting a structured response

Managing output structure using a schema

Defining and passing the schema

Inspecting structured output

Conclusion

Key takeaways:

Further reading