Day 2/30: LLM Tool Calls

I still remember the frustration of trying to integrate a large language model (LLM) into our support bot. The model could understand natural language, generate human-like responses, and even learn from feedback. However, it lacked the ability to interact with external tools and services, making it feel like a disembodied brain - incredibly smart, but unable to act on its knowledge. Our bot would often respond with phrases like "I can help you with that," but then fail to actually perform the task. It was as if the LLM had no "hands" to manipulate the world.

The problem was that our LLM was not designed to make tool calls from scratch. It relied on pre-existing frameworks and libraries to interact with external services, which limited its flexibility and autonomy. We needed a way to give our LLM the ability to call tools and services without relying on these frameworks, essentially giving it "hands" to manipulate the world.

To solve this problem, we turned to the Model Context Protocol (MCP) and LangGraph. MCP provides a standardized way to represent and interact with external tools and services, while LangGraph allows us to model complex workflows and decision-making processes. By combining these two technologies, we could create a system that enables our LLM to make tool calls from scratch, without relying on pre-existing frameworks.

Here's an example of how we used LangGraph and MCP to give our LLM "hands":

import langgraph
from langgraph import StateGraph, add_node, add_conditional_edges
from mcp import Tool, ToolCall

# Define a tool that our LLM can call
class SendMessageTool(Tool):
    def __init__(self, message):
        self.message = message

    def execute(self):
        # Simulate sending a message
        print(f"Sending message: {self.message}")

# Create a LangGraph state graph
graph = StateGraph()

# Add nodes to the graph
start_node = add_node(graph, "start")
message_node = add_node(graph, "send_message")
end_node = add_node(graph, "end")

# Add conditional edges to the graph
add_conditional_edges(graph, start_node, message_node, lambda x: x["action"] == "send_message")
add_conditional_edges(graph, message_node, end_node, lambda x: x["message"] is not None)

# Define a tool call
tool_call = ToolCall(SendMessageTool, "Hello, world!")

# Create an LLM that can make tool calls
class LLM:
    def __init__(self, graph):
        self.graph = graph

    def make_tool_call(self, tool_call):
        # Use the graph to determine the next action
        current_node = self.graph.get_node("start")
        while current_node != self.graph.get_node("end"):
            # Get the next node based on the current state
            next_node = self.graph.get_next_node(current_node, {"action": "send_message", "message": tool_call.tool.message})
            # Execute the tool call
            if next_node == self.graph.get_node("send_message"):
                tool_call.tool.execute()
            current_node = next_node

# Create an instance of the LLM and make a tool call
llm = LLM(graph)
llm.make_tool_call(tool_call)

This code defines a simple tool that sends a message, and an LLM that uses a LangGraph state graph to determine when to make a tool call. The LLM can make tool calls from scratch, without relying on pre-existing frameworks.

One practical gotcha we learned from this experience is that it's essential to carefully design the state graph and tool calls to avoid infinite loops. If the graph is not properly structured, the LLM may get stuck in an infinite loop, making repeated tool calls without achieving any meaningful outcome.

As we continue to develop our agentic AI system, we'll explore more advanced topics, such as integrating multiple tools and services, and using MCP to handle errors and exceptions. Tomorrow, we'll dive deeper into the world of agentic AI and explore new ways to give our LLM "hands" to manipulate the world.