Day 23: Tool Use and Function Calling — Connecting LLMs to APIs, Databases and Shells

By default a language model is an isolated reasoning engine. It cannot read the current time, query your private database or run a command. Function calling, also called tool use, is the bridge that connects that engine to real systems, so the model becomes a router that decides what to call while your software does the calling.

In plain terms: The model is a manager who can write instructions but cannot touch anything. It hands a clear, structured request to an assistant (your application), who does the task and reports back.

Diagram: On its own an LLM only has its trained weights. It cannot read the current time, query a private database, call a live API or run commands. Function calling is the bridge: the model asks for an action and your software performs it. See the animated version.

The four-step protocol

A common misconception is that the LLM runs the code. It does not. Function calling is a deterministic, four-step exchange.

1. Declare the tools. Your application sends the model a list of tools described with JSON schemas: the name, what it is for and its typed parameters.
2. Model invocation. If the question needs outside data or an action, the model replies with a structured tool call (the function name and exact arguments) instead of ordinary text.
3. Client-side execution. Your application intercepts that reply, validates it and runs the real code: a SQL query, a REST call, a script.
4. Observation injection. Your application sends the result back as a tool message, and the model uses it to finish its answer.

Diagram: In function calling the application declares tools as JSON schemas, the model replies with a structured tool call instead of text, the application validates and executes the real code, and the result is returned to the model as an observation so it can write the final answer. The model never runs code itself. See the animated version.

Diagram: The application describes a tool with a name, a purpose and typed parameters. The model answers with a structured JSON tool call naming the function and its arguments, and the application sends the real result back as an observation. The names and values are illustrative. See the animated version.

Type safety and validation

Free text breaks software pipelines. Use structured arguments and validate them with JSON Schema or a library such as Pydantic, so a malformed value is rejected before it reaches your backend. Then add checks the model cannot override: an allow-list of tools and a permission check for the current user.

Diagram: Before running a tool call, the application checks the arguments against a schema, checks the tool is on an allow-list, checks the user has permission, and only then executes and logs it. Bad or unsafe calls are rejected. Pydantic or JSON Schema keeps arguments type-safe. See the animated version.

Here is the loop with an OpenAI-compatible API. Ollama and vLLM from Day 16 expose the same interface, so the pattern works with a local model too. It assumes a model that supports tool calling, and get_gpu_status is a stand-in for your own function.

import json
from openai import OpenAI
from pydantic import BaseModel

client = OpenAI()                     # or OpenAI(base_url="http://localhost:11434/v1", api_key="x")

class GpuArgs(BaseModel):             # type-safe arguments
    node: str

def get_gpu_status(node: str) -> dict:
    return {"node": node, "status": "degraded"}   # replace with a real lookup

TOOLS = [{"type": "function", "function": {
    "name": "get_gpu_status",
    "description": "Returns GPU health for a node",
    "parameters": GpuArgs.model_json_schema(),
}}]
ALLOWED = {"get_gpu_status": (GpuArgs, get_gpu_status)}

messages = [{"role": "user", "content": "Is gpu-07 healthy?"}]
reply = client.chat.completions.create(model="gpt-4o-mini", messages=messages, tools=TOOLS)
msg = reply.choices[0].message

if msg.tool_calls:
    messages.append(msg)
    for call in msg.tool_calls:
        schema, fn = ALLOWED[call.function.name]                       # allow-list
        args = schema.model_validate_json(call.function.arguments)     # validate
        result = fn(**args.model_dump())                               # execute
        messages.append({"role": "tool", "tool_call_id": call.id,
                         "content": json.dumps(result)})               # observation
    final = client.chat.completions.create(model="gpt-4o-mini", messages=messages)
    print(final.choices[0].message.content)

Restricting the model to proposing structured arguments keeps execution under your control: authenticated, validated and logged inside your own application.

Coming Up Next

Day 24: Multi-agent orchestration: supervisor, worker and evaluator patterns.

FunctionCalling #ToolUse #APIs #Pydantic #SoftwareArchitecture

---

Originally published at https://sureshpallapothu.in/blog/day-23-function-calling, where this post includes animated diagrams.