Adding Function Calling to an LLM App

Function calling is the bridge between large language models and the real world. Rather than asking a model to describe the weather, you give it a structured tool, let it decide when to invoke the tool, and feed the result back into the conversation. This pattern is the foundation of agents, retrieval pipelines, and any LLM app that needs to do more than generate text.

How Function Calling Works

At a high level, function calling is a request/response loop. You define one or more tools as JSON schemas and include them in your chat completion request. If the model decides it needs external data, it responds with a tool call instead of plain text. Your application runs the function, appends the result to the message history, and sends the history back to the model for a final answer.

Defining Your Tools

A tool definition is just a JSON schema. The model uses the name, description, and parameter structure to decide what to call and how to fill the arguments.

weather_tool = {
    "type": "function",
    "function": {
        "name": "get_current_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The city and state, e.g. San Francisco, CA"
                },
                "unit": {
                    "type": "string",
                    "enum": ["celsius", "fahrenheit"]
                }
            },
            "required": ["location"]
        }
    }
}

The Execution Loop

The heavy lifting happens in the conversation loop. Because Oxlo.ai is fully OpenAI SDK compatible, you can use the official openai Python package and only change the base URL.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ.get("OXLO_API_KEY")
)

messages = [
    {"role": "user", "content": "What is the weather like in Boston today?"}
]

# First request: model decides to call a tool
response = client.chat.completions.create(
    model="qwen-3-32b",
    messages=messages,
    tools=[weather_tool],
    tool_choice="auto"
)

message = response.choices[0].message
messages.append(message)

# If the model made a tool call, execute it and append the result
if message.tool_calls:
    for tool_call in message.tool_calls:
        # In production, route to real functions. Here we mock the response.
        function_result = {
            "temperature": "14",
            "unit": "celsius",
            "condition": "Cloudy"
        }

        messages.append({
            "role": "tool",
            "tool_call_id": tool_call.id,
            "name": tool_call.function.name,
            "content": str(function_result)
        })

    # Second request: model synthesizes the final answer
    final_response = client.chat.completions.create(
        model="qwen-3-32b",
        messages=messages,
        tools=[weather_tool]
    )

    print(final_response.choices[0].message.content)

Notice that the second request includes the full message history, including the tool result. The model uses that context to produce the final, grounded response.

Parallel Tool Calls

Modern models can request multiple tools in a single turn. The snippet above already iterates over message.tool_calls, so it handles parallel calls naturally. Execute independent functions concurrently, collect the results, and append one tool message per call ID.

Why Oxlo.ai for Agentic Workloads

Function calling shines in agentic patterns where a single user request triggers multiple model turns, long system prompts, and large JSON contexts. On token-based providers, every tool call inflates the prompt and drives up cost. Oxlo.ai uses flat per-request pricing, so each turn costs the same regardless of how many tokens are in flight. For multi-step agents, that predictability adds up quickly.

Oxlo.ai supports function calling across its agent-focused lineup, including Qwen 3 32B, Llama 3.3 70B, GLM 5, and Minimax M2.5. It is fully OpenAI SDK compatible, so the code you just saw runs without modification, and there are no cold starts on popular models. You can explore the exact request-based rates on the Oxlo.ai pricing page.

Production Checklist

Before shipping, harden the loop:

• Validate arguments. Never trust raw model output. Run JSON schema validation before executing functions.
• Handle timeouts. Wrap external API calls in retries and circuit breakers so a slow tool does not hang the chat.
• Be idempotent. A tool call may be retried or replayed. Design side effects so they are safe to run more than once.
• Limit recursion. Cap the number of tool turns to prevent runaway loops and runaway spend.

Conclusion

Function calling is not an exotic feature. It is a basic building block of useful LLM applications. The implementation loop is straightforward: define schemas, let the model choose, execute, and return. With Oxlo.ai, you get an OpenAI-compatible API, a broad catalog of tool-capable models, and flat per-request pricing that keeps agentic workloads predictable. If you are building agents today, the only change you need to make is the base URL.