Language models are great at generating text. They are less predictable when you need them to return machine-readable data every single time. Function calling — sometimes called tool use — solves this by letting you describe functions in JSON Schema format, then having the model decide when and how to call them. Paired with structured output constraints, you get deterministic, parse-safe responses instead of hoping the model formats its reply correctly.
This matters in production. A model that occasionally returns "price": "twelve dollars" instead of "price": 12.0 breaks downstream parsing and requires fragile regex fallbacks. Function calling eliminates that class of bug.
What function calling actually does
When you send a request with function definitions, the model does not call anything. It returns a structured payload saying "I would call this function with these arguments." Your code then executes whatever that means — a database lookup, an API call, a local computation — and optionally feeds the result back to the model for a final natural-language response.
The key insight: function calling is a contract between you and the model about output format, not an execution mechanism. The model is a router, not an executor.
Defining a function schema
Major provider SDKs use JSON Schema for function definitions. Here is a minimal Python example:
import json
import openai
client = openai.OpenAI() # uses OPENAI_API_KEY from env
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a city",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "City name, e.g. 'Paris'"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"default": "celsius"
}
},
"required": ["city"],
"additionalProperties": False
},
"strict": True
}
}
]
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "What's the weather in Lyon?"}],
tools=tools,
tool_choice="auto"
)
message = response.choices[0].message
if message.tool_calls:
call = message.tool_calls[0]
args = json.loads(call.function.arguments)
print(f"Function: {call.function.name}")
print(f"Arguments: {args}")
# -> {'city': 'Lyon', 'unit': 'celsius'}
The "strict": True flag forces the model to conform exactly to your schema — no extra keys, no missing required fields. Enable it. Without it, the model may omit optional fields or add unexpected properties that break your deserialization code.
Handling the tool call loop
In a real agent, you feed the result back to continue the conversation. The pattern is always the same: execute the tool, append the result, call the API again, repeat until no tool calls remain.
def call_tool(name: str, args: dict) -> str:
"""Execute the tool and return a string result."""
if name == "get_weather":
# Replace with a real weather API call
return json.dumps({"city": args["city"], "temp_c": 18, "condition": "partly cloudy"})
raise ValueError(f"Unknown tool: {name}")
def run_agent(user_message: str, max_iterations: int = 10) -> str:
messages = [{"role": "user", "content": user_message}]
iterations = 0
while iterations < max_iterations:
iterations += 1
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=messages,
tools=tools,
tool_choice="auto"
)
msg = response.choices[0].message
finish_reason = response.choices[0].finish_reason
if finish_reason == "length":
raise RuntimeError("Response truncated — increase max_tokens or shorten schema")
messages.append(msg) # append assistant turn to history
if not msg.tool_calls:
return msg.content # done
for call in msg.tool_calls:
result = call_tool(
call.function.name,
json.loads(call.function.arguments)
)
messages.append({
"role": "tool",
"tool_call_id": call.id,
"content": result
})
raise RuntimeError(f"Agent did not terminate after {max_iterations} iterations")
print(run_agent("What's the weather in Lyon and should I bring a jacket?"))
Two things worth noting: always cap iterations, and always check finish_reason before parsing arguments. A truncated response mid-JSON will fail silently without that guard.
Structured outputs for extraction
Function calling is designed for "the model decides what to do next." If you just need to extract structured data from unstructured text, the Structured Outputs approach is cleaner — you pass a Pydantic model and get back a typed, validated object:
from pydantic import BaseModel
from openai import OpenAI
client = OpenAI()
class Invoice(BaseModel):
vendor: str
total_eur: float
due_date: str # ISO 8601
line_items: list[str]
completion = client.beta.chat.completions.parse(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": "Extract invoice data from the text."},
{
"role": "user",
"content": (
"Invoice from Acme Corp, 1250 EUR due 2026-09-30. "
"Line items: consulting 10h, travel expenses."
)
}
],
response_format=Invoice,
)
invoice = completion.choices[0].message.parsed
print(invoice.vendor) # Acme Corp
print(invoice.total_eur) # 1250.0
print(invoice.due_date) # 2026-09-30
print(invoice.line_items) # ['consulting 10h', 'travel expenses']
The model either returns a valid Invoice or raises a RefusalError. No try/except around json.loads, no regex fallbacks, no post-processing.
The decision rule is simple: use structured outputs for deterministic extraction, use function calling when the model needs to decide which action to take.
Common pitfalls
Over-engineering the schema. Keep schemas flat where possible. Deeply nested schemas with many optional fields increase the chance the model makes wrong choices. If you find yourself at three levels of nesting, split the task into two sequential calls.
Not handling refusals. Models can refuse to fill a field — especially for sensitive or ambiguous content. Always check message.refusal before accessing message.parsed:
if completion.choices[0].message.refusal:
raise ValueError(f"Model refused: {completion.choices[0].message.refusal}")
Putting too many tools in scope. Don't expose twenty functions at once and expect the model to pick correctly. For complex agents, gate which tools are available based on conversational state. More tools means more ambiguity, which means more routing errors.
Trusting arguments blindly. The model constructs tool arguments from user input. A user who types something adversarial could end up with unexpected values in your function arguments. Treat every argument as untrusted input — validate, sanitize, and check authorization before execution.
The takeaway
Function calling and structured outputs are complementary, not competing. Tool use handles agentic decision-making: which action, in what order, with what parameters. Structured outputs handle data extraction: get me a typed object, every time, without parsing brittle text.
For anything touching user data, access controls, or external services, apply defense-in-depth to every function the model can invoke. The model does not know your business logic — you do. Treat the LLM's function arguments exactly as you would treat user-supplied HTTP query parameters: validate types, enforce bounds, check permissions. A solid security hardening checklist applied at the tool layer prevents the most common agentic vulnerabilities before they reach production.
I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.
Top comments (0)