How to Implement Function Calling with Structured Outputs

Most language models return free-form text. That works for chatbots, but it breaks down when you need your application to actually do something with the response — parse a price, trigger an API call, update a record. Function calling (also called tool use) solves this by making the model return a structured payload instead of prose.

This guide shows you how to implement it in Python, covering both the standard function-calling pattern and structured output mode. By the end, you will have working code for extracting structured data, chaining tool calls, and handling errors gracefully.

What function calling actually does

The term is misleading. The model does not call your functions. What happens is:

1. You describe one or more tools (functions) in your API request — their names, parameters, and types.
2. The model decides whether to "call" a tool, and if so, returns a JSON object that matches the schema you described.
3. Your code parses that JSON and runs the actual function.
4. You feed the result back into the conversation.

The model is a JSON router, not an executor. Keeping this mental model clear matters when debugging — if the model returns bad JSON, the problem is usually in your schema definition, not your function logic.

Defining tools with JSON Schema

Most LLM providers accept tools in JSON Schema format. Here is a minimal example that extracts invoice data from raw text:

import json
import openai

client = openai.OpenAI()

tools = [
    {
        "type": "function",
        "function": {
            "name": "extract_invoice",
            "description": "Extract structured fields from an invoice text.",
            "parameters": {
                "type": "object",
                "properties": {
                    "vendor": {"type": "string", "description": "Company or person issuing the invoice"},
                    "amount": {"type": "number", "description": "Total amount in the invoice currency"},
                    "currency": {"type": "string", "enum": ["EUR", "USD", "GBP"]},
                    "due_date": {"type": "string", "description": "Due date in ISO 8601 format (YYYY-MM-DD)"},
                },
                "required": ["vendor", "amount", "currency"],
            },
        },
    }
]

def extract_invoice_data(raw_text: str) -> dict:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "user", "content": f"Extract the invoice data from this text:\n\n{raw_text}"}
        ],
        tools=tools,
        tool_choice={"type": "function", "function": {"name": "extract_invoice"}},
    )

    message = response.choices[0].message
    if message.tool_calls:
        return json.loads(message.tool_calls[0].function.arguments)
    return {}

Setting tool_choice to a specific function forces the model to always call it. Use "auto" when you want the language model to decide whether a tool call is appropriate — useful when not every user message requires a structured response.

Chaining tool calls in a conversation loop

Real agents rarely need just one tool call. A more common pattern is a loop where you keep feeding results back until the model stops requesting tools:

def run_agent(user_message: str, tools: list, tool_map: dict) -> str:
    messages = [{"role": "user", "content": user_message}]

    while True:
        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=messages,
            tools=tools,
            tool_choice="auto",
        )

        choice = response.choices[0]
        messages.append(choice.message)

        # No tool calls → model is done
        if choice.finish_reason == "stop":
            return choice.message.content

        # Execute each tool call and append results
        for tc in choice.message.tool_calls:
            fn_name = tc.function.name
            fn_args = json.loads(tc.function.arguments)

            if fn_name not in tool_map:
                result = {"error": f"Unknown tool: {fn_name}"}
            else:
                try:
                    result = tool_map<a href="**fn_args">fn_name</a>
                except Exception as e:
                    result = {"error": str(e)}

            messages.append({
                "role": "tool",
                "tool_call_id": tc.id,
                "content": json.dumps(result),
            })

tool_map is a plain dict mapping function names to callables. This keeps the loop generic — you add new tools by updating the dict, not by touching the loop logic. Always catch exceptions inside the loop and return them as JSON errors; the model can often recover from a tool error on its own.

Validating outputs with Pydantic

The model's JSON output is not guaranteed to match your schema exactly. Enums may come back in unexpected casing, required fields may be missing if the input was ambiguous. Validate before using:

from pydantic import BaseModel, field_validator
from typing import Optional
from datetime import date

class Invoice(BaseModel):
    vendor: str
    amount: float
    currency: str
    due_date: Optional[date] = None

    @field_validator("currency")
    @classmethod
    def normalize_currency(cls, v: str) -> str:
        return v.upper()

def safe_extract(raw_text: str) -> Invoice | None:
    raw = extract_invoice_data(raw_text)
    if not raw:
        return None
    try:
        return Invoice.model_validate(raw)
    except Exception as e:
        print(f"Validation failed: {e}")
        return None

Pydantic coerces types where it can (string "42.5" becomes float 42.5) and raises a clear error when it cannot. This single validation layer catches the majority of production failures.

Structured outputs: the stricter alternative

Some providers now support "structured outputs" — a mode where the model is constrained at the generation level to match a JSON Schema. Unlike regular function calling, this is a hard guarantee: invalid JSON becomes impossible.

schema = {
    "name": "invoice_extraction",
    "strict": True,
    "schema": {
        "type": "object",
        "properties": {
            "vendor": {"type": "string"},
            "amount": {"type": "number"},
            "currency": {"type": "string"},
        },
        "required": ["vendor", "amount", "currency"],
        "additionalProperties": False,
    },
}

response = client.beta.chat.completions.parse(
    model="gpt-4o-2024-08-06",
    messages=[{"role": "user", "content": "Extract: Invoice from Acme Corp, $1,250 USD"}],
    response_format={"type": "json_schema", "json_schema": schema},
)

data = json.loads(response.choices[0].message.content)

The strict: True flag enables constrained generation. It requires additionalProperties: false and all properties must be in required. This is the right choice when you know exactly what shape you need and cannot afford to handle malformed output at runtime.

For language models that don't natively support structured outputs, you can approximate this with careful prompting plus Pydantic validation and a retry loop — though you will never get the same guarantee.

Picking the right approach

Scenario	Recommended approach
Extract known fields from text	Function calling with tool_choice forced
Agent that may or may not need data	Function calling with tool_choice: auto
Critical data pipeline, no retries	Structured outputs (if supported)
Multi-step agentic loop	Conversation loop with tool_map

The security checklists at AYI NEDJIMI Consultants include a section on sandboxing and validating LLM tool outputs — worth reviewing if your tools touch sensitive systems or external APIs.

The takeaway

Function calling gives you a clean contract between your application logic and a language model. Define your schema with the minimum required fields, validate the output with Pydantic, and keep your execution loop generic. Structured outputs remove the validation burden entirely when your provider supports them — but they come with schema restrictions you need to plan around.

The common mistake is over-engineering the schema upfront. Start with the minimum fields your code will break without, and iterate from there.

---

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.