How Actually Your Functions Get Called By LLMs?

How do tool calls actually happen?

That was the question that started this investigation for me.

I kept seeing examples like this:

@agent.tool
def get_weather(city: str) -> dict:
    return weather_api.fetch(city)

Then the explanation would casually say:

“The LLM calls the get_weather function.”

But... how?

Does the LLM reach into my Python process and execute that function?

Does it know where the function exists in memory?

Can it directly access my database, local files, or private APIs?

If an LLM receives input and generates output, how can it possibly call anything?

And if model providers already support native tool calling, why do we need agent frameworks such as PydanticAI, LangChain, or LangGraph?

That led me to several more questions:

• Who actually executes the function?
• How does the function result get back to the model?
• Is tool calling just a sophisticated prompting technique?
• Is LangChain merely adding instructions around an LLM?
• Can I build the entire loop myself?
• What does an agent framework provide beyond the loop?
• Is MCP the same thing as tool calling?
• Do I need an agent framework for every LLM application?

My original mental model was simple:

input → LLM → output

That model seemed incompatible with phrases such as:

The LLM called a tool.

So I decided to stop treating the process like magic and follow the data through real framework code. 🔍

This article documents what I found.

---

First: how I verified the claims in this article 🔬

Before explaining the architecture, I want to make the evidence trail explicit.

This is not based only on diagrams, AI generated explanations, or second-hand tutorials. Sure I used help of AI creating this article, But I assure you that I validated the results during my study, so you can rely on the results.

I checked:

1. The public documentation for PydanticAI and LangChain.
2. The current agent orchestration implementations.
3. The code that validates model-generated tool arguments.
4. The exact lines that invoke registered tools.
5. The code that converts function results back into model messages.
6. Commit-pinned revisions so the evidence remains inspectable after main changes.

Source-review timestamp

The source review for this article was completed on:

August 3, 2026 at 20:42 CEST

Inspected source snapshots

The commits below are the latest revisions I found for the specific files at the time of review. They are file-specific revisions, not necessarily the repository-wide HEAD commits.

Project	File	Inspected commit
PydanticAI	_agent_graph.py	5dee1ea37c86df4764ff143ef4259dc8027ab81e
PydanticAI	tool_manager.py	fbabc0f2d04be7b67c00d18e7e2c4fcfe0310ed2
PydanticAI	_tool_execution.py	d8a1254a5ad1a86d0164d497f43f12616069d6cf
LangChain	agents/factory.py	9ef324c9abb534b4839bd2a42f4fb36b1ec3f3c3
LangGraph	prebuilt/tool_node.py	2b1abc807b282245211f5ba8f292aaf3e24f1e07

The relevant commit-pinned files are linked throughout the article.

That matters because internal framework code changes.

A link to main shows whatever the project contains today. A commit-pinned link shows exactly what I inspected while writing this article.

The practical examples use Python, but the architecture is not Python-specific.

The same loop appears in TypeScript, Go, C#, Rust, and other languages:

model request
→ structured tool request
→ runtime validation
→ ordinary function execution
→ tool-result message
→ another model request

Python simply gives us readable framework implementations to inspect.

---

The answer in one paragraph 💡

An LLM does not directly execute my function.

It generates output that may represent a request to call a function.

My application—or an agent framework running inside it—reads that request, matches it to an available tool, validates the arguments, checks permissions, executes the real function, captures the result, returns that result to the model, and continues the loop.

The model proposes an action. The runtime validates, governs, and executes it.

Even native tool calling is still structured model output.

A model response may contain something logically similar to:

{
  "name": "get_weather",
  "arguments": {
    "city": "Tokyo"
  }
}

At that moment:

• get_weather() has not run.
• No weather API has been contacted.
• No Python function has been invoked.
• No external side effect has happened.

The model has produced a structured request.

Something else must execute it.

That “something else” is application code, framework code, or—in the case of certain provider-hosted tools—provider-operated runtime infrastructure.

It is not the neural network itself.

---

The four actors involved 🧩

The architecture becomes much clearer when I separate four components:

1. The LLM.
2. The provider API or model adapter.
3. The application or agent framework.
4. The actual tool implementation.

They cooperate, but they do not perform the same job.

---

1. The LLM: generating tokens and action requests 🧠

At its core, the model performs something conceptually similar to:

input tokens → model computation → output tokens

Depending on the provider, those output tokens may be interpreted as:

• Natural-language text.
• JSON.
• Structured content blocks.
• A tool-call object.
• Several tool calls.
• A mixture of text and action requests.

The raw model does not directly:

• Run Python.
• Execute JavaScript.
• Query my database.
• Read arbitrary files from my computer.
• Send an email from my account.
• Call my private API.
• Modify application state.
• Refund a payment.
• Restart my production server.

It can generate text that describes any of those actions.

It can also generate structured output requesting that surrounding software perform one of them.

That distinction is the foundation of tool calling.

---

2. The provider API: giving tool requests a structure

Model providers define conventions for presenting tools to models and returning tool requests to applications.

A model request may include:

• Conversation messages.
• System instructions.
• Tool names.
• Tool descriptions.
• JSON Schemas for tool arguments.
• Tool-selection settings.

A provider response may contain:

• Normal assistant text.
• One tool call.
• Multiple tool calls.
• Provider-specific content blocks.
• Text combined with tool requests.

The provider API makes the generated action request easier for application code to interpret.

It does not automatically possess or execute a local function running inside my application.

The provider may know this:

{
  "name": "get_customer",
  "description": "Retrieve a customer by ID.",
  "parameters": {
    "type": "object",
    "properties": {
      "customer_id": {
        "type": "string"
      }
    },
    "required": ["customer_id"]
  }
}

But it does not automatically have this:

def get_customer(customer_id: str) -> dict:
    return my_private_database.find_customer(customer_id)

The schema and the implementation are different things.

One important exception: provider-hosted tools

Some providers expose tools that run on their own infrastructure, such as hosted web search, file search, or code execution.

In those cases, provider software may execute the tool.

But even then, the model weights are not directly operating the server.

The model generates an action request, and a controlled provider runtime executes it.

The architecture still contains a separation between:

model decision

and:

external execution

---

Prompt-based tool use versus native tool calling

Before native tool APIs became common, applications often told models:

If you need a tool, return JSON containing:
- the tool name
- the arguments

A model might answer:

{
  "name": "get_customer",
  "arguments": {
    "customer_id": "cust-1001"
  }
}

The application would then:

1. Parse the JSON.
2. Resolve the tool name.
3. Validate the arguments.
4. Execute the function.
5. Send the result back to the model.

With native tool calling, the application sends tool definitions through a dedicated API field, and the provider returns a dedicated tool-call structure.

Conceptually:

{
  "tool_calls": [
    {
      "id": "call_123",
      "name": "get_customer",
      "arguments": {
        "customer_id": "cust-1001"
      }
    }
  ]
}

Both approaches involve the model generating an action request.

Native tool calling generally provides:

• Better message structure.
• Provider-supported call IDs.
• More reliable argument handling.
• Easier support for multiple calls.
• Clearer separation between text and actions.
• Less fragile output parsing.

It does not eliminate the execution loop.

---

3. The application or framework: running the loop 🔁

The application or agent framework owns the orchestration.

It commonly performs these operations:

1. Send messages and available tool definitions to the model.
2. Read the model response.
3. Detect tool requests.
4. Match each requested name to an implementation.
5. Validate the arguments.
6. Check authorization and approval rules.
7. Execute the actual tool.
8. Capture its result or exception.
9. Create a tool-result message.
10. Send the updated conversation back to the model.
11. Continue until the model returns a final answer.

A production framework may also handle:

• Streaming.
• Parallel tool calls.
• Retries.
• Invalid-argument correction.
• Timeouts.
• Cancellation.
• Token budgets.
• Cost limits.
• Maximum-step limits.
• Dependency injection.
• Conversation state.
• Durable execution.
• Checkpoints.
• Human approval.
• Structured outputs.
• Logging and tracing.

This orchestration layer is where agent frameworks earn most of their value.

---

4. The tool: ordinary executable software 🛠️

The tool itself is normal code:

def get_weather(city: str) -> dict[str, object]:
    return weather_api.fetch(city)

It might instead be:

async function getWeather(city: string): Promise<Weather> {
  return weatherApi.fetch(city);
}

Or:

func GetWeather(ctx context.Context, city string) (Weather, error) {
    return weatherClient.Fetch(ctx, city)
}

The language does not change the architectural boundary.

The function runs because some runtime invokes it.

The neural network does not reach into the language runtime and call it directly.

---

Following one tool call from beginning to end

Let’s use one tool throughout the article:

def get_customer(customer_id: str) -> dict[str, str]:
    ...

The user asks:

What subscription plan is customer cust-1001 using?

Here is the full lifecycle.

---

Step 1: The application sends the model a tool definition

The application sends:

• The user’s message.
• Relevant instructions.
• The get_customer tool definition.
• A schema describing its arguments.

A simplified definition might look like:

{
  "name": "get_customer",
  "description": "Retrieve a customer by customer ID.",
  "parameters": {
    "type": "object",
    "properties": {
      "customer_id": {
        "type": "string"
      }
    },
    "required": ["customer_id"]
  }
}

The model can now infer that get_customer may help answer the question.

It still cannot directly execute the implementation.

---

Step 2: The model generates a tool request

The response may be normalized into:

{
  "tool_calls": [
    {
      "id": "call_123",
      "name": "get_customer",
      "arguments": {
        "customer_id": "cust-1001"
      }
    }
  ]
}

At this point, the customer function has still not run.

The model has effectively written a request ticket:

Please run get_customer with customer_id="cust-1001".

---

Step 3: The runtime validates the request

The application or framework asks:

• Does get_customer exist?
• Was it exposed to this model invocation?
• Is customer_id present?
• Is its value a string?
• Is this user allowed to access cust-1001?
• Does the operation require human approval?
• Has the agent exceeded its step limit?

Only after these checks should the runtime execute the tool.

---

Step 4: Ordinary software invokes the function

The framework performs something logically equivalent to:

result = get_customer(customer_id="cust-1001")

The function might return:

{
  "id": "cust-1001",
  "name": "Alice",
  "plan": "pro"
}

This is the moment when the actual work happens.

The database was queried by application code.

The model generated the proposal.

---

Step 5: The runtime creates a tool-result message

The function’s return value must be represented in the conversation:

{
  "role": "tool",
  "tool_call_id": "call_123",
  "content": {
    "id": "cust-1001",
    "name": "Alice",
    "plan": "pro"
  }
}

Provider formats differ, but the purpose is the same:

• Associate the result with the correct tool request.
• Make the result visible to the model.

---

Step 6: The model is called again

The next model request contains:

• The original question.
• The model’s earlier tool call.
• The tool result.

The model can now answer:

Customer cust-1001 is using the Pro plan.

The complete sequence looks like this:

sequenceDiagram
    actor User
    participant Runtime as Application / Agent Framework
    participant Model as LLM Provider
    participant Tool as Tool Implementation

    User->>Runtime: What plan is cust-1001 using?
    Runtime->>Model: Messages + get_customer schema
    Model-->>Runtime: Tool request: get_customer(cust-1001)

    Note over Runtime,Tool: The model requested an action.<br/>The function has not run yet.

    Runtime->>Runtime: Validate arguments and permissions
    Runtime->>Tool: Invoke get_customer("cust-1001")
    Tool-->>Runtime: {"plan": "pro"}

    Runtime->>Model: Tool-result message
    Model-->>Runtime: Customer cust-1001 uses the Pro plan
    Runtime-->>User: Final response

The most important detail is the missing direct connection:

LLM ─────X────→ local function

The runtime sits between the model and the executable code.

---

Building the loop without a framework

Once I understood the data flow, the basic loop looked surprisingly small.

The following is intentionally provider-neutral pseudocode:

async def run_agent(user_message: str) -> str:
    messages = [
        {
            "role": "user",
            "content": user_message,
        }
    ]

    for _ in range(10):
        response = await call_model(
            messages=messages,
            tools=tool_schemas,
        )

        messages.append(response.message)

        if not response.tool_calls:
            return response.text

        for call in response.tool_calls:
            tool = tool_registry[call.name]

            arguments = validate_arguments(
                tool=tool,
                arguments=call.arguments,
            )

            authorize_tool_call(
                tool=tool,
                arguments=arguments,
                current_user=current_user,
            )

            result = await execute_tool(
                tool=tool,
                arguments=arguments,
            )

            messages.append(
                {
                    "role": "tool",
                    "tool_call_id": call.id,
                    "content": serialize(result),
                }
            )

    raise RuntimeError("Maximum agent steps exceeded")

This is the heart of many agent frameworks.

What each stage does

call_model(...)

This converts application messages and tool schemas into the provider’s request format.

It also converts the provider response into objects the application can inspect.

response.tool_calls

These calls are model-generated output.

The model may:

• Select the wrong tool.
• Invent a tool name.
• Omit an argument.
• Use the wrong type.
• Request an unauthorized resource.
• Produce semantically dangerous arguments.

Structured output is not automatically trustworthy output.

tool_registry[call.name]

The runtime resolves a generated string to an executable implementation:

tool_registry = {
    "get_customer": get_customer,
    "search_orders": search_orders,
}

This is a crucial security boundary.

The model can only request tools that the application exposes and resolves.

validate_arguments(...)

Validation checks whether generated arguments match the expected structure.

For example:

class GetCustomerArguments(BaseModel):
    customer_id: str

This can prove that customer_id is a string.

It cannot prove that the current user is allowed to access that customer.

authorize_tool_call(...)

Authorization is a business decision, not a schema decision.

if customer_id not in current_user.allowed_customers:
    raise PermissionError("Customer access denied")

execute_tool(...)

This is where the actual side effect occurs.

The executor may need to support:

• Sync and async functions.
• Exceptions.
• Timeouts.
• Cancellation.
• Approval.
• Tracing.
• Sandboxing.
• Dependency injection.

The step limit

Without a stopping guard, the model could repeatedly request tools:

model → tool → model → tool → model → tool → ...

A maximum-step limit is ordinary defensive engineering.

The loop itself is not magic. The engineering around the loop is where frameworks earn their keep.

---

Proof from PydanticAI’s implementation 🔬

Now let’s move past conceptual pseudocode and inspect a real framework.

The public starting points are:

• PydanticAI repository
• PydanticAI tools documentation

The documentation shows that PydanticAI can derive a tool schema from a Python function’s signature and docstring.

A current-style example looks like:

import asyncio
import os

from pydantic_ai import Agent


CUSTOMERS = {
    "cust-1001": {
        "id": "cust-1001",
        "name": "Alice",
        "plan": "pro",
    }
}


agent = Agent(
    os.environ["MODEL"],
    instructions=(
        "Answer questions about customer subscriptions. "
        "Use get_customer when customer data is required."
    ),
)


@agent.tool_plain
async def get_customer(customer_id: str) -> dict[str, str]:
    """Retrieve a customer by customer ID."""
    customer = CUSTOMERS.get(customer_id)

    if customer is None:
        return {
            "id": customer_id,
            "error": "Customer not found",
        }

    return customer


async def main() -> None:
    result = await agent.run(
        "What subscription plan is customer cust-1001 using?"
    )
    print(result.output)


if __name__ == "__main__":
    asyncio.run(main())

The public API is convenient, but it does not show us who invokes the function.

For that, we need to follow the internal execution path.

---

PydanticAI step 1: the model response enters the graph

In the inspected revision of _agent_graph.py, a model response containing tool calls is routed to a CallToolsNode.

The relevant transition is logically represented by:

return CallToolsNode[DepsT, NodeRunEndT](last_message)

This tells us something important:

The model response does not execute the Python function by itself.

The response is handed to another runtime node responsible for processing tool requests.

---

PydanticAI step 2: validation and execution are separate operations

Inside the inspected revision of _tool_execution.py, the framework first validates the model-generated call:

validated = await self.tool_manager.validate_tool_call(call)

Then it executes the validated call:

tool_result = await self.tool_manager.execute_tool_call(validated)

That separation is excellent evidence of the actual boundary:

model-generated tool request
→ framework validation
→ framework execution

The model is not performing either operation.

---

PydanticAI step 3: the tool manager invokes executable code

The inspected revision of tool_manager.py eventually delegates the validated call to the registered tool set:

return await self.toolset.call_tool(
    name,
    validated.validated_args,
    validated.ctx,
    validated.tool,
)

That is the real execution handoff.

The framework has:

1. Received a generated tool request.
2. Resolved the tool.
3. Validated its arguments.
4. Invoked registered executable code.

There is no mystery left in this part of the path.

---

PydanticAI step 4: the result becomes a model message

After execution, _tool_execution.py creates a ToolReturnPart.

Conceptually, that part contains:

• The tool name.
• The returned content.
• The tool-call ID.
• Additional metadata.

The result is then included in a later model request.

The PydanticAI documentation also exposes this sequence through its message history:

ToolCallPart
→ actual tool execution
→ ToolReturnPart
→ later model response

That is the framework loop made visible.

PydanticAI’s execution path

Agent receives registered tool
        ↓
Tool schema is generated
        ↓
Provider adapter sends schema to model
        ↓
Model returns ToolCallPart
        ↓
Agent graph routes response to CallToolsNode
        ↓
ToolManager validates the call
        ↓
ToolManager executes the registered tool
        ↓
ToolReturnPart is created
        ↓
Result is sent in another model request
        ↓
Model produces final text or another tool call

Useful source symbols include:

ModelRequestNode
CallToolsNode
ToolManager
ToolCallPart
ToolReturnPart
validate_tool_call
execute_tool_call

These are internal implementation details, not stable public APIs.

If a symbol moves in a future release, search the repository rather than assuming the path remains permanent:

rg "CallToolsNode" pydantic-ai
rg "validate_tool_call" pydantic-ai
rg "execute_tool_call" pydantic-ai
rg "ToolReturnPart" pydantic-ai

---

Proof from LangChain and LangGraph 🔬

Now let’s follow the same operation through the current LangChain agent stack.

Official starting points:

• LangChain repository
• LangChain agents documentation
• LangChain tools documentation
• LangGraph repository

A current LangChain example for the same use case looks like:

import os

from langchain.agents import create_agent
from langchain.tools import tool


CUSTOMERS = {
    "cust-1001": {
        "id": "cust-1001",
        "name": "Alice",
        "plan": "pro",
    }
}


@tool
def get_customer(customer_id: str) -> dict[str, str]:
    """Retrieve a customer by customer ID."""
    customer = CUSTOMERS.get(customer_id)

    if customer is None:
        return {
            "id": customer_id,
            "error": "Customer not found",
        }

    return customer


agent = create_agent(
    model=os.environ["MODEL"],
    tools=[get_customer],
    system_prompt=(
        "Answer questions about customer subscriptions. "
        "Use get_customer when customer data is required."
    ),
)


result = agent.invoke(
    {
        "messages": [
            {
                "role": "user",
                "content": (
                    "What subscription plan is customer "
                    "cust-1001 using?"
                ),
            }
        ]
    }
)

print(result["messages"][-1].content)

Again, the public API hides most of the loop.

So let’s follow the source.

---

LangChain step 1: create_agent constructs a loop

In the inspected revision of LangChain’s agents/factory.py, the create_agent documentation describes its purpose directly:

"""Creates an agent graph that calls tools in a loop until a stopping condition is met."""

This is not merely a prompt template.

The factory constructs graph-based orchestration around:

• A model node.
• Tool routing.
• Tool execution.
• State updates.
• Stopping conditions.

The implementation imports LangGraph components including ToolNode and message types such as ToolMessage.

---

LangChain step 2: the model returns tool calls

The model does not execute the tool.

It returns an AI message containing structured tool-call data.

Conceptually:

{
  "name": "get_customer",
  "args": {
    "customer_id": "cust-1001"
  },
  "id": "call_123"
}

Graph routing examines that message.

If tool calls are present, execution moves to the tool node.

If no tool calls are present, the graph can stop and return the final answer.

---

LangGraph step 3: ToolNode invokes the registered tool

The decisive evidence appears in the inspected revision of LangGraph’s tool_node.py.

For synchronous tools, the execution path contains:

response = tool.invoke(call_args, config)

For asynchronous tools, it contains:

response = await tool.ainvoke(call_args, config)

That is the actual function-execution boundary.

The model generated a request.

ToolNode resolved the registered tool and invoked it through normal runtime code.

The source does not require us to guess who performs the execution.

It shows us.

---

LangGraph step 4: the result becomes a ToolMessage

After the invocation, ToolNode normalizes the result into a ToolMessage.

That message contains information such as:

• The returned content.
• The tool name.
• The tool-call ID.
• Status or error information.

The message is appended to graph state.

The graph then returns to the model node, allowing the model to read the tool result and decide whether to:

• Answer the user.
• Request another tool.
• Retry with different arguments.
• Continue another branch of the workflow.

LangChain and LangGraph’s execution path

Python function is converted to a tool
        ↓
Tool schema is bound to the chat model
        ↓
Model returns AIMessage with tool_calls
        ↓
Graph routing detects tool_calls
        ↓
ToolNode resolves the requested tool
        ↓
ToolNode calls tool.invoke() or tool.ainvoke()
        ↓
Result is converted to ToolMessage
        ↓
ToolMessage is added to graph state
        ↓
Graph returns to model node
        ↓
Execution stops when no tool call remains

Useful source searches include:

rg "create_agent" langchain
rg "bind_tools" langchain
rg "class ToolNode" langgraph
rg "tool.invoke" langgraph
rg "tool.ainvoke" langgraph
rg "ToolMessage" langchain langgraph

---

A note about old LangChain tutorials

Many older tutorials use concepts such as:

AgentExecutor
format_to_tool_messages
classic tool-calling agents
agent scratchpads

Those examples belong to earlier or classic LangChain APIs.

Modern create_agent examples increasingly expose graph-based execution built with LangGraph components.

That does not make the older material useless, but mixing classic APIs with modern APIs in one runnable example can create confusion.

When reading framework articles, always check:

• The publication date.
• The installed package version.
• Whether the example uses classic or current APIs.
• Whether the linked source still matches the explanation.

---

What these source tours prove

PydanticAI and LangChain/LangGraph organize their internals differently, but both implement the same fundamental loop:

model produces structured tool request
        ↓
framework reads request
        ↓
framework validates and resolves tool
        ↓
framework invokes executable code
        ↓
framework converts result into message
        ↓
framework calls model again

PydanticAI makes the sequence visible through concepts such as:

CallToolsNode
ToolManager
validate_tool_call
execute_tool_call
ToolReturnPart

LangChain and LangGraph expose the sequence through concepts such as:

create_agent
tool_calls
ToolNode
tool.invoke
tool.ainvoke
ToolMessage

Different abstractions, same architectural boundary.

The LLM creates the request. The runtime performs the side effect.

---

Why not keep a handwritten loop?

For a small application, I often should.

The handwritten loop is excellent when:

• Learning how tool calling works.
• Building a prototype.
• Using a single provider.
• Exposing one to three tools.
• Wanting maximum transparency.
• Having simple request-response behavior.

The complexity grows when I need:

• Provider-specific adapters.
• Streaming.
• Parallel calls.
• Tool-call retries.
• Validation-error feedback.
• Tool exception handling.
• Maximum-step limits.
• Token and cost budgets.
• Dependency injection.
• Durable state.
• Checkpoints.
• Human approval.
• Authorization.
• Observability.
• Structured outputs.
• Cancellation.
• Timeouts.
• User-specific tool visibility.
• Deterministic tests.

Eventually, my tiny loop contains:

• A state machine.
• A tool registry.
• Validators.
• Adapters.
• Retry policies.
• Middleware.
• Persistence.
• Tracing.
• Error translation.

At that point, I am building an agent framework—just one with a single maintainer and no documentation. 😅

---

PydanticAI versus LangChain/LangGraph

This is not a popularity contest.

They emphasize different developer experiences.

Concern	Handwritten loop	PydanticAI	LangChain/LangGraph
Learning the fundamentals	Excellent	Good	Good
Minimal dependencies	Excellent	Moderate	Lower
Type-driven development	Manual	Strong focus	Supported
Pydantic validation	Manual	Central design	Optional/integrated
Complex graph workflows	Manual	Possible	Strong focus
Provider support	Manual	Framework-supported	Broad ecosystem
State and persistence	Manual	Available patterns	Core LangGraph strength
Human-in-the-loop	Manual	Supported patterns	Strong graph support
Source-code surface	Small initially	Focused	Larger ecosystem
Production orchestration	Must build it	Provided abstractions	Extensive abstractions

Consider PydanticAI when

• The application is primarily Python.
• Strong runtime validation matters.
• You prefer type-oriented APIs.
• You want explicit dependency patterns.
• You need structured outputs.
• You like Pydantic’s development model.

Consider LangChain and LangGraph when

• The workflow branches.
• Execution must resume after interruption.
• Durable state and checkpoints matter.
• Many systems must be integrated.
• Middleware is important.
• Human approval is part of the workflow.
• You want graph-level control.

Consider neither when

• One model request solves the problem.
• No tools are needed.
• The workflow is tiny.
• A handwritten loop is clearer.
• Framework complexity would exceed application complexity.

Do not install an agent framework merely because your application contains an LLM.

A text summarizer does not need a robotic middle-management department. 🤖

---

What does the model decide?

The model commonly proposes:

• Whether the current information is sufficient.
• Whether a tool appears necessary.
• Which visible tool seems appropriate.
• Which arguments to generate.
• Whether another call is needed after seeing a result.
• How to phrase the final response.

These are probabilistic decisions.

The model may make a bad proposal.

---

What does the runtime decide? 🔐

The application or framework determines:

• Which tools are visible.
• How schemas are generated.
• Whether arguments are valid.
• Whether the user is authorized.
• Whether approval is required.
• Whether execution should be blocked.
• How the function is invoked.
• Which timeout applies.
• How exceptions are represented.
• How many iterations are permitted.
• Which state is persisted.
• When execution stops.
• What gets logged.

For example:

if tool_call.name == "refund_payment":
    require_human_approval()

if current_user.role != "admin":
    hide_admin_tools()

if step_count >= 8:
    stop_agent()

if requested_amount > current_user.refund_limit:
    reject_tool_call()

The model may propose a refund.

The application owns the money.

A prompt such as this is not an authorization system:

Please never issue an unauthorized refund.

Prompts influence model behavior.

They do not replace deterministic controls.

---

Are agent frameworks just prompt templates?

Not really.

Some early agents and custom systems relied heavily on prompts such as:

When you need a tool, return JSON.

Modern providers expose structured tool definitions and tool-call responses.

Frameworks still use instructions to influence:

• When tools should be used.
• Which tools are appropriate.
• How outputs should be formatted.

But their main responsibility is orchestration:

request
→ response
→ tool detection
→ validation
→ authorization
→ execution
→ result
→ next request
→ stopping condition

The analogy I now use is:

• The LLM is the planner.
• The framework is the dispatcher.
• The tool is the worker.

Or:

• The model writes a request ticket.
• The runtime checks and routes it.
• Executable software performs the operation.

---

Making the invisible loop visible 🔍

Agent debugging becomes much easier when I stop logging only the final answer.

I want to see:

MODEL REQUEST
AVAILABLE TOOLS
MODEL RESPONSE
MODEL TOOL CALL
VALIDATED ARGUMENTS
AUTHORIZATION RESULT
TOOL START
TOOL RESULT
MODEL FOLLOW-UP
FINAL RESPONSE

A useful trace may look like:

[model] user asks about cust-1001
[model] requests get_customer({"customer_id": "cust-1001"})
[validation] arguments accepted
[authorization] access granted
[tool] executing get_customer
[tool] returned {"id": "cust-1001", "plan": "pro"}
[model] follow-up includes tool result
[model] final answer generated

When execution behaves strangely, I check:

1. Did the model receive the correct tool schema?
2. Did it generate a tool call?
3. Was the requested name correct?
4. Were the arguments valid?
5. Was the user authorized?
6. Did the runtime invoke the tool?
7. Did the function succeed?
8. Was the result linked to the correct call ID?
9. Did the next request contain the result?
10. Why did the loop stop?

I also recommend:

• Enabling framework tracing.
• Inspecting raw provider responses.
• Reading the full message history.
• Placing breakpoints inside tools.
• Testing tools independently.
• Using fake models for orchestration tests.
• Setting iteration limits.
• Recording tool latency and failures.

Replacing this:

The agent got confused.

with this:

The second model response contained no tool_calls.

is a major debugging upgrade.

Mystery reduced. Bug acquired. 🐛

---

Where MCP fits

Tools do not have to be local functions.

They may be exposed by another process through a protocol such as MCP.

MCP can standardize how an application:

• Discovers tools.
• Reads their schemas.
• Sends tool requests.
• Receives results.

But MCP does not change the model loop:

model requests tool
→ application receives request
→ application invokes or forwards request
→ application receives result
→ application returns result to model

MCP and tool calling are not the same thing.

• Tool calling is the model-facing action-request mechanism.
• MCP is one possible application-facing protocol for connecting to external capabilities.

MCP is the next layer of this story, but it deserves its own article.

---

Ten misconceptions worth correcting

1. “The LLM executes my function.”

It does not.

The model generates an action request. Runtime software executes the function.

2. “Native tool calling gives the model operating-system access.”

It does not.

It provides a structured convention for tool definitions and model-generated tool requests.

3. “The framework decides everything.”

It does not.

The model commonly proposes which tool to use and which arguments to generate.

4. “The model decides everything.”

Definitely not.

The runtime controls exposure, validation, authorization, execution, state, and stopping rules.

5. “Agent frameworks are just prompt templates.”

They may use prompts, but they also implement adapters, execution loops, validation, state, retries, middleware, and tracing.

6. “MCP and tool calling are the same thing.”

They operate at different architectural layers.

7. “Every AI application needs LangChain.”

It does not.

Many applications need one model request and no agent loop.

8. “An agent is automatically autonomous and intelligent.”

An agent may simply be a model running inside a loop with tools and stopping conditions.

The label does not guarantee good judgment.

9. “Giving a tool to a model makes execution safe.”

It does not.

Safety requires deterministic controls such as authorization, approval, isolation, limits, and auditing.

10. “Tool-call arguments are guaranteed to be valid and authorized.”

They are not.

Tool calls are generated output and should be treated as untrusted input.

---

The mental model I am keeping 💡

I started with these questions:

How do tool calls actually happen?

Does the LLM really execute my function?

If providers support tool calling, why do we need agent frameworks?

My answer now is that generating an action request and operating a reliable tool-using system are different problems.

Here is the mental model I keep:

LLM

Generates text and structured action requests.

Provider adapter

Translates between application-level messages and provider-specific formats.

Agent framework

Runs the loop, manages state, validates calls, executes tools, handles failures, and returns results to the model.

Tool

Performs the real operation.

My application

Owns permissions, business rules, safety, and the user experience.

The framework does not give the model magical hands.

It creates a controlled runtime around model-generated proposals.

Once I followed the actual code, “agentic AI” stopped looking like a mysterious new type of computer.

It looked like something much more familiar:

A model proposes the next operation, while ordinary software validates, executes, records, and controls it.

That is less magical.

It is also much easier to understand, test, secure, and debug. 🛠️

In the next article, I’ll follow the same approach with MCP: no magic words—just the client, server, transport, and the actual messages moving between them.

Comments

[Reid Marlow]

The mental model I use is that the model never calls the function. It emits a structured request, and the host runtime decides whether to execute it. That boundary is where auth, validation, rate limits, and logging belong. If a framework makes that look invisible, I get nervous.