DEV Community

Devanshu Biswas
Devanshu Biswas

Posted on

Function calling: how a model that can't do math answers "what's 23*47?"

#ai #llm #agents #beginners

An LLM can't multiply reliably, doesn't know today's weather, and can't send an email. So how do ChatGPT-style assistants and every "AI agent" pull it off? Function calling. The model doesn't answer — it asks your code to run a tool, reads the result, and continues.

🛠️ Step through the loop (live): https://dev48v.infy.uk/ai/days/day28-function-calling.html

The one idea

Give the model a menu of tools. Each tool is a JSON schema: a name, a description, and typed parameters. Instead of replying in prose, the model can return a structured tool call:

{ "name": "calculator", "arguments": { "expr": "23*47" } }
Enter fullscreen mode Exit fullscreen mode

That's not the answer. It's a request. Your code runs the real function, gets 1081, feeds it back into the conversation, and calls the model again. Now the model has a real number to work with.

Why the model needs this at all

Three gaps, one fix:

  • No live data. Weather, prices, your database — none of it is in the model's frozen weights.
  • No actions. It can't book a flight or write a row. It only predicts text.
  • Shaky arithmetic. It does math by pattern-matching tokens, so big multiplications come out subtly wrong.

Tools hand the model a calculator, eyes, and hands. The model's only job is to decide which tool and what arguments — from the schema descriptions alone. It never sees your code.

The loop

This is the whole mechanism:

user question
  → model returns a tool_call
    → YOUR CODE runs the tool
      → feed the result back as a message
        → call the model again
          → another tool_call, or the final answer
Enter fullscreen mode Exit fullscreen mode

Repeat until the model stops asking for tools and just answers. In Python:

while True:
    resp = llm(messages, tools=TOOLS)
    if not resp.tool_calls:
        return resp.content              # final answer -> stop
    for call in resp.tool_calls:
        result = TOOLKIT[call.name](**call.arguments)   # you run it
        messages.append(tool_msg(call.id, result))      # feed back
Enter fullscreen mode Exit fullscreen mode

The messages array grows

The conversation is an array the model re-reads every turn. Tool use adds two roles: an assistant message carrying tool_calls, and a tool message carrying each result, keyed by a matching id.

0 user       "23*47 and is it raining in Paris?"
1 assistant  tool_calls: [calculator]
2 tool       (id c1) -> 1081
3 assistant  tool_calls: [get_weather]
4 tool       (id c2) -> {condition: "Light rain", temp_c: 14}
5 assistant  "23 x 47 = 1081, and yes, it's raining in Paris."
Enter fullscreen mode Exit fullscreen mode

Parallel vs sequential

If two calls are independent — the multiplication and the Paris weather don't depend on each other — the model can emit both at once and you run them together. One round-trip instead of two.

But when one call needs another's output first (search for a city, then get its weather), the calls have to be sequential. The demo lets you flip between the two and watch the round-trip count change.

The part people skip: this is untrusted input

The arguments come from the model, and the model can be steered by prompt injection hidden in a web page or document it read. "The model asked for it" is not authorization.

  • Validate every argument against the schema before running anything.
  • Sandbox and allow-list. The calculator in the demo only accepts digits and operators — it parses arithmetic, it never eval()s arbitrary code.
  • Least privilege. Give each tool the narrowest capability it needs.
  • Human approval for anything irreversible: payments, deletes, sends.

And when a tool throws, don't crash the loop — return the error as the tool result. The model reads it and usually fixes its own call next turn. Errors are just more feedback.

Why this matters

Strip an "AI agent" down to its engine and this is what's left: a model in a loop, calling tools and reading results until the goal is met. Add more tools, some planning, and memory across turns and you have an agent — but the primitive underneath is exactly this call → run → feed-back loop.

It's also structured output in disguise. The tool call is JSON constrained to match your schema, the same machinery behind "JSON mode." Reliable tool use and reliable structured output are the same trick: make the model emit a schema, not prose.

The interactive version runs a real calculator (23*47 = 1081, no eval) and a mocked weather tool, and shows the message array filling in one step at a time:

👉 https://dev48v.infy.uk/ai/days/day28-function-calling.html

Part of AIFromZero. 🌐 https://dev48v.infy.uk

Top comments (0)