How I Built a Tool-Calling AI Agent from Scratch (Python) — No Framework Required

The hottest topic in 2026 might be: "AI can already write code, but how do we make it actually do things?" For the past two months I've offloaded a few repetitive manual tasks on my team — checking the weather, querying inventory, sending notifications — to an agent. The biggest lesson: whether an agent can actually work isn't about how smart the model is, it's about how cleanly you hand it the "tools."

This post skips the theory and walks through a runnable example of how Function Calling (a.k.a. tool calling) actually works, plus the real pitfalls I hit. The code is copy-paste ready — once you run it, you'll understand the layer underneath every major agent framework (LangChain, AutoGen, OpenAI Agents SDK).

---

1. First, what is Function Calling?

One sentence: the model doesn't execute your function — it outputs a structured "call request," and your code does the actual execution.

A lot of first-timers assume "the model runs the function for me." It doesn't. The real flow is:

1. You tell the model "here are the tools you can use" — what each one looks like (name, parameters, description).
2. The user asks something. The model decides "this needs a tool" and returns a JSON block: {"name": "get_weather", "arguments": {"city": "Xi'an"}}
3. Your program receives that JSON and actually calls get_weather("Xi'an"), then gets the result.
4. You feed the result back into the conversation, and the model composes a natural-language answer based on it.

The model only decides which tool and what arguments. Execution always stays in your hands. Once you internalize this, every agent framework is just a wrapper around this loop.

---

2. Environment setup

You only need an OpenAI-compatible Python SDK:

pip install openai

The key parameter when initializing the client is base_url. Any endpoint that follows the OpenAI API spec can plug in here — a local gateway, a cloud inference service, or your team's existing unified access layer. Swap in your address:

from openai import OpenAI

client = OpenAI(
    api_key="sk-your-key",
    base_url="https://easy88ai.com/v1"  # replace with your OpenAI-compatible endpoint
)

Tip: Hardcoding your key like this is only for demo. In real projects use an env var os.getenv("OPENAI_API_KEY") and never commit keys to a repo.

---

3. Define a tool: give the model a "capability list"

Tools are described with JSON Schema. The model uses description to decide when to call a tool, so writing a clear description matters more than anything. Here's a weather-lookup tool:

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Query the current weather for a specified city. Use when the user asks about the weather, temperature, or rainfall of a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "City name, e.g.: Xi'an, Shanghai, Beijing"
                    }
                },
                "required": ["city"]
            }
        }
    }
]

The description says when to use it, not what the function does. Those are two different things — the model uses the former to make decisions and the latter to understand boundaries.

---

4. Full runnable code

This is a minimal closed loop: send a message → the model may return a tool call → you execute it → feed the result back → the model summarizes. Copy and run:

import json
from openai import OpenAI

client = OpenAI(
    api_key="sk-your-key",
    base_url="https://easy88ai.com/v1"
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Query the current weather for a specified city. Use when the user asks about the weather, temperature, or rainfall of a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string", "description": "City name, e.g.: Xi'an"}
                },
                "required": ["city"]
            }
        }
    }
]

# A mock weather function (swap in your real API call in production)
def get_weather(city: str) -> str:
    fake_db = {"Xi'an": "Sunny, 26°C, southeast wind 2级", "Shanghai": "Cloudy, 30°C, high humidity"}
    return fake_db.get(city, f"{city}: weather data unavailable")

messages = [{"role": "user", "content": "How's the weather in Xi'an today? Good for going out?"}]

# Round 1: model decides whether to call a tool
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages,
    tools=tools
)

choice = response.choices[0].message

# If the model wants to call a tool
if choice.tool_calls:
    # 1. Append the model's reply (with tool_calls) back — many people miss this step
    messages.append(choice)

    for call in choice.tool_calls:
        args = json.loads(call.function.arguments)
        result = get_weather(args["city"])

        # 2. Feed the tool result back as a tool message
        messages.append({
            "role": "tool",
            "tool_call_id": call.id,
            "content": result
        })

    # 3. Round 2: model generates the final answer based on the tool result
    final = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=messages
    )
    print(final.choices[0].message.content)
else:
    print(choice.content)

Output looks roughly like:

Xi'an is sunny today, 26°C with a light southeast wind — feels quite pleasant, good for going out.

See? The model never actually "checked" the weather. It just correctly decided to call get_weather('Xi'an'). The real lookup was done by your get_weather function.

---

5. The pitfalls I actually hit (the valuable part)

Running the example above is easy. Wiring it into real business logic is hard. Here are the ones that bit me:

Pitfall 1: Vague tool description → random calls. Early on I wrote the description as "get weather info," and the model tried to call the weather tool even when the user asked "where should I travel tomorrow?" Later I changed it to "Use when the user asks about the weather, temperature, or rainfall of a location," and false calls dropped sharply. The description is the model's decision boundary — spend 10 minutes polishing it.

Pitfall 2: Forgetting to echo back the assistant's tool_calls message. This is the #1 beginner error — tool_call_id mismatch. The message returned in round 1 carries tool_calls; you must append it to messages verbatim, then append the role: "tool" result. Skip any step and round 2 throws a 400.

Pitfall 3: No validation before executing. The model returns a JSON string — after parsing, always validate field types and required fields. I had one production incident: the model passed city as an array ["Xi'an", "Shanghai"], but my function only accepted a string and crashed. Now every tool entry gets a pydantic / hand-written validation layer first.

Pitfall 4: Tools need timeout and retry. Real tools sit behind external APIs — they hang, they're slow. The time I didn't set a timeout, a weather API blocked for 40 seconds and the whole agent froze. Now every tool call is wrapped with timeout + at most 2 retries; on failure it returns "tool temporarily unavailable" so the model degrades gracefully instead of the whole chain dying.

Pitfall 5: Multi-turn tool calls need a loop, not an if. The example above calls one tool. Real scenarios may chain several (check weather → check transit → check ticket price). The correct pattern wraps "model decides → execute → feed back" in a while loop that runs until the model stops returning tool_calls.

---

6. Advanced: organizing multiple tools

In production you'll have a dozen tools. Two lessons:

• Don't overload tools. Stuffing 20 tools at once drops decision quality. Load relevant tools dynamically per scenario (e.g., the "ordering" scenario only gets menu/payment tools).
• Use a strategy pattern to pick different models for different tasks. Cheap model for simple queries, big model for complex reasoning. Same idea as "multi-model routing" — but that's a topic for another post.

---

7. Three conclusions from actually building this

1. Function Calling is the foundation of an agent, not decoration. To make AI actually do work, cleanly describe your tools in JSON Schema first — bigger ROI than buying a more expensive model.
2. Execution power always stays on your side. The model only makes decisions; the real side effects (sending messages, mutating databases, calling APIs) must be guarded by your code with validation and timeouts. That's the production-ready baseline.
3. Hand-write the minimal loop before adopting a framework. LangChain / Agents SDK are just wrappers around that while loop above. Run it yourself once, then read the framework docs — comprehension speed is completely different.

The code in this post is the most bare-bones version. Once you internalize it, you'll read any agent framework's source and think "oh, so that's all it is."

---

I'm building easy88ai, a unified API gateway that routes GPT, Claude, Gemini and 200+ models through one OpenAI-compatible endpoint — which is what I use as the base_url in the examples above. Happy to swap notes on LLM tooling in the comments.