What I Learned Building an Agentic AI System with Python and OpenAI Function Calling

Ever felt like your chatbot is smart, but just not useful enough? I’ve been there. You give it context, fine-tune its prompts, maybe even add a little function-calling magic—but at the end of the day, it still just spits out text. If you’ve ever wanted your AI to actually do things, like check the weather, send emails, or pull live data—autonomously—you know the pain. I spent weeks wrangling with OpenAI’s function calling and Python, stumbling over dead ends, and finally got an agentic system running that can use tools on its own. Here’s what I wish I’d known before I started.

What’s Agentic AI, and Why Should You Care?

When people say “agentic AI,” they’re talking about AI that doesn’t just answer questions, but can figure out what actions to take—like which APIs to call and in what order—based on your goals. Think of it as going from “Siri, what’s the weather in Paris?” to “Book me a flight to Paris if the weather is nice this weekend.”

Most LLMs out-of-the-box are basically fancy text predictors. Agentic workflows mean plugging the model into a tool-use loop, so it can decide, “Hmm, I need to check the weather and look up flights,” and then call your code to actually do those things.

The trick is connecting those dots—getting the model to select and call functions, then giving it the results and letting it plan the next step. That’s what OpenAI’s function calling API is really about, and it’s a game-changer if you set it up right.

How the Agentic Workflow Actually Works

Here’s the high-level loop I ended up with:

1. User sends a message (“Book me a table for two at an Italian restaurant tonight”)
2. LLM decides which functions to call (e.g., search restaurants, make reservation)
3. Your Python code runs those functions
4. Feed results back to the LLM (it might ask clarifying questions, or plan the next action)
5. Repeat until the task is complete

The magic is in step 2 and 4: turning the LLM’s “intent” into actual function calls, and then giving it the info it needs for the next step.

Setting Up OpenAI Function Calling in Python

Let’s see how to wire up a simple agent. Suppose you have two tools: one to get the weather, and one to recommend an activity.

import openai

openai.api_key = "sk-..."  # Replace with your OpenAI API key

# Define the functions your agent can use
function_definitions = [
    {
        "name": "get_weather",
        "description": "Get the weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string", "description": "City name"}
            },
            "required": ["city"]
        }
    },
    {
        "name": "suggest_activity",
        "description": "Suggest an outdoor activity based on the weather.",
        "parameters": {
            "type": "object",
            "properties": {
                "weather": {"type": "string", "description": "Weather condition"}
            },
            "required": ["weather"]
        }
    }
]

# Your function to call OpenAI with function definitions
def run_agent(message):
    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo-0125",  # Any OpenAI function-calling model
        messages=[{"role": "user", "content": message}],
        functions=function_definitions,
        function_call="auto"
    )
    return response

result = run_agent("What should I do in Berlin today?")
print(result)

What’s going on here? You’re telling the model “Here are the tools you can use—call them if you want.” The model will decide if it needs to call get_weather first, and how to call it (it’ll fill in the city for you). You’ll get back a function_call in the response, which you’ll need to actually execute.

Executing Functions and Feeding Results Back

Here’s where most people (me included) trip up: You can’t just stop at “the model wants to call get_weather.” You have to run your actual Python code, get the result, and then continue the conversation so the model can take the next step.

Here’s a minimal loop to wire it together:

import json

# Dummy implementations for demonstration
def get_weather(city):
    # In a real system, call a weather API here
    return f"sunny in {city}"

def suggest_activity(weather):
    if "sunny" in weather:
        return "How about a picnic in the park?"
    return "Maybe visit a museum?"

def run_tool(tool_name, args):
    if tool_name == "get_weather":
        return get_weather(**args)
    if tool_name == "suggest_activity":
        return suggest_activity(**args)
    return "Unknown tool"

def agent_loop(user_message):
    messages = [{"role": "user", "content": user_message}]
    while True:
        response = openai.ChatCompletion.create(
            model="gpt-3.5-turbo-0125",
            messages=messages,
            functions=function_definitions,
            function_call="auto"
        )
        msg = response['choices'][0]['message']

        # If the model wants to call a function
        if msg.get("function_call"):
            fn_name = msg["function_call"]["name"]
            args = json.loads(msg["function_call"]["arguments"])
            result = run_tool(fn_name, args)

            # Add the function call and result back to conversation
            messages.append({
                "role": "assistant",
                "content": None,
                "function_call": msg["function_call"]
            })
            messages.append({
                "role": "function",
                "name": fn_name,
                "content": result
            })
        else:
            # Model has a final answer
            return msg["content"]

print(agent_loop("What should I do in Berlin today?"))

Key things happening here:

• The loop keeps going until the model decides it’s done (i.e., no more function_call in the response).
• Every time the model wants to call a function, you execute it in Python, then add its result as a new message.
• This lets the agent chain multiple actions. For example, it can call get_weather to figure out the weather, then use that info to call suggest_activity, then finally summarize the plan for the user.

Why Not Just Do It All in Python?

I’ll be honest—when I first saw this, I thought: “Why not just write a regular program that calls the weather API and then suggests an activity?” You could, but here’s why I stuck with the agentic approach:

• Flexibility: The agent can handle more open-ended requests (“I want something fun to do if it’s not raining”) without you having to hardcode all possible flows.
• Extendability: Add new tools without rewriting your logic; just add another function definition.
• Handling ambiguity: The model can ask clarifying questions, rather than just failing or guessing.

That said, plain procedural code will always be faster and more predictable for very narrow tasks. Use agents when you want flexibility, not just for the sake of it.

Making It More Real: Connecting Real APIs

Here’s how I plugged in a real API. Let’s wire up OpenWeatherMap for the weather function:

import requests

def get_weather(city):
    api_key = "YOUR_OPENWEATHERMAP_KEY"
    url = f"http://api.openweathermap.org/data/2.5/weather?q={city}&appid={api_key}&units=metric"
    resp = requests.get(url)
    data = resp.json()
    # Extract a simple weather description
    desc = data["weather"][0]["description"]
    temp = data["main"]["temp"]
    return f"{desc}, {temp}°C in {city}"

# The rest of the agent_loop stays the same

A couple of gotchas:

• Always sanitize inputs before passing them to APIs (here, the model might hallucinate weird city names).
• Handle API failures gracefully, or your agent will break in weird ways.

Common Mistakes (I’ve Made Them All)

1. Forgetting to add function results as messages.
   
   If you don’t feed the function’s output back as a function message, the agent can’t “see” the result and will get stuck or repeat itself. I spent a weekend debugging an agent that kept calling the same function over and over.

2. Not validating arguments.
   
   OpenAI’s models can make up function arguments that don’t make sense. Always check and sanitize before hitting external APIs, or you’ll quickly burn through API credits (or worse, get banned).

3. Assuming the agent will always “do the right thing.”
   
   The LLM is smart, but not magical. Sometimes the agent will call functions in a weird order, or skip steps. You need to log and sanity-check outputs, especially as you add more tools.

Key Takeaways

• Agentic AI lets your LLM use real tools in sequence, autonomously—but only if you wire up the loop right.
• Always feed function results back to the model, or your agent will get stuck in a loop.
• Validate everything before executing real-world code (user inputs, model arguments, API outputs).
• The agentic approach is more flexible, but less predictable—log everything while you’re building.
• Don’t overcomplicate: for simple tasks, plain Python is faster and easier.

Wrapping Up

If you want your AI to actually do things, not just chat, agentic workflows are the way forward. It’s a bit of a mindset shift—and takes real engineering—but it's worth the effort. Give it a try, and save yourself the trial and error I went through.

---

If you found this helpful, check out more programming tutorials on our blog. We cover Python, JavaScript, Java, Data Science, and more.