DEV Community

Chaitasi Patel
Chaitasi Patel

Posted on

From Local LLM to AI Agent: Building Your First Tool-Using Agent with Python and Ollama

You don't need a cloud API or a complex agent framework to start experimenting with AI agents.

I wanted to understand the fundamentals of tool-using agents by building a small project that runs entirely on my local machine.

In this walkthrough, I'll show how I connected Qwen3 8B, Ollama, and Python to build a simple agent that can use a calculator tool to solve a math problem.

The goal is to understand the core mechanics—not build another framework.

Explore the complete project and source code on GitHub →

What we're building

A small Python agent that:

  • Runs Qwen3 8B locally using Ollama.
  • Exposes a Python calculator function as a tool.
  • Lets the model request the tool when needed.
  • Returns the calculation result to the model.
  • Generates a final response.

For example:

Calculate (125 * 8) + 37

The expected result is 1037.

How it works

The architecture is intentionally simple:

       User Task
           |
           v
    Local Qwen3 Model
           |
           v
   Requests Calculator
           |
           v
    Python Tool Executes
           |
           v
     Tool Result
           |
           v
    Model Final Response
Enter fullscreen mode Exit fullscreen mode

The important distinction is that the model decides when to request a tool, but the Python application executes it.

This separation is one of the fundamental ideas behind tool-using agents.

A few engineering decisions

I kept the implementation small so I could focus on three things:

1. Local inference

Ollama runs Qwen3 8B locally, so this configuration doesn't require a separate hosted AI API key.

2. Controlled tool execution

The calculator supports basic arithmetic. Instead of using Python's unrestricted eval(), it parses expressions with the AST module and evaluates only explicitly supported operations.

3. Bounded agent loop

The agent has a maximum execution limit of five steps. It handles tool requests, returns results to the model, and stops when the model produces a final response or reaches the limit.

These are basic safeguards, but they help make the execution flow easier to understand and control.

Try it yourself

You'll need Python 3.10+, Ollama, and enough available memory to run the model.

Download Qwen3:

ollama pull qwen3:8b
Enter fullscreen mode Exit fullscreen mode

Clone the project:

git clone https://github.com/chaitasipatel25/local-ai-agent.git
cd local-ai-agent
Enter fullscreen mode Exit fullscreen mode

Install the Python dependency and run:

python -m pip install ollama
python agent.py
Enter fullscreen mode Exit fullscreen mode

The complete setup instructions, implementation, and calculator tests are in the repository:

View local-ai-agent on GitHub →

What I'd explore next

This is an educational starting point, not a general-purpose autonomous assistant. The task is currently predefined and the agent has only one tool.

Some natural next steps:

  • Accept dynamic user prompts.
  • Add more tools, such as unit conversion.
  • Improve structured logging and error handling.
  • Explore multi-step tasks and conversation history.
  • Add approval gates before executing actions that modify files or system state.

Final thoughts

Building this small project will help better understand the interaction between a language model, an application, and external tools.

For anyone getting started with AI agents, I'd recommend building something small and running it locally before reaching for a framework.

Start with one model, one tool, and one execution loop. Understand how they work together, then build from there.

If you try the project or extend it with another tool, I'd love to hear what you build.


Explore more of my work:

Top comments (0)