DEV Community

Cover image for AI Coding Agents: A Practical Guide to Claude Code with Ollama
Ismail Ajjal
Ismail Ajjal

Posted on

AI Coding Agents: A Practical Guide to Claude Code with Ollama

#ai #claude #llm #agents

Want to try an AI coding agent without sending your project to a hosted model? This guide connects Claude Code to a local Ollama model. Commands are provided for Windows CMD, PowerShell, Linux and macOS.

Our test setup: Windows, 8 GB RAM, Ollama, Claude Code and qwen2.5-coder:1.5b. Results and limitations from this setup are called out separately.

What you’ll do: install the tools → download a model → connect Claude Code → test a real file operation.

1. Install Ollama

Ollama runs the model locally. Use the command for your operating system, or use the official download page for a manual installer.

Windows — PowerShell (run PowerShell, not CMD):

irm https://ollama.com/install.ps1 | iex
Enter fullscreen mode Exit fullscreen mode

Windows — CMD / manual installation:

CMD does not run the PowerShell command above. Download and run the Windows installer from ollama.com/download, then reopen CMD.

macOS or Linux — Terminal:

curl -fsSL https://ollama.com/install.sh | sh
Enter fullscreen mode Exit fullscreen mode

Manual installers and Linux-specific instructions: Ollama downloads · Linux manual install.

2. Install Claude Code

Windows — PowerShell:

irm https://claude.ai/install.ps1 | iex
Enter fullscreen mode Exit fullscreen mode

macOS / Linux — Terminal:

curl -fsSL https://claude.ai/install.sh | bash
Enter fullscreen mode Exit fullscreen mode

Official instructions: Claude Code setup.

3. Verify the commands

Close and reopen the terminal, then run:

ollama --version
claude --version
Enter fullscreen mode Exit fullscreen mode

If a command is not found, reopen the terminal first. If it still fails, check that the installer completed and that the executable is on your PATH.

4. Download and test the local model

For this experiment, we used a small coding model to fit a computer with 8 GB RAM:

ollama pull qwen2.5:1.5b
Enter fullscreen mode Exit fullscreen mode

Run the model and ask a simple question to confirm that it responds, then type /bye to exit the model chat.

ollama run qwen2.5:1.5b
Enter fullscreen mode Exit fullscreen mode

Check installed models with:

ollama list
Enter fullscreen mode Exit fullscreen mode

The first download needs an internet connection. After the model is downloaded, inference runs on the local machine. Speed and quality depend on your hardware and the model.

5. Connect Claude Code to Ollama

There are two methods. Start with Method A; use Method B only if you need to configure the connection manually. Do not mix the two configurations unnecessarily.

Method A — Ollama launcher (recommended) 

ollama launch claude --model qwen2.5-coder:1.5b
Enter fullscreen mode Exit fullscreen mode

This uses Ollama’s Claude Code integration and selects the local model. If launch is not recognized, update Ollama using the current installer from ollama.com/download, then reopen the terminal.

Method B — Environment variables (manual setup)

Claude Code can be pointed at Ollama’s local API with environment variables. These variables are not required when Method A works. They are a way to configure the manual connection.

Windows CMD — current window only:

set ANTHROPIC_BASE_URL=http://localhost:11434
set ANTHROPIC_AUTH_TOKEN=ollama
set ANTHROPIC_API_KEY=
claude --model qwen2.5-coder:1.5b
Enter fullscreen mode Exit fullscreen mode

Windows PowerShell — current window only:

$env:ANTHROPIC_BASE_URL = "http://localhost:11434"
$env:ANTHROPIC_AUTH_TOKEN = "ollama"
$env:ANTHROPIC_API_KEY = ""
claude --model qwen2.5-coder:1.5b
Enter fullscreen mode Exit fullscreen mode

Linux / macOS — current shell only:

export ANTHROPIC_BASE_URL=http://localhost:11434
export ANTHROPIC_AUTH_TOKEN=ollama
export ANTHROPIC_API_KEY=""
claude --model qwen2.5-coder:1.5b
Enter fullscreen mode Exit fullscreen mode

These commands apply to the terminal session in which you run them. Close that session and the temporary variables disappear.

Do I need to set them every time?

  • With ollama launch claude: usually no manual variable setup is needed.
  • With set, $env: or export: yes, set them again in each new terminal session.
  • Persistent variables are possible, but are not needed for this tutorial. Avoid setx unless you deliberately want Windows variables to persist; it affects future sessions, not the current CMD window.

6. Test a real file operation

Create an empty test folder and enter it. For example, on Windows:

Launch the agent using Method A or Method B, then give it a specific task:

Create hello.py containing print("Hello from local Ollama"). Show me the file contents and explain how to run it. Do not modify any other files.

After the agent finishes, verify the result yourself.

7. Our results and limitations

On our Windows computer with 8 GB RAM, qwen2.5:1.5b was usable for basic chat but did not consistently behave as a reliable coding agent. Some tool calls were malformed, and the workflow was slow. More generally, tests with heavier models showed significant slowdowns on this machine.

This is a result from one setup, not a universal benchmark. A small model can answer prompts yet still struggle with an agent’s tools, multi-step tasks and system instructions. Do not claim reliable file creation unless you have verified it.

8. Local vs Cloud: same prompt, different results

A local model runs on your own computer, while a cloud model runs on remote infrastructure. The main difference is not only where the model runs, but also how well it can handle complex coding-agent tasks on your available hardware.

Feature Local model — qwen2.5:1.5b Cloud model — gemma:31b-cloud
Execution On your computer through Ollama On remote infrastructure
Local resources Uses your RAM and CPU/GPU Most inference resources are provided remotely
Internet Required for the initial download; usually not for local inference Generally required
Privacy Prompts can stay on your machine, depending on the tools and configuration Requests are sent to the provider
Performance on a low-resource PC Small models use less memory but may struggle with complex tasks Can be more practical for demanding tasks on a low-resource computer
Agent actions May explain what to do instead of completing the action May handle multi-step actions more successfully, depending on the setup

Same prompt, different behavior

We tested both configurations with the same request:

Create hello.py containing print("Hello from local Ollama"). Show me the file contents and explain how to run it. Do not modify any other files.

Local test — qwen2.5:1.5b

The model explained how to create the file manually, but did not create the requested file in our observed test. This shows an important limitation: generating a plausible explanation is not the same as successfully performing a file operation.

Cloud test — gemma:31b-cloud

In our separate test, the agent carried out the requested actions in the workspace. The resulting file and its contents could be checked directly.

What does this mean for an 8 GB RAM computer?

On our Windows computer with 8 GB RAM, the small local model was suitable for basic conversation, but it was not consistently reliable for agentic coding tasks. Larger local models were also slow in our tests.

A cloud model may be a better option when a task requires reliable tool use or several coordinated file operations, because the main inference workload runs remotely. However, this is not a universal rule: results depend on the model, its configuration, available tools, network conditions and the task itself.

Important: these observations come from our particular tests, not a controlled benchmark of every local and cloud model.

Conclusion

Ollama makes it possible to experiment with Claude Code using a local model, without a hosted model API for inference. The key lesson from our test is that local execution does not automatically mean fast or reliable agent behaviour. Start small, verify every change, and choose a model that your hardware can handle.

References

Top comments (0)