DEV Community

Cover image for AI Coding Agents: A Practical Guide to Claude Code with Ollama
Ismail Ajjal
Ismail Ajjal

Posted on Edited on

AI Coding Agents: A Practical Guide to Claude Code with Ollama

#ai #claude #agents #llm

Want to try an AI coding agent without sending your project to a hosted model? This guide connects Claude Code to a local Ollama model. Commands are provided for Windows CMD, PowerShell, Linux and macOS.

Our test setup: Windows, 8 GB RAM, Ollama, Claude Code and qwen2.5:1.5b. Results and limitations from this setup are called out separately.

What you’ll do: install the tools → download a model → connect Claude Code → test a real file operation.

1. Install Ollama

Ollama runs the model locally. Use the command for your operating system, or use the official download page for a manual installer.

Windows — PowerShell:

irm https://ollama.com/install.ps1 | iex
Enter fullscreen mode Exit fullscreen mode

Manual installation:
Download and run the Windows installer from ollama.com/download, then reopen CMD.

macOS or Linux — Terminal:

curl -fsSL https://ollama.com/install.sh | sh
Enter fullscreen mode Exit fullscreen mode

Manual installers and Linux-specific instructions: Ollama downloads · Linux manual install.

2. Install Claude Code

Windows — PowerShell:

irm https://claude.ai/install.ps1 | iex
Enter fullscreen mode Exit fullscreen mode

macOS / Linux — Terminal:

curl -fsSL https://claude.ai/install.sh | bash
Enter fullscreen mode Exit fullscreen mode

Official instructions: Claude Code setup.

3. Verify the commands

Close and reopen the terminal, then run:

ollama --version
claude --version
Enter fullscreen mode Exit fullscreen mode

4. Download and test the local model 

ollama pull qwen2.5:1.5b
Enter fullscreen mode Exit fullscreen mode

Run the model and ask a simple question to confirm that it responds, then type /bye to exit the model chat.

ollama run qwen2.5:1.5b
Enter fullscreen mode Exit fullscreen mode

Check installed models with:

ollama list
Enter fullscreen mode Exit fullscreen mode

5. Connect Claude Code to Ollama

There are two methods. Start with Method A; use Method B only if you need to configure the connection manually. Do not mix the two configurations unnecessarily.

Method A — Ollama launcher (recommended) 

ollama launch claude --model qwen2.5:1.5b
Enter fullscreen mode Exit fullscreen mode

This uses Ollama’s Claude Code integration and selects the local model. If launch is not recognized, update Ollama using the current installer from ollama.com/download, then reopen the terminal.

Method B — Environment variables (manual setup)

Claude Code can be pointed at Ollama’s local API with environment variables. These variables are not required when Method A works. They are a way to configure the manual connection.

Windows CMD — current window only:

set ANTHROPIC_BASE_URL=http://localhost:11434
set ANTHROPIC_AUTH_TOKEN=ollama
set ANTHROPIC_API_KEY=
claude --model qwen2.5:1.5b
Enter fullscreen mode Exit fullscreen mode

Windows PowerShell — current window only:

$env:ANTHROPIC_BASE_URL = "http://localhost:11434"
$env:ANTHROPIC_AUTH_TOKEN = "ollama"
$env:ANTHROPIC_API_KEY = ""
claude --model qwen2.5:1.5b
Enter fullscreen mode Exit fullscreen mode

Linux / macOS — current shell only:

export ANTHROPIC_BASE_URL=http://localhost:11434
export ANTHROPIC_AUTH_TOKEN=ollama
export ANTHROPIC_API_KEY=""
claude --model qwen2.5:1.5b
Enter fullscreen mode Exit fullscreen mode

These commands apply to the terminal session in which you run them. Close that session and the temporary variables disappear.

6. Test a real file operation: Local vs Cloud

After connecting Claude Code to Ollama, we tested two configurations using the same prompt: one with a local model and one with a cloud model.

Test prompt:

Create hello.py containing print("Hello from local Ollama"). Show me the file contents and explain how to run it. Do not modify any other files.

Test A — Local model: qwen2.5:1.5b

We ran the task using the local model on our Windows computer with 8 GB RAM.

The model explained how to create the file manually, but it did not create the requested file in our observed test.

Local model test result

Test B — Cloud model: gemma:31b-cloud

We then repeated the same task using gemma:31b-cloud.

In this separate test, the agent carried out the requested actions in the workspace, and we could verify the resulting file and its contents.

Cloud model test result

After each test, verify the file directly in the workspace instead of relying only on the agent's final message.

7. Comparison and limitations

The two tests produced different results. The table below summarizes the main differences between the configurations and what we observed.

Feature Local — qwen2.5:1.5b Cloud — gemma:31b-cloud
Where it runs On our computer through Ollama On remote infrastructure
Local resources Uses local RAM and CPU/GPU Most inference resources are remote
Internet Needed for the initial download; usually not for local inference Generally required
Privacy Prompts can stay local, depending on the tools and configuration Requests are sent to the provider
Result in our file test Explained how to create the file, but did not create it Carried out the requested actions in our test
Main limitation Performance and reliability depend on the model and hardware Depends on the service, model, network and task

Conclusion

Ollama makes it possible to experiment with Claude Code using a local model, without relying on a hosted model API for local inference.

Our comparison highlights an important lesson: running a model locally does not automatically make an AI coding agent fast or reliable.

Start with a small task, verify every file operation, and choose a model that fits your hardware and requirements. Consider privacy, Internet access and performance before deciding between a local and a cloud configuration.

References

Top comments (1)

Collapse
 
marwane_chiahni_f603805de profile image
Marouane Chiahni •

Nice jobe 🔥🔥