DEV Community

Deduped
Deduped

Posted on Originally published at dedupedai.com

How to run local LLMs with Ollama and plug them into your coding agent

#ai #llm #ollama #tutorial

Ollama runs open models such as Gemma, Qwen, DeepSeek, Kimi and gpt-oss on your own machine, with a CLI, a local REST API and official Python and JavaScript libraries. The newest trick is the one most people miss: ollama launch wires a coding agent like Claude Code, Codex or OpenCode straight to Ollama models.

This guide goes from install to a local chat, then to calling the model from code, then to a coding agent backed by Ollama.

Install 

# macOS and Linux
curl -fsSL https://ollama.com/install.sh | sh

# Windows (PowerShell)
irm https://ollama.com/install.ps1 | iex
Enter fullscreen mode Exit fullscreen mode

Prefer not to pipe a script into your shell? There are manual downloads for macOS (Ollama.dmg) and Windows (OllamaSetup.exe), manual Linux instructions at docs.ollama.com, and an official Docker image, ollama/ollama, on Docker Hub.

Run your first model 

ollama run gemma4
Enter fullscreen mode Exit fullscreen mode

The first run downloads the model, then you chat in the terminal. Download size and speed depend on the model and your hardware, so start small. The full list is at ollama.com/library.

Not sure where to start? Just run:

ollama
Enter fullscreen mode Exit fullscreen mode

It prompts you to run a model or connect Ollama to your existing agents and apps, such as Claude Code, OpenClaw, OpenCode, Codex and Copilot.

Launch a coding agent on Ollama 

ollama launch claude
Enter fullscreen mode Exit fullscreen mode

Pick a model when prompted, then work in your repo as usual. Supported coding integrations listed in the README are Claude Code, Codex, Copilot CLI, DeepSeek Harness, Droid and OpenCode. Each link has the specific setup for that agent.

Be realistic about quality: small local models are much weaker at agentic coding than frontier models. Local is great for privacy, offline work and cheap experiments; give the agent bigger models for real multi-file tasks if your hardware allows.

There is also ollama launch openclaw, which turns Ollama into a personal assistant across WhatsApp, Telegram, Slack, Discord and more.

Call the model from your own code

Ollama's REST API listens on localhost by default:

curl http://localhost:11434/api/chat -d '{
  "model": "gemma4",
  "messages": [{
    "role": "user",
    "content": "Why is the sky blue?"
  }],
  "stream": false
}'
Enter fullscreen mode Exit fullscreen mode

Python 

pip install ollama
Enter fullscreen mode Exit fullscreen mode 
from ollama import chat

response = chat(model='gemma4', messages=[
  {'role': 'user', 'content': 'Why is the sky blue?'},
])
print(response.message.content)
Enter fullscreen mode Exit fullscreen mode

JavaScript 

npm i ollama
Enter fullscreen mode Exit fullscreen mode 
import ollama from "ollama";

const response = await ollama.chat({
  model: "gemma4",
  messages: [{ role: "user", content: "Why is the sky blue?" }],
});
console.log(response.message.content);
Enter fullscreen mode Exit fullscreen mode

That is a full AI feature prototype with no API bill. When you need more, the API docs, CLI reference, Modelfile reference and importing models guide cover the rest.

Under the hood

Ollama lists llama.cpp as its supported backend. If you want lower-level control, such as an OpenAI-compatible server with your own flags, llama.cpp is the engine to look at directly.

Use cases with copy-paste prompts

Each page has who it is for, the steps, and a prompt you can paste into your coding agent.

Use case: Launch a coding agent backed by Ollama

Use case: Run an open model on your machine in one command

Use case: Call a local model from Python or JavaScript

Get one email every Monday with the week's hottest AI repos and exactly how to use them: subscribe free at dedupedai.com

Top comments (0)