baboon

Posted on Jan 22

With OllaMan, Even Beginners Can Run LLMs

#mcp #ai #llm #vibecoding

A beginner-friendly guide to running AI models on your own computer. Get from zero to chatting with a......

You've probably heard of ChatGPT, Claude, and Gemini. They're powerful, but they all run in the cloud — meaning your conversations travel through someone else's servers.

What if you could run AI models entirely on your own machine? Local LLMs make this possible: complete privacy, no internet required, and zero API costs.

The catch? Setting up local models usually involves command lines, environment variables, and technical know-how that scares off most people.

That's where OllaMan comes in.

Let's clear up these two terms first:

Ollama: The Engine

Ollama is the open-source project that actually runs AI models on your computer. It supports all the popular open-source models:

Llama 3 — Meta's flagship open model
Mistral — The lightweight European alternative
DeepSeek — Exceptional reasoning capabilities
Gemma — Google's efficient open model

Ollama is fantastic, but it only offers a command-line interface. Great for developers, intimidating for everyone else.

OllaMan: The Dashboard

Think of Ollama as the kitchen, and OllaMan as the restaurant's beautiful front-of-house.

OllaMan is a desktop app that wraps Ollama in a modern graphical interface. With it, you can:

👀 Browse all your installed models at a glance
🖱️ Download new models with a single click
💬 Chat with models like you would with ChatGPT
🎨 Enjoy polished dark and light themes

Ollama runs the models. OllaMan makes it delightful.

Step 1: Install Ollama

First, get Ollama running on your machine:

Visit ollama.ai
Download the installer for your OS (macOS / Windows / Linux)
Run the installer — it's a standard"Next, Next, Finish" setup

Once installed, Ollama runs silently in the background.

💡 Note: Don't expect a window to pop up — Ollama runs as a background service. That's normal.

Step 2: Install OllaMan

Next, grab OllaMan:

Head to ollaman.com
Download the app for your platform
Install and launch OllaMan

OllaMan automatically detects your local Ollama service. If everything's working, you'll land on the dashboard.

Download a Model

A fresh Ollama installation has no models yet. Let's fix that:

Click "Discover" in the left sidebar
Browse the model library — you'll see dozens of options
Pick something like Llama 3 or Mistral
Click into the model details page
Choose a size (we recommend 7B or 8B for beginners — lower hardware requirements)
Hit the "Pull" button to start downloading

While downloading, you can:

Watch real-time progress on the Downloads page
See download speed and completion percentage
Queue up multiple models simultaneously

⏰ Download time: Depends on your internet speed and model size. A 4GB model takes roughly 5 minutes on a 100Mbps connection.

Start Chatting

Once downloaded, getting to your first conversation is straightforward:

Click "Chat" in the sidebar
Select your newly downloaded model from the top bar
Type a message and hit Enter

That's it. You're now chatting with a local AI.

Create an Agent for Repeated Tasks

After using OllaMan for a while, you might notice you're typing the same instructions repeatedly:"Act as a coding assistant"or"Always respond in a friendly tone."

Agents solve this. An Agent is a pre-configured AI persona with:

A system prompt (the AI's role)
A default model
Custom generation parameters

To create one:

Go to the Chat page
Click the current Agent card in the left sidebar
Click the "+" button
Set a name, icon, and system prompt
Save

Here are some Agent ideas:

Agent Name	Use Case	System Prompt Snippet
Code Buddy	Programming help	"You're a patient coding mentor who explains concepts clearly..."
Writing Coach	Content creation	"You're a creative writing assistant who helps brainstorm and polish text..."
Study Helper	Learning	"You're a friendly tutor who breaks down complex topics into simple terms..."

Once created, switching Agents instantly changes your AI's personality and defaults.

Tip 1: Attach Files Instead of Pasting

Need the AI to analyze code or a document? Skip the copy-paste.

Click the 📎 attachment button in the input area and select files directly. OllaMan supports:

Code files: .py, .js, .ts, .java, and more
Documents: .txt, .md, .json
Images (with vision models): .png, .jpg

Tip 2: Enable Thinking Mode

Some models (like DeepSeek R1 or QwQ) support "thinking mode" — they'll show their reasoning process before giving an answer.

If your model supports this, you'll see a "Think" toggle near the input. When enabled:

Responses split into "thinking" and "answer" sections
The thinking section is collapsible
Great for complex reasoning tasks

Tip 3: Tune Generation Parameters

The settings panel on the right side of each chat lets you adjust:

Parameter	What It Does	Recommendations
Temperature	Controls creativity	Code/factual: 0.1-0.3 Creative writing: 0.8-1.2
Top P	Sampling range	Usually keep at 0.9
Top K	Candidate token count	Usually keep at 40