You can run a free AI agent on your own Windows PC with Hermes Agent, Ollama and Qwen 3.8. It needs no API key and no subscription, and your files never leave your machine. This guide walks through the full Hermes Agent setup in 5 steps, then tests it on 3 real tasks.
What you will do:
- Install Ollama on Windows
- Download Qwen 3.8
- Set the context length to 64K (the step most people miss)
- Install Hermes Desktop
- Connect Hermes to Ollama ## 🎥 Watch the Video
👉 Watch on YouTube: I Built a FREE AI Agent on My Windows PC
🧩 How Hermes, Ollama and Qwen Work Together
A local AI agent has three parts: an agent that does the work, a model that thinks, and an engine that runs the model.
| Part | Role | What it does |
|---|---|---|
| Hermes Agent | The worker | Decides what to do and uses tools: reading files, running commands, memory, skills |
| Qwen 3.8 | The brain | The AI model that does the thinking |
| Ollama | The engine | Runs the model on your own PC |
Hermes Desktop is the app window for Hermes, so you do not need the terminal for day-to-day use. Hermes is open source and built by Nous Research.
🖥️ Hardware Requirements for Qwen 3.8
Qwen 3.8 27B runs smoothly on a GPU with about 24 GB of VRAM. On less, part of the model spills into system RAM and replies slow down. The 4-bit Ollama download is about 17 GB, and Unsloth lists 17–19 GB VRAM for 4-bit quants.
- GPU: 24 GB VRAM recommended (for example RTX 4090)
- RAM: 32 GB minimum
- Disk: about 20 GB free
- OS: Windows 10 or 11 My test machine: RTX 4070 (12 GB VRAM), 32 GB RAM, 1 TB SSD, Windows 11. That is below the recommendation, so my timings show the slow end.
Which model for your GPU
| Your GPU VRAM | Model to pull | Download size |
|---|---|---|
| 24 GB | qwen3.8:27b |
~17 GB |
| 12–16 GB | qwen3.5:9b |
~6.6 GB |
| 8 GB | qwen3.5:4b |
~3.4 GB |
Every step below stays the same; only the model name changes.
🦙 Step 1: Install Ollama on Windows
Download the Windows installer from ollama.com and run it. If Ollama is already installed, update it: Qwen 3.8 needs a recent Ollama release. Check your version in PowerShell:
ollama --version
🧠 Step 2: Download Qwen 3.8
Pull the model in PowerShell. It is about 17 GB, so the time depends on your internet speed.
ollama pull qwen3.8:27b
Why not regular Qwen 3?
Hermes needs at least 64,000 tokens of context and refuses to start below that. On Ollama, qwen3:8b, qwen3:14b and qwen3:32b are listed with a 40K context window, so Hermes rejects them. Qwen 3.8 supports 256K, so it clears the limit easily.
⚠️ Step 3: Set Ollama Context Length to 64K
This is the step that breaks Hermes for most people. Ollama sets its default context by VRAM: 4K under 24 GB, 32K from 24–48 GB, and 256K above that. Hermes needs 64K, because every turn it sends the model its tools, skills and memory. One guide puts that at around 13,000 tokens per turn before your question.
On most Indian gaming GPUs you get 4K by default. Raise it one of two ways:
- Ollama app: Settings → Context length slider → 64k
-
Environment variable: set
OLLAMA_CONTEXT_LENGTHto64000, then restart Ollama Verify it worked. Load the model, then run:
ollama ps
The CONTEXT column should read around 64000, not 4096. The PROCESSOR column shows whether the model sits fully on your GPU or is split with the CPU.
New to context size? I explain it in What Is Context Size in Ollama?
🪽 Step 4: Install Hermes Desktop on Windows
Hermes Desktop installs the app and the Hermes agent together. Get it from the Hermes desktop page.
- Windows 11: install from the Microsoft Store or the downloaded installer
- Windows 10: use the PowerShell installer, then open the app
iex (irm https://hermes-agent.nousresearch.com/install.ps1)
Ollama also documents a one-command route, ollama launch hermes-desktop, in its Hermes Desktop integration guide. For Windows, it calls the desktop installer the recommended path.
🔌 Step 5: Connect Hermes Agent to Ollama
When Hermes asks for a model provider, choose the custom, OpenAI-compatible endpoint option. The label can differ slightly between versions. Enter these values:
| Setting | Value |
|---|---|
| Base URL | http://127.0.0.1:11434/v1 |
| API key | leave empty |
| Model | qwen3.8:27b |
| Context length | 64000 |
Save, send a quick "hi", and check that Hermes replies. If it cannot connect, confirm the Ollama app is running and the URL ends in /v1.
🛠️ 3 Real Things a Local AI Agent Can Do
I tested Hermes on three everyday tasks. Copy the prompts and change the paths to your own folders.
1. Find large files eating your disk space
Find the 10 largest files in C:\Users\YourName\Downloads, list them with their sizes, and tell me which ones look safe to delete. Do not delete anything.
The last line matters. An agent can run commands on your PC, so always say what it is not allowed to do.
2. Summarise a PDF privately
Read the PDF in this folder and give me a five point summary in simple English.
This is where local AI shines. Bank statements, offer letters and client documents never leave your PC.
3. Generate a README for a project
Look at the project in C:\Users\YourName\Projects\my-app. Read the files to understand what it does, then create a README.md in the same folder with: project name and description, features, tech stack, prerequisites, how to install, how to run, and folder structure. Only create README.md. Do not change or delete any other file.
This is a true multi-step agent task: Hermes reads several files, works out the project, then writes a new file.
⚖️ Local AI vs Cloud AI: Honest Verdict
On a 12 GB GPU, Qwen 3.8 works for tasks you can wait on, not quick back-and-forth chat. The 27B model does not fit fully on my RTX 4070, so part of it runs on the CPU. In an earlier test on this machine, the same model took 34 minutes to build a snake game in Claude Code.
| Local (Hermes + Ollama) | Cloud API | |
|---|---|---|
| Cost | Free after hardware | Pay per use or subscription |
| Privacy | Files stay on your PC | Data goes to the provider |
| Speed | Depends on your GPU | Fast |
| Long multi-step tasks | Weaker on small GPUs | Stronger |
With a 24 GB GPU, Qwen 3.8 is a strong pick. With less, use the smaller Qwen 3.5 models for faster replies.
🧯 Hermes Agent + Ollama: Common Errors and Fixes
| Error | Cause | Fix |
|---|---|---|
| Context window below the minimum required by Hermes | Ollama still on 4K, or a 40K model like qwen3:8b | Redo Step 3, check ollama ps, use Qwen 3.8 or 3.5 |
ollama command not found |
PowerShell opened before install | Restart PowerShell |
| Very slow replies | Model larger than VRAM, split across CPU and GPU | Check PROCESSOR in ollama ps; use a smaller model |
| Hermes cannot connect | Ollama not running or wrong URL | Start the Ollama app; URL must end in /v1
|
| Model forgets your request | Context too small, prompt truncated | Set context to 64K and restart Ollama |
❓ FAQ
Is Hermes Agent free?
Yes. Hermes Agent is open source from Nous Research. With a local model through Ollama, there are no API costs.
Can I run Hermes Agent on Windows?
Yes. Hermes Desktop installs natively on Windows 11 and Windows 10. The command-line route through ollama launch hermes needs WSL2 on Windows.
What is the minimum context length for Hermes Agent?
64,000 tokens. Below that, Hermes refuses to start.
Which Qwen model is best for Hermes Agent?
Qwen 3.8 27B on a 24 GB GPU. On 12–16 GB use qwen3.5:9b, and on 8 GB use qwen3.5:4b.
Does it work offline?
The model runs entirely on your PC, so your prompts and files are not sent to a cloud AI.
Takeaways
Hermes Desktop, Ollama and Qwen 3.8 give you a real AI agent on your own Windows PC for ₹0. The one setting to remember is context length: set it to 64K, or Hermes will not work.
What would you automate with a free local AI agent? Tell me in the comments on the video. Thanks for reading, and see you in the next one.
Sources
- Ollama docs: Context length
- Ollama docs: Hermes Desktop
- Ollama library: qwen3
- Unsloth: Qwen3.8
- Capital & Compute: Best local models for Hermes Agent
- LumaDock: Hermes Ollama context fix
Originally published on The Code Street. I make practical AI, local LLM and AI agent tutorials on YouTube. Follow me here on dev.to for more.
Top comments (0)