I Stopped Asking One AI to Do Everything. Here's What Happened.

For months I had one AI agent. One model. One endpoint. I'd throw every question at it — code review, article drafts, debugging, random "what's the weather" questions — and expect it to handle it all.

It didn't. Not really. It answered everything, but it answered everything the same way. The same tone, the same depth, the same blind spots. When I needed a surgical code review, I got a friendly chat response. When I needed a casual summary, I got a three-paragraph essay.

So I split it into three specialized agents. Not because it's trendy. Because general-purpose AI is a compromise, and I was tired of compromising.

The Problem: One Brain, Too Many Jobs

I run a home lab on a Mac Mini M4, a Windows PC with an RTX 3060, and an Ubuntu box. All three have Ollama installed. For the first six months, I just used the biggest model I could fit and routed everything to it.

That model was Qwen 3 Coder 30B — a coding specialist. Great for refactoring. Great for debugging. But when I asked it to "write a friendly summary of my day," it would respond with something that sounded like a technical spec document. Because that's what it was trained to do.

The reverse was just as bad. When I used a general chat model for coding questions, it would hallucinate APIs, miss edge cases, and write code that looked right but subtly violated conventions.

I was asking a brain surgeon to do therapy, and a therapist to do surgery.

The Split: Three Agents, Three Jobs

I didn't build a microservices architecture. I didn't install Kubernetes. I added two more Ollama endpoints and a 20-line router.

Here's what I ended up with:

Celebi (Mac Mini, Qwen 3.5 9B) — The generalist. Handles routing, scheduling, daily summaries, weather, quick questions. Response time: 1-2 seconds. Perfect for "what's on my calendar today" and "summarize these emails."

ProgrammierMinna (Windows PC, Qwen 3 Coder 30B) — The coder. Handles code generation, refactoring, debugging, PR review. Response time: 8-15 seconds. When I need a function written or a bug found, this is where the query goes.

DocMinna (Mac Mini, Granite 3.2 8B) — The writer. Handles documentation, article drafts, READMEs, technical specs. Response time: 3-5 seconds. This model is worse at coding but surprisingly good at structure and flow.

The router is embarrassingly simple:

def route_query(query: str) -> str:
    coding_keywords = ["code", "function", "bug", "refactor", "debug", "pr", "review"]
    writing_keywords = ["write", "draft", "article", "readme", "doc", "summary", "blog"]

    if any(k in query.lower() for k in coding_keywords):
        return "http://192.168.1.106:11434"  # ProgrammierMinna
    elif any(k in query.lower() for k in writing_keywords):
        return "http://192.168.1.102:11434"  # DocMinna
    else:
        return "http://192.168.1.102:11434"  # Celebi

Is it perfect? No. "Write a Python script that sends emails" gets routed to DocMinna because of "write," when it probably should go to ProgrammierMinna. I fix those manually when I catch them. But it works 90% of the time, and that's enough.

What Actually Changed

Quality went up immediately

Before the split, I'd ask for a code review and get generic advice like "consider adding error handling." After the split, ProgrammierMinna would say "this async function doesn't handle TimeoutError — add a try/except around line 47, and use asyncio.wait_for with a 5-second timeout."

Same question. Different model. Different depth.

I stopped over-explaining

With the generalist, I'd have to add context like "please be thorough, this is for production code" or "keep it casual, this is a blog post." The specialists already know their role. I don't need to prompt-engineer the tone. It's baked into the model choice.

Parallel processing became possible

I can now fire off a coding task to ProgrammierMinna and a writing task to DocMinna simultaneously. They're running on different machines with different GPUs. No queue. No waiting for one to finish before the other starts.

Fallbacks got simpler

When the Windows PC is offline ( asleep, rented out on Vast.ai, or I'm traveling), queries that would normally go to ProgrammierMinna fall back to Celebi with a note: "PC offline — answering with generalist model, quality may vary." The system degrades gracefully instead of just failing.

The Honest Downsides

Three models to manage. Updates, storage, keeping track of which version is on which machine — it's overhead. Each model is 4-20GB. My model folder went from 40GB to 120GB across three machines.

Routing mistakes happen. I mentioned the "write a Python script" example. There are others. "Debug this article" (writing + debugging) confuses the router. I hit maybe 5% mis-routes, and I notice them because the response feels slightly off.

More endpoints to monitor. Instead of one Ollama instance to check, I have three. I built a simple health check script that pings each one and sends me a Telegram alert if any are down. It took 30 minutes to build. It runs forever.

Context doesn't transfer. If I'm in a long coding session with ProgrammierMinna and then ask Celebi "what did we just decide about the database schema?" — Celebi has no idea. Each agent has its own conversation history. I work around this by copying relevant context when I switch, but it's friction.

The Numbers

Metric	One Generalist	Three Specialists
Avg response time (coding)	8-15s	8-15s (same model)
Avg response time (writing)	8-15s	3-5s (smaller, faster model)
Avg response time (general)	8-15s	1-2s (tiny model)
Code review quality	6/10	9/10
Draft writing quality	5/10	8/10
Daily summary quality	7/10	8/10
Monthly cost	$0 (local)	$0 (local)
Setup complexity	Low	Medium
Maintenance overhead	Low	Medium

The quality jumps are subjective but real. I measured them by how often I had to ask for a redo. With the generalist, maybe 30% of responses needed a follow-up clarification. With specialists, maybe 5%.

When This Makes Sense (And When It Doesn't)

Do this if:

• You have multiple distinct task types (coding + writing + analysis)
• You have the hardware to run multiple models (even small ones)
• You care about quality more than simplicity
• You're already hitting the limits of your current model

Don't do this if:

• You're just casually chatting with AI
• You only have one machine with limited RAM
• Your tasks are all similar (all coding, all writing)
• You value simplicity over marginal quality gains

The Setup in 30 Minutes

If you want to try this, here's the fastest path:

1. Install Ollama on two machines (or twice on one machine if you have the RAM)
2. Pull different models: a coder model on one, a general model on the other
3. Write a 10-line router (like the Python snippet above)
4. Point your scripts at the router instead of directly at a model
5. Adjust keywords as you find mis-routes

Total time: 30 minutes. Total cost: $0.

The Real Win

The biggest change isn't technical. It's mental.

Before, I had one AI "employee" who was mediocre at everything. Now I have three specialists who are genuinely good at their jobs. When I send a query, I know which expert is handling it. I trust the output more. I spend less time verifying and fixing.

It's the difference between a Swiss Army knife and a actual toolbox. The knife fits in your pocket. But when you need to build something real, you want the right tool.

Sam Hartley is a solo dev running a multi-agent AI setup on a 3-machine home lab. Writes about the infrastructure that makes local AI actually usable.

→ Custom automation setups on Fiverr
→ Follow CelebiBots on Telegram

ai #agents #automation #selfhosted #ollama #productivity #buildinginpublic