For months I was the guy chasing a single model to rule them all. Every weekend I'd pull down the newest quantized weights, run a few test prompts, mutter, and load the old one back. Meanwhile the thing I do for a living kept smacking me in the face: nobody in IT runs one tool for everything. You keep the right wrench for the right bolt. Cloud models trained that reflex out of me, at first.
You sign into the cloud one, you get the frontier model, and you never ask yourself again. Then I went fully self-hosted at home, and I realized I didn't have to pick one at all. Owning a model means you get to split what it does.
So I run three. Not more. Each is quantized small enough to sit on hardware I already own, and each one is bad at the thing the other two are good at. That's the job.
The chat one is the workhorse. It stays warm in Open WebUI all day and takes the boring questions — "write this email," "make this doc shorter," "what's wrong with this YAML." It doesn't need to be 70 billion to do that. My family uses the same one and doesn't know it's running in the living room; it's just fast and it doesn't ask about their stuff.
The coding one is picky. It gets the jobs that are half-written already and just need to be careful — keep the imports right, don't refactor the whole file while you're in there. Slower, but it argues less.
And the math one is the one I almost never admit to keeping. It's stubborn and wrong a surprising amount. But when it's right, it's right, and I got tired of a generalist confidently mangling my numbers. A specialist is wrong in a predictable lane; a generalist is wrong everywhere.
This reads like overkill, I know. But the whole point of self-hosting is that a second small model costs almost nothing. Some disk, a rounding error of power. What I got back was my attention. No more model-hop weekends, no more second-guessing whether the tool is the thing that's wrong. Small model, small job, move on.
What shifts when you stop renting the biggest brain is the question you're even asking. It stops being "is this as smart as the cloud's best?" and becomes "is this better than what I was doing last week?" And that one you can answer on a Tuesday with normal hardware in the corner of your desk, without sending a single token anywhere.
If you want that exact setup without wiring it from twenty GitHub tabs, I packaged what I settled on — LM Studio, Open WebUI, Docker, the quantized models, plus the monitoring and backups so you aren't running this in a vacuum. Pay what you want, $9.99 suggested, nothing leaves the building. https://symshah.gumroad.com/l/selfhosted-ai-homelab
Sit with the smaller models. You'll find the fast one does, honestly, most of it.
Top comments (0)