Last month our office line dropped for a full morning. VPN went first, then Teams, then every chat app started spinning its little circle. The usual meltdown. And somewhere in the middle of it, someone asked me to pull up a summary of a document we'd batted around the week before.
I opened Open WebUI, asked the model, and it answered. No circle, no “connecting”, no error toast. It just worked, because the whole thing lives on a machine under my desk, not a server a thousand miles away.
That's the moment the point of this setup stopped being abstract for me.
I run a small container stack at home and at work — LM Studio for the models, Open WebUI as the front end, a bit of Docker glue to keep it tidy. Quantized models, because a 13B at 4-bit does the job for most of what I throw at it and I don't need a GPU that costs more than my car. It's not breaking any benchmark records. It also doesn't care if the ISP is having a bad day.
I got into this for privacy. I didn't like pasting company documents into a free chatbot that slurps everything and maybe one day ships it somewhere I can't see. That's still the main reason it exists. But the resilience is the part I didn't expect to value as much.
Here's the thing about renting an AI from someone else: your access to it is a low-priority SLA like everything else you subscribe to. When the network dies, your AI dies with it. When the provider ships a model you don't like, that's the one you get. When they quietly change their terms, your data is their data now. All of it happens out of your control, and you'd have to read a changelog to even know.
Self-hosting flips that. The failure modes aren't eliminated, but they're now local and mine. Disk fills up, I see it. Model runs slow, I swap it. A mid-morning outage is just an afternoon of reading rather than a reminder that I pay rent on a service that's down.
I'm not saying everyone needs this. If you're a hobbyist doing a one-off, renting is fine and cheap. But I do this for a living, and enough of my day touches AI now that I'd rather it live where the rest of my tools live: on hardware I can reach.
I put together the exact stack I run as a self-contained bundle a while back — configs, model suggestions, the Docker wiring, the bits I had to figure out the hard way. It's pay-what-you-want at https://symshah.gumroad.com/l/selfhosted-ai-homelab if you'd rather copy someone's homework than rebuild it from scratch.
The next time your line drops, your AI doesn't have to drop with it.
Top comments (0)