Every thread about self-hosting AI collapses into the same argument within ten replies: "you can't self-host a frontier model on a $6 VPS." That's true. It's also answering a question nobody useful is asking.
I build a self-hosted personal assistant commercially, so treat this as a field report from an interested party rather than neutral analysis. But the terminology confusion here costs people real decisions, and it's worth untangling.
Two different things wear the same word
Self-hosting the model means the weights run on hardware you control. Real, valuable, and expensive: you need a GPU, and what you get is a model measurably weaker than what you're used to. For most people this trade fails on day one, they go back to the hosted product, and conclude self-hosting AI is a fantasy.
Self-hosting everything around the model is a different proposition entirely. The model stays an API call. What moves onto your machine is:
- the memory of who you are and what you've discussed
- the full conversation archive, searchable
- scheduled jobs and automation
- skills, integrations, credentials
- the orchestration logic that decides what context the model even sees
That runs comfortably on a cheap VPS, because none of it is inference.
Why the second one is the interesting one
Think about what actually accumulates value over years of using an assistant. It isn't the model — model weights get replaced every few months and you don't own them either way. It's the context: what it knows about you, what corrections you've made, what it has concluded about how you work.
When that lives inside a vendor product, three things follow:
- Its lifetime is a business decision, not an engineering one. Retention windows change. Features get deprecated. Nobody consults you.
- Switching cost compounds. Every month you use it, leaving gets more expensive — which is the point, from the vendor's side.
- You can't inspect or restructure it. You get whatever memory abstraction they ship.
Move that layer onto your own box and those three invert. The schema is yours, the backups are yours, the migrations are your problem — which is a real operational cost, not a rhetorical one.
The honest accounting
Being precise, because vague claims here are how people get burned:
What never leaves your machine: stored memory, conversation history, scheduling state, skill configuration, credentials for the services you connect.
What does leave: the current prompt, going to the model provider under their terms.
So if your threat model is "the model provider must not see my text," this does not solve it, and no amount of self-hosting around the edges will. If your threat model is "my accumulated context must not become someone else's asset, and must survive their product decisions" — that's exactly what it solves.
What it costs in practice
A VPS with at least 4 GB of RAM handles it — roughly $5–15/month depending on provider and billing period. Below 4 GB the install starts fighting for memory, so that is the real floor, not the price. The model subscription is separate and is the dominant cost. Setup is an install script and roughly half an hour if nothing goes sideways; when it does go sideways it's usually DNS or a firewall rule, same as any other self-hosted service.
Worth being blunt: if you use AI a few times a week, this is not worth the operational overhead. We wrote that case ourselves — the arithmetic is in is self hosting AI worth it, including where the honest answer is "just use the hosted product."
The broader tradeoff list, including what's overrated about self-hosting, is in self-hosted AI benefits.
The point
"Self-hosted AI" being read as "local weights" quietly kills the option that's actually practical for most people. You can have data sovereignty over the part that matters — everything that accumulates — while still using a frontier model through an API.
Those are separable concerns. Treating them as one thing is how the conversation keeps going nowhere.
Curious how others draw this line: if you run any part of your AI stack yourself, which part did you decide to own, and what made that the cutoff?
Top comments (0)