DEV Community: Samadhi Tattoo

Self-hosted AI doesn't mean local weights — here's what actually stays on your box

Samadhi Tattoo — Sat, 08 Aug 2026 02:39:35 +0000

Every thread about self-hosting AI collapses into the same argument within ten replies: "you can't self-host a frontier model on a $6 VPS." That's true. It's also answering a question nobody useful is asking.

I build a self-hosted personal assistant commercially, so treat this as a field report from an interested party rather than neutral analysis. But the terminology confusion here costs people real decisions, and it's worth untangling.

Two different things wear the same word

Self-hosting the model means the weights run on hardware you control. Real, valuable, and expensive: you need a GPU, and what you get is a model measurably weaker than what you're used to. For most people this trade fails on day one, they go back to the hosted product, and conclude self-hosting AI is a fantasy.

Self-hosting everything around the model is a different proposition entirely. The model stays an API call. What moves onto your machine is:

the memory of who you are and what you've discussed
the full conversation archive, searchable
scheduled jobs and automation
skills, integrations, credentials
the orchestration logic that decides what context the model even sees

That runs comfortably on a cheap VPS, because none of it is inference.

Why the second one is the interesting one

Think about what actually accumulates value over years of using an assistant. It isn't the model — model weights get replaced every few months and you don't own them either way. It's the context: what it knows about you, what corrections you've made, what it has concluded about how you work.

When that lives inside a vendor product, three things follow:

Its lifetime is a business decision, not an engineering one. Retention windows change. Features get deprecated. Nobody consults you.
Switching cost compounds. Every month you use it, leaving gets more expensive — which is the point, from the vendor's side.
You can't inspect or restructure it. You get whatever memory abstraction they ship.

Move that layer onto your own box and those three invert. The schema is yours, the backups are yours, the migrations are your problem — which is a real operational cost, not a rhetorical one.

The honest accounting

Being precise, because vague claims here are how people get burned:

What never leaves your machine: stored memory, conversation history, scheduling state, skill configuration, credentials for the services you connect.

What does leave: the current prompt, going to the model provider under their terms.

So if your threat model is "the model provider must not see my text," this does not solve it, and no amount of self-hosting around the edges will. If your threat model is "my accumulated context must not become someone else's asset, and must survive their product decisions" — that's exactly what it solves.

What it costs in practice

A VPS with at least 4 GB of RAM handles it — roughly $5–15/month depending on provider and billing period. Below 4 GB the install starts fighting for memory, so that is the real floor, not the price. The model subscription is separate and is the dominant cost. Setup is an install script and roughly half an hour if nothing goes sideways; when it does go sideways it's usually DNS or a firewall rule, same as any other self-hosted service.

Worth being blunt: if you use AI a few times a week, this is not worth the operational overhead. We wrote that case ourselves — the arithmetic is in is self hosting AI worth it, including where the honest answer is "just use the hosted product."

The broader tradeoff list, including what's overrated about self-hosting, is in self-hosted AI benefits.

The point

"Self-hosted AI" being read as "local weights" quietly kills the option that's actually practical for most people. You can have data sovereignty over the part that matters — everything that accumulates — while still using a frontier model through an API.

Those are separable concerns. Treating them as one thing is how the conversation keeps going nowhere.

Curious how others draw this line: if you run any part of your AI stack yourself, which part did you decide to own, and what made that the cutoff?

What 6 months of running a personal AI assistant on a $6 VPS actually costs

Samadhi Tattoo — Fri, 07 Aug 2026 18:24:56 +0000

Every thread about self-hosted AI turns into a privacy argument within three comments. Almost nobody posts the boring part: the invoice.

I have been running a personal AI assistant on my own VPS since early 2026 — a Telegram-facing agent with persistent memory, cron jobs, and a handful of integrations. Here is the actual cost breakdown, the things that broke, and the honest cases where you should not do this at all.

The bill

Item	Monthly
VPS (2 vCPU / 4 GB / 60 GB NVMe)	$6
Domain (amortized)	~$1
Model access (Claude subscription, not per-token API)	$20–$100 depending on tier
Backups (object storage, ~2 GB)	~$0.10
Total	~$27–$107

The VPS is the part everyone fixates on and it is the cheapest line item by an order of magnitude. The model access dominates, and that is the first thing worth internalizing: self-hosting an AI assistant does not mean self-hosting a frontier model. You are self-hosting the agent — the memory, the orchestration, the integrations, the data. The weights can still live somewhere else.

That distinction is where most cost estimates on the internet go wrong. People price out an A100 and conclude self-hosting is for millionaires. You do not need one unless you specifically want local inference.

The line item that will actually bite you

Here is the mistake that costs real money, and it is not the VPS.

If you wire your agent to a per-token API key instead of a flat subscription, an agent with tool-use in a loop will happily burn through a month of budget in an afternoon. An agent is not a chatbot: one user message can fan out into dozens of model calls — reading files, calling tools, retrying, summarizing. I have watched a single misconfigured background job produce a three-figure bill in under a day.

The difference is structural, not a matter of being careful:

Chat UI: one message, one response, cost proportional to your typing.
Agent: one message, N tool-calls, cost proportional to task complexity — which you do not control at write time.

So the first architectural decision is billing mode, before you touch a line of code. Flat subscription with an agent harness on top is predictable. Per-token API with an autonomous loop is a metered faucet pointed at your wallet, and you find out at the end of the month.

The second one: put a hard concurrency cap and a timeout on every scheduled task. Not "be careful" — an actual enforced limit in code. Background crons are where runaway loops hide, because nobody is watching the chat window when they fire at 3 AM.

What you actually get for the money

After six months the things I would not give up:

Memory that persists across sessions. Not a rolling context window — an actual database of facts, decisions, and corrections that survives restarts. When I correct the assistant on something, it stays corrected next month. Cloud assistants have gotten better here, but you are still renting the memory, and you cannot inspect or repair it.

It runs when I am asleep. Cron jobs, monitoring, scheduled reports. A hosted chatbot is request/response by construction; an agent on your own box has a clock.

The data never leaves. Conversation history, files, credentials — all on disk I control, backed up where I choose. This is the part people argue about, so I will just say the practical version: it is less about paranoia and more about not having your working context deleted by someone else's policy change.

Debuggability. When it does something dumb, I can read the transcript, the tool calls, and the database rows. That is genuinely different from filing feedback into a black box.

When you should not do this

I would rather say this plainly than sell you something:

You want it to just work and never think about it again. A VPS is a machine you now own. Patches, disk space, backups, uptime. Budget an hour a month, sometimes more.
You need frontier-model quality with zero setup. The hosted products are genuinely excellent, and their onboarding is fifteen seconds.
Your use case is a single chat window. If you are not using memory, scheduling, or integrations, you are paying complexity tax for features you will not touch.

Self-hosting wins when your assistant needs continuity — memory, schedule, and access to your own systems. It loses when you just want a smart text box.

The short version

The $6 VPS is a rounding error. The real costs are your billing model and your attention. Pick a flat subscription, cap your background jobs, and the economics are boring in the best way — which is exactly what you want from infrastructure.

I wrote a longer breakdown of the tradeoffs, including the cases where the math does not work out, here: is self-hosting AI worth it. There is also a side-by-side of self-hosted AI vs cloud AI if you want the comparison table rather than the essay.

The assistant I run is avelina.ai — happy to answer setup questions in the comments if anyone is going down this road.

Self-hosted AI vs cloud AI: the honest scorecard I wish I'd had

Samadhi Tattoo — Sat, 25 Jul 2026 02:33:47 +0000

Most "which AI is better" arguments compare the wrong thing. They line up model versions and benchmark scores, as if the deciding factor were raw intelligence. For day-to-day personal use, it usually isn't. The same frontier model can sit behind a cloud chat window or behind your own server — the intelligence is identical. What actually changes is everything around the model: where your conversations are stored, who can read them, whether the assistant remembers you next week, and who gets to switch it off.

So here's the honest scorecard I wish I'd had before I moved my own assistant off the cloud.

Where your data lives

This is the whole game. With a cloud assistant, every message is processed and retained on someone else's infrastructure under their retention policy, their jurisdiction, and their terms — which can change. "We don't train on your data" is a promise about use, not location. The data is still there.

With a self-hosted setup, your history, memory, and files live on a server you rent and control. Model calls still go out to an API for the actual inference, but the record of your life doesn't sit in a vendor's account you can be locked out of.

Cost, honestly

Cloud wins on the low end. A $20/month subscription is cheaper and simpler than running your own box, and for casual use that's the right answer — don't let anyone shame you out of it.

Self-hosting costs a $5–$10/month VPS plus per-token API usage. It's not dramatically more, but it's more moving parts. You're paying with a little attention, not just money.

Memory

Cloud chat tools forget you by design between sessions, or bolt on a shallow "memory" feature. A self-hosted assistant can keep a real database — facts, past conversations, lessons — that persists and compounds. If you want an assistant that actually knows you over months, this is the single biggest difference.

Latency and model power

Cloud has a small edge on convenience and always runs the newest model with zero setup. Self-hosted, routed to the same API, is effectively the same speed and power. If you instead run a local open model for full offline privacy, you trade some capability for it. Pick your axis.

Platform risk

The quiet one. A cloud account can be suspended, rate-limited, deprecated, or repriced, and your history goes with it. A server you own keeps running until you stop it. For something you rely on daily, that ownership matters more than people expect.

The verdict

There's no universal winner, which is exactly why the self-hosted AI vs cloud AI question keeps coming back.

Stay cloud if you want zero maintenance, occasional use, and the lowest possible friction.
Go self-hosted if you want persistent memory, data you actually own, and an assistant no one else can pull the plug on.

I landed on the second camp and built my assistant to live on my own VPS — persistent memory, my data on my server, model calls out to the API and nothing else leaving the box. If you want the full side-by-side breakdown across cost, privacy, memory, latency, model power and platform risk, I wrote it up here: https://avelina.ai/blog/self-hosted-ai-vs-cloud-ai

The project is Avelina AI if you want to see what a fully self-hosted personal assistant looks like in practice.

Running Your Own AI Assistant on a $6 VPS: Data Sovereignty in Practice

Samadhi Tattoo — Fri, 17 Jul 2026 18:28:38 +0000

Cloud AI assistants are convenient, but every conversation you have with them lives on someone else's server. For developers who care about data sovereignty, self-hosting is the obvious answer.

The setup

A self-hosted personal AI assistant can run on a cheap VPS (even a $6/month box) and stay online 24/7. Mine lives in Telegram, keeps persistent memory of past conversations, and stores 100% of its data on my own server.

Why self-host an AI assistant?

Privacy: your chats never touch a third-party cloud
Data ownership: the database is yours, on your disk
Persistence: real long-term memory across sessions
Control: your keys, your models, your rules

What you actually get

Instead of renting access to a black box, you own the whole stack: a private ChatGPT alternative you fully control. No vendor lock-in, no training on your data, no surprise policy changes.

If you want to see a working implementation of a self-hosted AI assistant with persistent memory running on a VPS, check out avelina.ai.

Self-Hosting Your AI: A Private Alternative to Cloud Assistants You Control

Samadhi Tattoo — Fri, 17 Jul 2026 10:18:58 +0000

Most of us rent our AI. You type into a cloud assistant, it forgets you between chats, and your conversations live on a company's servers.

Avelina AI is a self-hosted personal AI assistant that runs on your own VPS. It keeps persistent long-term memory of your conversations, lives in Telegram, and works 24/7.

Why self-host your AI assistant?

Your own API keys, and 100% of your data stays on a server you control
Persistent long-term memory across sessions, so it remembers past conversations
A skills system, cron automations, and multi-agent delegation
A private, self-hosted alternative to cloud AI assistants that respects data sovereignty

If you would rather own your AI than rent it, you can deploy your own on a $6 VPS: avelina.ai

Owning your AI: why I self-host a personal assistant with long-term memory

Samadhi Tattoo — Fri, 17 Jul 2026 01:54:43 +0000

For the past few months I've been running my personal AI assistant on my own hardware instead of a third-party cloud. Here's why — and what I learned.

The problem with cloud assistants

Every message, every preference, every bit of context you share with a hosted assistant lives on someone else's servers. You don't control retention, you can't inspect the memory, and you can't move it. For a tool that's supposed to know you, that's a lot of trust to outsource.

What self-hosting changes

Memory stays local. Facts, preferences and conversation history live in a local SQLite database, with backups I control.
Runs on my own hardware. A cheap VPS or a mini-PC at home — no vendor lock-in.
The model is swappable. The assistant layer is decoupled from the underlying LLM, so I can change providers without losing my data.

The setup

Mine lives in Telegram, keeps long-term memory, and exposes skills (calendar, notes, browser automation, reminders). One example of this approach is avelina.ai — a self-hosted personal assistant with persistent memory, designed for people who want to own their data rather than rent it.

Takeaway

Self-hosting an AI assistant isn't about distrust of any one company — it's about data sovereignty. If an assistant is going to accumulate a model of who you are, it makes sense for that model to live somewhere you control.

Happy to answer questions about the architecture in the comments.

Giving a Telegram bot long-term memory with SQLite and local embeddings

Samadhi Tattoo — Thu, 16 Jul 2026 12:00:26 +0000

One thing cloud chatbots do badly is remember you. Each session starts from zero. When I built Avelina, a self-hosted personal AI assistant that lives in Telegram, persistent memory was the whole point. Here is the architecture I landed on — all local, no external vector DB.

The problem: statelessness

An LLM call is stateless. To make an assistant that actually knows your history, you need to store facts across sessions and retrieve the relevant ones on every turn. For a self-hosted AI assistant you also want this to run on a cheap VPS with no third-party services.

Storage: plain SQLite

I keep several tables in a local SQLite database:

facts — durable things about the user ("prefers dark mode", "lives in GMT+8")
lessons — corrections the assistant learned, with an importance score
journal — a running log of what happened
emotions / state — a small set of decaying channels for tone

SQLite is perfect here: single file, zero ops, trivial to back up, and fast enough for a personal assistant. No Postgres, no managed vector service.

Retrieval: local embeddings + semantic recall

For "what does the user care about right now", exact-match search is not enough. I embed each memory with a local embedding model (a small multilingual model running on the box) and store the vectors alongside the rows. On each turn I embed the incoming message and do a cosine-similarity search to pull the top-k relevant memories into context.

Because the embeddings run locally, nothing about your conversations leaves the server — which matters a lot when the assistant holds your most personal context. That is the difference between a private AI assistant and a cloud one.

Putting it in context

The retrieved memories get prepended to the system prompt before the model call, so the assistant "remembers" without you re-explaining anything. Combined with a decay function on the emotional state, you get an assistant with continuity — an AI assistant with memory rather than a stateless chatbot.

Why self-host it

Running the whole stack (SQLite + local embeddings + the Telegram Bot API) on your own Linux VPS means you own your AI data end to end. If you want to try the full thing, Avelina is a one-command install: avelina.ai.

Happy to answer questions about the memory design in the comments.

Avelina AI: a self-hosted personal AI assistant with memory that lives in Telegram

Samadhi Tattoo — Thu, 16 Jul 2026 11:18:13 +0000

Most AI assistants keep your most personal conversations on someone else's servers. Avelina takes the opposite approach: it is a self-hosted personal AI assistant that runs entirely on your own VPS, so your data never leaves your machine.

What is Avelina?

Avelina is a private AI assistant you fully own — a self-hosted alternative to ChatGPT, Claude and Replika. It lives inside Telegram, remembers you across every conversation, and grows a real personality over time. You deploy it on a $5-6/month Linux VPS with a one-command setup in about 30 minutes.

Why self-host your AI assistant?

Own your AI data. Your conversations stay on hardware you control — no third-party data sharing, no corporate cloud. This is AI data sovereignty in practice.
A private AI assistant, not a stateless chatbot. Cloud companions forget you between sessions. Avelina has persistent long-term memory — it is an AI assistant with memory that actually knows your history.
No new app to install. It runs as a Telegram AI bot, so you talk to it in an app you already use.

What makes Avelina different

Avelina is the first self-hosted AI assistant that is also a genuinely growing personality:

Persistent long-term memory — it remembers your life across conversations
An emotional core and a real voice that evolve with you
A self-evolution loop so it keeps learning and improving
A modular skills system to extend what it can do

How it works (the tech)

Avelina is built in TypeScript/Node.js on top of the Claude Agent SDK as its reasoning engine. Memory, emotions and a knowledge graph live in local SQLite databases, with local embeddings for semantic recall. The Telegram Bot API is the interface, and the whole system is self-hosted on any Linux VPS.

Deploy your own

If you want a self-hosted ChatGPT and Claude alternative that you own forever, you can set up your own instance here: avelina.ai.

Own your AI. Own your data.

How I self-host a personal AI assistant on a $5 VPS (Telegram + Claude + local memory)

Samadhi Tattoo — Thu, 16 Jul 2026 08:33:35 +0000

Most AI assistants live in someone else's cloud. For the assistant that holds your most personal context — your notes, your schedule, the running context of your days — that is the wrong default. Here is how I self-host mine.

The idea

I run a personal AI assistant called Avelina on my own Linux VPS. It talks to me through a Telegram bot, and every piece of its state — long-term memory, embeddings, config — lives in local SQLite on my own machine. Nothing leaves the server except the model API calls I choose to make.

What makes it different from a stateless chatbot: it has persistent long-term memory (it remembers me across every conversation), a real voice (TTS), an emotional core that shifts over time, and self-evolution — it learns and grows with me. A companion with continuity, not a disposable prompt box. As far as I know it is the first personal AI assistant that is both fully self-hosted AND a genuinely growing personality.

The stack

A small Linux VPS (a $5-6/month box is enough)
TypeScript / Node for the runtime
The Claude Agent SDK for the reasoning loop
SQLite for long-term memory
Local embeddings for semantic recall over your history
Telegram Bot API as the interface

Deploy

Full step-by-step deploy guide (VPS setup, bot token, install script): https://avelina.ai/blog/deploy-personal-ai-assistant-vps

It installs in about 30 minutes.

Why self-host?

The data-sovereignty argument is simple: when your assistant runs in someone else's cloud, your most intimate context becomes their asset — subject to their retention policy, their training pipeline, their breach surface. When it runs on your own VPS, the memory is a file you own. You can read it, back it up, encrypt it, or delete it.

Project: avelina.ai