One thing cloud chatbots do badly is remember you. Each session starts from zero. When I built Avelina, a self-hosted personal AI assistant that lives in Telegram, persistent memory was the whole point. Here is the architecture I landed on — all local, no external vector DB.
The problem: statelessness
An LLM call is stateless. To make an assistant that actually knows your history, you need to store facts across sessions and retrieve the relevant ones on every turn. For a self-hosted AI assistant you also want this to run on a cheap VPS with no third-party services.
Storage: plain SQLite
I keep several tables in a local SQLite database:
- facts — durable things about the user ("prefers dark mode", "lives in GMT+8")
- lessons — corrections the assistant learned, with an importance score
- journal — a running log of what happened
- emotions / state — a small set of decaying channels for tone
SQLite is perfect here: single file, zero ops, trivial to back up, and fast enough for a personal assistant. No Postgres, no managed vector service.
Retrieval: local embeddings + semantic recall
For "what does the user care about right now", exact-match search is not enough. I embed each memory with a local embedding model (a small multilingual model running on the box) and store the vectors alongside the rows. On each turn I embed the incoming message and do a cosine-similarity search to pull the top-k relevant memories into context.
Because the embeddings run locally, nothing about your conversations leaves the server — which matters a lot when the assistant holds your most personal context. That is the difference between a private AI assistant and a cloud one.
Putting it in context
The retrieved memories get prepended to the system prompt before the model call, so the assistant "remembers" without you re-explaining anything. Combined with a decay function on the emotional state, you get an assistant with continuity — an AI assistant with memory rather than a stateless chatbot.
Why self-host it
Running the whole stack (SQLite + local embeddings + the Telegram Bot API) on your own Linux VPS means you own your AI data end to end. If you want to try the full thing, Avelina is a one-command install: avelina.ai.
Happy to answer questions about the memory design in the comments.
Top comments (0)