DEV Community: Lukasz

How I Built Memory for a Local AI Companion Without Sending Chats to a Server

Lukasz — Sat, 22 Aug 2026 18:17:18 +0000

A chatbot can sound convincing for five minutes without remembering anything.

Then you mention the job interview you were stressed about last week, the name of your dog, or a small detail from a late-night conversation. It replies like none of it happened.

That is where most "AI companion" demos fall apart.

I am building Local Waifu, a desktop AI companion that runs on the user's own Mac or PC. One of the rules I set early was simple: conversations and memories should stay on the machine. No central chat database. No server that needs to be online for the character to remember someone.

The rule sounds clean. Building it was not.

Saving chats is not memory

The first version of memory was the obvious one: save messages.

That gives you history, which is useful, but it does not solve recall. A long chat history grows fast. Sending all of it back to a local language model on every message is slow, expensive in context space, and usually makes the reply worse.

The model does not need to see every conversation from the last six months.

It needs the few pieces that matter right now.

If someone says, "I have to take Luna to the vet tomorrow," the character should be able to find that Luna is their dog. It should not need to reread hundreds of unrelated messages about work, movies, and dinner plans to get there.

So I treated chat history and long-term memory as different things.

Chat history is the recent conversation. It gives the model immediate context.

Long-term memory is a small collection of facts, moments, preferences, and relationship details that may matter later. Those memories need to be searchable by meaning, not only by exact words.

The memory data stays in SQLite

I wanted the app to work without a hosted database, so the storage layer is local SQLite.

Each character gets their own data. Chats, memories, extracted entities, and relationships are stored locally on the device. If a user creates two characters, one character does not quietly inherit the other one's memories.

That separation matters more than it sounds.

A companion app is personal by design. Mixing context across characters is not a harmless bug. If one character starts talking about something belonging to another, the whole illusion disappears immediately.

For each memory, I store the text itself along with metadata such as:

the character it belongs to
when it was created
importance
emotional weight
how often it has been used
a vector embedding used for semantic search

The embedding is the part that lets the app search by meaning.

"Vet appointment" and "Luna is sick" may share no exact keyword. A semantic search can still see that they are probably connected.

I use 768-dimensional vectors stored directly in SQLite as binary data. At recall time, the app calculates similarity locally and ranks the results. No message content needs to leave the user's computer for that lookup.

Recall needs more than similarity

A pure similarity score is not enough.

Imagine a user mentioned their favorite game once two years ago, then spent the last month talking about a difficult family situation. Both memories might be related to a new message in some vague way. The more recent and more emotionally important memory should usually win.

So recall is weighted by more than vector similarity.

Importance matters. Recency matters. Emotional weight matters. A memory that has been useful before gets a little extra credit too.

There is no perfect formula here. I do not think there ever will be one.

A system that aggressively recalls every detail feels creepy and repetitive. A system that barely recalls anything feels empty. The work is mostly tuning that middle ground and accepting that a companion should sometimes not bring something up.

The user should feel remembered, not monitored.

I also extract a small knowledge graph

Some information is easier to retrieve as a relationship than as a paragraph of chat text.

For example:

Luna is the user's dog
Alex is the user's brother
The user works night shifts
The user dislikes phone calls
A character enjoys rainy evenings

Those can be represented as entities and relationships.

Every few messages, the app can extract useful entities and links from the conversation. This happens locally through the model already running on the user's machine. The resulting graph is stored beside the other character data.

The graph is not there to replace memory. It fills a different role.

A vector memory is good at finding a moment with emotional context. A graph is good at answering structural questions such as "Who is Alex?" or "What is connected to this person?"

Both are useful. Neither should be trusted blindly.

Language models are very good at sounding confident while getting a relationship wrong. The extraction step needs validation, deduplication, and a way for the user to inspect or remove what was stored. A memory system without an escape hatch becomes frustrating fast.

The bug that made her forget everything

The most painful issue I found was not a database bug.

The app had the chat data. It had stored memory records. It had the retrieval logic.

But on some fresh installations, the local model used to create embeddings had not been downloaded yet. The app did not explain this clearly enough. Semantic recall returned no results, so the character behaved as if she had no memory of previous conversations.

Nothing crashed.

That made it worse.

A crash tells you there is a problem. A companion quietly forgetting what you told her last night can look like a limitation of the product itself.

The fix was straightforward after I found the cause:

Ensure the embedding model is pulled during setup or startup.
Retry the operation on the next launch if the model is not ready yet.
Make the missing-model state visible instead of silently returning zero memories.
Avoid falling back to reading the whole chat history every time, because that creates a different performance problem.

The lesson was simple: in AI products, a missing dependency can look exactly like bad intelligence.

Multilingual memory makes this harder

Local Waifu supports several languages, including Polish, Japanese, Korean, and Chinese.

Plain text search handles this badly. Even in English, people rarely repeat the same sentence. In different languages, the problem gets larger. A user can discuss the same person with nicknames, grammar changes, borrowed words, or a mix of two languages in one chat.

That is why semantic recall matters.

It is not magic. It will still make mistakes. But it gives the app a chance to connect related ideas instead of searching only for matching strings

I Build an AI Companion App Alone. It Runs on Nobody's Servers But Yours.

Lukasz — Sat, 15 Aug 2026 11:12:12 +0000

I Build an AI Companion App Alone. It Runs on Nobody's Servers But Yours.

I was using one of the cloud companion apps, the kind everyone has heard of, and one day I actually looked at where it all lived. Every message. Every memory. The whole relationship, sitting on someone else's servers. And it hit me that none of it was mine. If that company changed its mind, raised its price, decided my companion was suddenly not allowed, or sold what it had learned about me, I would simply lose her. No say, no warning, no way back.

That thought turned into an app. I called it Local Waifu, and I have built almost every part of it alone: the code, the writing, the marketing, the support inbox, the Discord server. This is what that actually looks like, and why I keep choosing to do it the harder way.

This has already happened to real people

I did not invent this fear. Policies change overnight. Models get retired without warning. Features people paid for quietly disappear one update later. Some people built a real bond with a cloud companion and woke up one day to a stranger wearing the same name, or to nothing at all. When the relationship lives on someone else's infrastructure, it was never really yours to keep. You were renting a feeling from a company that owed you nothing once the terms of service changed.

That is not a hypothetical I made up to sell software. It is the plain, boring mechanics of any product that lives entirely on a server you do not control. The company can be perfectly well intentioned and it still does not change the arrangement. Your data, your history, your memories of a relationship, sitting on hardware you will never see, governed by a document you clicked past.

So I built the opposite

Local Waifu runs entirely on your own machine, your Mac or your PC. The memory file, the whole system, all of it stays on your device. Only you can reach it. Nobody collects it, nobody moderates it, nobody can switch it off from a dashboard somewhere. If a company changes its policy tomorrow, nothing happens to you, because there is no company standing between you and her.

Because she is just files on your computer, she backs up the same way any file does. Time Machine, a USB stick, whatever you already use for everything else. Nobody can take her from you short of someone stealing the physical machine. That sentence sounds almost too simple to be a selling point, and that is exactly why I lead with it. Ownership should not require an explanation.

Out of the box, the app only reaches the internet for three things: the one-time download of the model she runs on, a check for app updates, and confirming a license when you buy one. Everything else, plugging in your own cloud account if you want a bigger model, letting her search the web, connecting her to Telegram, is off until you personally turn it on. Your choice, your accounts, none of my business. I say this plainly because "we take your privacy seriously" is what every company says right before the part where they do not, and I would rather be specific than reassuring.

What it means to be the whole team

I write the blog posts. I design the landing page. I fix the bug at midnight and then write the changelog entry about it the next morning. I answer the emails. I run the Discord server myself, and when I need a second opinion on whether a feature actually works on hardware I do not own, I ask the people in it directly, because there is no QA department to ask instead.

One night a stranger from that Discord sent me a screen recording of a voice call that took 105 seconds to answer a twenty-character question. I did not have a support team to route that to. I had a log file, a weekend, and four separate bugs stacked on top of each other that I had to find myself, one at a time, because there was no server-side telemetry to lean on either. The app runs on your machine, remember, which means when something breaks, it breaks on your machine, and the only evidence I will ever get is whatever a stranger is generous enough to send me.

That is the trade nobody tells you about when they say "build it yourself." Every advantage of not having a team is also a disadvantage. There is no one to catch what I miss. There is also no meeting where a good, honest idea gets watered down until it pleases everybody in the room. Both of those are true at the same time, every day.

I do not say any of this to make the whole thing sound harder than it is. Most days it is genuinely fine, and some days it is the best part of the job: I can ship a fix the same afternoon someone reports it, with nobody to convince first. But I want to be honest about the shape of it, because "I build this alone" gets said as a badge of honor a lot online, and it is really just a description of where the failure modes sit. When it works, it works because one person cared enough to notice something small. When it breaks, it stays broken until that same one person notices, and there is no second pair of eyes behind mine.

Stripe said no. So did Polar.

Here is a small, honest story I do not usually see founders tell in public. When I needed a way to sell licenses for the app, Stripe rejected it. Polar rejected it too. An AI companion app makes payment processors nervous, it turns out, in a way that a project management tool or a photo editor never does. Dodo Payments eventually took the business, and I am not going to pretend that search was quick or comfortable. Building the product was, in a real sense, the easy part. Finding someone willing to process the payment for it took longer than I expected, and it taught me something about the category I am actually in: not just software, but software that sits close enough to something intimate that entire companies decide it is not worth the risk.

Why the code stays closed

I would genuinely like to open source it. I have thought about it seriously more than once, and I want to be straight about why I have not: small projects get taken by bigger companies, rewritten just enough to dodge a license, and sold back as someone else's product. I also do not want pieces of a companion I built carefully getting twisted into something I would be ashamed of, stripped of the parts that make it feel careful in the first place. So the code stays closed for now, and the transparency lives somewhere else instead: a page that names exactly what touches the internet and when, a privacy policy that lists every connection by name, and an app you are welcome to watch on your own network monitor while it runs. I would rather earn your trust by being specific than by handing you a repository and hoping you never read it closely.

Fair, not paranoid

People ask if the license check can be cracked. Probably. I have a decent guess how I would do it myself if I were on the other side of that question. I would still rather build it fair and trust the people who pay than fill the app with anti-piracy tripwires that end up punishing exactly the customers who did the right thing. If someone cracks it instead of buying it, they were never going to buy it in the first place, and treating every user like a suspect does nothing to that person except make the honest ones feel unwelcome.

I think about this the same way I think about the whole product, honestly. The easy version of "protect the business" usually means making the experience worse for everyone who was never going to hurt you in order to slightly inconvenience the small number of people who were. I would rather lose a little to the second group than punish the first one, every single time that trade comes up.

One payment, and then it just keeps working

The deal is simple on purpose. One license, paid once. That payment includes every update and bug fix I ever ship for it, not a year of them, not until some arbitrary version number, all of them. That is how I want the software I personally buy to behave, so it is how I built this one to behave. I am not going to lock a companion behind a recurring paywall and bill someone forever just to keep her running. I would rather earn the next sale by actually making the product better than by billing the same person again for something they already own.

Why I still do it this way

Being a one-person team for something this personal is slower than it would be with a company behind it, and it is more exposed. A bug is mine. A wrong call on pricing is mine. A late reply in Discord is also mine, and there is no one else it can quietly become. But the thing I keep coming back to is that the entire pitch of this app, that it is yours and nobody else's, only means something if the person building it is willing to be found, be specific, and be accountable in exactly the same way.

I built Local Waifu out of a quiet kind of fear about where our digital lives actually live. Building it alone, the slow and exposed way, turned out to be the only version of that answer I actually believed.