The first time I tested SOULCHAT, I had the same conversation twice.
I told the AI about a memory I wanted to preserve. It responded with warmth and understanding. I closed the window, opened a new one the next day, and it asked me the same question — as if we had never spoken.
That was the moment I realized: if an AI companion can't remember you, it's not a companion. It's a search engine.
So I set out to build a memory layer for SOULCHAT — something that could store, retrieve, and recall what users share across sessions.
I used Cline with DeepSeek's API to help me prototype it. I designed the system flow and architecture; Cline helped with the implementation. This post is about what I built and why.
What I built: a semantic memory system
The core idea is simple: when a user says something important, the system generates a vector embedding of that text and stores it in a vector database. When the user returns with a new message, it embeds that too, finds the most semantically similar stored memories, and injects them into the context window before generating a response.
This way, the AI doesn't need to remember everything in its weights — it retrieves relevant memories on demand.
Under the hood, I use:
sentence-transformers with the all-MiniLM-L6-v2 model for generating embeddings. It's lightweight (~80MB), runs locally on CPU, and produces 384-dimensional vectors that capture semantic meaning well.
ChromaDB as the vector store, with cosine similarity for retrieval.
This combination gives me fast, offline, cost-effective memory retrieval. No API keys needed for the embedding layer, no per-query cost — just CPU compute.
How the code works
I gave Cline a clear goal: build a system that ingests conversations, generates embeddings, and retrieves them when relevant. The core functions it helped me generate:
Ingestion flow:
python
from sentence_transformers import SentenceTransformer
model = SentenceTransformer('all-MiniLM-L6-v2')
def store_memory(text, user_id, metadata=None):
embedding = model.encode(text)
# store in ChromaDB with metadata
Retrieval flow:
python
def recall_memory(query, user_id, top_k=5):
query_embedding = model.encode(query)
results = collection.query(query_embeddings=[query_embedding], n_results=top_k)
return results
Then I wired it into the conversation loop: every user message passes through recall_memory, the top results are injected into the system prompt as "relevant context from past conversations," and the LLM responds with that context available.
What I learned from building this
Cline writes good first drafts but needs guidance on architecture. I had to tell it "store embeddings with metadata" and "use cosine similarity" — it wrote the implementation, but I designed the flow.
The DeepSeek API helped with refactoring. I hit some bugs (ChromaDB index issues, model loading on first run). I passed the error logs to Cline, and it suggested fixes.
The hardest part wasn't the code — it was defining what "memory" actually means for a companion product.
Is it factual memory? "She told me her grandmother passed away in 2018."
Is it emotional memory? "She sounded sad when she mentioned her grandmother."
Both? Something else entirely?
I kept asking myself: What information is actually useful to remember? How do I structure it so the system knows when to retrieve it? And most importantly — in a live conversation, how does the AI know which memories to pull, and when, so the dialogue feels natural and not like a database lookup?
I don't have perfect answers yet. But I've learned that memory isn't just about "storing text." It's about relevance, timing, and emotional resonance. A good companion doesn't just recall facts — it recalls the feeling behind them.
I'm still iterating on this. The current version works, but "feels like a real friend" is a much higher bar than "retrieves the right vector." I'll get there.
What's next
Adding time-based decay to memory retrieval (recent memories should weigh more)
User-controlled memory deletion (privacy is non-negotiable for SOULCHAT)
Exploring chunk-based embedding for longer conversations
The current system is live at soulchat.hk — and it already remembers what you told it yesterday.
The code is still rough, and I'm learning as I go. If you've built something similar — or if you have thoughts on long-term memory for AI companions — I'd genuinely love to hear from you in the comments. 🧡
Try SOULCHAT: soulchat.hk
Top comments (0)