Building Memoir: a health app that's actually supposed to know you
I didn't set out to build a memory system. I set out to build a health app I'd actually want to use — medications, symptoms, workouts, mood, all in one calm place instead of scattered across four different apps that all feel like they were designed by a hospital's IT department in 2011. Warm colors. No dashboards screaming red numbers at you. Something that felt private, not like it was quietly compiling a dossier to sell to an insurer.
The AI chat was supposed to be the easy part. Bolt on Gemini, let people ask it questions about their health data, done. It was not the easy part. It ended up being the part that taught me the most, mostly by breaking in ways I didn't see coming.
The AI that knew a stranger named Himanshu
The first version of the chat "worked" in the sense that it responded to messages. But early on I actually read the code path it used for context, and found this sitting there, dead serious, as the entire personalization layer:
User: Himanshu, 25yo Male, 175cm, 72kg
Medications: Metformin 500mg (twice daily, 93% adherence), Lisinopril 10mg...
Hardcoded. Every single user, every single conversation, got told they were a 25-year-old man on three medications they'd never taken, with an adherence percentage somebody had just made up because it looked convincing in a demo. It wasn't lying to me specifically — it would've confidently told anyone the same fake life story. That's a specific kind of uncomfortable to discover in a health app: the AI was more confident about a stranger's medication schedule than it should ever be about a real one.
The fix was mechanically simple — build the context string from whatever's actually in the person's profile and medication list instead of a fabricated one — but it changed how I thought about the whole feature. An AI health assistant that isn't grounded in your real data isn't a lesser version of the feature. It's a different, worse feature wearing the same UI.
Giving it memory made it worse before it made it better
Once the chat was honest about the current data, I wanted it to be smart about history too — notice that you've mentioned being stressed for a few days running, or that a headache pattern lines up with bad sleep, without you having to re-explain your whole week every time you open the chat. So I wired in Cognee, which turns stored text into an actual knowledge graph and lets you ask it things in plain language instead of writing a query for every pattern you can think of ahead of time.
And it worked, right up until it worked too well. A few days into testing, the AI told me — with total confidence — that I was taking Metformin, Lisinopril, and something called a "Painkiller," and that my steps that day were approximately one hundred million. None of that was true. What had actually happened was that all my earlier testing, including that fake Himanshu profile, had gotten permanently absorbed into the memory graph, and nothing was telling the model that current, live data always outranks whatever the memory graph half-remembers. The graph doesn't forget, and it doesn't know that a fact from three weeks ago might not be a fact anymore.
The actual fix was one paragraph in a system prompt, explicitly telling the model the live profile is the single source of truth and the memory graph is historical color commentary, never permission to state something as current fact. It's a small instruction carrying a lot of weight, and it's the kind of thing that's very easy to quietly delete in a future refactor because it looks like it's not doing anything. It's doing everything.
Some bugs are dumb, some are just weird
Not every bug taught me something profound. The "mark medication as taken" button, for instance, simply had no click handler on it. None. It looked completely real — hover states, color changes on press, the works — and did absolutely nothing when you tapped it. Sometimes the deep lesson is "read your own code more carefully," and that's fine.
Other bugs were genuinely strange. The landing page had a rotating headline — "Your Personal Health OS," then Fitness OS, then Wellness OS — except the animated word was completely, invisibly gone. Not clipped, not the wrong color. Gone. It took actually screenshotting the rendered DOM to work out why: the gradient-text effect was applied to a wrapper whose children mixed position: absolute and position: relative, and browsers just refuse to paint a clipped gradient through that combination. The fix was applying the gradient to each word individually instead of the shared wrapper — three lines changed, half an hour of screenshotting weird invisible text to get there.
I got the closely related lesson twice more before I stopped needing it a third time: once when a modal that should've been dead-center was instead offset by exactly half its own width and height (Framer Motion was quietly overwriting my manual centering transform), and again when a calorie input on mobile just stopped existing past a certain screen width, because two flex-1 inputs don't actually shrink below their own content size unless you tell them they're allowed to. CSS has more of these than I remembered.
The moment I remembered other people exist
For a long stretch, Memoir was built and tested as if there would only ever be one person using it, ever, on one browser, forever — a very easy trap when you're building alone. Then someone asked, reasonably: shouldn't a second person signing up get their own data instead of seeing whatever the first person left behind?
The honest answer was no. Everything — medications, symptoms, diary entries, even the AI's memory graph — sat in one shared bucket with no concept of "whose is this." Fixing it meant going back through storage, the account system, and every API call that talked to the AI's memory, threading a real per-account ID through all of it. It's not a glamorous fix. It's the kind of thing that doesn't show up in a demo at all, right up until it's the whole reason the product is or isn't trustworthy.
"Free" is doing a lot of work in that sentence
Somewhere around adding email confirmations and a real "Continue with Google" button, I ran into the quieter reality of building something meant for other people, not just a screen recording: free tiers all have an asterisk. Resend gives you real email sending for free, but only to your own address until you verify a domain. Gemini's free tier is generous until it isn't — and it turns out the model spends part of its own answer budget on invisible "thinking" before it writes a word you can see, which explained why AI insights kept cutting off mid-sentence with no warning. Even Google sign-in, needing no backend at all for a client-side flow, still needs you to go create real credentials in a real console before a button click does anything.
None of that is a complaint. It's just the gap between "I built a feature" and "a stranger can use this feature," and it's wider than it looks from the inside.
What's still not real
I'd rather say this plainly than let the app oversell itself. There's no appointment reminder that actually fires on the day — that needs a real server-side database and a scheduled job, and Memoir doesn't have either, by design, not as a placeholder for something bigger later. Google sign-in works, but it's a client-side identity handshake, not a full session system. Honest tradeoffs for what this is right now.
What actually surprised me, looking back, is that almost none of the hard problems were about the AI itself. Gemini did what I asked it, basically every time. The hard problems were about trust — trusting the right data, trusting it to the right person, and being honest about what a free tier actually promises versus what it sounds like it promises. That turned out to be most of the job.
Top comments (1)
Cognee makes the most sense when memory is treated as a graph of working context, not just a nicer vector store. The useful question is what gets promoted into memory and what should expire.