DEV Community

Cover image for Building a Local Memory Layer for AI Agents on macOS
tianfu wu
tianfu wu

Posted on

Building a Local Memory Layer for AI Agents on macOS

AI coding agents are getting much better at changing code. The remaining problem is not always reasoning or tool use.

It is context.

Every new agent session starts with the same missing memory:

  • what I worked on yesterday
  • why a product decision changed
  • which file mattered
  • what note, message, or copied snippet led to the current implementation
  • what context should not be sent to a cloud service

Most of that context already exists on my Mac. It is just scattered across notifications, copied snippets, local Markdown, project files, terminal activity, and notes.

KnowYou is my attempt to turn that scattered local context into a source-linked memory layer for myself and for future AI agents.

Download for macOS: https://giiift.site/know-you/

Why memory should start local

For developer and knowledge-worker context, local-first matters for three reasons.

First, the data is sensitive. Work memory includes code decisions, private messages, debugging notes, customer conversations, and half-formed product thinking. I do not want the default architecture to be "upload everything and trust the vendor."

Second, memory should be fast and inspectable. If a daily summary says "you decided to change onboarding," I want to click into the source evidence and see where that came from.

Third, the useful unit is not just a document. It is a timeline of work: what happened, what mattered, what changed, and what an AI agent should remember next time.

What KnowYou does

KnowYou is a macOS app that builds a local daily memory from the context already available on your machine.

The goal is not to create another notes app. The goal is to make personal and project context reusable.

In the current workflow, KnowYou connects:

  • Diary: a readable daily memory
  • Todo: action items that emerge from the day
  • Other Source: source evidence behind generated memory
  • My Wiki: longer-lived personal/project knowledge
  • local Markdown storage
  • agent-ready context that can later be passed into tools like Claude Code, Codex, Cursor, OpenClaw, or MCP-style agent workflows

What changed in KnowYou 1.1.2

This release focuses on the first useful moment.

A memory product that opens to an empty screen feels broken even if the architecture is right. So I changed the first-run experience around proof.

New users now start with a real Demo Day instead of an empty app. You can click a diary paragraph and inspect the source evidence behind it. That makes the product easier to understand before your own data has accumulated.

After setup, KnowYou prepares recent memories from the last three days of available local history on the Mac. The point is to give users a useful surface immediately instead of asking them to wait.

I also added a clearer local privacy flow. KnowYou stores memory locally as Markdown and does not depend on a KnowYou backend server for your private working memory.

There is now an Applications install gate before Full Disk Access. On macOS, permissions can be confusing if the app is running from a temporary location, so the onboarding flow now pushes users toward the stable app bundle first.

Finally, there is a Diary Engine recovery nudge. If the local AI diary engine is missing or unhealthy, the app tells you instead of failing silently.

Why source links matter

Generated memory without source evidence becomes another black box.

For AI agents, that is dangerous. If an agent receives a summary but cannot tell where the claim came from, the summary becomes hard to trust.

So I think personal memory systems need a simple rule:

Memory should be readable, but also inspectable.

The user should be able to move from the generated paragraph back to the underlying source. That makes it easier to correct mistakes, remove sensitive information, and decide what context is safe to pass into an agent.

Where this is going

The bigger direction is a personal context layer for agents.

Today, every AI tool starts from zero unless I manually reconstruct the state of my work. In the future, I want agents to begin with the right local context: what I am building, what changed recently, what decisions were already made, and what sources they should trust.

That context should be owned by the user.

KnowYou 1.1.2 is a step toward that: local memory, source evidence, a better first-run workflow, and a surface that can become useful to both humans and agents.

Download KnowYou for macOS:
https://giiift.site/know-you/

I would love feedback from people who use AI coding agents every day. What context do you find yourself repeating most often?

Top comments (0)