DEV Community

frog404
frog404

Posted on

I built an AI-friendly knowledge base so coding agents stop forgetting project context

I have 20+ personal projects. Every time I start a new AI session, I have to re-explain the project structure, what was decided last time, and what to do next. Switching between Claude Code, Codex, and other tools makes it worse — the context doesn't follow.

I built ContextMixer to fix this: a knowledge base where AI agents search, read, and update project documents through MCP or REST API, and humans review them in a web UI.

Why not Notion / Obsidian / local Markdown?

Local Markdown + Git worked for CLI agents but couldn't be accessed from chat-based tools like Claude.ai.

Notion had MCP support, but fetching pages was slow and token-heavy. Its rich block structure is great for humans but inefficient for AI access.

Obsidian is local-first, which rules out chat-based access.

I needed something accessible from both chat and CLI, lightweight for AI, and editable by humans. Nothing fit, so I built my own. It runs on Cloudflare Workers + D1 + R2, entirely within the free tier.

Progressive retrieval

The core design idea: agents choose how much to read.

View Returns Tokens
meta title + summary tens
outline heading structure hundreds
section one specific section hundreds–thousands
full entire document all

A typical flow for "check the auth design":

1. list_docs(collection_id)        → document list with metadata
2. get_doc(doc_id, view="outline") → heading structure
3. get_doc(doc_id, view="section", section="Auth") → just what's needed
Enter fullscreen mode Exit fullscreen mode

Three API calls, but hundreds of tokens instead of thousands. Most queries resolve at metasection without ever reading the full document.

AI Cortex: structuring project memory

ContextMixer is a generic container. The structure is up to you.

I keep four documents per project:

  • context — current phase, recent work, unresolved issues, handoff notes for the next session
  • spec — confirmed goals, requirements, tech stack (nothing tentative)
  • decisions — design choices with reasoning, including rejected alternatives
  • notes — library pitfalls, bug workarounds, implementation tips

I call this pattern AI Cortex. The agent reads context at session start and updates it at the end. The next session picks up where the last one left off.

The biggest win: I no longer re-explain previous sessions. The agent reads context and resumes. decisions also helps — "why did we drop Vue?" is answered by the document, not by me repeating myself.

But letting AI write to a knowledge base has risks. Without rules, agents write tentative ideas into spec, create unconfirmed decisions, or duplicate documents. Writing permissions aren't enough — you need writing rules. In my setup, CLAUDE.md specifies things like "don't write to decisions without user confirmation" and "search before creating new documents." Still a work in progress.

Karpathy's LLM Wiki

After I started building ContextMixer, I came across Karpathy's LLM Wiki pattern — where an LLM builds and maintains a structured wiki from raw sources instead of doing RAG lookups every time.

It resonated, but the use cases differ. LLM Wiki is a research library: accumulate and organize what you've read. ContextMixer is a project whiteboard: manage ongoing work across sessions, tools, and access points.

Links

📦 GitHub
🔗 Demo
📄 Project page (detailed design notes)
📝 Japanese article on Zenn

AI agents don't need to remember everything. They need a good place to read from.

Top comments (2)

Collapse
 
aldo_cve profile image
Aldo

Dealing with context window limitations and the 'forgetfulness' of LLMs across sessions became a significant architectural challenge for us when we started embedding AI features into our core SaaS product. We quickly realized that a simple prompt wasn't going to cut it for any kind of persistent, intelligent agent. Your approach to an AI-friendly knowledge base sounds like a solid step towards building that durable, context-aware memory layer.

What kind of structure did you settle on for your project context? We've experimented with a few RAG patterns – everything from simple chunking of markdown files with metadata tags, to more structured knowledge graphs where entities and relationships are explicitly defined. The trade-off between retrieval latency, embedding costs, and the quality of the retrieved context is a constant balancing act. Finding the right granularity for chunks, especially when dealing with codebases, can be tricky. Too small, and you lose context; too large, and you blow your token window or introduce noise.

One area we're still refining is how to keep this knowledge base current without manual intervention becoming a bottleneck. For personal projects, you might be the primary editor, but in a team setting, integrating updates from source control, documentation, or even ticket systems automatically becomes crucial. Have you considered how you'll manage versioning of your project context, especially as projects evolve? It's easy for an agent to start hallucinating if its understanding of the project drifts too far from reality.

Collapse
 
frog404 profile image
frog404

Sorry for the late reply!

To answer your first question: I went in a different direction from RAG. The hypothesis behind this project is that you may not need complex retrieval machinery at all — if documents are structured well and the retrieval operations are lightweight enough, the agent can compensate with multiple calls. So ContextMixer doesn't use embeddings or vector search. Documents are structured so the agent chooses how deep to read: meta → outline → section → full. There's no fixed chunk size — the "chunk" is whatever level of detail the agent asks for. The trade-off is extra round trips, but since each call is cheap, that's exactly what the hypothesis accepts.

On keeping the knowledge base current: honestly, not solved yet. What I'm experimenting with is rule-based cascade updates — I have rules in the agent's skill files that tell it to update related documents when it changes one. As a future idea, I'm thinking about a server-side process that periodically scans project documents with an LLM to detect contradictions and raise alerts before the drift starts to hurt.

Curious how the knowledge graph approach worked out for you — did the explicit entity/relationship modeling pay off, or did maintenance eat the benefit?