DEV Community

Mizoko
Mizoko

Posted on

Why I Let an LLM Wiki Write Our Decision Log

TL;DR

  • I had an LLM write our product decision log into a wiki. Every decision carries its reason and a source link back to the original discussion.
  • The LLM writes only two things: decisions with their reasons, and flows that cross mobile and backend. Everything else stays in its own source of truth.
  • The first hand-made pages hold 41 sourced decisions. The log is now easy to search and trace, for humans and for AI.

Why the wiki stopped working

I am a PM on a mobile app for beauty salons. The feature in this story is a skin-diagnosis feature.

Our wiki was supposed to record why we chose one technology over another. Updates fell behind, and the reasons stopped being recorded. Code tells you how things work. It does not tell you why a decision was made, or how a flow crosses mobile and backend.

Those were the only two things missing. So those are the only two things I asked the LLM to write.

The model: Karpathy's LLM Wiki

In April 2026, Andrej Karpathy published a gist called "LLM Wiki". Most LLM document workflows are RAG: every question re-finds the knowledge, and nothing accumulates. The LLM Wiki puts a persistent, interlinked wiki between the raw sources and the reader, maintained entirely by the LLM.

Three layers: raw sources (collected by humans, immutable), the wiki (owned by the LLM), and a schema (a document describing structure and rules). Three operations: Ingest, Query, and Lint.

Human wikis die because maintenance grows faster than value. Here, the LLM carries all of that maintenance.

How I mapped it

  • Raw sources: the mobile and backend code on release branches, PR bodies and reviews, Backlog tickets and comments. Instead of collecting sources by hand, I used what development already produces.
  • Wiki: the GitHub Wiki of the app repository. The LLM adds pages to the wiki humans already use, instead of introducing a new tool.
  • Schema: a skill file, SKILL.md, for a skill called update-llm-wiki. It restricts what gets written to the two things above.
  • The index is _Sidebar.md. The log is the wiki's git history.

What the LLM writes, and what it never touches

Decisions and their reasons. Only reasons actually written in a source, each with a citation to a Backlog comment or a PR.

Cross-boundary flows between mobile and backend, as mermaid sequence diagrams.

Everything else stays where it already lives: visuals in Figma and code, API specs in OpenAPI, decisions whose reasons sit in code comments stay in code.

Two examples

Format: one line for the decision, then the reason and the source underneath.

  • The 20-item cap is checked on the backend only, and we accept that concurrent creation can produce 21 items.
  • Reason: a counter column or an exclusive lock does not fit a member-facing form. Even at 21 it grows no further, and deleting one item returns to the cap.
  • Source: 3 comments across 2 tickets
  • For numeric fields, store the normal range at the time of saving together with the value.
  • Reason: so that later changes to the normal range do not rewrite past judgments.

Results

  • 41 sourced decisions across the two hand-made pages (24 on the OCR import page, 17 on the custom fields page), counted from the source lines on those pages.
  • First commit: 2026-11-17 19:37, 5 files, 270 lines added.
  • Pages created: the feature overview (with API and 8 tables), two decision pages, and two sequence diagrams in the data-flow page.
  • Whether these pages have actually been used is not yet confirmed.

The other half: an entity graph

On 2026-10-02 I had added an entity graph to the same wiki: screens, APIs, use cases and tables as nodes and edges (30 files, 20,947 lines), used for incident investigation and onboarding, queried by a skill called ask-entity-graph.

The split: the graph holds structure that can be extracted from code mechanically. The wiki holds reasons that exist nowhere in code.

A comment on the gist argues that code knowledge is fundamentally about relations, so Markdown is the wrong tool and a graph is better. Running both is where I landed.

The follow-up article covers the automation: how the wiki now updates itself with no human in the loop.

Top comments (0)