The Tweet That Changed How I Think About AI + Knowledge
In early April 2026, Andrej Karpathy (OpenAI co-founder, former Tesla AI Director) posted something deceptively simple:
"Something I'm finding very useful recently: using LLMs to build personal knowledge bases for various topics of research interest."
He followed it up with a GitHub gist titled LLM Wiki an "idea file" describing a pattern for building knowledge bases that actually compound over time instead of rediscovering the same information on every query.
I decided to build it for a real production problem.
The Problem: RAG Has No Memory
In My organization, we were running an SEO monitoring pipeline for our landing site. It used Cognee (a knowledge graph framework) backed by Neo4j + ChromaDB on a $35/month VM.
Every day it would:
- Pull Google Search Console data
- Scrape our pages for SEO issues
- Query the knowledge graph
- Post a Slack report
It worked. But it had a fundamental flaw the same flaw Karpathy describes in every RAG system.
Every time it ran, it was rediscovering knowledge from scratch. It had no memory of what it found yesterday. It couldn't connect "we deployed this fix on March 13" with "clicks went up 81% on March 18." It couldn't say "this keyword has been declining for 3 weeks here's why." It just answered the current query and forgot everything.
On top of that: the VM broke 3+ times. Neo4j config issues. ChromaDB API path changes. Sidecar containers failing silently. We spent more time fixing the pipeline than reading its output.
Karpathy's Core Insight
His idea is simple but changes everything:
Instead of retrieving from raw documents at query time compile knowledge once, keep it current, and query the compiled result.
The architecture has 3 layers:
raw/ → immutable source data (never edited)
wiki/ → LLM-maintained markdown knowledge base
AGENTS.md → schema/rules file telling the LLM what to do
The LLM reads new raw data → integrates it into the wiki → cross-references it with existing knowledge → flags contradictions → the wiki gets richer every day.
His exact framing: "Obsidian is the IDE. The LLM is the programmer. The wiki is the codebase."
No vector database. No embeddings. No $35/month VM. Just markdown.
What I Built: LLM Wiki for SEO
I took Karpathy's pattern and applied it specifically to SEO monitoring for vibetrader.com.
The 3 Layers
raw/ : daily immutable snapshots:
raw/gsc/2026-04-17.json ← Google Search Console (clicks, CTR, positions, queries)
raw/audit/2026-04-17.json ← page audit (H1, meta, schema, canonical checks)
raw/commits/2026-04-17.json ← landing-site git log (what code changed)
wiki/ : LLM-maintained knowledge:
wiki/overview.md ← "story so far" updated daily
wiki/log.md ← append-only daily log
wiki/topics/keywords.md ← keyword clusters + position tracking
wiki/topics/issues.md ← open SEO issues with severity
wiki/topics/recommendations.md ← history of recs + acted/pending status
wiki/topics/code-changes.md ← code change impact tracker
wiki/topics/performance.md ← CAUSAL CHAINS: issue → fix → metric improvement
wiki/topics/competitors.md ← weekly competitor analysis
wiki/topics/lint-report.md ← weekly wiki health check
AGENTS.md : the schema file. Tells the LLM exactly how to update each page, what cross-links to write, what format to follow.
The Flow
3:00 PM IST daily
─────────────────
gsc_pull.py → pulls GSC data via API → raw/gsc/today.json
audit.py → fetches vibetrader.com, checks H1/meta/schema → raw/audit/today.json
→ posts Slack audit digest (SEO/GEO/AEO scores)
ingest.py → reads ALL raw/ + ALL wiki/
→ sends full context to Azure OpenAI
→ LLM updates all wiki pages with cross-links
→ appends to log.md
→ posts enriched Slack report
git commit → wiki/ + raw/ committed back to main
4:00 PM IST daily (1 hour after ingest)
─────────────────────────────────────────
fix_agent.py → reads wiki/issues + recommendations
→ LLM classifies which issues are auto-fixable
→ skill functions run deterministically (not LLM-generated code)
→ raises PR to landing-site dev branch
→ never auto-merges
11:30 AM IST Saturday
───────────────────────
lint.py → audits wiki for staleness, contradictions, orphan pages
→ health score /100
→ saves lint-report.md + posts Slack
2:30 PM IST Sunday
────────────────────
competitor_analysis.py → scrapes competitor sites
→ LLM analysis → updates competitors.md
The Knowledge Graph in Plain Markdown
The key innovation over standard LLM Wiki I added a performance.md page that tracks causal chains:
### ✅ 2026-03-18 — API response fix
- **Cause:** [[topics/issues]] API returning incomplete pagination data
- **Fix:** API response items + pagination as separate fields → [[topics/code-changes]]
- **Before:** 127 clicks/day
- **After:** 229.7 clicks/day
- **Impact:** +81% clicks
- **Keywords moved:** [[topics/keywords]] — all branded queries improved
- **Confidence:** HIGH — spike confirmed same week as deploy
Every entry answers: what caused this? what moved? how confident?
This is what Cognee was trying to do with Neo4j store causal relationships between entities. We're doing it in plain markdown, with [[wiki-links]] that Obsidian renders as a visual knowledge graph.
Obsidian as the Visualization Layer
Open Obsidian → point it at your wiki/ folder → graph view instantly shows all cross-connections between pages:
-
performance↔keywords↔issues↔recommendations↔code-changes
As the wiki accumulates data daily, the graph gets richer. After 30 days, clicking any node shows a real story: "this keyword dropped because of this issue, which was fixed by this code change, which resulted in this metric improvement."
That's the knowledge graph. No Neo4j required.
Interactive Query Mode
Beyond the daily Slack push, I added a local query script:
# one-shot
python scripts/query.py "what keywords are improving this week?"
# REPL with follow-up context
python scripts/query.py
you> which code change had the most SEO impact?
wiki> Based on code-changes.md:
March 18 — API response fix
Before: 127 clicks/day | After: 229.7 clicks/day | Impact: +81%
This is the highest confirmed impact change in the wiki.
you> why did that happen? ← follow-up, remembers context
wiki> ...
The LLM answers from the accumulated wiki context not by re-reading 30 days of raw JSON. This is exactly what Karpathy's "query" operation describes.
Weekly Lint Pass
One gap in most LLM Wiki implementations no health check. I added lint.py that runs every Saturday:
- Staleness detection : claims in the wiki that contradict newer log entries
- Contradiction detection : two pages saying opposite things
- Orphan pages : pages with no cross-links pointing to them
-
Missing concept pages:
[[wiki-links]]referenced but never created - Stale recommendations: recs pending > 30 days
Posts a health score /100 to Slack and saves lint-report.md.
The Fix Agent (Skill-Based, Not LLM-Generated Code)
The fix agent reads the wiki, identifies fixable issues, and raises PRs to the landing-site repo. The key design decision: the LLM only classifies issues it never writes code.
Instead, deterministic "skill" functions handle each fix type:
skills/add_jsonld.py ← checks schema not already present before adding
skills/add_internal_link.py ← validates page exists in app/ first
skills/fix_meta_tags.py ← only updates if explicit value given
skills/fix_robots_txt.py ← safe append-only sitemap fix
The LLM outputs {"skill": "add_jsonld", "page": "app/layout.tsx"}. The skill runs. No hallucinated file paths. No duplicate schemas. No business logic touched.
PRs always target dev branch. Never auto-merge. Human review required.
vs Cognee: Honest Comparison
After running both systems in parallel for a week:
| Cognee | LLM Wiki | |
|---|---|---|
| Infrastructure | Neo4j + ChromaDB + VM | Plain markdown files |
| Monthly cost | ~$35 | ~$0 (just Azure OpenAI tokens) |
| Times it broke | 3+ | 0 |
| Files to maintain | 6 JS pipeline files | 5 Python scripts |
| History awareness | Inconsistent | Grounded references exact dates |
| AEO accuracy | 3/10 (missed JSON-LD already added) | 7/10 (reflected actual state) |
| Knowledge compounds | ❌ | ✅ |
LLM Wiki won on every dimension except one: Cognee had richer graph relationships out of the box. But we replicated that with performance.md and [[wiki-links]].
What Karpathy Got Right
The core insight is correct and the more I use it the more obvious it becomes:
RAG = studying for an exam by re-reading all your textbooks every time you get a question.
LLM Wiki = you already made notes, highlighted the important parts, drew arrows between connected ideas. Now you just read your notes.
The wiki grows daily. The LLM gets smarter about your domain the more data accumulates. By day 30, query.py answers questions with 30 days of grounded context. By day 90, performance.md has a dozen confirmed causal chains real institutional knowledge about what moves your metrics and why.
That's what no RAG system gives you. And you don't need a vector database to get there.
Try It Yourself
The pattern works for any domain where you're accumulating knowledge over time:
- SEO monitoring (what I built)
- Competitor tracking
- Research — papers, articles, building a thesis over weeks
- Engineering team wiki — fed by Slack threads, PRs, incident reports
- Personal second brain — journal entries, articles, book notes
The only thing you need:
- A
raw/folder where data lands - A
wiki/folder for LLM-maintained markdown - An
AGENTS.md(orCLAUDE.md) telling the LLM the rules - A cron job that runs ingest daily
Start with one raw data source and one wiki page. Let it run for a week. The compounding effect becomes obvious fast.
Top comments (0)