📌 The full version (with FIG.5 interactive Hindsight loop simulator and 5 SVG figures) lives on my blog:
👉 https://okikusan-public.dev/obsidian-as-ai-memory.en
This dev.to post is the condensed version. The interactive simulator and the layered SVG visuals are only on the canonical page.
Introduction
A developer's offhand comment changed everything.
"I want Obsidian to be not just a note-taking tool, but the AI's long-term memory device itself."
This single sentence crystallized a fully local stack: Ollama + Hindsight + PostgreSQL + Obsidian. Not just a combination of tools, but a new layer where the infrastructure audits itself and humans and AI mutually extend each other's cognition.
Four prerequisite terms
This article assumes the following background. If any of these are unfamiliar, skim here first.
- Hermes Agent — Nous Research's OSS autonomous AI agent. Runs resident as CLI / gateway and handles dialogue with the user.
- daily-chats/ — A folder inside the Obsidian Vault. Every Hermes session end auto-exports that day's raw conversation log (questions, responses, tangents, hesitations, code snippets — all unedited) as markdown. The entry point of AI's long-term memory.
- knowledge/ and MOC — A separate Vault folder. Summaries and insights extracted from daily-chats/ get organized into themed MOCs (Map of Content — index notes that bundle related notes). The destination layer.
- Hindsight — Hermes's memory mechanism. Periodically scans daily-chats/, summarizes via Ollama, persists into PostgreSQL, and accumulates summaries of summaries in a self-referential engine. The protagonist of this article.
TL;DR
- Real-world benchmark with Gemma3 on Ollama: 23.4 tokens/sec — not "tolerable," but "daily usable."
- Hindsight repeatedly summarizes past summaries, forming a self-referential feedback loop. Infrastructure self-auditing begins to run entirely locally.
- Obsidian's daily-chats/ gets redefined as AI's primary persistent memory device.
- The loop that gradually converts "hesitation, hypothesis, discomfort" into explicit knowledge is starting to function as a cognitive OS.
What 23.4 tokens/sec actually means for "being local"
Gemma3 on Ollama benchmarks at 23.4 tokens/sec. In an era of cloud dependency, this is not "tolerable speed" but "daily usable speed."
What matters is not the speed itself. It's the fact that every process completes locally. Raw conversation logs accumulated in daily-chats/ are instantly summarized by the LLM, structured by Hindsight, and persisted into PostgreSQL. No external APIs participate at all.
The entire history of thought stays closed inside one's own machine. This is more than privacy. It's an experiment in how deeply AI can understand context under the constraint "never leak externally."
Hindsight's self-referential infrastructure
Conventional RAG stops at "retrieve and answer." Hindsight is different. It repeatedly summarizes past summaries, accumulating meta-metadata from summaries of summaries. The vector and metadata layers etched into PostgreSQL gradually self-organize over time.
This is "infrastructure self-auditing":
- Does today's daily-chat contradict last week's summary?
- How have the themes the user repeatedly touches evolved over the long term?
- To what extent do AI-generated summaries distort the original context?
The foundation is in place for the AI itself to periodically verify these. No external human reviewer needed. The system relativizes itself and proposes corrections — a loop already turning, entirely locally.
FIG.2 — User ⇄ Hermes dialogue flows into daily-chats/, summarized via Ollama, promoted into MOC, and returns to Hermes as context.
"Making Obsidian the memory" — the ultimate human-AI co-creation
The developer didn't stop at treating Obsidian as a "second brain." They redefined it as "the primary memory device for AI." daily-chats/ is no longer a graveyard of miscellaneous logs. Every piece of text accumulated there is structured through Hindsight and functions as AI's long-term memory.
This is not a relationship where humans ask AI to "remember this for me."
It's a relationship where humans design the environment itself: "I'll write it here, so you go ahead and structure it on your own."
AI only becomes intelligent within the context it is given. The core of this pipeline lies in the reversed idea — humans prepare the field that transforms context into "persistent, searchable, and self-referential memory."
The loop that turns tacit into explicit
What gets written in daily-chats/ is not polished ideas, but "hesitation, hypotheses, discomfort, wavering judgment." Hindsight gradually converts that ambiguity into explicit knowledge. Eventually, Obsidian's knowledge/ layer becomes not just a collection of MOCs, but a "knowledge graph that AI itself is cultivating."
When that happens, what does the developer witness?
- The moment a discomfort felt in the past is rediscovered by AI
- The moment a rough note suddenly gains meaning in a different context
- The moment the infrastructure quietly points out: "This theme contradicts what it was three months ago"
Everything local, everything persistent, everything self-referential — such a state may already be moving at hand.
FIG.3 — The left side (doubt, hypothesis, discomfort) gets passed into daily-chats/. Hindsight accumulates summaries of summaries, growing the right-side knowledge/ MOCs over time.
This is not about tools — it's about a cognitive OS
Ollama + Hindsight + PostgreSQL + Obsidian.
What makes this combination special is not that any one piece is superior. It's because the circuit that converts the fluidity of human thought into a form AI can handle has finally closed.
The moment the developer decided to "make Obsidian the memory itself," technology transcended mere means. AI and humans have begun, without interference from anyone, to build with their own hands an OS for becoming wise together while compensating for each other's weaknesses.
Every layer is in your own hands. That's why this deserves the name "cognitive OS."
FIG.4 — Human → Hermes → Hindsight → Obsidian + PostgreSQL → Ollama → local machine. Every layer in your own hands.
This loop has only just begun.
📌 The full version with the FIG.5 interactive Hindsight loop simulator (click to iterate the recursion 1 → 2 → 3 and watch the knowledge graph grow) is on the original:
👉 https://okikusan-public.dev/obsidian-as-ai-memory.en
How are you connecting your notes to AI memory? Would love to hear how the "summary of summary" idea lands in your own setup. 🦄 💬
Top comments (0)