Open Brane Annotated: 8 Columns, 80-Line Write Path, One SQLite File

Yesterday I open-sourced Open Brane — a personal event-log brain with one SQLite table, one write path, and an MCP server agents can query. This post walks through how it works.

If you haven't read the first post, the short version: every source you care about gets normalized into one append-only table. That table is the source of truth. Every downstream view — Obsidian pages, vector search, compiled journals, dashboards — is rebuilt from it. Agents hit a local MCP server, never the database directly.

If you like videos: https://www.youtube.com/watch?v=uoNe2_OexCc

This post is the implementation. The goal is that by the end you've read the whole system and could write the missing adapters yourself.

The One Table

CREATE TABLE events (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    ts              TEXT    NOT NULL,
    source          TEXT    NOT NULL,
    type            TEXT    NOT NULL,
    actor           TEXT,
    payload_json    TEXT    NOT NULL,
    attachment_uri  TEXT,
    ingested_at     TEXT    NOT NULL
);

That's it. No joins, no foreign keys, no migrations. 3 GB on my production disk at 942,068 rows.

Each column does exactly one thing:

• id — monotonic primary key.
• ts — when the event happened in the real world. A git commit's timestamp, a Claude session's started-at, a Fitbit reading's minute. Not when it was ingested.
• source — which adapter wrote it. git, claude-laptop, gdrive, fitbit. Load-bearing. The dedup story depends on it.
• type — what kind of event. commit, reply, document-chunk, sleep-score, query. This is the axis I filter on most.
• actor — an opaque dedup fingerprint. For a git commit it's the SHA. For a Claude reply it's the session ID plus message index. The adapter picks. If two rows share (source, actor) they're duplicates.
• payload_json — the whole event as JSON. No schema on write. The adapter decides the shape.
• attachment_uri — relative path into blobs/ for large binary attachments. Voice recordings, PDFs, images. Kept outside the DB.
• ingested_at — when the event hit the database. Useful for catching up after an ingester was offline.

An index on (source, type, ts) handles every query I've ever needed. No composite indexes, no full-text index — semantic search takes the role full-text would in a traditional event store.

WAL mode is on. SQLite in WAL mode handles 10,000 writes/sec on commodity hardware. My heaviest backfill day moved tens of thousands of events through the gate without drama. I'm nowhere close to the limit.

The One Write Path

Every write in the brain goes through scripts/record_event.py. 80 lines. Here's the shape:

# simplified — real version has error handling and payload validation
def record_event(source, type, payload, ts=None, actor=None, attachment_uri=None):
    ts = ts or datetime.utcnow().isoformat()
    ingested_at = datetime.utcnow().isoformat()

    payload_str = json.dumps(payload, sort_keys=True)
    actor = actor or _fingerprint(source, type, payload_str)

    with db() as conn:
        conn.execute(
            "INSERT OR IGNORE INTO events "
            "(ts, source, type, actor, payload_json, attachment_uri, ingested_at) "
            "VALUES (?, ?, ?, ?, ?, ?, ?)",
            (ts, source, type, actor, payload_str, attachment_uri, ingested_at)
        )

INSERT OR IGNORE on the unique (source, actor) constraint is the entire dedup story. Adapters re-run freely; duplicates collapse.

The script exposes two interfaces — a Python function for when you're importing it, and a command-line entry point that accepts --payload-stdin for when an adapter shells out. Shell-out is the recommended mode. It means the adapter cannot touch the database, cannot acquire a lock, cannot hold a connection open. Each event is its own subprocess. Predictable, debuggable, impossible to corrupt.

Why this constraint matters. Every state mutation in the brain is expressible as "some adapter produced an event row at time T." If you want to know what changed, you query events. If you want to trace a problem, you follow the event IDs. No mutable state exists anywhere else in the system that can drift from this log.

The Adapter Pattern

An adapter is a Python script that reads a source, emits events, exits. Here's the skeleton:

# scripts/ingest_mysource.py
def main():
    config = load_config()
    state = load_state()  # last-seen cursor, optional

    for item in fetch(config, since=state.last_seen):
        payload = normalize(item)
        record_event.record_event(
            source="mysource",
            type="item",
            payload=payload,
            ts=item["timestamp"],
            actor=item["stable_id"],
        )

    save_state(state)

Four rules every adapter follows:

1. Idempotent. Running it twice in a row produces zero new events the second time. Dedup via actor fingerprint.
2. Pure function of source state. No hidden internal state that changes behavior between runs. If you must track a cursor, persist it to a state file that's rebuildable.
3. Crash safe. If the adapter dies mid-run, the next run picks up from the last successful event. Events are committed one at a time.
4. Shell-out to record_event.py. Don't touch the DB directly. Use the command-line interface with --payload-stdin.

The repo ships three canonical adapters: ingest_gdrive.py (the most complex — Drive traversal, doc extraction, chunking), ingest_claude.py (Claude Code session JSONL parsing), ingest_git.py (gh api + git log over a list of repos).

An hour is the right budget for a new adapter after you've written your first. The fetch loop is the only bespoke part. Dedup, payload shape, write path are all copy-paste from the canonical adapters.

The MCP Server

Agents never hit the database. They hit an MCP server. scripts/mcp_server.py exposes eight tools on stdio:

record_event        — write (typically called by agents to save decisions)
query_events        — filtered reader (by source, type, time window)
semantic_search     — vector search + payload join
get_journal         — compiled daily summary for a date
compile_journal     — force-rebuild a journal for a date
list_wiki_pages     — enumerate the curated wiki
get_wiki_page       — fetch one wiki page
health_check        — probe Ollama + Qdrant + SQLite

Claude Code on my laptop runs the server as stdio. An mcp-proxy wrapper exposes the same tools over HTTP/SSE on port 7778 so remote agents — Kai on the production VPS in Germany, Scout on the local agent box — call the same tool set without running their own server.

The tools compose. query_events narrows by source/type/time. semantic_search finds conceptually related rows. Agents chain them: "find all events on 2026-04-12 that mention the butterfly pipeline, then pull the git commit rows from those sessions."

The Views

Three views sit on top of the event log. Each is rebuildable.

Qdrant vector DB. embed_events.py runs on cron every five minutes. Finds new events, builds a text summary from payload_json, sends it to Ollama running nomic-embed-text locally on the RTX 3090, upserts a 768-dimensional vector into Qdrant. Zero external API calls. If Qdrant's disk dies I rebuild overnight.

Compiled journal. compile_journal.py --date 2026-04-14 groups every event from that date by source and outputs a readable markdown brief. Used by agents to answer "what did I do last Tuesday" without reading the raw log.

Curated wiki. Markdown tree with nine regions — agents, clients, daily-briefs, decisions, people, products, projects, systems, topics. 41 pages. compile_wiki.py reconciles it against the event log and surfaces new entities that should probably have a page. Each page is human-editable. The wiki is the curated layer, events are the raw layer.

Nothing stops you adding more views. Slack digest? Write a script that queries events and posts. PDF export of yesterday? Same. The views are small because the source of truth is the event log.

The Stack

Five components, all boring.

Component	Why
SQLite (WAL)	Fits on a USB stick. Never an operational issue at personal scale.
Ollama + nomic-embed-text	Local embeddings, 768-dim, no API cost.
Qdrant	Single Docker container, self-persists. Swap for pgvector if you prefer.
MCP (Model Context Protocol)	Lingua franca for agent↔tool. Works with Claude Code, Cursor, Codex, custom.
Python 3.11+	stdlib handles 90% of the work. Only deps: httpx, qdrant-client, mcp.

Every piece is sovereign. Nothing in the critical path talks to a vendor API. If a cloud goes down, nothing in the brain changes.

Failure Modes I've Hit

Ingester breaks, adapter reports success. Pull failed (auth expired, API changed) but the adapter caught the exception and exited cleanly. Zero new events, zero error log. I now require every adapter to write a health_check event per run whether it found new data or not. A missing heartbeat in the events table flags a broken adapter within a day.

Summary string too short, vector has no signal. embed_events.py originally built embeddings from a truncated payload summary. Narratives — long-form — were getting chopped before the meaningful bit. Different event types need different summary budgets. Fixed by adding per-type summary templates.

Auto-commit with empty commit message. The brain's own git history has dozens of commits titled snapshot 2026-04-12T01:01:29Z. Semantically invisible. The brain's development history is harder to search than my actual work. Still unfixed. On the todo list.

Qdrant drift. Vector DB and event log got out of sync after a disk event. Fixed by treating Qdrant as fully rebuildable and running a nightly consistency job. If Qdrant's count does not equal events count under the embedding policy, rebuild the missing range.

All four were caught only because the event log made them queryable. A system where health data lives in the same substrate as work data is a system where every problem has a query that finds it.

What I Didn't Build

A dashboard. I query the events table with SQL when I want to see something. If I ever want a dashboard, I'll write a view. Not a dependency.

A workflow engine. Cron handles scheduling. The adapter pattern handles retries. I do not have a DAG. I do not want one.

An auth layer. Network boundary is auth. MCP binds to localhost or my tailnet IP. If someone has network access they have data access. This is correct for a personal brain. If you're building a multi-user system, don't inherit this choice.

A multi-tenant schema. The brain is single-owner. Events have no owner column. A second user would need a second brain. This is a deliberate choice — the simpler schema pays for itself every day I don't have to think about tenancy.

How to Run It Today

Ubuntu or Debian, fifteen minutes:

sudo apt install -y sqlite3 python3-venv
curl -fsSL https://ollama.com/install.sh | sh
ollama pull nomic-embed-text
docker run -d --name qdrant -p 6333:6333 \
    -v $HOME/qdrant_storage:/qdrant/storage qdrant/qdrant

git clone https://github.com/cgallic/open-brane /var/lib/open-brane
cd /var/lib/open-brane
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

export BRAIN_DB=$(pwd)/events.db
sqlite3 $BRAIN_DB < SCHEMA.sql

echo '{"summary":"first event"}' | ./scripts/record_event.py \
    --source manual --type note --payload-stdin
./scripts/query_events.py --limit 5
./scripts/health_check.py

If the last command prints "all_healthy": true, you're running. Wire up the first adapter. Ingest Claude Code sessions (ingest_claude.py) — you already have the data on disk, and it's the highest-value source for most agent users.

The Pattern, Generalized

Open Brane is a specific implementation of a more general pattern: one append-only log, one write path, many views, agents query through a narrow tool surface.

That pattern works for more than personal memory. Incident logs. Customer interaction history. Content pipelines. Anywhere you have heterogeneous sources, mutable state that drifts, and agents that need consistent context — the pattern helps.

The reason it works is counterintuitive. Most data architectures keep the write layer rich and the read layer simple. The brain does the opposite. The write layer is as dumb as possible — no schema, no validation beyond "is it JSON." The richness is entirely in the views, which are cheap to throw away and rebuild.

This is the shape of systems that survive a year of use.

Repo: https://github.com/cgallic/open-brane

What's the one source you'd ingest first? Reply with it — I'll tell you whether there's a canonical adapter you can crib from or whether it's a new pattern worth documenting.